Enterprise Architecture Score Card

When is an enterprise architecture good enough?

One of the major principles in enterprise architecture is described best in this quote:

“An enterprise architect knows he has achieved the perfect solution not when there is nothing left to add, but when there is nothing left to take away.” [Saint-Exupery]

Before reviewing an enterprise architecture, we must define the criteria of how to review it. The criteria have a strong dependency on the goals and objectives of what has to be achieved with that enterprise architecture. Therefore, before starting an enterprise architecture study, the evaluation criteria must be defined. This document defines the criteria by which the Oklahoma Department of Human Services (OKDHS) enterprise architecture efforts will be measured.

The Enterprise Architecture Score Card™ is based on a methodological approach for the different enterprise architecture results of different enterprise architecture process steps. Six areas are reviewed and scored on an annual basis by the EPMO office, and the results are posted on the OKDHS Web site. Along with the results, focus areas and goals are set for the following year.

Explanation of the used criteria and terminology

The Enterprise Architecture Score Card™ is used as a measurement instrument to check the quality of the enterprise architecture efforts by answering the questions based on the assessed status, with the goals and objectives of the enterprise architecture program in mind.

Every question is assessed for the areas of:

• Business
• Information
• Information Systems
• Technology Infrastructure

A special item focuses on the level of alignment or integration between these areas or how holistic the approach was and how holistic was the documentation.
For each of these areas, the result of each question is assessed, using three statuses:

• Status 0 = Unknown and not documented (red)
• Status 1 = Partly known and partly documented (yellow)
• Status 2 = Fully known and well documented (green)

In addition to these three values, the level of alignment or integration for each question is also assessed. The answer for each question encompasses the elements of knowledge and documentation. Having the knowledge without documenting it means maintenance cannot be done, and the knowledge is not transferable to others.

Subtotals and totals reflect the valuation for the quality of the assessed enterprise architecture results, and for the addressed completeness of the enterprise architecture process phases. A more in-depth insight and overview of the quality of the enterprise architecture effort can be derived based on this approach, and steering can be done in areas with less quality.

Questions with status 1 must be examined more in more depth to get more information about the availability, dependency, quality, and level of documentation. It is important to document the rationales of decisions made during the enterprise architecture process.

Questions with a status 0 must be concentrated on for development.

Listed are the questions and the 2006 results.

Question	2006 Score	2007 Focus
Are the mission, vision, goals, and objectives of the enterprise architecture set?	0.58
Is the scope of the enterprise architecture program defined?	0.87
Is the form and function level of deliverables defined?	0.12	X
Are the business and IT strategies defined?	0.62
Are the guiding principles and drivers defined?	0.05
Are the key performance indicators defined?	0.58
Are the critical success factors defined?	0.16	X
Are the critical stakeholders defined?	0.71
Are the collaborative parties involved defined?	0.29
Are the contractual agreements defined?	0.19
Are the interoperability standards defined?	0.04	X
Are the related law and regulations defined?	0.41
Are the functional requirements defined?	0.58
Are the non-functional requirements (quality attributes) defined?	0.16	X
Are the concepts in use defined?	0.33
Are the security requirements defined?	0.50
Are the governance requirements defined?	0.66
Are there deliverables at the logical level?	0.54
Are the critical logical design decisions defined?	0.08	X
Are the critical logical design decisions traceable?	0.08	X
Are the logical description methods and techniques defined?	0.04	X
Are you using tools for modeling at the logical level?	0.66
Are there logical standards?	0.41
Are there deliverables at the physical level?	0.58
Are the critical physical design decisions defined?	0.12	X
Are the critical physical design decisions traceable?	0.04	X
Are the physical description methods and techniques defined?	0.12
Are tools for modeling used at the physical level?	0.54
Are the physical standards set?	0.58
Are the critical design decisions defined?	0.04	X
Is the organizational impact known?	0.24
Are the costs and consequences known?	0.08	X
Is the security impact known?	0.24
Is the governance impact known?	0.24	X