Basics About Zachman framework

With inspirations from traditional architectural and engineering discipline, John Zachman started the Zachman framework with its first publication in 1987. Zachman framework provides a simple (at least it looks simple) but well-structured approach that supports many ways of architectural presentation using a common vocabulary and viewpoints.

To be more specific, Zachman framework describes a model of an enterprise system from six perspectives, Planner, Owner, Designer, Builder, Subcontractor and the working system.  The major feature in Zachman framework is this matrix with 36 cells (example picture from Wikipedia) indicating all "possible" aspects to an enterprise system.

Here are the rows of the matrix:
Scope, Business Model, System Model, Technology Model, Components, Working system

Here are the columns of the matrix:
Who, when, why, what, how, where

As indicated in the publications about Zachman framework The major principles behind Zachman framework are:

1. "A complete system can be modeled by depicting answers to the following questions: why,
who, what, how, where, and when."
2. "The six perspectives capture all the critical models required for system development."
3. "The constraints for each perspective are additive; those of a lower row are added to those of
the rows above to provide a growing number of restrictions."
4. "The columns represent different abstractions in an effort to reduce the complexity of any
single model that is built."
5. "The columns have no order."
6. "The model in each column must be unique."
7. "Each row represents a unique perspective."
8. "Each cell is unique."
9. "The inherent logic is recursive."

This framework with 36 cells can be used recursively to management the overall enterprise systems and each components inside it. One can build the 36 cells in different levels to keep views of an enterprise system at different levels.  Implementing Zachman framework must recognize two critical challenges in current enterprises.

1. One must beginning making the descriptive representation of enterprise more explicit.  This means to populate the various cells of Zachman framework
2. Formalize a system to keep populate the different cells in a disciplined way.

There can be two different approaches to get a company a quick-start on using the Zachman framework.

1. Start developing the cells with the purpose of developing an enterprise architecture strategy
2. Start developing row 1 of the matrix in order to establish an implementation road map.

As claimed by different parties, one drawback of this framework is that it does not have so much public, open documentation as compared to TOGAF. However, from my own experience, this is also a good part of a framework where certain spaces are left for the enterprise itself to find out and define the discipline.   To many enterprise architects, working by using everything available makes their life easier and simple. While to some other enterprise architects, having more space to dig out valuable works for the enterprise is of more fun and self-satisfying.

Understanding TOGAF

The Open Group Architectural Framework (TOGAF) is considered to be the most often used framework for the future. Here are some notes that I've taken after reading the first introduction of TOGAF. 

TOGAF support the most commonly accepted architecture types, i.e., business (or business process) architecture, application architecture, data architecture, technology architecture. A key concept built-into TOGAF is that it is driven to create an open systems-based architecture so that solutions made at an enterprise are not relying on any single suppliers but focused on ensuring an open and integrated solution such that multiple vendors and heterogeneous systems can be added at any time without a major cost or risk.
 

TOGAF consists of three parts. 1) The TOGAF architecture development method, which explains how to build an enterprise architecture for a specific enterprise in order to fulfill the business requirements. 2) The Enterprise Continuum whch is a repository of all architecture assets such as models, patterns, etc. 3) The TOGAF Resource Base which is a set of resources (templates, guidelines, handbooks, etc.) that facilitates the development of the architecture.
 

In the TOGAF dictionary, "architecture" is often interpreted as "view" of things. For example, there  will be a business view, a data view, or a technology view of enterprise systems. This is quite typical in the enterprise architecture world. Just like the architecture of a building, it is always useful to present the architecture through different "views" to suit the need of different purposes, such as a construction view, a electricity supply view, a networking view, a security view, etc. 

In fact, creating and using views is the one of the key activities in TOGAF. Seeing the enterprise as a whole thing, different people (in other words, the stakeholders) tend to looking at this "whole thing" from different view points (i.e., perspectives) because they are different "concerns" (which is the key interest that defines the acceptability of any systems inside this whole thing). The focus inside TOGAF is to ensure that all necessary viewpoints are identified and satisfied. The following three steps have to be repeated very often.

    - Choose key stakeholders
    - Understand and document their concerns
    - Figure out how one models and deals with those concerns

TOGAF provides a taxonomy of views but it does not mean every enterprise (who employed the TOGAF concept) must develop all items provided. In fact, it is always important to "pick to use" what is beneficial to an enterprise, which is determined by the architect team. One of the key points to ensure the success of making different views to different stakeholders is to ensure that everyone use the same language (i.e., architecture terms, elements, definitions) to communicate.

Purpose of having an Enterprise Architecture

Enterprise Architecture is closely connected to an overview of the whole. It does not matter if an enterprise means a large corporation, or a division, or a department. Enterprise Architecture is to keep an eye on the overall interest of this “enterprise.”

So what’s the business case of having an Enterprise Architecture? One thing for sure is that people tend to follow the trace of “standardization” when an EA process is driving. And this leads to a better commoditization of software and systems within an organization, i.e., controllable operating cost and predictable development time. When having an established EA process, people tend to have better communication effects and are able to make design decisions in shorter time (this is extremely important for large organizations where escalations and politics are popular). And, the most important of all, an EA tells the people, especially the leaders, the directions to go, the initiatives to take, and the strategies to rely on.

Perhaps a better way t to look at this is to see the situation when EA is not in place. Without EA, an organization definitely experiences one or more of the followings.

• Many locally optimal, rather than globally optimal solutions exist and cannot removed
• Very high operating cost due to the fact that things are not globally shared and re-used
• Too much dependency on a vendor or a legacy application (no one knows who's using it, no one knows anything about it, no one dares to close it down)
• Most solutions are created for short term purpose and limits the long term benefit

• IT standards are not respected

It is worth noting that even lots of IT functionalities are outsourced to different vendors, EA is still vital and even more critical in such situations. With more than one vendors pressing the CFO for different payments, the EA team must guardian the enterprise to avoid being a cash-cow kind.

6 Questions for Understanding Enterprise Architecture

Many companies are implementing enterprise architecture these years. IT employees or “senior” IT employees are upgraded to the “shining” title of “Enterprise Architect.” Is it true that Enterprise Architect is just another rank name in deciding the seniority of employees or is it something really creating benefit for a company?

The best way to find out all these answers is to understand what is Enterprise Architecture.  

Wait a minute. I am not going to start writing massive content about the definitions, theories, and so on. Instead, I will list the key questions that one should ask about Enterprise Architecture. How about the answers? Well, they are in wiki pages, google results, books, presentations, and most important of all, your own analysis and judgements.

1. To understand Enterprise Architecture, the first question is “what is Enterprise Architecture?” Or, one could start even one step back. What is “Enterprise?” Is it a multi-nation corporation or the IT department or a whole business division? And what is “architecture” or what is “IT architecture?”

2.  Why do we need Enterprise Architecture? Of course we know that it is about fulfilling the business requirements. But there can be different ways of solving business requirements. Why do so many of us choose to use the EA approach?

3. When talking about IT architecture or software architecture, many books and articles mentioned about different levels or levels of abstraction. For example, there can be business architecture, solution architecture, reference architecture, data or information architecture, operation architecture, technology architecture, etc. What are the right ways of understanding these levels? How are they connected?

4. Are there any existing concepts, frameworks, guidelines, and systems about Enterprise Architecture (oh yes, there’s plenty and more coming). What are their differences?

5. How about a short history of Enterprise Architecture? Why did we come to this era that we all need the EA. What was it before? How did enterprise IT evolve over the past many years? Besides the history, how’s the progress and new topics in this discipline? Where do I get the latest trend and what are the trend now?

6. What about the business case for implementing Enterprise Architecture? How and why did so many companies accept to implement EA?

OK. Enough about the questions. Here are two exercises if all the above questions are answered.

E1. Consider IT systems and solutions as an commodity is a natural way to implement cost-saving for any enterprise. More and more companies are seeing the need of using "out of the box" software rather than self- developing IT systems as a "piece of art" which can only serve specific purposes. In parallel to this, we have been seeing the new evolution of IT software that enables simple and quick customizations which makes a standard software quickly fit well into an organization with a compromise between the end-users "never satisfied needs", the business sponsors' "cost-reduction while shorter time-to-market demand focus" and the IT technicians' "quality and easy to maintenance awareness”. Providing this scenario (which is very typical in companies), how is EA’s role in such a situation? How should EA solve or resolve or manage such scenarios?

E2. Many companies consider cost-saving and EA is a great area to focus on cost-saving. But is this enough? How about getting more profit? Can we use EA to gain profit for the company?

The Need of Overviewing Enterprise Data Lineage

The concept of data lineage comes to proliferation in the Information Management world due to the need for more detailed, quality-focused metadata of the data flows and deliveries across enterprise IT systems.

In a large enterprise where regulatory reporting, dash boards, score card and analytics are widely used, different IT systems need to delivery their data to fulfill the various purpose of business intelligence applications. As a best practice, such enterprise would already have established a data warehouse team to maintain a central repository of all kinds of data. Use of data warehouse makes the architecture of enterprise data flows quite simple. The warehouse is just like a centralized hub that collects all kinds of input and delivers outputs to various parties. And in such case, managing data flows and data deliveries across the enterprise seems simple (although still very challenging) because all management focus can be put around the data warehouse. Data lineage in such an architecture form is focused on the lineage of data pre- and post- the data warehouse.

An very good example is the data lineage tools inside the IBM InfoSphere Metadata Workbench. In the data lineage tool, a source delivery file can be connected to an ETL job that takes this file to the data warehouse table. Then the table can be connected to the ETL job that loads the data warehouse table to the data mart tables which are then extracted and loaded to BI reports. With a naive (yes, not native but naive) support of defining external applications and extension mappings, the data lineage can be (manually) extended to external systems and tools to include the whole data life cycle, i.e., starting from the front-end system where the data was initially created, to the deliveries, the data warehouse, the data mart, and the BI applications, not to mention the various ETL jobs, data quality checks, etc., during the "life cycle" of this data-to-information flow. 

The data lineage functionality in InfoSphere Metadata Workbench does give us an good example of how enterprise users can benefit from having such a view across different architectural elements. Here is a short list:

1. Traceability of technical elements brought by the data lineage functionality can reduce the cost of system maintenance and support. For example, a developer in the system management group will easily find out all the relevant tables and jobs that are related to a piece of  "problematic" data. 
2. The cost of doing impact analysis can be dramatically reduced if the data lineage information is available. Whenever a change is to be applied on an architectural elements, estimating the impact in terms of cost or even possibility  can save a great budget. 
3. The capability of aligning architecture designs and to considering optimal refactoring options can be further strengthened if a detailed data lineage map is available. It is much easier for an IT architect to make the design options if the relevant data flows, together with the performance and throughput data, are presented on a diagram (which makes the job of IT architect more or less similar to an construction architect's work).

After seeing the benefits of data lineage, we need to come back and challenge if we need to have an overview of the whole data lineage across the enterprise IT systems instead of only those around the data warehouse. 

Typically, most of the data need can be fulfilled by requesting a data delivery from the data warehouse to any business intelligence applications. In many enterprises, lots of IT systems act as a "hub" of certain group of enterprise data and generate outputs by adding "values" to the data. For example, a CRM system or an accounting system normally need to take data like customer, branches, agreements from other systems and then create output data with more "corrected," "aligned," "calculated" information. From an enterprise point of view, the output from such "intermediate" systems is also a main data source for the data warehouse. And the data flows from and to these systems are equally or even more "mission-critical" than the data warehouse.

Another example is the case when there is much need on making "operational reporting" or "operational business intelligence" in a real-time or "near real time" manner.  In such cases, event-based architecture is used to send data directly to the business intelligence applications. No data warehouse is involved here (except that an end-of-day summary may be delivered to the data warehouse through a batch-window). Such data flow often requires great architecture care due to its mission-critical purpose.

So, it does make lots of sense to maintain an overview of the whole enterprise data lineage on top of all IT systems, not just around the data warehouse. As up to now and up to my knowledge, there has not been any great software product that can fulfill this need completely. We need to cross our fingers and see what the vendors (I expect IBM, ASG, and maybe some start-ups) can bring to the table in the future.

How to start the implementation of InfoSphere Business Glossary

InfoSphere Business Glossary (IBG or BG) is a very useful "Data Concept Management Tool" to connect all users of data at an enterprise. BG is part of the IBM Information Server tool suite and is intended to be used as a enterprise glossary of business concepts and terms. With the metadata interchange support and the Information Server Metadata Server, a business term described in BG can be connected to a technical asset such as a column in a table, an ETL job that populates a table, a field member of a Business Intelligence report, or flat file contained in a delivery between enterprise applications. Another key feature of BG is the use of data steward. A data steward (person, role or group) can be assigned as an owner of a business term defined in BG. This provides an opportunity of creating an enterprise data governance structure. The ownership concept provides an enterprise-wide awareness of data and quality of data.
 
Implementation of BG normally requires a team of allocated resources (e.g., a project team or a task force) to start an initial "enabling" phase to define the work process and to actually implement a substantial amount of key business terms and concepts. The following is a list of key issues or topics that such team must work with in order to create a successful initial implementation of Business Glossary.

1. Creating a data owner/steward role in the enterprise
    Apparently, making terms without knowing if it is correct or wrong does not make any sense at all. The enterprise must have employees that care and use the data that a term describes. Designing a role like data steward will not cost the employee a huge amount of extra working hours. On the long term perspective, the data owner can guardian the data and quality of data which will, in return, helps the enterprise to make the right business decisions based on the reports/dashboards/mining results that are based on the data.

2. Designing a basic category structure in BG
   Business Glossary is a very simple tool. All business terms are located in categories. All categories are having a tree structure. A term physically located in one category can be "referred" by another category. A term can be connected to other terms as synonyms.
   Generally, a good structure in BG contains top categories that classifies the business concepts. For example, there can be a category called "Customer" which includes all kinds of business terms and concepts regarding customers to this enterprise. If all the top categories in BG are about such kind of major business conceptual areas, for example, "Customer," "Branch," "Product," "Agreements," etc., any users in the enterprise will find it not difficult to browse around to understand all the key business areas and business concepts. Besides, the search functionality in BG is well enough for any newcomers to look around.

3. Designing the work processes around BG
    Typically, when users start to use BG, there will be requirements on how to let data steward edit the terms, how to manage the life cycle of terms (in BG, a term can be "standard," "accepted," "candidate," or "deprecated"), and how to make sure that all contents are having back-ups etc. There can be many more such questions to the maintenance team of the BG content.
    In addition, when BG is used and maintained in a strict enterprise environment where there can be TEST, FREEZE and PRODUCTION instances, the work processes seem to be more important.

4. Marketing and making organizational implementation
    A tool like Business Glossary does not contain any fancy and complicated functionality such that all users of the enterprise can suddenly start enjoying and cheering up every time they find it useful. The team must go out and communicate with almost everyone in the enterprise in order to have a good start on the tool. Here, the data stewards team should be the first group that get familiar (and enjoyable) with the tool and start (with their charms) to influence other enterprise users as "Business Glossary advocates" until everybody likes it.

5. Starting populating the terms
    Normally an enterprise must have several key business concepts that are used throughout almost all activities/documentations, such as customer, internal units, employee, etc. It is very unlikely that one cannot find any employee that cares about terms and concepts in these core conceptual areas. In other words, one should find it possible and possibly easy to define and describe terms and assign data stewards in these core concepts. Giving a dedicated period and resource to populate terms in such core business conceptual areas gives a huge positive impact on the success of Business Glossary.

The concept of having a Business Glossary is to use the "crown sourcing" power to improve and align the organization on the data and information usage. It paves the way for successful business intelligence implementation in the long run. Hopefully, we could expect more social and collaborative features from Business Glossary in the future.

The dilemma of consistency in technology architecture

As one of the key elements of enterprise architecture, the "platform" or "technology" architecture is about what kind of software and hardware tools and systems to use across the whole organization.

One can imagine that there are ways of maintaining a strategy of having a "buying list" of these systems for an enterprise. Historically or even nowadays, large enterprises tend to put a lot of efforts on maintaining the "open system" concept. Here the "open system" concept means that an enterprise use software systems that accepts and enables  interoperability, portability and open software standards. But in reality, not all software vendors give full-support to open standards and using systems from various technology and vendor background means extra cost of ownership to the IT managers. Thus, the following two phenomenas show the general dilemma of most enterprises towards technology choices.

• Due to business and economical requirements, different departments are becoming more and more vendor dependent in spite of the enterprise-wide strategy of supporting open system concept. 
• Although most departments are using standard products, no matter if they come from a single vendor a or limit group of them, different local units still choose to use packaged solutions (often called appliances)  in situations where they often find it hard (or expensive) to use the standard tools.

So should an architect put all her/his effort to ensure the consistency of the usage of standard tools?

Although consistency is definitely a personal virtue, it is absolutely not expandable to IT architecture. Instead of wasting all the effort to keeping the consistency, it is more useful to focus on maintaining the architectural strategy in a timely manner and getting ready to accept new ideas and changes from time to time.

In terms of integration of different systems, it is always important to consider a set of protocols in the infrastructure when adding new items in the buying list.

1. Middlewares and gateways that enable the integration;
2. Communication protocols such as web services
3. Information brokers, such as those that transform the data-types, char-sets (ASCII to EBCDC), or XML-based transformation
4. BPM tools that cope with processes at various frequencies;
5. Event and alert management tools;
6. Message-based systems, i.e., solutions that can keep messages for various systems. 
7. Application-oriented adapters that support integration of applications with other existing solutions.