Meta Data: The Key To Decision Support
By David Marco
This article marks the first monthly column dedicated to meta data and data administration. During the coming months we will examine such topics as meta data ROI, architecture (distributed vs. centralized vs. decentralized), data administration staffing and organizing, meta data delivery, and the future trends shaping the meta data and data administration arena. In addition, we will examine some “best practice” case studies of companies that are doing meta data well. If you believe your company is one of them please feel free to contact me.
Building a meta data repository is critical for accessing, maintaining, and controlling the vital information stored in our decision support systems (DSS). While meta data has always been a central covenant of data warehousing, over the last couple of years it has been brought further into the spotlight as most Fortune 1000 companies have some sort of DSS system currently in place, most for several years. The vast majority of these companies have had to struggle with the task of managing the exponential growth of these DSS systems over time. Without meta data the task of managing this growth becomes overly difficult and time consuming. This need has driven many major software vendors like Microsoft, Platinum, Oracle, and IBM to enter the meta data marketplace with significant product offerings. This month’s column focuses on clearly presenting the benefits of implementing a meta data repository to support a company’s decision support system efforts.
Reduces Development Costs
DSS systems grow very rapidly, as a result the data warehouse will need to be modified through a process of iterative steps. Each of these steps will require an analysis of the current warehouse environment. The repository will significantly reduce the cost of development and the time frame needed to do it in. It accomplishes this by documenting the data transformation rules, data sources, data structures, and the context of the data in the data warehouse and data marts. This is critical because without the repository the transformation rules would only be contained in the IT staff’s memory. The meta data significantly aids the analyst as they examine the impact of proposed changes into the DSS environment. This benefit will reduce the costs of future releases and help to reduce the propensity of new development errors.
Improved Error Resolution
The meta data repository will reduce the turnaround time for production related problem resolution. If a DSS production problem is identified the development team can use the repository to quickly gather information related to the problem. This is very valuable, as the business users have come to depend on the information contained within the data warehouse to make their strategic decisions. The less “down time” the warehouse experiences the greater payback the business users will experience.
Delivering Business Intelligence
A central objective of any corporation’s business intelligence strategy is to improve the value that the information in the DSS system provides to the business user. The ultimate goal of the meta data repository is to drive the business user’s access to the information stored in the DSS system. This can be achieved as the business and technical meta data are directly linked to the information stored in the DSS system. This greatly increases the usability of the DSS systems to the business users.
To understand meta data’s vital role in the data warehouse, consider the purpose of a card catalog in a library (See Figure I). The card catalog identifies what books are in the library and where they are physically located. It can be searched by subject area, author, or title. By showing the author, number of pages, publication date, and revision history of each book, the card catalog helps you determine which books will satisfy your needs. Without the central card catalog information system, finding books in the library would be a cumbersome and time-consuming chore.
Meta data is the card catalog in a data warehouse. By defining the contents of a data warehouse, it helps the user to locate relevant information for analysis. In addition, the meta data allows the user to trace data from the data warehouse to its operational source (drill-down) and to related data in other subject areas (drill-across). By managing the structure of the data over a broad spectrum of time, it provides a context for interpreting the meaning of the information. As meta data is extracted and stored over several years, snapshots of the data exist for each year. In order to accomplish this the meta model tables need to be captured with a “From” and “”To” date on each column. This will allow the users to easily trace back through the repository to past versions of the meta data.
Figure 1: Meta Data Repositories and Card Catalogs
A meta data repository build with the business users in mind and created on a technologically sound architecture lifts the data warehouse from a stovepipe application to a true business intelligence system. Even with the immature state of the meta data repository marketplace, the alternative of not building a repository will not satisfy the needs of the business users or the data warehouse staff that will need to maintain the DSS system over time. This challenge of implementing a meta data repository is one of the chief mitigating factors that have prevented most organizations from achieving successful data warehouse and data mart implementations.