Managed Meta Data Environment: A Complete Walkthrough (part 6 of 8)
By David Marco
This article is adapted from the book “Universal Meta Data Models” by David Marco & Michael Jennings, John Wiley & Sons
Over the last several part of this series, I have been presenting the six major components of a Managed Meta Data Environment (MME):
-
Meta Data Sourcing Layer
-
Meta Data Integration Layer
-
Meta Data Repository
-
Meta Data Management Layer
-
Meta Data Marts
-
Meta Data Delivery Layer
In last month’s column I presented the Meta Data Repository and Meta data Integration Layers. This installment will discuss the MME’s fourth component the Meta Data Management Layer.
Meta Data Management Layer
The Meta Data Management Layer provides systematic management of the Meta Data Repository and the other MME components (see Figure 1). As with other layers, the approach to this component greatly differs whether a meta data integration tool is used or if the entire MME is custom built. If an enterprise meta data integration tool is used for the construction of the MME, than a meta data management interface is most likely built within the product. This is almost never the case; however, if it is not built in the product, than you would be doing a custom build. The Meta Data Management Layer performs the following functions:
-
Archive
-
Backup
-
Database modifications
-
Database tuning
-
Environment management
-
Job scheduling
-
Load statistics
-
Purging
-
Query statistics
-
Query and report generation
-
Recovery
-
Security processes
-
Source mapping and movement
-
User interface management
-
Versioning
Figure 1: Meta Data Management Layer
Archive
The archive function allows the meta data architect to set the criteria or event that triggers the MME archiving process. It should be able to archive all of the meta data in the Meta Data Repository and the related Meta Data Marts, and it should allow for specific meta data tables to be archived when necessary.
Backup
Backup functionality is often confused with archiving. Archiving targets the storage of older and less needed versions of the MME, while backups refer to the process of making sure the current MME is stored in a separate database so that if the production version of the MME is corrupted or if any of its components fail, a backup version can be brought online. Often the backup strategy is implemented at a hardware level through the use of disk mirroring. Best practices in this area include storing the copy in a different physical location than the production version of the MME.
Database Modifications
Since the meta model is implemented in an open, relational database, often tables and columns within the meta model need to be added, modified or deleted. The Meta Data Management Layer needs to not only assist in this process, but also track the changes that have been made to the MME.
Database Tuning
Tuning of the Meta Data Repository and its associated Meta Data Marts is a very important part of the Meta Data Management Layer. First, identifying indexes ensures that reports run efficiently. When analyzing physical meta model structures it is common to only see indexes on primary keys. This is typically a sign of poor or no indexing strategies.
Second, database tuning helps you identify and remove dormant meta data within the repository. A large MME that has been in production for a few years commonly has a good deal of dormant meta data. A sound MME will contain meta data that provides operational measurement statistics on the use of meta data in the MME to assist in the identification of dormant meta data.
Environment Management
Many meta data professionals make the mistake of believing that when they implement a MME that they are implementing and maintaining one system. In actuality they are building and maintaining three (possibly more) systems:
-
Production
-
Testing
-
Development
The production version of the MME is the system that is in the “production environment” of an organization and is the version of the MME that the end users are accessing. The testing environment is the version used to test changes to system “bugs” found in the Production version of the MME. The development version of the MME is used to test future, major MME enhancements.
The names and number of MME environments differ based on the organizations internal IT standards; however, the three environments mentioned above are the most common. In any event, a good meta data management layer can handle whatever number of environments and names that are required. The environment management portion of the meta data management layer needs to organize and control the management and migration between these three system versions.
Job Scheduling
The program and process jobs that are executed to load the MME and to access the MME need to be scheduled and managed. The job scheduling portion of the Meta Data Management Layer is responsible for both event-based and batch triggered job scheduling.
Load Statistics
The meta data extraction and integration layers of the MME generate a great deal of valuable MME loading statistics. These MME load statistics need to be historically stored within the meta data repository portion of the MME. Examples of the most common type of load statistics include:
-
How long did it take a particular process tread to run (clock time and CPU time)?
-
How long did the entire meta data extraction and integration layers take to run (both clock and CPU time)?
-
What errors were encountered in the meta data extraction and integration layers?
-
What were the categories (e.g. informational, warning, severe, critical, etc.) of the errors that were logged?
-
How many rows were inserted, changed, or deleted in each table of the meta model?
In next month’s column I will finish walking through the functions of the Meta Data Management Layer.