This article is adapted from the book “Universal Meta Data Models” by David Marco & Michael Jennings, John Wiley & Sons

In part 6, I presented the Meta Data Management Layer of a Managed Meta Data Environment (MME). This past column discussed the following Meta Data Management Layer functions:

• Archive
• Backup
• Database modifications
• Database tuning
• Environment management
• Job scheduling
• Load statistics

In part 7 I will complete my walkthrough of the Meta Data Management Layer by discussing the remaining functions:

• Purging
• Query statistics
• Query and report generation
• Recovery
• Security processes
• Source mapping and movement
• User interface management
• Versioning

Purging

This part of the Meta Data Management Layer defines the criteria for MME purging requirements. Your MME’s purging specific purging requirements and criteria will be governed by its business requirements. As a general rule, meta data that is inaccurate or improperly loaded should be purged; all other meta data should be archived.

Query Statistics

Once the various Meta Data Delivery Layer reports and queries are generated, it is important to capture the user statistics associated with the access to these reports and queries. At a minimum, the Meta Data Management Layer needs to historically capture:

• Who is accessing the report or query?
• What report or query are they accessing?
• When are they accessing the report or query?
• How often are they accessing the report or query?
• How long do they access the report or query?

Query and Report Generation

The reports and queries used in the meta data delivery layer are defined and generated in the report generation section of the meta data management layer. How this is accomplished depends on whether a custom meta data delivery application is developed or a meta data access tool is implemented. This component also needs to manage any publish and subscribe capabilities that are required.

Recovery

There are many situations that can cause a company to have to recover or reload a previous version of their MME: hardware failure, database corruption, power outage, errors in the Meta Data Integration Layer. The recovery process needs to be tightly tied to the backup and archiving facilities of the Meta Data Management Layer. Recovery processes my be manually built or utilize existing recovery processes within the meta data integration tool. Both approaches need to be integrated into the organization’s existing application recovery processes.

Security Processes

The Security processes manage:

• Creation of new MME users
• Grouping of MME users
• Setting MME user privileges and profiles
• Firewalls/Infrastructure
• Password management
• User location verification
• Meta data masking (data level or access level)

This extensiveness of security processes is dependent on the business requirements of the MME. For instance, the security requirements around a Department of Defense or Federal Bureau of Investigation (FBI) MME are far greater than that of a bank.

Source Mapping and Movement

Meta data sources need to be mapped to the correct attributes and entities of the Meta Data Repository. This mapping and movement process needs to be captured and managed in the Meta Data Management Layer of the MME.

User Interface Management

This is the processes for the building, managing and maintaining the Web site (recommended user interface) that the end users will visit to navigate through the MME. Typically the view (version of the Web site) end users see depends on their security privileges and profiles. A business user would not be interested in looking at program code changes, so it makes sense to not have meta data reports or queries related to highly technical meta data available to the traditional business user through control of role access.

Versioning

As meta data is changed, added, and purged it will be important for the meta data management layer to historically track (or version) it. There are two common techniques for meta data versioning. The first is the use of date stamps. Date stamping — assigning a date to each row of your meta model entities — allows a firm to make whatever changes are necessary to the meta data, while maintaining its historical relevance.

The second technique is to assign version numbers to each row of the meta model. Version numbers are merely generated unique values (e.g. 1.1, 1.2, 1.a, etc.). Version numbers are more limiting than date stamping, which offers more meaning to MME users. Version numbers can be associated with specific dates and times; however, this adds additional complexity to the loading of the meta data and an additional join in the access queries and reports.

Another challenge of using versioning is capturing meta data that will become current at some point in time in the future. For example, a new physical table may be moved into the production environment at a future date. To handle these versioning situations an “effective dated rows” can be useful. Effective dated rows is a process that allows data to be entered into a group (table) for subsequent processing when the effective date becomes current. Effective dated data can be historical, current, or future. Below are the key effective dated rows concepts:

Effective Data – The date on which the row of data becomes effective; the date it can be acted on. Effective Status – Allows a application to select the appropriate effective dated rows when combined with the effective date (Domain value list: “Active” or “Inactive”). Current Row – First row of data with an effective date equal to or less than the system date and an effective status of “active”. Only one row can be in this state. Future Row – Meta data rows that have a date greater than the system date.

Historical row – Rows of meta data that have an effective date less than the current row.

Table 1: Effective Dated Rows

Next month I will conclude our eight part MME odyssey by reviewing the Meta Data Marts and the Meta Data Presentation Layer.