Managed Meta Data Enviornmnet: A Complete Walkthrough (part 8 of 8)
By David Marco
This article is adapted from the book “Universal Meta Data Models” by David Marco & Michael Jennings, John Wiley & Sons
Over the last seven columns I have been presenting the six major components of a Managed Meta Data Environment (MME):
Meta Data Sourcing Layer
Meta Data Integration Layer
Meta Data Repository
Meta Data Management Layer
Meta Data Marts
Meta Data Delivery Layer
This eight and final installment will discuss the MME’s fifth and sixth components; Meta Data Marts and the Meta Data Delivery Layer.
Meta Data Marts
A Meta Data Mart is a database structure, usually sourced from a Meta Data Repository, designed for a homogenous meta data user group (see Figure 1). “Homogenous meta data user group” is a fancy term for a group of users with like needs.
Figure 1: Meta Data Marts
There are two reasons why an MME may need to have meta data marts. First, a particular meta data user community may require meta data organized in a manner other than what is in the Meta Data Repository component. Second, an MME with a larger user base often experiences performance problems because of the number of table joins that are required for the meta data reports. In these situations it is best to create meta data mart(s) targeted specifically to meet those user’s needs. The Meta Data Marts will not experience the performance degradation because they will be modeled multidimensionally. In addition, a separate meta mart provides a buffer layer between the end users from the Meta Data Repository. This allows routine maintenance, upgrades, and backup and recovery to the repository without impacting the availability of the Meta Data Mart.
Meta Data Delivery Layer
The Meta Data Delivery Layer is the sixth and final component of the MME architecture. It delivers the meta data from the Meta Data Repository to the end users and any applications or tools that require meta data feeds to them (Figure 2).1
Figure 2: Meta Data Delivery Layer
The most common targets that require meta data from the MME are:
Data warehouses and data marts
End users (business and technical)
Messaging and transactions
Meta data marts
Web sites and e-commerce
Quite often applications (customer relationship management (CRM), enterprise resource planning (ERP), and so on) need to receive meta data from the MME for their own use. In these situations it is most common to have the meta data repository create an extract file that can be brought into the application. Typically the repository will generate a flat file and place it in a holding area that, when the application is ready, can read it in.
Data Warehouses and Data Marts
The Meta Data Delivery Layer data warehouses and their associated data marts (usually query and reporting are executed at the data mart level) are separate from applications because of some subtle differences in the use of the meta data. Figure 3 shows the data mart query and report bringing in meta data from the MME. Typically data marts are accessed via front-end tools (Business Objects, Cognos, Hyperion, Microstrategy, and so on). These tools generate SQL code. Since the Meta Data Repository component is stored on an open, relational database it is easy enough to “point” these tools at the repository and bring the meta data directly into the query/report (See Figure 3 for an example).
Figure 3: Meta Data Delivery Layer: Data Warehouse and Data Marts
For some data warehousing applications where the data in the data mart(s) is voluminous or the number of end users is high, the overhead involved with going to a separate database may create too great a time delay for end users. These technical implementation issues can be remedied by loading the meta data directly into the data marts (Figure 4) during the marts load cycle.
Figure 4: Loading Meta Data Directly Into The Data Marts
The meta data delivery layer typically brings meta data directly to both business and technical end users. Usually this meta data is delivered directly to the user’s computer in the form of a document or spreadsheets or through a “thick” or “thin” client front-end meta data access tool.
Messaging and Transactions
As previously discussed, many companies use some form of messaging and transactions — whether EAI or XML — to transfer data from one system to another. Although most companies are not very advanced in their use and application of EAI or XML, these types of applications do utilize meta data. If companies continue to grow these applications, their need for meta data will continue to rise.
Meta Data Marts
As discussed earlier in this article, there are situations where meta data will be extracted from the repository and brought into a Meta Data Mart component. These Meta Data Marts are a database structure designed for a homogenous group of meta data users.
Sharing and interchange of meta data among various software tool’s internal meta data repositories is particularly desirable for global enterprises with dispersed teams trying to solve similar or related data analysis problems using an integrated computing approach.2 Hopefully, industry meta model standards like Common Warehouse Metamodel (CWM) and ISO11179 should make this effort easier. Today, most companies have to analyze the software tool’s meta models and then build technical processes that will share meta data between these tools. Anyone that has had the opportunity to build and maintain these types of processes will attest to how difficult it is.
Some MMEs need to send meta data to third parties; vendors or suppliers, customers, government agencies or regulatory bodies, and business partners (Figure 5). Typically when meta data is exchanged with these third parties, it is done through the generation of a flat file; however, more and more companies are beginning to use XML as transportation syntax.
Figure 5: Meta Data Delivery Layer: Third Parties
Web sites and E-Commerce
Web sites and e-commerce applications also need meta data. Meta data-related Web sites are a very effective way to present meta data to the end users.3 E-commerce — with its trend towards XML and its need to interact with customers and partners — will continue to trend towards needing meta data in their processes.