Top 10 Mistakes to Avoid When Developing a Meta Data Repository (part 1 of 2)
By David Marco
Building a meta data repository is critical for accessing, maintaining, and controlling the vital information stored in our decision support systems (DSS). While meta data has always been a central covenant of data warehousing, over the last couple of years it has been brought further into the spotlight as most Fortune 1000 companies have some sort of DSS system currently in place, most for several years. The vast majority of these companies have had to struggle with the task of managing the exponential growth of these DSS systems over time. Without meta data the task of managing this growth becomes overly difficult and time consuming. This need has driven many major software vendors like Microsoft, Platinum, Oracle, and IBM to enter the meta data marketplace with significant product offerings.
Now before you run out and start implementing a repository, it is important to note that up to this point too few companies have successfully implemented a meta data repository that serves the needs of their business and technical users. This column initiates a two-part series of the top ten mistakes to avoid when developing a meta data repository. In part 1 we will review the first five mistakes.
1. Not defining the tangible business and technical objectives of the meta data repository
Quite often data administration teams will neglect to clearly define the specific business and technical value that their meta data repository will provide. These objectives are critical to define up front as they will guide all proceeding project activity.
Clear business and technical objectives are definable and measurable. This activity is imperative since once the meta data repository is completed the management team will have to justify the cost expenditures. Keep in mind a meta data repository, like a data warehouse is NOT a project, it is a process. The repository will need to grow to support the ever-expanding role of the data warehouse/data marts that it supports. In addition, as business users become more sophisticated their demands will substantially increase. Once a cost justification can be quantified for the initial release of the repository, the process for gaining funding for the follow up releases is greatly simplified.
2. Examining meta data tools before defining requirements
This is THE top mistake that most companies make. It is surprising how often I receive calls from companies requesting me to suggest a meta data tool for their repository project. My standard response is “what are your repository requirements”? Typically the reply from the other line is silence. This situation is highly concerning. The meta data repository requirements must guide the tool selection process, not precede it.
As we discussed, clear requirements for the meta data project are critical, as they provide the lighthouse for all subsequent project activities. Without this beacon it becomes all too probable for the project’s course to go awry.
3. Selecting a repository tool without conducting an evaluation
All of the major meta data vendor tools maintain and control the repository in a different manner. Finding the tool that best suites your company requires careful analysis. An educated consumer will be the most satisfied one because they understand exactly what they’re buying and what they’re not buying.
Remember whichever tool is purchased none of them make meta data “easy”, regardless of the marketing materials or salesperson’s hype. To be successful in your meta data project it takes
knowledge, discipline, talented employees, and good old fashion hard work, just like an other major IT endeavor. While none of the tools eliminate these needs, for most companies it is far better to purchase a tool and work around its limitations, as oppose to building everything from scratch.
4. Not creating a Data Administration team
Very often companies neglect forming a dedicated data administration team. This team will be responsible for maintaining, controlling, and providing access into the meta data repository. The typical data management team, at full staff will consist of 1 – 2 data modelers, 2 meta data integrators, 2 meta data access developers, 1 – 2 business analysis, meta data repository architect, and a project leader. Keep in mind some of the roles can be fulfilled by the same resource, depending on the size and schedule of the effort.
5. Having too many manual processes in the meta data integration architecture
The process for loading and maintaining the meta data repository needs to be as automated as is possible. Less than successful meta data implementations will typically contain far too many manual processes in their integration architectures. The task of manually keying in meta data becomes much to time consuming for the data administration team. With careful analysis and some development effort the vast majority of these manual processes can be removed.
Very often much of the business meta data will require some sort of manual activity just to capture the information. Additional processes will most likely need to be developed to allow the business leaders and analysts to modify the business meta data. Unfortunately, some companies manually key in a great deal of their business meta data, which makes the repository non-scalable and impossible to maintain over time.