Top 10 Mistakes to Avoid When Developing a Meta Data Repository (part 2 of 2)
By David Marco
Many companies are realizing the invaluable role that the meta data repository plays in any successful data warehousing effort. This conclusion has occurred as corporations came to the discovery that a data warehouse implementation is a process and not a project. Typically an operational system is ran and built as a project. It has a specific end date and rarely undergoes major functional changes once implemented. On the other hand, decision support systems (DSS) function as a process. Once the warehouse is initially built is when the “real” work just begins. Once end users start uncovering the wealth of knowledge in their DSS systems they begin to ask for more and more functionality from their data warehouse. It is typical to see a data warehouse double in database size and in number of end users in the first 12 months of production. Without a carefully developed meta data repository the long-term grow of a data warehouse is greatly impeded.
This column is the second and concluding article on the top ten mistakes to avoid when developing a meta data repository. In the March issue of DM Review we examined mistakes 1 through 5. In this concluding installment we will present mistakes 6 through 10.
6. Having the Data Administration team report to the project manager of the DSS team
The head of the data administration team should report to the same manager as the head of the DSS team does. When the data administrator reports to the DSS manager quite often the meta data repository is forgotten or cast aside. The data administration team and the DSS team must work together as both of there work impacts the other. A muddled data warehouse architecture will directly impact the quality of the meta data repository. Conversely, a poorly designed repository will greatly reduce the effectiveness of the DSS system.
7. Letting the meta data tool vendors manage your project
All too often companies will be convinced to allow the meta data integration tool vendor to manage and implement their repository project. This is a critical mistake, as these vendors tend to be highly tool focused, as they rightfully should be. While the meta data integration tool is at the heart of the meta data process, it takes a lot more than a tool to create a fully functional repository. In addition, typical software vendor’s consulting staffs are not true integrators; instead they are tool experts.
8. Failing to have an experienced meta data project manager/architect leading the project
An experienced data administrator keeps the vision of the project in concert with the real-world reality of meta data and decision support. In addition, the architecture of the repository must be scalable, robust, and maintainable so that is can accommodate the expanding, and changing DSS, and meta data requirements. These fundamental challenges require a highly experienced, senior level individual.
If a consultant is used to initially get the project up and running it is imperative that the person is highly skilled at knowledge transfer and that an in-house employee has been assigned to shadow the consultant from the onset of the project. Be wary of consultants without real-world, hands on experience. It’s one thing to be able to write or speak about meta data; it’s entirely something else to have the experience needed to navigate through the political quagmires and the knowledge of what it takes to physically build a meta data repository.
9. Trivializing the meta data repository effort
All to often companies do not realize the amount of work it takes to build a meta data repository. Everything you need to do to build a data warehouse, you need to do to build a meta data repository. These tasks include defining business/technical requirements, data modeling, source system analysis,
source data extraction/capture, source data transformation, data cleansing, data loading, and end user access. In addition, it is best to develop the meta data repository iteratively. You don’t have to do everything all at once. However, when doing a project iteratively you must have the end result in mind at all times, as it will be your guiding wind.
It is important not to overlook the political challenges of the meta data effort. Politics cause the best-planned meta data and DSS projects to go astray. Remember cooperation will be needed from multiple IT and business teams to support the meta data effort.
10. The Data Administration team creates standards none of the supporting teams can follow
In order to capture much of the key business and technical meta data the data administration team will need to develop standards that the DSS team and business users can easily follow. Quite often the data administration team makes the processes and procedures for following these standards far too complex and tedious. When this situation occurs, the data administration team becomes viewed as a bottleneck to the DSS development process. At this point, normally it is a matter of time before the data administration team is disbanded. Make sure to keep all processes and procedures simple and easy to follow. In addition, keep the amount of time needed to complete them to a minimum and do not neglect to create a feedback loop so that the other teams can let you know how you’re doing.
11. Contractually obligate the meta data software vendor to provide a named architect
It’s impossible to encapsulate all of the meta data and data administration traps in a series of only 10 points, so here is a bonus tip.
As with all software, a meta data integration tool comes with a high learning curve. These learning curves have sunk more then one project. What will greatly reduce your risk of failure is to have a person that understands how to architect with the specific meta data integration tool. On the other hand, be prepared to pay a hefty fee for this person. Top of the line people are in high demand and more than make up for the investment in the amount of time and pain they can save. As a result, before the software is purchased I interview the proposed architect and we make that person’s time a condition of the sale.
Realize that these mistakes are all too common. If they can be avoided the likely hood of success for your meta data and data warehousing effort increase dramatically.