Staffing A Data Warehouse – Part II

By Bruce Johnson

In my last article, I began a 3-part series designed to outline criteria for staffing a data warehouse program within a corporation.  Part I focused on breaking down the various roles, thoughts on how to organize those resources, and considerations for handling support and development.

In part II, we will look deeper into how to build and retain a strong organization.  This is a significant challenge that corporations face, and more often than not, struggle with.


Building The Right Organization

As you can imagine, there could be unlimited thoughts and opinions as to how to best build the right organization.  Some criteria are consistent across all organizations, while others need to be specific to the type, size, and style of solution that your organization is building.  Regardless, if your approach is something you make up without any feedback or input by those that have seen success, you are likely to be challenged to achieve success.  If you do look for help here, be cautious of looking for advice from consultants that have never been permanent employees in successful data warehousing organizations.  They typically don’t understand corporate politics, organizational dynamics, and operational needs.

In part I, we outlined the various roles that were present in a data warehouse effort or program.  Based on the roles required for your organizational requirements, we can now begin an abbreviated outline of some thoughts about how to go about staffing that area.

The Hardest Roles To Fill  
Some roles are invariably tough to fill no matter what.  Here are a few that I have had the hardest time staffing.  An IT manager with proven experience leading and staffing a successful data warehouse effort is a must.  Yet, if they were good, they likely either get promoted or leave for other greater opportunities.  A lead architect role – to be successful, you have to select someone that understands many design alternatives and can fit the right architecture to your business needs.  An ETL architect and DBA – ETL resources are some of the most technical that you will require.  Thus, good ETL architects are very hard to find – most are consultants.  A BI Architect with experience with many tools and technologies and the ability to work with users to interpret and define what solutions they really need to their business challenges.  Lastly data modelers with experience in data warehouses are rarely available.  They are very helpful when they can work side by side with the ETL developers and DBAs on one team.

What skills are you really looking for?  
Quite often the focus is on the specific technical toolsets that you deploy – ETL, BI, Meta data, etc…  While this can be important, your odds of finding key resources that know exactly your tool and your data is almost nil.  If you find someone has experience with those specific tools, it is likely that they do not have architecture or design experience.  Putting someone in charge of analysis and design of any individual area is probably the most significant decision that will impact your end success.

Attracting The Right Employees
There are many different web sites you can use to highlight your open positions.  Make sure you have a good story.  These candidates are going to have many opportunities available to them and they need to know why yours is the one they will enjoy and appreciate the most.  Leverage users groups and lists (another reason to have your organization active in local users groups – others will hear your story).  When you do get a strong lead, follow-up quickly and decisively.  Networking can be invaluable and having leadership that is well connected will expose additional opportunities.

Supplementing With Consultants And Contractors
If your goal is to get up and running quickly, you will likely have to bring in some outside help.  I separate consultants as those that will provide guidance and strategy, while contractors will work with your employees or alone to get the work done.  While it is desirable to have internal employees learn all of the facets of what it takes to deliver a data warehouse solution, training alone cannot accomplish this.  I have been an internal employee for some large corporations facing just this challenge and invariably I find the best solution is to bring in some external guidance and let my key resources know that they have an opportunity to learn from them.  That way you make progress while developing expertise with on the job seasoning.

Retaining A Strong Team
As you have found in trying to recruit skilled resources, the available resources that have what you need are very small.  When you are successful in developing sound skills and experience, it is critical that you keep a focus on retaining those resources.  I have picked a few key items that I have seen that will help resources stay engaged.

Treat This Like A Profession  
I went to school for IT and have tremendous respect for how large the field of IT now has grown.  It is impossible to know every aspect of technology today, other than at a cursory level.  Data warehousing specifically is challenging in part because it is treated as just another application to develop.  If you do not learn the high level concepts of data warehousing and then have a chance to implement these components multiple times for multiple needs, you will not have a significant understanding of how/why/what it takes to be successful.  Once resources are trained and experienced, they desire to feel appreciated for those special skills.  Even though data warehousing has now been around for almost 20 years, many organizations have specific salary adjustment criteria for this special skill.

Professional Growth Opportunity
Many, but not all employees want to know that as they are successful in their job and demonstrate effort and desire to succeed that they have an opportunity to receive a reward/promotion.  I certainly hope this sounds ridiculously simple management to any manager reading this.  Unfortunately, I frequently run into people that do not reward these resources with special skills. Some managers are more concerned about taking everything the team does for their personal gain.  Want to have employees move out quickly, look out only for yourself and you will find yourself alone or surrounded by resources who are not very good at their job.  If employees start leaving, don’t reward the ones that are not strong skilled and hard workers, change your approach.

Many resources want to learn and recognize the value that seasoned experts in this field provide.  Having access even on a limited basis to these types of resources helps your employees grow and they know it.  I have often kept some strong architectural consultant on retainer where I can bring them in here and there for a week or even a few days to help employees with some planning, design, or just process discussion.  This helps provide that extra knowledge that helps employees grow and gain confidence.

Lastly, an additional area to provide learning is in training.  For almost all resources, technical and business, there is an unlimited amount of growth in data warehousing.  Once you learn, you need to practice and demonstrate.  Since there are so many areas of growth in fully understanding every aspect of data warehousing, learning can be a never-ending experience.  The resources that desire this type of learning are exactly the resources that you need to support – they are your future.  You need to encourage attendance at users groups – these are free activities that usually only take a day out of the office on a quarterly basis.  There are also seminars, conferences, and classes that will help those individuals to grow.  I have always tried to explain to IT leadership that data warehousing areas need extra large training and travel budgets as there is a lot to learn and the learning continues.  It is also thus important that you require resources who attend training to share their findings with the remainder of the team, helping everyone to grow.



Building up an effective organization is almost always a challenge and it takes time.  Those wanting to be fully up and running in an instant are relegated to either paying very high salaries for unproven resources or using consultants to get the ball rolling.  In either case, getting internal or external employees to adopt your strategy and define processes that will make your solution effective is not easy and cannot happen quickly.  Skirting the critical steps of defining good processes will only lead to either delivering little value (can you see funding disappear) or having to rework your solutions.

About the Author

Bruce has over 20 years of IT experience focused on data / application architecture, and IT management, mostly relating to Data Warehousing. His work spans the industries of healthcare, finance, travel, transportation, retailing, and other areas working formally as an IT architect, manager/director, and consultant. Bruce has successfully engaged business leadership in understanding the value of enterprise data management and establishing the backing and funding to build enterprise data architecture programs for large companies. He has taught classes to business and IT resources ranging from data modeling and ETL architecture to specific BI/ETL tools and subjects like “getting business value from BI tools”. He enjoys speaking at conferences and seminars on data delivery and data architectures. Bruce D. Johnson is the Managing director of Data Architecture, Strategy, and Governance for Recombinant Data (a healthcare solutions provider) and can be reached at

Free Expert Consultation