How To Keep Small Efforts Small
Small, focused analytical efforts can be of great benefit to an organization. They are often used to serve a specific need, such as build a pilot solution to prove value for a larger future effort, or provide an enhancement to an existing solution. Have you ever been challenged by what should be a short, quick, analytics project that turns into a time and cost nightmare? Unfortunately, one area that many organizations struggle with is the ability to take a small, reasonably sized analytics project and deliver it in the estimated time frame. Let’s take a look at some of the key criteria required to realize the rewards a small project can deliver.
Keys To Success
Sizing The Effort Properly
Users are satisfied by a system if it meets the data they need and has a method of utilizing that data that is efficient and easy to use. Some efforts are deemed small due to the fact there is a limited number of fields that need to be put into the system – even though the various methods of using the data are huge. Or vice versa – an effort only has a couple of front end deliverables, but there is a voluminous amount of data that must be migrated, integrated, validated, and deployed for usage. Make sure that your organization is being honest with itself when saying this effort is small and should be delivered in 60, 90, or 120 days. You know your organization, culture, resources, and past project history. Ease of use and limited options when using will help speed adoption and drive business requests for more data and solutions.
Start The Clock Once The Effort Starts
It sounds pretty simplistic to say that it doesn’t start until you get going. Yet, it is common for management to look at the date of approval of an effort as the start date. When you bring your proposals forward for funding/endorsement, make sure to clarify that there is project setup that needs to take place before the project will actually start. Frequently that takes over 2 weeks and sometimes more. It is also crucial that you put a focus on duration before you start – i.e. the project will last for 12 weeks. Then, once the project charter is approved and the requirements are defined, you give a specific end-date (it should either be less than 12 weeks or you should explain that you found additional requirements and the scope is now slightly enlarged). Ideally, you have a formal project status report that is delivered upwards on a weekly basis on an effort like this. That will assist in keeping leadership aware of the now defined timeframe. It will also expose potential roadblocks to success at the earliest possible time, enabling them to step in and assist before the project falls past the critical path for success.
Tools and Infrastructure
If you are going to build an effort rapidly, you have to have the right tools and technologies in place. Most corporations have a hard time defining hardware and toolsets to use in these types of efforts. Let’s look at this from a few angles:
- Infrastructure – having hardware platforms, servers, disk, network connectivity, and protocols in place prior to starting will go a long way to getting this project off and running. If you have to define your infrastructure, purchase the various layers, wait to receive them, then install, setup, and validate these solutions, you are probably already past what I would call a timeline for a small solution. If you have a solution set that you can easily plug into your existing infrastructure in short order, or have an existing server to host the solution on, you are better set for success.
- Tools – Having the various data warehousing, business intelligence tools for building and deploying your solution already in house is a great starting point. For example, don’t plan on trying to do an evaluation of ETL tools as a first component in your project. This is a project in and of itself and for most companies it is not a small project. Since there are many layers of technology possible in these options, you need to simplify any new technology you are looking at, to as few different vendors as possible. I always tell clients that do not have an ETL tool to build all ETL for a small project in house. ETL solutions are hard to implement, hard to learn, and hard to deploy. When you are working with a large data warehouse and have resources that will constantly be using these tools, they become invaluable. When you don’t, they are ridiculously costly and have a severe negative cost to the project from a timeframe perspective.
- Data Quality – All data warehouses expose data quality issues, whether it is in population of the data into the warehouse or what the user encounters using the front end. For smaller efforts, you can afford less time to perfect the data. Too often technical resources and business resources say their data in their warehouse must be perfect. This is a noble gesture, but it is usually not an option if you want to deliver analytics in a reasonable timeframe. It also covers up the flaws in your systems and may not accurately reflect the business you are running. For example, if you reject a sales record for some client address validation, you then won’t have the $ from that transaction reflected in your reports. Did you receive that money? Yes. Should it be reflected on financial reports? Yes. Is there a data flaw? Yes. On smaller efforts, data quality is indeed important, but correcting all data problems prior to deploying a solution will cause you to miss date after date and eventually turn into a credibility issue with the business.
Technical Leadership / Project Management
Project management of day to day tasks on small projects is best done by the lead designer that knows exactly what is to be produced. They are the ones that know the needs and the design well enough to keep focused. The challenge these resources often encounter is keeping the business from coming up with more and more ideas, thoughts, creative uses as the project proceeds. This lead designer should not even be the one spending time validating scope change, all evaluation of scope change in and of itself negatively impacts project progress. If it is absolutely necessary, scope can be changed, however, then the timeframe, resources, or cost must also change.
If you asked most data warehousing consultants, they would likely say that requirements definition would be the main factor in keeping this type of effort on track. In many ways, I agree. However, defining a key portion of data, and then determining a few select ways of using that data is really the secret. A wise person I worked with at the Mayo Clinic used to advise project managers that “Good Enough” needed to be balanced against having the optimal solution. As much as we would all prefer to build the perfect solution for any business need, in almost every instance, it is simply not practical. This is even more important when trying to manage a small focused effort. If you keep asking questions to validate the business needs, you will continue to come up with valid business needs that would indeed be of benefit. Adding data sources, derivatives, views, tools/techniques for using data, etc… This is no different than trying to come up with a plan to build your dream house. The more you review the plan with people once it is done, the more ideas you come up with and the more changes you have. It is much easier for us to feel the impact of those changes in our house design all throughout the effort than it sometimes is when working with our users. It hits our pocket book and delays us from getting what we ultimately want, our new house. This sounds to me like one way we can get a perspective of how a user feels when they see a project spiral out of sight on cost and timeframe (even if it is their changing requirements that caused it).
One advantage of keeping a focus on the data provided and only selected analysis solutions is that you can deliver on time and then continue to add reports/analytics to this existing data as the user starts to work with it. They will naturally come up with what would be the most valuable next solution to provide. If need be, you can enhance it later by adding additional data as well.
You can still realize the long-term solution you want to build, if you plan for the future and carefully manage change control throughout your project. Strive for perfection and you may never get that first solution delivered successfully.
Too Many Cooks In The Kitchen
For smaller efforts, you must have someone in charge that has proven success with delivering this type of effort. It is then imperative that you give them the freedom and support to manage the project as their seasoning leads. If multiple people try to oversee or direct the actions of someone with proven experience, you are essentially overriding that experience that you brought in, negating the value. Instead, try having your resources (including management) watch and learn. This will help you build up a staff of resources that, in turn, can lead similar projects and successfully deliver. Generally, the people that I know that do this really well do it really well almost all the time. The only exception is when they are directed by those that don’t have specific experience and knowledge to be giving that level of direction.
Small, focused efforts can be the right solution to your business analytics need. Too often companies look to an enterprise data warehouse when the right answer for them is a specific solution. Right sizing your infrastructure, architecture, and development toolsets can only be defined once you understand what problem you are trying to solve. Then it is time to start the project clock ticking. The ability of your project managers to keep the scope contained and to communicate within the team and through management, will enable the project to stay on track with expectations. By closely managing change, and keeping distractions out of the way, your resources will be able to focus on designing a database to capture the data needed. Using the available front-end tools, they can build a couple of reports/analytics to get the adoption started. Like all great solutions, your business will then continue to mature in how you use that data and the value that is realized.
About the Author
Bruce has over 20 years of IT experience focused on data / application architecture, and IT management, mostly relating to Data Warehousing. His work spans the industries of healthcare, finance, travel, transportation, retailing, and other areas working formally as an IT architect, manager/director, and consultant. Bruce has successfully engaged business leadership in understanding the value of enterprise data management and establishing the backing and funding to build enterprise data architecture programs for large companies. He has taught classes to business and IT resources ranging from data modeling and ETL architecture to specific BI/ETL tools and subjects like “getting business value from BI tools”. He enjoys speaking at conferences and seminars on data delivery and data architectures. Bruce D. Johnson is the Managing director of Data Architecture, Strategy, and Governance for Recombinant Data (a healthcare solutions provider) and can be reached at firstname.lastname@example.org