Healthcare data can present a series of challenges not often found in other industries, sometimes based on the disparate systems and processes
Data management in many industries, such as travel, transportation, retail, finance, can seem similar to many practitioners. However, healthcare data management can be quite different, with many challenges in the nature of the systems and data they produce.
Data Capture
Healthcare’s unique data challenges start with the capture of data. Most businesses have core capture systems to collect transactions that define the nature of their business. Their sales and marketing departments use that data to adjust strategy, increase revenue, margin, and ultimately, profitability. In healthcare, the nature of the services provided has required many different systems be built to accommodate the business processes. Specific service areas often require quite different processes; from the Emergency Room to a clinical setting to surgery to labs. The medical practice areas also may operate separately; information about patients in oncology, orthopedics, ophthalmology, and other areas all require different data and data types.
Noting the need for many levels of data capture, healthcare has been busy over the years building many individual systems. Thus, the focus on an integrated system has undeniably been desired, but ultimately slow to develop enthusiastic support. The Electronic Medical Record (EMR) is the term used for a system that can capture comprehensive patient record and transaction data. Although EMR systems are common in practices, hospitals, etc., they do not represent patient record or transaction data uniformly. Many medical practices and hospitals have had to change business processes to fit the technical solution and the EMR processes’ flow. The challenge remains to leverage it as a single system of record, and to manage data across providers.
Data Types
All businesses have complexities within the data that is collected that go beyond what is expected initially. It would be simplistic to think of banking requiring only information about money that flows in and out of accounts. Upon investigation, it is easy to see the need for significant information about clients, demographics, and operations, etc. As businesses look for that advantage against competition, it reinforces a need to understand additional aspects of the influencers that affect their specific business.
In healthcare, there is incredible complexity just in the identification and storage of transactional data, for patients, providers, incidents, etc. Analysis of the data requires additional layers of data types and more complexity.
Unstructured Data
The clinical note is the basis for capturing a patient’s experience with a healthcare provider; in it, the patient visit is documented. Providers have their own style for recording the note’s content. More importantly, each system approaches the structure of a note differently. On the surface, it would seem that the healthcare experience could be improved if all systems were to adopt a common set of drop downs, check boxes, and note formats to standardize the capture of data.
In late 2005 at the Healthcare Data Warehousing Association annual conference, a physician spoke on the value of these notes. He stated, “We don’t want structured data, until we have to have it”. Medical professionals want to be able to leverage the information around clinical events so they can analyze and learn from it. However, they do not want to be restricted to the kind of structured system that would have them entering data into a unified format since they are not comfortable with it. Perhaps this physician and his colleagues were telling IT to hurry up and figure out how to transform unstructured clinical notes into structured data because providers cannot easily change how they enter it or analyze it.
Unstructured data also includes medical images – x-rays, MRIs, ECGs, video, etc. As medical technology evolves, more automation and procedure recordings will require additional and possibly more complex unstructured data.
Data Volumes
Many businesses have transaction volumes that create challenges in processing 24 hours of data in a 24 hour window just so they can put all of their data into an enterprise data warehouse. This “depth” of data is truly a significant challenge. For many large healthcare providers, they also have that “depth” of data to process.
In addition to “depth,” most providers have a “breadth” of data that may not exist in other industries. Like customer data, patient data exists about every aspect of a patient; their surroundings and demographics, family history, insurance coverage, etc. Consider all the various tests, lab results, medications, procedures, treatments, etc., that are captured for every patient. Each of these consists of many fields of surrounding data. One piece of important post-procedure data is the angle of the patient’s bed after certain procedures have been performed.
Specific Needs
Healthcare also has several other critical criteria to consider when designing a data management program.
- The need to reproduce results as they looked at the time of the analysis requires that data be versioned to a point in time.
- Metadata in healthcare takes on additional meaning from most other industries. There may not be the capability to search across the actual images, x-rays, tissue samples, and other pieces of unstructured data that healthcare has captured. Yet, searches can be performed across the metadata of that unstructured data.
- Consistent terminology – there are many standards and terminologies applicable to healthcare. That is the challenge: there is no single terminology that accommodates all of healthcare, to the business and to the patient. An enterprise data model is necessary to mitigate this issue. The time, resources, and experience to develop an enterprise data model with its business data definitions does not fit with the business demands for solutions that may be more urgently needed to save lives.
- Medical research is an enormous field starving for more information but challenged with navigating possible options for collecting, storing, and using data and metadata properly.
- Future trends in healthcare provide their own challenges for data management, including genomic and proteomic data.
Conclusion
All industries have their own unique needs that make data management challenging. Through the mountains of systems that exist in healthcare, achieving a mature data management environment and practice may seem unreachable. Before building solutions for healthcare, make sure to focus on two things: 1) The unique challenges faced by healthcare data management and the organization first; 2) Leverage proven experience in data management as the foundation for success.
