Module 4: Improving and Maintaining the Quality of EMR System Data
Welcome to this module on Improving and Maintaining the Quality of EMR System Data. Data quality plays an essential role in evidence-based decision making. Strong data quality supports strong decision making, leading to an increased demand for data. Poor data quality erodes confidence in the decisions made with such data and reduces the likelihood that decision makers will continue to seek out data to inform their decisions. Read the learning objectives to discover what you will learn in this module.
Learning Objectives
After taking this module, you’ll be able to:
- Identify key threats to the quality of EMR system data and how they affect the quality of health services
- Identify appropriate solutions when data quality is threatened
- Explain how EMR system features promote data quality
- Check paper and electronic forms for data quality
Learning Activities
-
Video: Improving and Maintaining the Quality of EMR System Data (2 min)
Welcome to this module on improving and maintaining the quality of EMR system data. We will start this module with an example of using an EMR for decision making.
Instructions: Click or tap the video to play.
-
Reading: Data and the Real World (5 min)
Let’s begin by answering a fundamental question. What is quality?
Quality is a state of being free from defects or deficiencies. It is the extent to which something has features which are good or bad.When we talk about data quality, we refer to the degree to which the data represents reality. The more the data mirrors the actual state of a patient’s or population’s health, the stronger we consider the data. When we talk about poor data quality, we mean that the data does not reflect the real world.
The quality of the data that is collected, stored, and managed by an health information system―whether it uses an EMR system or not―has an impact on the quality of the data that is reported, analyzed, and used for decision making.
-
Reading: Aspects of Data Quality (5 min)
Instructions: To learn more about these aspects, click or tap each button.
Accuracy
Also known as validity. Accurate data are considered correct; the data measure what they are intended to measure. Accurate data minimize error (e.g., recording or interviewer bias, transcription error, sampling error) to a point of being negligible.
Reliability
The data generated by a program’s information system are based on protocols and procedures that do not change according to who is using them and when or how often they are used. The data are reliable because they are measured and collected consistently. Example: Duplicate patient records exist because staff create new patient files without fully searching for an existing record, spell names differently, and other data entry practices. As a result of poorly followed practices, the reliability of the patient record is at risk.
Completeness
An information system from which the results are obtained is appropriately inclusive. It represents the complete list of eligible persons or units and not just a fraction of the list.
Precision
The degree to which repeated measurements under unchanged conditions show the same results. For example, if only the pediatric age field for “age in years” is available rather than “age in months” and / or “age in days” then this can result in measures which are imprecise. A healthcare worker cannot enter an infant’s age as “6 months” or “18 months.” The “age in years” field means that she must enter 6-month or 18-month-old infants as “1 year old.”
Timeliness
Data are timely when they are up-to-date (current) and when the information is available on time. Timeliness is affected by:- The rate at which the information system is updated
- The rate of change of actual program activities
- When the information is actually used or required.
Integrity
Data generated by an information system are protected from accidental manipulation or deliberate bias and/or manipulation for political or personal reasons. For example, only those with data management rights have access to the part of the system where patient records can be merged. This prevents others from making changes to a patient’s record. Another example is inflating the daily workload numbers in order to create demand for extra health personnel in a certain department.
-
Activity: Data Quality Under Attack (15 min)
Any aspect of data quality can be compromised. Let's look at what happens in three different scenarios where data quality is at risk. For each scenario, you will first hear about how data is being used by a decision maker. Then, you will be asked to reflect on what might happen as a consequence of using the data available in the situation. Finally, you will have a chance to choose what might have caused the data quality issues.
Once you have gone through the consequences and causes of one data quality scenario click or tap the “More Causes of Poor Data Quality” to reveal other potential threats to data quality.
- Checking patients out of visit
- Counting on CD4 count results
- Program enrollment
- More causes of poor data quality
Checking patients out of visit
Scenario:When a facility uses the EMR system at point-of-care, KenyaEMR records the date and time when patients are checked in for a visit and when they are checked out at the end of a visit. As patients arrive at the CCC each day, they go to Reception, where they are checked in for their visit. At the end of the patient’s clinical visit, some providers will check the patient out. At the end of some days, reception will check out any patients who have not been checked out yet. Some patients remain checked in. They may be checked out several days or weeks later.
Counting on CD4 count results
Scenario:During a patient’s six month follow up visit, Dr. Onyango orders the same, routine laboratory tests that he has ordered for this HIV positive patient for the last two years. Two weeks later, Dr. Onyango reviews his patient’s laboratory test results in the EMR System and is concerned to learn that the patient’s CD4 count is 170- low enough to consider placing his patient on ART. He begins writing out a prescription for ARVs and up-dating the patient’s chart. In up-dating the chart, his notes remind him of the patients’ clinical presentation, which did not include any symptoms that would suggest such a low CD4 count.
Dr. Onyango goes to the laboratory and asks the laboratory technician to review the results from his patient’s tests. The Laboratory Results Reporting Form that the laboratory technician gives to the clinician lists the patient’s absolute CD4 count as 710, not 170.
Program enrollment
Scenario:One of Nurse Wanyama’s MCH patients recently gave birth at the maternity. Nurse Wanyama knows from the patient’s EMR record that she is HIV positive and that she had been receiving PMTCT services. She recognizes that the patient’s infant son may have been exposed to HIV and proceeds to set up an EMR record for the child in order to document and link the infant to the right services. Nurse Wanyama enters child’s family history and then enrolls the infant in the HIV program - instead of the MCH program.
More causes of poor data quality
These three scenarios brought common threats to data quality to light. More causes of poor data quality may include:- Person recording or entering data accidentally transposes numbers
- Accidentally skip a field and enter data into the next (wrong) field
- Poor or lack of understanding of how fields are to be filled
- Workload of person recording or reporting data is so great that they do not gather all of the data
- Wrong keystrokes
- No follow up with missing data on registers or records when compiling reports
- Importance of a particular piece of data is not known or well understood, and therefore not collected
- Quality of the data collection tools may be affected by frequent changes, insufficient or untimely supply and illegible content
- Particular pieces of data are assumed to be collected at other points in the process (age, sex, weight, drugs dispensed, lab results)
- Records or registers do not collect all of the data needed for reporting or decision making
- The system is not working as expected
- Poor understanding of medical terminology and or program indicators
- Handwriting in registers or records is illegible
-
Quiz: How EMR Systems can help (5 min)
Many of the same causes of data quality issues in a paper system arise where an EMR system is also in use. However, the likelihood that some of the same causes will occur is reduced―though not entirely eliminated―by several unique features of an EMR system.
Instructions: See if you can match the cause of poor data quality with the data quality supports found in an EMR system. Click or tap the drop-down menus to match each cause with the correct data quality support.
-
Reading: Improving and Maintaining Data Quality (5 min)
Once we know the probable causes of poor data quality, it is easier to assess, improve, and maintain data quality. Formal processes have been designed to detect existing data quality problems, remedy them, and prevent new problems with data quality from developing: data quality assurance and data quality control.
What, when, and how data quality is assessed usually indicates whether national or program-specific data quality assurance or data quality control procedures are being followed. For instance, formal data quality assurance activities, such as data quality audits, take place periodically (quarterly, semi-annually, annually, etc.). They typically involve assessing data at multiple sites and administrative levels in order to identify system-related issues or problems. These activities use a set of data quality indicators for each aspect of data quality as well as specific data quality assessment or audit tools to measure the quality of data.
Facilities may also have their own plan or practices for monitoring and maintaining data quality on a daily, weekly, monthly, or quarterly basis.
-
Reading: When is Data Quality Assessed? (5 min)
Data quality can be assessed at any three points in time: before, during, or after data collection. Some assessments will look at different aspects of data quality during these periods. For instance, a form’s design can be assessed before data is collected to determine if it supports the reliable collection of correct data.
Instructions: Click or tap each button to see examples of what can be assessed at each point in the timeline.
-
Activity: Taking Action to Improve Data Quality (15 min)
While most of the assessment methods you just learned about are used in formal data quality assurance and data quality control activities, users still play an active role in assuring data quality. Let’s return to our three data quality scenarios and see what options are available to staff to address the data quality issues that you identified earlier.
Instructions: Click or tap each button to see what action the data clerks and clinicians can take in response to the data quality threats they encountered earlier.
Checking patients out of visits
Counting on CD4 Count Results
Program Enrollment
-
Reading: What users can do to improve data quality (5 min)
Remember, you don’t have to wait until the MOH team arrives at your facility for a data quality audit to do something about the quality of the data in your EMR system’s database. Any of these actions will increase the likelihood that data in the database will reflect reality.
- Check data entry before saving the form
- Check that the correct patient record is open before entering data
- Flag paper forms with missing or inaccurate data
- Alert providers or others to complete or correct data when errors are detected
- Collect data using established and consistent protocols and procedures
- Respect time schedules for data entry and reporting as indicated in your facility’s SOPs
- Give feedback and discuss data quality during review meetings, routine staff meetings and CMEs
- Make user manuals and SOPs accessible to staff using the EMR system
-
Reading: What data managers can do to improve data quality (5 min)
HRIOs and others who manage data quality can take additional actions to correct and improve data. Once data has been incorrectly entered into the system, responsibility for correcting the error no longer lies with the user. This is taken up by data managers. In addition to the actions taken by users, those who manage data quality may:
- Create a quality assurance plan
- Visually inspect completed forms
- Run EMR system data quality reports to identify missing data
- Make data quality reports available to users
- Routinely (daily, weekly) check EMR system data against a random selection of paper forms
- Correct inaccurate data and fill in incomplete data detected
- Control user correction of data errors
- Give feedback of data quality results and recommendations
- Resolve identified causes of data errors, such as merging duplicate patient records
- Implement identified system changes
- Communicate with users
-
Reading: Data Quality and EMR Systems (5 min)
Good quality data for decision making is the result of good data entry, data quality control, and assurance mechanisms. In addition to the actions users can take to monitor and maintain data quality, EMR systems have built in features that are designed to support the entry of quality of EMR system data.
Notice that these features either guide users to enter quality data or enable users to identify at-risk data. EMR systems do not magically prevent or fix data quality issues. The responsibility for ensuring data quality remains with users and those responsible for assuring and controlling data quality.
-
Reading: Where Does Responsibility Lie for Data Quality? (5 min)
Instructions: Click or tap each button to learn what responsibilities lie with the EMR user or the EMR system.
EMR System Users
- Check data entry before saving the form
- Check that the correct patient record is open before entering data
- Routinely (daily, weekly) check EMR system data against a random selection of paper forms
- Run EMR system data quality reports to identify missing data
- Flag paper forms with missing or inaccurate data
- Collect data using established and consistent protocols and procedures
- Respect time schedules for data entry and reporting
- Learn to use the EMR system and follow data quality procedures
EMR System
- Use menus or checkboxes to standardize responses
- Use standardized date format, ranges, calculations
- Use data concepts from a standard data dictionary
- Use alerts to flag required fields to ensure completeness
- Prevent duplication of patient records by using unique patient identifiers
- Compile data quality reports to identify key data missing from records
-
Activity: You make the call (15 min)
Instructions: Let’s see how well you can spot problems with data quality. Click or tap each button to learn more about these data quality issues.
Mixed up patient files
Scenario:You are the data manager at Dangazeli District Hospital. Your current aim is to reduce the number of duplicate records in KenyaEMR. Each week, you review possible duplicate records and merge those that belong to the same patient. During your review, you come across these two patient records. Take a close look: do these two records belong to the same patient?
Missing data
Scenario:Tomorrow is the weekly staff meeting and you want to give feedback on how well staff are picking up KenyaEMR and entering complete data. You check the total number of patients in the database (121) and run Patients Missing Entry Point and Patients Missing date confirmed HIV+ reports. Less than 10% of patients are missing either piece of data.
Entering data retrospectively
Instructions: Test your knowledge with 2 questions about entering data retrospectively
Question 1
Can you find the four data entry errors on this MOH257?
Click or tap on the data fields to find the data entry mistakes.
Reason
The facility where the patient was confirmed HIV positive is missing. The patient record is incomplete.
Regimen
This family member is HIV positive; the In Care and CCC Number fields should be filled. The patient's family history is incomplete.
Who
The date the patient is enrolled in HIV care is missing.
Height
The patient's ART history has not been filled. This results in an incomplete patient record.
Try again
Please try another spot on the form.
Question 2
Can you find the four data entry errors on the EMR system’s triage form?
Click or tap the data fields to flag the data entry mistakes.
Date
This data is not entered in a timely manner. The form is being filled on 28 November when the patient was checked in on the 26 November.
BP
This data is not complete since the clinician did not fill both fields.
Respiratory Rate
This data could be inaccurate if the digits were transposed.
Vital Signs
The vital signs did not change from the signs recorded at the patient’s January visit. If the provider copied these signs instead of recording the patient’s actual vital signs, then the data’s integrity is at risk.
Try again
Please try another spot on the form.
Case Scenarios Conclusion
Let’s look at the different dimensions of data quality that can be compromised on the MOH257 or in the EMR system’s triage form. Both forms can have problems with accuracy, completeness, and precision. What about the other dimensions?
Both forms can have any number of fields with missing data. You can see here on the MOH257 that these include the date the patient was enrolled in the HIV program and where. On the EMR system’s triage form the blood pressure was not completely filled.
Here you can see that the MOH257 form included a birth date with an illegible birth year. One of the advantages of an EMR system is that entries are always legible. While the precision of the data recorded on the triage form may not be clearly at risk, free text fields where entries can lack adequate detail and precision present a risk.
The MOH257 does not indicate when the form was filled, making it difficult to know if it was completed in a timely manner.
In many cases, it is difficult to know from looking at the forms what data entry errors are the result of staff receiving poor training or the lack of processes and procedures to follow. If staff were not aware what fields must always be filled, for instance, then the data’s reliability will weaken over time.
Both forms can have problems with the integrity of their data. An incomplete patient clinic number on the MOH257 could mean that the wrong data is being recorded on the wrong record. And here, the vital signs entered on the EMR’s triage form appear to be copied from a previous visit, meaning that they may not reflect the patient’s current condition.
-
Reading: Summary (5 min)
When clinicians and managers have access to quality data, they can be confident in the decisions they make using that data. The scenarios that you have worked with in this session shows that poor data quality can have serious consequences, including failure to provide patients with the care and treatment that they need. You have also learned what causes different data quality problems and the actions you can take to improve data quality.
EMR systems can make data management easier in many ways. EMR systems have some features to support data quality, such as alerts, reports, and patient record merging. Remember that EMR systems cannot always prevent data entry errors or other data quality issues and that EMR systems do not always detect or fix such problems either. Responsibility for data quality lies with the people using the EMR system: receptionists, nurses, clinicians, data clerks, and managers. Now go on to the final quiz.
-
Quiz: Post-Test (15 min)
After you take the module, please be sure to take the post-test. Read each question and select the correct answer. Please note that the post-test will open in a new website page. You will need access to the Internet to complete the post-test.
Click to go to the post-test Post-test Module 4