Module 4: Improving and Maintaining the Quality of EMR System Data

Welcome to this module on Improving and Maintaining the Quality of EMR System Data. Data quality plays an essential role in evidence-based decision making. Strong data quality supports strong decision making, leading to an increased demand for data. Poor data quality erodes confidence in the decisions made with such data and reduces the likelihood that decision makers will continue to seek out data to inform their decisions. Read the learning objectives to discover what you will learn in this module.

Learning Objectives

After taking this module, you’ll be able to:

  1. Identify key threats to the quality of EMR system data and how they affect the quality of health services
  2. Identify appropriate solutions when data quality is threatened
  3. Explain how EMR system features promote data quality
  4. Check paper and electronic forms for data quality

Learning Activities

  • Video: Improving and Maintaining the Quality of EMR System Data (2 min)

    Welcome to this module on improving and maintaining the quality of EMR system data. We will start this module with an example of using an EMR for decision making.

    Instructions: Click or tap the video to play.

  • Reading: Data and the Real World (5 min)

    Let’s begin by answering a fundamental question. What is quality?
    Quality is a state of being free from defects or deficiencies. It is the extent to which something has features which are good or bad.

    When we talk about data quality, we refer to the degree to which the data represents reality. The more the data mirrors the actual state of a patient’s or population’s health, the stronger we consider the data. When we talk about poor data quality, we mean that the data does not reflect the real world.

    The quality of the data that is collected, stored, and managed by an health information system―whether it uses an EMR system or not―has an impact on the quality of the data that is reported, analyzed, and used for decision making.

    /data-real-world
  • Reading: Aspects of Data Quality (5 min)

    Instructions: To learn more about these aspects, click or tap each button.

    Accuracy

    Also known as validity. Accurate data are considered correct; the data measure what they are intended to measure. Accurate data minimize error (e.g., recording or interviewer bias, transcription error, sampling error) to a point of being negligible.

    Reliability

    The data generated by a program’s information system are based on protocols and procedures that do not change according to who is using them and when or how often they are used. The data are reliable because they are measured and collected consistently. Example: Duplicate patient records exist because staff create new patient files without fully searching for an existing record, spell names differently, and other data entry practices. As a result of poorly followed practices, the reliability of the patient record is at risk.

    Completeness

    An information system from which the results are obtained is appropriately inclusive. It represents the complete list of eligible persons or units and not just a fraction of the list.

    Precision

    The degree to which repeated measurements under unchanged conditions show the same results. For example, if only the pediatric age field for “age in years” is available rather than “age in months” and / or “age in days” then this can result in measures which are imprecise. A healthcare worker cannot enter an infant’s age as “6 months” or “18 months.” The “age in years” field means that she must enter 6-month or 18-month-old infants as “1 year old.”

    Timeliness
    Data are timely when they are up-to-date (current) and when the information is available on time. Timeliness is affected by:
    • The rate at which the information system is updated
    • The rate of change of actual program activities
    • When the information is actually used or required.
    Integrity

    Data generated by an information system are protected from accidental manipulation or deliberate bias and/or manipulation for political or personal reasons. For example, only those with data management rights have access to the part of the system where patient records can be merged. This prevents others from making changes to a patient’s record. Another example is inflating the daily workload numbers in order to create demand for extra health personnel in a certain department.

  • Activity: Data Quality Under Attack (15 min)

    Any aspect of data quality can be compromised. Let's look at what happens in three different scenarios where data quality is at risk. For each scenario, you will first hear about how data is being used by a decision maker. Then, you will be asked to reflect on what might happen as a consequence of using the data available in the situation. Finally, you will have a chance to choose what might have caused the data quality issues.

    Once you have gone through the consequences and causes of one data quality scenario click or tap the “More Causes of Poor Data Quality” to reveal other potential threats to data quality.

    Checking patients out of visit
    Scenario:
    When a facility uses the EMR system at point-of-care, KenyaEMR records the date and time when patients are checked in for a visit and when they are checked out at the end of a visit. As patients arrive at the CCC each day, they go to Reception, where they are checked in for their visit. At the end of the patient’s clinical visit, some providers will check the patient out. At the end of some days, reception will check out any patients who have not been checked out yet. Some patients remain checked in. They may be checked out several days or weeks later.
    CheckIn

    Question: When clinic staff check patients out at different times in the CCC's workflow, which of the following do you think will happen? Select all that apply.


    Question: What do you think causes such poor quality data? Select all that apply.

    Counting on CD4 count results
    Scenario:
    During a patient’s six month follow up visit, Dr. Onyango orders the same, routine laboratory tests that he has ordered for this HIV positive patient for the last two years. Two weeks later, Dr. Onyango reviews his patient’s laboratory test results in the EMR System and is concerned to learn that the patient’s CD4 count is 170- low enough to consider placing his patient on ART. He begins writing out a prescription for ARVs and up-dating the patient’s chart. In up-dating the chart, his notes remind him of the patients’ clinical presentation, which did not include any symptoms that would suggest such a low CD4 count.

    Dr. Onyango goes to the laboratory and asks the laboratory technician to review the results from his patient’s tests. The Laboratory Results Reporting Form that the laboratory technician gives to the clinician lists the patient’s absolute CD4 count as 710, not 170.

    Question: What do you think will happen as a result of the problems with the data? Select all that apply.


    Question: What do you think could have caused the poor data on CD4 count? Select all that apply.

    Program enrollment
    Scenario:
    One of Nurse Wanyama’s MCH patients recently gave birth at the maternity. Nurse Wanyama knows from the patient’s EMR record that she is HIV positive and that she had been receiving PMTCT services. She recognizes that the patient’s infant son may have been exposed to HIV and proceeds to set up an EMR record for the child in order to document and link the infant to the right services. Nurse Wanyama enters child’s family history and then enrolls the infant in the HIV program - instead of the MCH program.
    Enroll

    Question: What do you think will happen as a result of enrolling the infant in the HIV program instead of the MCH program? Select all that apply.


    Question: What do you think could have caused the poor data on program enrollment? Select all that apply.

    More causes of poor data quality
    These three scenarios brought common threats to data quality to light. More causes of poor data quality may include:
    • Person recording or entering data accidentally transposes numbers
    • Accidentally skip a field and enter data into the next (wrong) field
    • Poor or lack of understanding of how fields are to be filled
    • Workload of person recording or reporting data is so great that they do not gather all of the data
    • Wrong keystrokes
    • No follow up with missing data on registers or records when compiling reports
    • Importance of a particular piece of data is not known or well understood, and therefore not collected
    • Quality of the data collection tools may be affected by frequent changes, insufficient or untimely supply and illegible content
    • Particular pieces of data are assumed to be collected at other points in the process (age, sex, weight, drugs dispensed, lab results)
    • Records or registers do not collect all of the data needed for reporting or decision making
    • The system is not working as expected
    • Poor understanding of medical terminology and or program indicators
    • Handwriting in registers or records is illegible
  • Quiz: How EMR Systems can help (5 min)

    Many of the same causes of data quality issues in a paper system arise where an EMR system is also in use. However, the likelihood that some of the same causes will occur is reduced―though not entirely eliminated―by several unique features of an EMR system.

    Instructions: See if you can match the cause of poor data quality with the data quality supports found in an EMR system. Click or tap the drop-down menus to match each cause with the correct data quality support.

  • Reading: Improving and Maintaining Data Quality (5 min)

    Once we know the probable causes of poor data quality, it is easier to assess, improve, and maintain data quality. Formal processes have been designed to detect existing data quality problems, remedy them, and prevent new problems with data quality from developing: data quality assurance and data quality control.

    What, when, and how data quality is assessed usually indicates whether national or program-specific data quality assurance or data quality control procedures are being followed. For instance, formal data quality assurance activities, such as data quality audits, take place periodically (quarterly, semi-annually, annually, etc.). They typically involve assessing data at multiple sites and administrative levels in order to identify system-related issues or problems. These activities use a set of data quality indicators for each aspect of data quality as well as specific data quality assessment or audit tools to measure the quality of data.

    Facilities may also have their own plan or practices for monitoring and maintaining data quality on a daily, weekly, monthly, or quarterly basis.

  • Reading: When is Data Quality Assessed? (5 min)

    Data quality can be assessed at any three points in time: before, during, or after data collection. Some assessments will look at different aspects of data quality during these periods. For instance, a form’s design can be assessed before data is collected to determine if it supports the reliable collection of correct data.

    Instructions: Click or tap each button to see examples of what can be assessed at each point in the timeline.

    Before

    Form design, staff training on data collection and data quality

    During

    Check data entry for completeness and accuracy

    After

    Gauge the completeness, timeliness, and accuracy of collected data

  • Activity: Taking Action to Improve Data Quality (15 min)

    While most of the assessment methods you just learned about are used in formal data quality assurance and data quality control activities, users still play an active role in assuring data quality. Let’s return to our three data quality scenarios and see what options are available to staff to address the data quality issues that you identified earlier.

    Instructions: Click or tap each button to see what action the data clerks and clinicians can take in response to the data quality threats they encountered earlier.

    Checking patients out of visits

    Question: What are some ways to address this problem? Select all that apply.

    Counting on CD4 Count Results

    Question: What are some ways the software can be improved to address these problems? Select all that apply.

    Program Enrollment

    Question: What are some ways to address these problems? Select all that apply.

  • Reading: What users can do to improve data quality (5 min)

    Remember, you don’t have to wait until the MOH team arrives at your facility for a data quality audit to do something about the quality of the data in your EMR system’s database. Any of these actions will increase the likelihood that data in the database will reflect reality.

    • Check data entry before saving the form
    • Check that the correct patient record is open before entering data
    • Flag paper forms with missing or inaccurate data
    • Alert providers or others to complete or correct data when errors are detected
    • Collect data using established and consistent protocols and procedures
    • Respect time schedules for data entry and reporting as indicated in your facility’s SOPs
    • Give feedback and discuss data quality during review meetings, routine staff meetings and CMEs
    • Make user manuals and SOPs accessible to staff using the EMR system
  • Reading: What data managers can do to improve data quality (5 min)

    HRIOs and others who manage data quality can take additional actions to correct and improve data. Once data has been incorrectly entered into the system, responsibility for correcting the error no longer lies with the user. This is taken up by data managers. In addition to the actions taken by users, those who manage data quality may:

    • Create a quality assurance plan
    • Visually inspect completed forms
    • Run EMR system data quality reports to identify missing data
    • Make data quality reports available to users
    • Routinely (daily, weekly) check EMR system data against a random selection of paper forms
    • Correct inaccurate data and fill in incomplete data detected
    • Control user correction of data errors
    • Give feedback of data quality results and recommendations
    • Resolve identified causes of data errors, such as merging duplicate patient records
    • Implement identified system changes
    • Communicate with users
    data-quality-emr-systems
  • Reading: Data Quality and EMR Systems (5 min)

    Good quality data for decision making is the result of good data entry, data quality control, and assurance mechanisms. In addition to the actions users can take to monitor and maintain data quality, EMR systems have built in features that are designed to support the entry of quality of EMR system data.

    Notice that these features either guide users to enter quality data or enable users to identify at-risk data. EMR systems do not magically prevent or fix data quality issues. The responsibility for ensuring data quality remains with users and those responsible for assuring and controlling data quality.

  • Reading: Where Does Responsibility Lie for Data Quality? (5 min)

    Instructions: Click or tap each button to learn what responsibilities lie with the EMR user or the EMR system.

    EMR System Users
    • Check data entry before saving the form
    • Check that the correct patient record is open before entering data
    • Routinely (daily, weekly) check EMR system data against a random selection of paper forms
    • Run EMR system data quality reports to identify missing data
    • Flag paper forms with missing or inaccurate data
    • Collect data using established and consistent protocols and procedures
    • Respect time schedules for data entry and reporting
    • Learn to use the EMR system and follow data quality procedures
    EMR System
    • Use menus or checkboxes to standardize responses
    • Use standardized date format, ranges, calculations
    • Use data concepts from a standard data dictionary
    • Use alerts to flag required fields to ensure completeness
    • Prevent duplication of patient records by using unique patient identifiers
    • Compile data quality reports to identify key data missing from records
  • Activity: You make the call (15 min)

    Instructions: Let’s see how well you can spot problems with data quality. Click or tap each button to learn more about these data quality issues.

    Mixed up patient files
    Scenario:
    You are the data manager at Dangazeli District Hospital. Your current aim is to reduce the number of duplicate records in KenyaEMR. Each week, you review possible duplicate records and merge those that belong to the same patient. During your review, you come across these two patient records. Take a close look: do these two records belong to the same patient?
    PatientMerge

    Question: Patient 1 and Patient 2 are the same patient.


    Question: How do you know these are the same patient?

    Missing data
    Scenario:
    Tomorrow is the weekly staff meeting and you want to give feedback on how well staff are picking up KenyaEMR and entering complete data. You check the total number of patients in the database (121) and run Patients Missing Entry Point and Patients Missing date confirmed HIV+ reports. Less than 10% of patients are missing either piece of data.
    missing-data

    Question: What do these two reports tell you about the current quality of the data in KenyaEMR?

    Entering data retrospectively

    Instructions: Test your knowledge with 2 questions about entering data retrospectively

    Question 1

    Can you find the four data entry errors on this MOH257?

    Click or tap on the data fields to find the data entry mistakes.

    MOH257-hotspot-image
    Where In care Date enrolled Yes-No try again
    Reason

    The facility where the patient was confirmed HIV positive is missing. The patient record is incomplete.

    Regimen

    This family member is HIV positive; the In Care and CCC Number fields should be filled. The patient's family history is incomplete.

    Who

    The date the patient is enrolled in HIV care is missing.

    Height

    The patient's ART history has not been filled. This results in an incomplete patient record.

    Try again

    Please try another spot on the form.

    Question 2

    Can you find the four data entry errors on the EMR system’s triage form?

    Click or tap the data fields to flag the data entry mistakes.

    EMRFilled
    Date BP Respiratory Rate Other Recordings try again
    Date

    This data is not entered in a timely manner. The form is being filled on 28 November when the patient was checked in on the 26 November.

    BP

    This data is not complete since the clinician did not fill both fields.

    Respiratory Rate

    This data could be inaccurate if the digits were transposed.

    Vital Signs

    The vital signs did not change from the signs recorded at the patient’s January visit. If the provider copied these signs instead of recording the patient’s actual vital signs, then the data’s integrity is at risk.

    Try again

    Please try another spot on the form.

    Case Scenarios Conclusion

    Let’s look at the different dimensions of data quality that can be compromised on the MOH257 or in the EMR system’s triage form. Both forms can have problems with accuracy, completeness, and precision. What about the other dimensions?

    Both forms can have any number of fields with missing data. You can see here on the MOH257 that these include the date the patient was enrolled in the HIV program and where. On the EMR system’s triage form the blood pressure was not completely filled.

    Here you can see that the MOH257 form included a birth date with an illegible birth year. One of the advantages of an EMR system is that entries are always legible. While the precision of the data recorded on the triage form may not be clearly at risk, free text fields where entries can lack adequate detail and precision present a risk.

    The MOH257 does not indicate when the form was filled, making it difficult to know if it was completed in a timely manner.

    In many cases, it is difficult to know from looking at the forms what data entry errors are the result of staff receiving poor training or the lack of processes and procedures to follow. If staff were not aware what fields must always be filled, for instance, then the data’s reliability will weaken over time.

    Both forms can have problems with the integrity of their data. An incomplete patient clinic number on the MOH257 could mean that the wrong data is being recorded on the wrong record. And here, the vital signs entered on the EMR’s triage form appear to be copied from a previous visit, meaning that they may not reflect the patient’s current condition.

  • Reading: Summary (5 min)

    When clinicians and managers have access to quality data, they can be confident in the decisions they make using that data. The scenarios that you have worked with in this session shows that poor data quality can have serious consequences, including failure to provide patients with the care and treatment that they need. You have also learned what causes different data quality problems and the actions you can take to improve data quality.

    EMR systems can make data management easier in many ways. EMR systems have some features to support data quality, such as alerts, reports, and patient record merging. Remember that EMR systems cannot always prevent data entry errors or other data quality issues and that EMR systems do not always detect or fix such problems either. Responsibility for data quality lies with the people using the EMR system: receptionists, nurses, clinicians, data clerks, and managers. Now go on to the final quiz.

  • Quiz: Post-Test (15 min)

    After you take the module, please be sure to take the post-test. Read each question and select the correct answer. Please note that the post-test will open in a new website page. You will need access to the Internet to complete the post-test.

    Click to go to the post-test Post-test Module 4