Model for Improvement: Establishing Measures
Model for Improvement question: How will we know that a change is an improvement?
Measurement is a critical part of knowing if we have made a difference, what the impact of the changes are, if we have met our aim, and future action to take. When selecting measures, it is vital to include the people whose lives will be impacted by the improvement to have a voice in deciding what measures are important from their perspective. While aims often center around a quantitative aim or target, it’s important to measure your improvement work using quantitative and qualitative data.
Measurement for improvement should not be confused with measurement for research. This difference is outlined in the table below.
| Measurement for Research | Measurement for Learning and Improvement | |
| Purpose | To discover new knowledge | To bring new knowledge into daily practice |
| Tests | One large "blind" test | Many sequential, observable tests centered on learning |
| Biases | Control for as many biases as possible | Design data collection to stabilize bias |
| Data | Gather as much data as possible, "just in case" | "Just enough" data gathered through small sequential samples |
Three Types of Measures
Use a set of measures (typically 4 to 10) in order to track change and learning over time across different areas of the system (or processes). The three types of measures commonly used include: outcome, process, and balancing measures.
Disaggregating data (stratification) to show potential systemic inequities can be an important strategy for ensuring that improvement efforts close rather than maintain or widen equity gaps (e.g., race, ethnicity, ancestry, language, sexual orientation, gender identity).
Outcome Measures
Linking back to the numeric goal within the aim statement, the outcome measure indicates how the system is working, specifically the impact on the customers/patients. Typically there is one outcome measure for an improvement initiative, and at times, there may be a need for two.
Examples:
- For diabetes: Average hemoglobin A1c level for BIPoC patients with diabetes
- For access: Number of days to 3rd next available appointment
- For critical care: Intensive Care Unit (ICU) percent unadjusted mortality
- For medication systems: Adverse drug events per 1,000 doses
- For population health: Number of children under 10 years old experiencing hunger
Process Measures
Are the parts/steps in the system performing as planned? Are we on track in our efforts to improve the system?
Examples:
- For diabetes: Percentage of BIPoC patients whose hemoglobin A1c level was measured twice in the past year
- For access: Average daily clinician hours available for appointments
- For critical care: Percent of patients with intentional rounding completed on schedule
- For population health: Number of children under 10 years old receiving school meals
Balancing Measures (looking at a system from different directions/dimensions)
Are changes designed to improve one part of the system causing new problems in other parts of the system?
Examples:
- For reducing time patients spend on a ventilator after surgery: Make sure reintubation rates are not increasing
- For reducing patients' length of stay in the hospital: Make sure readmission rates are not increasing
Tips for Effective Measures
Plot Data Over Time
Improvement requires change, and change is, by definition, a temporal phenomenon. Much information about a system and how to improve it can be obtained by plotting data over time, such as data on length of stay, volume, patient satisfaction — and then observing trends and other patterns. Tracking a few key measures over time is the single most powerful tool a team can use. [See additional information below.]
Seek Usefulness, Not Perfection
Remember, measurement is not the goal; improvement is the goal. In order to move forward to the next step, a team needs just enough data to know whether changes are leading to improvement. Also, try to leverage existing measures the team is already collecting data for and reporting on so it’s also easier and not additional work.
Use Sampling
Sampling is a simple, efficient way to help a team understand how a system is performing. In cardiac surgery, the patient volume is typically low enough to allow tracking of key measures for all patients. However, sampling can save time and resources while accurately tracking performance. For example, if accurate data is not easily available through automated administrative databases, instead of monitoring the time from catheterization to cardiac surgery continuously, measure a random sample of 10 to 20 cardiac surgery patients per month.
Example: Use Sampling
Below is an example of using sampling in measuring the time for transfer from the emergency department (ED) to an inpatient bed.
Rapid movement from the ED after a decision to admit the patient is critical flow for entry to the entire system for emergent patient care. It represents the ability of patients with various illnesses to get into the system through the most common admission route.
Sampling approach: The measurement will consist of 6 weekly data collections of 25 patients each. The patients can be sampled in several ways, such as in one of the three ways described as follows:
- 5 patients per day for 5 days of the week (patients must be consecutive and at least one day must be a weekend day)
- 25 consecutive patients regardless of any specific day, except that it must include some weekend admissions
- If there are fewer than 25 admissions for a week, the total admissions for the week should be included in the sample
The time is measured from the decision to admit to the physical appearance of the patient into the inpatient room. The destination cannot be a "holding area" but must be a "real inpatient bed." The sample collection should be done in real time, so a data collection process needs to be worked out by members of the team to achieve this goal. The collections must be done weekly and summarized as the percentage of patients in the sample that achieved the goal for that week. Six weeks of data needs to be collected and six data points placed on a run chart.
Integrate Measurement into the Daily Routine
Useful data are often easy to obtain without relying on information systems. Don’t wait two months to receive data on patients’ average length of stay in the hospital from the information systems department. Develop a simple data collection form and make collecting the data part of someone’s job. Often, a few simple measures will yield all the information you need.
Use Qualitative and Quantitative Data
In addition to collecting quantitative data, be sure to collect qualitative data, which often are easier to access and highly informative. For example, ask the nursing staff how weaning from medications is going or how to improve the sedation protocol. Or, in order to focus your efforts on improving patient and family satisfaction, ask patients and their families about their experience of the cardiac surgery process.
Plotting Data Over Time
Plotting data over time using a run chart is a simple and effective way to determine whether the changes you are making are leading to improvement. Annotate the run chart to show the changes you made.
