By Nelson Tuazon, DNP, DBA, RN, NEA-BC, CENP, CPHQ, CPPS, FNAP, FACHE
Vice President and Associate Chief Nursing Office, University Health System
President, South Central Texas Organization of Nurse Executives
The concepts of the Triple Aim (Berwick, Nolan & Whittington, 2008) and the Iron Triangle in Healthcare (Kissick, 1994) have been used authoritatively in optimizing performance amidst the limitations and constraints in delivering healthcare. The competition in healthcare, the challenges with reimbursement and the pursuit of excellence have provided the impetus for the increased focus on outcomes. In the IOM report entitled "Crossing the quality chasm: A new healthcare system for the 21st century," key performance measures were established that address safe, effective, patient-centered, timely, efficient and equitable care (Committee on Quality Healthcare in America, Institute of Medicine, 2001). These performance measures have been used in developing strategies that address the needs of patients and in evaluating and monitoring improvements through balanced score cards (Bisbe & Berrubes, 2012; Ippolito & Zocolli, 2013). To effectively develop and establish organizational key performance indicators, leaders should clearly define and focus on organization strategic plans, tactics and imperatives (Bedgood, 2017).
A Measure by Any Name
There are three types of metrics or measures used to monitor and evaluate improvement efforts. Using the Donabedian model, measures that assess and compare performance are classified as structure, process or outcome measures. Generally, structural measures indicate organizational capacity, systems and processes to deliver high-quality care. Process measures reflect the steps or approaches to maintain or improve care. Outcome measures provide an indication on the impact of the services or interventions that address patient needs (Agency for Healthcare Research and Quality, 2011). The emphasis on achieving balance in the performance of healthcare organizations has led to the transformation of the nomenclature of these measures or metrics. To reflect this paradigm, these have evolved into outcome, process and balancing measures or metrics (Institute for Healthcare Improvement, 2019).
Are We Leading or Are We Lagging?
The need to differentiate leading from lagging metrics has faced some criticisms (Manuele, 2009). This skepticism is a result of a lack of full explanation and understanding of the relationship of these metrics. There are also differences in the way by which these various metrics are defined, interpreted and used. The commonly held definition of leading metrics indicate process, whereas lagging metrics indicate results. Lagging measures have also been referred to as trailing indicators (Minnick & Wachter, 2019).
Performance improvement initiatives have target end points. Leading and lagging metrics are useful as indicators to assess organizational performance (Minnick & Wachter, 2019). Leading measures predict organizational performance, whereas outcome measures are indicative of organizational strategy. Process measures tend to be more tactical and have primary impact on the results of the actions. Pawlowska (2015) distinguishes leading measures as factors used for assessing performance and are indicators for inputs, whereas lagging measures as factors used for evaluating performance and are indicators for measuring outcomes. These operational definitions are in concordance with the applications of the leading as being proactive and lagging measures as being retrospective (Almost et al., 2018). In economic terms, leading metrics change before the changes in economy occur while lagging metrics occur after the overall economy has changed (Manuele, 2009).
Organizations have the inclination to rely upon lagging measures (outcomes) to evaluate the sustainability of improvement plans and initiatives. Pojasek (2009) offers a different view on the use of leading metrics in driving sustainability. He posited that lagging metrics simply reflect the outcomes but do not measure the performance of the organization in a direct way. To achieve long-lasting sustainability, organizations should use a combination of leading and lagging measures. Payne, Bergman, Beus, Rodriguez and Henning (2009) suggested that a particular factor or construct such as safety climate could be viewed both as a leading or lagging metric depending on whether a prospective or retrospective design is used.
Limitations of Lagging Measures
Although lagging measures reflect the final outcomes, they may not provide a complete picture to inform the organization of future actions and direction to ensure overall sustainability. Pojasek (2009) has argued that lagging metrics are insufficient because of the lag in time between the implementation of the interventions and the ultimate outcomes. Other factors may also affect the outcomes other than the intended actions. The results may be insignificant to provide feedback on the process measures. Lagging measures may not meet the current needs of the stakeholders (Bedgood, 2018). Raben, Bogh, Viskum, Mikkelsen and Hollnagel (2018) examined the development of proactive methods in improving patient safety rather than the traditional reliance upon adverse events, errors and past failures. Leaders should consider the complexity of healthcare processes when identifying the processes required to get things done the right way. Using lagging key performance indicators may lead to false sense of security because of the possibility for minor issues to exacerbate into larger problems before they are detected (Bedgood, 2018). There could be latent issues occurring in the early stage that lagging measures may not identify (Pojasek, 2009).
The Myth of Measuring and Managing
Healthcare leaders have subscribed to Drucker’s thoughts on “what is managed gets done." Deming has also been incorrectly credited to the saying “you can’t manage what you can’t measure” (Hunter, 2015). However, although Deming stressed the importance of data as a statistician himself, he also believed that managers and leaders must still make decisions on things that they may not be able to measure using data (Hunter, 2015). Ryan (2014) has posited that a vast majority of the important things with which managers and leaders deal are not measurable.
Minnick and Wachter (2019) have reported that the use of leading metrics have become as common as the use of lagging metrics. In terms of patient safety, the use of leading and lagging metrics have to be carefully examined with regards to their direct impact on risks. Raben et al (2018) have posited that due to the nature of patient safety in the avoidance, prevention and amelioration of adverse outcomes, the use of leading measures (process metrics) may reveal issues that may be related to lack of safety, thereby predicting and anticipating actions to avoid, prevent or ameliorate adverse events.
Balancing Measures
Although the concept of the balanced scorecard has been in the literature for over three decades, research has shown that the use of balancing measures is rare and there is paucity in reporting of their application. Planners of performance improvement may fail to report unfavorable or unintended consequences on planned interventions or data about the outcomes may be inadequate (Toma, Dreischulte, Gray, Campbell & Guthrie, 2018).
As the iron triangle in healthcare demonstrates, there is competing pressure in achieving access to care, quality of care being delivered and cost of healthcare (Kissick, 1994). To improve the quality of care while decreasing or reducing the cost of healthcare, leaders must learn how to balance these competing demands (Kaboli & Mosher, 2014). There is a need to create equilibrium between the leading and lagging measures by establishing balancing measures. To exemplify this interplay between the three measures, consider the following scenario. Discharging patients before noon is a leading indicator (process measure) to decrease the length of stay of patients in the emergency department (outcome measure). These measures may have impact on patient satisfaction of the patients both in the in-patient units (i.e. decreased patient satisfaction if the patients feel that they are rushed to leave) and the patients in the ER (i.e. improved experience because of the decreased waiting time for a bed).
Research on patient safety has shown a relationship between leading and lagging metrics. Sheehan, Donohue, Shea, Cooper and Cieri (2016) have found that safety leadership and a focus on prevention impact the conformance of employees with leading measures and lagging measures, i.e. interventions to prevent safety events and ameliorate safety programs. These authors support the idea of decreasing reliance upon lagging metrics; rather, increasing the focus on the preventive nature of leading measures.
Final Words
In this data-driven and information-powered healthcare environment, it is easy for healthcare leaders to be mired by a multitude of metrics. Drucker’s thought on measuring and managing data have influence leaders in evaluating performance; however, leaders also take Drucker’s advice that we are expected to get the right things done (Drucker, 1967). The new metrics for healthcare leaders will be those that are beyond the leading and lagging metrics - metrics tell a story, but not the whole story (The new metrics: Part One; 2017; The new metrics: Part 2, 2017). Leaders should determine the impact of both the leading and lagging metrics and find ways to mitigate their possible negative effects. Ultimately, the use of balancing measures allows the concurrent monitoring of the processes and the determination of the outcomes of the improvement plan.
References
Agency for Healthcare Research and Quality (AHRQ). (2011). Types of health care quality measures. Rockville, MD: Agency for Healthcare Research and Quality, Retrieved from https://www.ahrq.gov/talkingquality/measures/types.html.
Almost, J. M., VanDenKerkhof, E. G., Strahlendorf, P., Caicco Tett, L., Noonan, J., Hayes, T., … Silva e Silva, V. (2018). A study of leading indicators for occupational health and safety management systems in healthcare. BMC Health Services Research, 18(1), N.PAG. https://doi.org/10.1186/s12913-018-3103-0
Bedgood, C. (2017). Cracking the KPI code in healthcare: Design the right metric system to find out if you are leading or lagging. ISE: Industrial & Systems Engineering at Work, 49(12), 40–44. Retrieved from https://search.ebscohost.com/login.aspx?direct=true&db=mth&AN=126283586&site=eds-live&scope=site
Berwick, D. M., Nolan, T. W., & Whittington, J. (2008). The triple aim: Care, health, and cost. Health Aff (Millwood), 27(3), 759-769.
Bisbe, J., & Barrubes, J. (2012). The balanced scorecard as a management tool for assessing and monitoring strategy implementation in healthcare organizations. Revista Espanola de Cardiologia, 65(10), 919-927.
Committee on Quality Health Care in America, Institute of Medicine. (2001). Crossing the quality chasm: A new health system for the 21st century. Washington, D.C.: National Academy Press.
Drucker, P. (1967). The effective executive: The definite guide to getting the right things done. New York: NY: Harper Collins.
Ippolito, A., & Zoccoli, P. (2013). Theoretical contribution to develop the classical Balanced Scorecard to health care needs. International Journal of Healthcare Management, 6(1), 37–44. https://doi.org/10.1179/2047971912Y.0000000025
Hunter, J. (2015). Myth: If you can measure it, you can’t manage it. Retrieved from https://blog.deming.org/2015/08/myth-if-you-cant-measure-it-you-cant-manage-it/
Institute for Healthcare Improvement (IHI). (2019). Science of improvement: Establishing measures. Retrieved from http://www.ihi.org/resources/Pages/HowtoImprove/ScienceofImprovementEstablishingMeasures.aspx
Kaboli, P.J., & Mosher, H.J. (2014). Using balanced metrics and mixed methods to better understand QI interventions. BMJ Quality & Safety, 23, 437-439.
Kissick, W. (1994). Medicine’s dilemma. New Haven and New London, CT: Yale University Press.
Manuele, F. A. (2009). Leading & Lagging Indicators. Professional Safety, 54(12), 28–33. Retrieved from https://search.ebscohost.com/login.aspx?direct=true&db=bth&AN=47054742&site=eds-live&scope=site
Minnick, W. D., & Wachter, J. K. (2019). The Role of Leading & Lagging Indicators in evaluating OSH professionals’ performance. Professional Safety, 64(1), 32–36. Retrieved from https://search.ebscohost.com/login.aspx?direct=true&db=bth&AN=133975749&site=eds-live&scope=site
Pawłowska, Z. (2015). Using lagging and leading indicators for the evaluation of occupational safety and health performance in industry. International Journal of Occupational Safety & Ergonomics, 21(3), 284–290. https://doi.org/10.1080/10803548.2015.1081769
Payne, S. C., Bergman, M. E., Beus, J. M., Rodríguez, J. M., & Henning, J. B. (2009). Safety climate: Leading or lagging indicator of safety outcomes? Journal of Loss Prevention in the Process Industries, 22(6), 735–739. https://doi.org/10.1016/j.jlp.2009.07.017
Pojasek, R. B. (2009). Using leading indicators to drive sustainability performance. Environmental Quality Management, 18(4), 87–93. https://doi.org/10.1002/tqem.20228
Prusak, L. (2010). What can’t be measured. Harvard Business Review. Retrieved from https://hbr.org/2010/10/what-cant-be-measured.
Raben, D. C., Bogh, S. B., Viskum, B., Mikkelsen, K. L., & Hollnagel, E. (2018). Learn from what goes right: A demonstration of a new systematic method for identification of leading indicators in healthcare. Reliability Engineering and System Safety, 169, 187–198. https://doi.org/10.1016/j.ress.2017.08.019
Ryan, L. (2014). If you can’t measure it, you can’t manage it: Not true. Retrieved from https://www.forbes.com/sites/lizryan/2014/02/10/if-you-cant-measure-it-you-cant-manage-it-is-bs/#783ce6087b8b
Sheehan, C., Donohue, R., Shea, T., Cooper, B., & Cieri, H. D. (2016). Leading and lagging indicators of occupational health and safety: The moderating role of safety leadership. Accident Analysis & Prevention, 92, 130–138. https://doi.org/10.1016/j.aap.2016.03.018
The new metrics: Part one. (2017). Healthcare Leadership Review, 36(9), 1–7. Retrieved from https://search.ebscohost.com/login.aspx?direct=true&db=rzh&AN=125017014&site=eds-live&scope=site
The new metrics: Part 2. (2017). Healthcare Leadership Review, 36(10), 1–7. Retrieved from https://search.ebscohost.com/login.aspx?direct=true&db=rzh&AN=125697783&site=eds-live&scope=site
Toma, M., Dreischulte, T., Gray, N. M., Campbell, D., & Guthrie, B. (2018). Balancing measures or a balanced accounting of improvement impact: A qualitative analysis of individual and focus group interviews with improvement experts in Scotland. BMJ Quality & Safety, 27(7), 547–556. https://doi.org/10.1136/bmjqs-2017-006554