Do Quality Measures Really Indentify Quality Care?

BY STEVEN B. LITTLEHALE, MS, APRN, BC, AND SAM SIMON, MA

The six chronic care QMs are intended to measure conditions germane to long-stay residents of long-term care facilities. The proportion of residents with a particular condition allegedly comments on the quality of care provided in the facility. Because excluding admission assessments lessens the impact of pre-existing chronic conditions on QM reports, the 90-day (quarterly) assessment is considered “ideal” for the chronic care population. The practice of excluding 5-day Medicare assessments has the same effect for post-acute measures. In this case, 14-day Medicare assessments are targeted for analysis (although this is not the case for all post-acute measures).

As a practical matter, excluding admission assessments (required by both OBRA and Medicare) does exclude many pre-existing conditions. Pressure ulcers, delirium, and pain upon admission are not considered in the calculation of QMs, for example. However, the very nature of a chronic condition can result in pre-existing conditions being included in QM calculations. Consider, for instance, a resident admitted with a stage III pressure ulcer. It is important to recognize that most facilities will not “cure” such an ulcer within 90 days. Indeed, debridement may make a wound appear to have worsened; other appropriate “best practices” might cause the ulcer to reduce to stage II. In either case, the pressure ulcer will be reflected in the facility’s QM score once it is coded on the “postadmission” MDS. Our firm, LTCQ, found empirical evidence of this. We recently studied 3,700 residents from a large national nursing home chain. The study found that 362 residents (9.8%) had triggered the pressure ulcer QM, but more than half of those residents (54.4%) had a pressure ulcer upon admission. Thus, for more than half of all residents who triggered the pressure ulcer QM, facility care or lack thereof was not necessarily responsible for the development of the ulcer.

Although the intention of using the QMs’ postadmission MDS is to target Medicare 14-day and OBRA 90-day assessments, other assessments not anticipated but mandated when a resident change occurs (e.g., significant change and corrections assessments) also are included in QM results. These significant change or correction assessments can cause pre-existing conditions to trigger a QM as soon as within the first 10 days of residency. In another study of a large multistate chain, LTCQ identified 2,379 residents who, over a one-year period, had been discharged from long-term care “with a return anticipated” and readmitted with a “significant change” assessment. Of these residents, 540 (22.7%) developed a new pressure ulcer while in the hospital. These new pressure ulcers were included in the facilities’ pressure ulcer QM.

CMS uses two risk adjustment practices that attempt to “level the playing field” when calculating QMs: exclusions and covariates. Each practice has merit and, when used together, they come closer to accurate assessment and more fair comparison.

Exclusions remove select residents from the QM analysis-for example, as mentioned above, not counting admission assessments is one form of exclusion. Additionally, few would argue that there is value in measuring a resident’s “improvement in walking” when he or she is near death. Most QMs have identified exclusions, and providers incorporating QMs into their quality improvement activities should be versed in these technicalities. CMS provides many excellent resources for understanding the subtle nuances of the QMs (see “Historical Background).

While exclusions remove many “inappropriate” residents from QM calculations, in practice CMS’ approach may not be enough. Consider residents with end-stage diseases. They are excluded from the calculation of the ADL decline, walking improvement, and delirium QMs, and yet the end-stage MDS item itself is often undercoded. Checking “end-stage” on the MDS requires a physician to document “six months or fewer” life expectancy, and this requirement has created a disincentive to code for end-stage disease. Many physicians hesitate to make such predictions and, as a result, often refuse to complete the required documentation. Close examination of MDS data illustrates this tendency to undercode “end of life.” Using a large dataset containing 13,000 MDS assessments, LTCQ found that 358 (2.7%) of the residents had an MDS indication of end-stage disease. A validated MDS-based death index that included weight loss, shortness of breath, very low body mass index, swallowing problems, congestive heart failure, and advanced age identified 938 (7.2%) residents who were likely to die within one year.1 However, among the residents identified by the death index, the MDS item identified only 7% as having an end-stage disease. Given its poor agreement with a validated measure of proximity to death, the validity of the MDS end-stage disease item must be called into question. With “end-stage disease” being undercoded, residents who clinically should be excluded from QM calculations are included.

Resident-level covariates are functional or clinical conditions that should not exclude the resident from QM consideration, yet warrant different treatment. An example would be the pain QM for chronic care. The resident-level covariate for this QM is an MDS item indicating cognitive impairment, with the rationale being that residents with greater cognitive impairment are less likely to report pain. Therefore, to account for the fact that residents with impaired cognition will report less pain, the pain QM is adjusted to account for the level of cognitive impairment. In this way, a facility with a higher-than-average resident population with intact cognition (and therefore more likely to be able to report pain) will not be “dinged” unfairly in comparison with facilities that have many more cognitively impaired residents. A resident-level covariate is also employed when calculating the post-acute delirium QM.

Risk adjustment is an attempt to account for facility case-mix. Providers should be aware of how their case-mix impacts their QMs (and QIs). A facility’s mission statement can result in its attracting a unique concentration of select residents and generating potentially less desirable QMs. If one’s mission is, for example, to serve a traditionally under-served population-poor inner-city elders, residents with head trauma, or residents with multiple sclerosis-the facility will most likely have QMs differing from more traditional nursing facilities. Specialty care units impact case-mix, as well. Although the term specialty care unit isn’t used uniformly, use of the term creates a perception that often contributes to the facility being a “magnet” for select types of residents. Facility location also impacts case-mix, as do referring relationships and sources.

While it is unsound to use QM reports without additional insights and analysis for purposes of litigation and risk appraisal, this may occur nevertheless. Allegations of poor care should be evaluated through superior risk adjustment and incidence-based metrics, which would include appropriate regional and national benchmarks to reflect local and regional variances (e.g., staffing numbers and patterns). These adjustments and comparisons sift through the impact of case-mix and regional differences and shed light on quality of care. Because the QM approach to analysis appears to prevail, facility management should strive to be clear about their quality status in addressing the survey process, public relations materials, and other forums. This means presenting specific information about your improvement and cure rates-ideally compared to similar facilities-to offset the negative tenor of the QMs.

Despite these limitations, it would do providers and consumers a disservice to write off the QMs as fatally flawed. An educated eye reviewing them can spot trends in performance over several months. Aberrant behavior that cannot be attributed to case mix should be investigated, beginning with a review of MDS data integrity related to items used in QM calculation and ending with an analysis of the resident’s care plan and the facility’s relevant policies and procedures. CMS is committed to continually improving these measures and the associated Quality Improvement initiative. That is a commendable journey, and we must take it together. NH

The goals for the Quality Measures (QMs) are to enable consumers to compare facilities with others in their state or locality, and to give nursing homes useful measures for internal quality improvement programs. In April 2002, CMS, under the auspices of the Department of Health and Human Services, published QMs from its six-state pilot study, both in local newspaper advertisements and on the Nursing Home Compare Web site (www.medicare.gov/NHCompare/home.asp). Based on this pilot experience, CMS selected a final set of 10 measures for national implementation in November 2002. For more information about the QM project, its stakeholders, and the process of QM development, go to https://cms.hhs.gov/providers/nursinghomes/nhi/final_qm.pdf.

Reference
1. Flacker JM, Kiely DK. Mortality-related factors and 1-year survival in nursing home residents. J Am Geriatr Soc 2003;51:213-21.