The challenges of developing healthcare quality measures based on ICD-10-CM Codes

Inappropriate outpatient antibiotic prescribing is a primary driver of antimicrobial resistance. Designing and evaluating antibiotic stewardship initiatives requires common measures of inappropriate prescribing that can be applied at the level of prescribers, health systems, and geographic regions. To this end, several research groups have estimated the prevalence of inappropriate prescribing in the US using measures based on diagnostic codes contained in International Classification of Diseases, ninth revision, clinical modification (ICD-9-CM).

However, on 1 October 2015, the US healthcare system transitioned to ICD-10-CM. This transition has hampered research on antibiotic prescribing that uses recent data since ICD-10-CM-based measures of antibiotic appropriateness have not been developed. To address this problem, our team created a classification scheme determining whether each of the diagnosis codes in the 2016 version of ICD-10-CM were indications for outpatient antibiotics. To develop this scheme we had to overcome challenges inherent in any effort to assess the appropriateness of medical decision-making, such as how to assess antibiotic appropriateness for ambiguous diagnosis codes. At the same time, many other challenges were specific to ICD-10-CM, for example:

The sheer number of codes in the 2016 version of ICD-10-CM—91,738 versus 17,553 in ICD-9-CM—required a painstaking, time-consuming process involving multiple iterations of the classification scheme over several months.
Previous ICD-9-CM-based schemes did not naturally translate to ICD-10-CM. In some cases, ICD-9-CM diagnosis codes were more detailed than ICD-10-CM. For example, non-infected superficial wounds such as insect bites and abrasions were differentiated from their infected counterparts in ICD-9-CM, but not ICD-10-CM. In other cases, ICD-10-CM codes were more detailed than the corresponding ICD-9-CM codes.
Few ICD-10-CM codes have been validated against medical chart review. Thus, our team often lacked information on the clinical scenario that a clinician was trying to convey by using a given diagnosis code. As an example, a clinician using a diagnosis code of thrombophlebitis might be treating a patient with an uncomplicated case who would not require antibiotics, or a patient with infected thrombophlebitis who would require antibiotics.

The classification scheme that we ultimately developed will hopefully facilitate comprehensive assessments of outpatient antibiotic appropriateness in the US. Furthermore, this scheme could potentially be adapted for use in other countries that are already using ICD-10, provided that outpatient claims with reliable diagnosis codes are available in these countries.

Our experiences may be relevant to other healthcare researchers developing ICD-10-CM-based measures of healthcare quality. On the basis of these experiences, we make the following three recommendations. Firstly, researchers should be prepared to invest even more time than before to develop quality measures, due to the size of ICD-10-CM. Given this increased time commitment, it will be crucial to consider from the outset whether a measure will be important, valid, and feasible to implement.

Secondly, researchers should exercise great caution if they attempt to adapt previously developed measures using crosswalks between ICD-9-CM and ICD-10-CM. Although these crosswalks may be a useful starting point, careful examination of all ICD-10-CM codes is necessary to account for relevant codes excluded from crosswalks, while careful examination of codes included in crosswalks is necessary to ensure that these codes still represent the construct of interest in ICD-10-CM.

Finally, researchers should recognize that quality measures will need to be revised as ICD-10-CM codes are updated and validated. As we did in our study, researchers should publish all codes used in measures to facilitate transparency and future revisions.