P2740 - Diagnostic Codes Performance for Identification of Eosinophilic Gastrointestinal Diseases Varies by Reference Standard Definition and Disease Subtype
University of Pennsylvania Health System Philadelphia, PA
Corey J. Ketchem, MD1, Anwesh Dash, MD2, Uğurcan Vurgun, PhD, MA3, Agnes Wang, MS4, Sunil Thomas, MBA2, Ashley Batugo, BS2, Gary Falk, MD, MS5, Kristle L.. Lynch, MD6, Evan S. Dellon, MD, MPH7, Danielle Mowery, PhD, MS8, James Lewis, MD, MSCE9 1University of Pennsylvania Health System, Philadelphia, PA; 2University of Pennsylvania, Philadelphia, PA; 3University of Pennsylvania, Mill Creek, WA; 4Univeristy of Pennsylvania, Philadelphia, PA; 5Division of Gastroenterology and Hepatology, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, PA; 6The University of Pennsylvania, Philadelphia, PA; 7Center for Esophageal Diseases and Swallowing, University of North Carolina School of Medicine, Chapel Hill, NC; 8Perelman School of Medicine at the University of Pennsylvania, Philadelphia, PA; 9University of Penn, Philadelphia, PA Introduction: Eosinophilic gastrointestinal diseases (EGIDs) are understudied. Internal Classification of Diseases (ICD) codes are a potential tool to facilitate EGID research, but their accuracy is unknown. We aimed to evaluate the ICD performance metrics for identifying EGIDs using multiple case definitions and across subtypes. Methods: We identified GI pathology reports containing “eosinophil” and paired them with preceding clinic notes from a large health system. Reference standard diagnoses were assigned via chart review and compared to ICD codes. The EoE reference standard required positive histology with variable definitions of esophageal dysfunction and diagnostic history. Defining reference standard for EoG and EoN included liberal (any eosinophils plus symptoms) and restrictive (increased eosinophils plus symptoms) criteria; only liberal criteria were used for EoC due to limited detail. For non-EoE EGIDs, a rule-based NLP approach captured numeric or descriptive eosinophil increases. EoG and EoN, lacking distinct ICD codes, were evaluated individually and combined under the eosinophilic gastroenteritis code. Performance metrics included sensitivity, specificity, PPV, NPV, accuracy, Cohen’s kappa, and F1 score (harmonic mean of precision and recall). Results: Defining EoE reference standard with symptoms of dysphagia and food impaction only, the ICD achieved an F1 score of 0.70 (95% CI: 0.59-0.80), with a PPV of 0.64 and sensitivity of 0.77. Including additional esophageal dysfunction symptoms yielded a modest improvement in F1 score (ΔF1 = 0.02, 95% CI: 0.0 to 0.08). Further addition of history of EoE documented in note free text resulted in higher PPV (0.95), specificity (0.98), and F1 score (0.84, 95% CI: 0.78-0.90), with similar sensitivity (0.76). The diagnostic performance of ICD codes for identifying EoG or EoN demonstrated low PPV (0.17-0.33) and sensitivity (0.08-0.33) across individual subtypes and in combination, with slightly better performance for restrictive definitions. No patients with EoC received the corresponding ICD code, resulting in zero sensitivity and undefined PPV. Discussion: The diagnostic performance of ICD codes for EoE is influenced by the specific criteria and clinical data elements incorporated in the reference standard definition. The ICD codes for non-EoE EGIDs demonstrate high false positive and false negative rates. These findings highlight the need for improved case identification methods to facilitate EGID research.
Figure: Table 1. Performance metrics of EoE ICD code using different combinations of diagnostic criteria for EoE classification. The ΔF1 column shows the absolute difference in F1 score compared to the referent criteria set.
Figure: Table 2. Diagnostic performance metrics of ICD codes for identifying non-EoE EGIDs across different subtypes using restrictive or liberal case definitions. A total of 12 and 6 cases met the liberal and restrictive criteria for eosinophilic gastritis (EoG), respectively, while 11 and 4 cases met the liberal and restrictive criteria for eosinophilic enteritis (EoN). For eosinophilic colitis (EoC), 8 cases met the liberal criteria.
