Natural Language Processing-Optimized Case Selection for Real World Evidence Studies

ASCO 2022 - Tempus-authored
Authors Jacob Koskimaki, Jenny Hu, Yiduo Zhang, Jose Mena, Nehanda Jones, Elizabeth Lipschultz, Vivek Prabhakar Vaidya, Gabriel Altay, Vance Andrei Erese, Krishna Kumar Swaminathan, Emma Mendonca, Tarun Dutt, Kuldeep Singh, Tian King, Vinay Phani Santosh Lakkimsetty, Hussein Al-Olimat, Brittany Manning, George Anthony Komatsoulis, Simon Chu, Jeff Ottens

Much information describing a patient’s cancer treatment remains in unstructured text in electronic health records and is not recorded in discrete data fields. Accurate data completeness is essential for quality care improvement and research studies on de-identified patient records. Accessing this high-value content often requires manual and extensive curation review.

Methods: AstraZeneca, CancerLinQ, ConcertAI, and Tempus have developed a natural language processing (NLP)-assisted process to improve clinical cohort selection for targeted curation efforts. Hybrid, machine-learning model development included text classification, named entity recognition, relation extraction and false positive removal. A subset of nearly 60,000 lung cancer cases were included from the CancerLinQ database, comprised of multiple source EHR systems. NLP models extracted EGFR status, stage, histology, radiation therapy, surgical resection and oral medications. Based on the results, cases were selected for additional manual curation, where curators confirmed findings of the NLP-processed data.

Results: NLP methods improved cohort identification. Successfully returned cases using the NLP method ranged from 75.2% to 96.5% over more general case selection criteria based on limited structured data. For all cohorts combined, 84.2% of the cases sent out for NLP curation were returned with curated content (Table). Each cohort contained a range of NLP-derived elements for curators to further review. In comparison, more general case selection criteria yielded a total of 3,878 cases returned out of 41,186 lung cancer cases sent for curation, for a success rate of only 9.6%.

Cohort Cohort Description Number of
from NLP
identification methods
Number of
cases sent to Tempus and
ConcertAI for curation
Number of cases
returned to CancerLinQwith
Percent of successfully curated
1A NSCLC, stage I, II, III, EGFR+,
408 408 341 83.60%
1B NSCLC, non
squamous, stage I, II, III, EGFR wild
type/unknown, complete
4313 1500 1285 85.70%
2A NSCLC, stage III, unresectable,
curative radiation to the chest total dose > = 50 Gy, did receive
852 620 466 75.20%
2B NSCLC, stage III, unresectable,
curative radiation to the chest total dose > = 50 Gy,
did not receive Imfinzi
3050 750 724 96.50%
3 SCLC, received Imfinzi or
559 500 402 80.40%
4 NSCLC, received Tagrisso as first line treatment 971 812 647 79.70%
Total: 10153 4590 3865


Conclusions: NLP-driven case selection of six distinct, complex lung cohorts resulted in an order of magnitude improvement in eligibility over candidate selection using structured EHR data alone. This study demonstrates NLP-assisted approaches can significantly improve efficiency in curating unstructured health data.