An accessible, efficient, and accurate natural language processing method for extracting diagnostic data from pathology reports.

TitleAn accessible, efficient, and accurate natural language processing method for extracting diagnostic data from pathology reports.
Publication TypeJournal Article
Year of Publication2022
AuthorsLam H, Nguyen F, Wang X, Stock A, Lenskaya V, Kooshesh M, Li P, Qazi M, Wang S, Dehghan M, Qian X, Si Q, Polydorides AD
JournalJ Pathol Inform
Volume13
Pagination100154
Date Published2022
ISSN2229-5089
Abstract

CONTEXT: Analysis of diagnostic information in pathology reports for the purposes of clinical or translational research and quality assessment/control often requires manual data extraction, which can be laborious, time-consuming, and subject to mistakes.

OBJECTIVE: We sought to develop, employ, and evaluate a simple, dictionary- and rule-based natural language processing (NLP) algorithm for generating searchable information on various types of parameters from diverse surgical pathology reports.

DESIGN: Data were exported from the pathology laboratory information system (LIS) into extensible markup language (XML) documents, which were parsed by NLP-based Python code into desired data points and delivered to Excel spreadsheets. Accuracy and efficiency were compared to a manual data extraction method with concordance measured by Cohen's κ coefficient and corresponding P values.

RESULTS: The automated method was highly concordant (90%-100%, P<.001) with excellent inter-observer reliability (Cohen's κ: 0.86-1.0) compared to the manual method in 3 clinicopathological research scenarios, including squamous dysplasia presence and grade in anal biopsies, epithelial dysplasia grade and location in colonoscopic surveillance biopsies, and adenocarcinoma grade and amount in prostate core biopsies. Significantly, the automated method was 24-39 times faster and inherently contained links for each diagnosis to additional variables such as patient age, location, etc., which would require additional manual processing time.

CONCLUSIONS: A simple, flexible, and scaleable NLP-based platform can be used to correctly, safely, and quickly extract and deliver linked data from pathology reports into searchable spreadsheets for clinical and research purposes.

DOI10.1016/j.jpi.2022.100154
Alternate JournalJ Pathol Inform
PubMed ID36605108
PubMed Central IDPMC9808011
Related Faculty: 
Hansen Lam, M.D.

Pathology & Laboratory Medicine 1300 York Avenue New York, NY 10065 Phone: (212) 746-6464
Surgical Pathology: (212) 746-2700