AI wins some -- and loses some -- in test against pathologists

2019 08 15 16 15 3581 Dcis Ai Elmore 20190815163052

A novel artificial intelligence (AI) algorithm outperformed pathologists in differentiating ductal carcinoma in situ (DCIS) from atypical hyperplasia, a job that is "considered the most diagnostically difficult task in clinical practice," researchers wrote in a paper published online August 9 in JAMA Network Open.

In a U.S. National Cancer Institute-supported study of 240 breast cancer cases, the researchers compared the performance of machine learning with the interpretation of digitized images by 87 practicing pathologists. A machine-learning algorithm that analyzed two features (tissue distribution and structure) had lower sensitivity than the pathologists when it came to distinguishing between invasive and noninvasive cancer. But sensitivity was higher for the algorithm when it came to differentiating atypia and DCIS from benign results, as well as DCIS from atypia (see table).

"The findings suggest that [machine-learning] methods are potentially suitable as diagnostic support systems in differentiating challenging preinvasive lesions of the breast," wrote senior author Dr. Joann Elmore, a professor of medicine at the David Geffen School of Medicine at the University of California, Los Angeles (UCLA), and colleagues (JAMA Netw Open, August 2019, Vol. 2:8, e198777).

Machine learning vs. practicing pathologists, breast cancer diagnoses
Diagnostic feature Sensitivity Specificity Accuracy
Invasive vs. noninvasive cancer
Machine-learning distribution feature: Captures distribution and simple spatial relationships of tissues 0.70 0.95 0.94
Machine-learning structure feature: Summarizes architectural changes in ductal structures based on semantic segmentation of tissue types 0.49 0.96 0.91
Pathologists 0.84 0.99 0.98
Atypia and DCIS vs. benign results
Tissue distribution feature 0.79 0.41 0.70
Structure feature 0.85 0.45 0.70
Pathologists 0.72 0.62 0.81
DCIS vs. atypia
Tissue distribution feature 0.88 0.78 0.83
Structure feature 0.89 0.80 0.85
Pathologists 0.70 0.82 0.80
Source: JAMA Network Open.

The answer to overtreatment?

The analysis just published was performed as a follow-up to another study that found a low level of agreement between practicing pathologists who regularly handle breast cancer cases and a consensus diagnosis by experts in the field for evaluating atypia in biopsy specimens (Journal of the American Medical Association, March 17, 2015, Vol. 313:11, pp. 1122-1132).

"Preinvasive lesions are an integral part of the diagnostic evaluation of breast biopsy specimens and present a more difficult classification scenario for pathologists than the binary classification task of determining whether a breast biopsy sample shows invasive cancer," the authors explained in JAMA Network Open.

An AI system outperformed doctors in differentiating DCIS (pictured) from atypia, one of the more challenging diagnoses in breast cancer cases. Image courtesy of Elmore Lab/UCLA.An AI system outperformed doctors in differentiating DCIS (pictured) from atypia, one of the more challenging diagnoses in breast cancer cases. Image courtesy of Elmore Lab/UCLA.

It's estimated that more than 1 million people in the U.S. will get a breast biopsy this year, and the diagnosis and treatment of early-stage lesions that would never actually become breast cancer, including types of DCIS, has long been a concern.

"Although an exact estimate of overdiagnosed cases is unknown, several studies have hypothesized and estimated its prevalence in breast cancer diagnosis," Elmore and colleagues wrote.

The new study used the pathologists' performance data from the 2015 JAMA study, but this time the researchers compared diagnostic interpretations with machine learning. The AI methods were designed to "emulate the behavior of pathologists" as they interpret cases, they noted.

Age of whole slide imaging

"[Whole slide imaging], a technology that captures the contents of a glass slide in a multiresolution image, is revolutionizing diagnostic medicine by providing researchers with tools to study diagnostic missteps and develop diagnostic support systems," the authors noted.

The first approval by the U.S. Food and Drug Administration of a digital pathology system in 2017 paved the way for greater use of computer-aided diagnosis. The U.S. health system is now undergoing a "major shift toward digital pathology, and the resulting need for automated diagnosis tools that can lead to computer-aided diagnostic support systems will be significant," they suggested.

"With further improvements in the [machine-learning] techniques we used -- or even improvements at the cellular level -- it may be possible for computer vision methods to be used in conjunction with pathologists' intuition to improve diagnostic performance," the authors concluded.

Page 1 of 12
Next Page