An artificial intelligence system applied to dual-stain cervical cancer testing was accurate and could reduce unnecessary follow-up colposcopies, with potentially transformative effects for patients, according to a study published June 25 by National Cancer Institute (NCI) researchers.
The NCI researchers described positive experiences with Cytoreader, an automated system for evaluating digital dual-stained whole-slide images for the protein biomarkers p16 and Ki-67, using data from three epidemiology studies of patients with cervical and anal cancers who were positive for human papillomavirus (HPV). The automated system assesses the number of dual-stained positive cells and flags the ones that exceed predefined cutoff points.
The study involved data from a total of 4,253 patients treated at Kaiser Permanente Northern California and the University of Oklahoma (Journal of the National Cancer Institute, Vol. 113:1, pp. 1-8). One part of the evaluation of Cytoreader involved comparing two different cutoff levels for positivity with AI dual-staining (one or two cells), versus manual dual-staining and Pap testing for triaging women who took part in cervical cancer screening at Kaiser. The presence of p16 and Ki-67 in the same cell is strongly associated with cervical cancer, and detection of the pair is helpful for triaging patients who are positive for HPV to determine if they need a follow-up colposcopy, the authors noted.
With one of the automated dual-staining cutoff points, the number of referrals for colposcopy would be reduced by one-third (41.9% versus 60.1%), a statistically significant result, the authors reported. Sensitivity at this cutoff point was similar to manual dual-staining, but with significantly higher specificity (see table). Furthermore, the researchers found similar results when they evaluated the system for women who had been vaccinated for HPV.
"Our results demonstrate how automation and machine learning can transform cervical cancer screening that is currently undergoing major changes," Dr. Nicolas Wentzensen, PhD, a staff scientist in the NCI's Division of Cancer Epidemiology and Genetics, and colleagues wrote.
AI accuracy for detecting cervical precancer (n = 3,095) | |||
Method | Colposcopy referral (%) | Sensitivity | Specificity |
Pap cytology | 1,860 (60.1%) | 85.8% | 41.9% |
Manual dual-stain | 1,536 (49.6%) | 90% | 52.6% |
Automated dual-stain, 2-cell cutoff | 1,298 (41.9%) | 88.1% | 61.5% |
Automated dual-stain, 1-cell cutoff | 1,741 (56.3%) | 91.8% | 46.5% |
Transformative potential
Primary cervical cancer screening has shifted to detection of HPV, but a positive result may flag an infection that would have passed on its own, with no need for treatment. HPV tests may be followed up with a Pap test, but that is prone to error, with limited sensitivity and reproducibility, resulting in frequent retesting, the authors noted. The U.S. Food and Drug Administration (FDA) approved Roche's Cintech Plus cytology test, which simultaneously detects p16 and Ki-67, in March, a sign of further changes in the space.
"Automated [dual-staining] evaluation removes the remaining subjective component from cervical cancer screening and delivers consistent quality for providers and patients," the authors wrote. "Moving from Pap to automated [dual-staining] substantially reduces the number of colposcopies and also achieves excellent performance in a simulated fully vaccinated population."
In addition to the analysis of triage in cervical cancer screening for Kaiser patients, the NCI researchers evaluated the accuracy of Cytoreader using data from the Biopsy Study of women referred for colposcopy at the University of Oklahoma and the Anal Cancer Screening Study at San Francisco Kaiser Permanente Northern California.
"The classifier was robust, showing comparable performance in [two] cytology systems and in anal cytology," the authors reported.
Performance of AI automated dual-staining in cervical, anal precancers (n = 3,095) | ||
Dual-stain detection method | Sensitivity | Specificity |
Biopsy Study, colposcopy population (n = 409), cervical precancer (CIN3+) | ||
Manual | 87% | 40.5% |
AI automated | 87% | 45.6% |
Anal Cancer Screening Study (n = 299), anal precancer (AIN2+) | ||
Manual | 92.8% | 36.1% |
AI automated | 91.3% | 46.1% |
Though digital pathology has long been talked about and has been viewed as important, it is currently not used as much as it could be in routine practice, and patients are missing out on potential benefits, the authors suggested. The system evaluated at the NCI is cloud-based and could be globally accessible.
"Cloud-based evaluation provides ample computational capacity and storage space and can provide diagnostic procedures where sufficient personnel, expertise, or infrastructure is lacking," Wentzensen et al wrote.