Point-of-care antibody tests for COVID-19 have performance issues and should not be used beyond research purposes, according to a systematic review and meta-analysis published July 1 in the BMJ.
The report from an international team of researchers is among the latest to draw attention to flawed serology products. The study evaluated the evidence for different types of antibody tests:
- Enzyme linked immunosorbent assays (ELISAs)
- Lateral flow immunoassays (LFIAs)
- Chemiluminescent immunoassays (CLIAs)
- Immunoglobulin (Ig) class: IgG, IgM, or both
Researchers assessed overall sensitivity and specificity as the primary outcome of the study, using data published between January 1 and April 30 in Medline, bioRxiv, and medRxiv. The analysis included 40 studies that included a comparison of serology to a reference standard of viral culture or reverse transcription polymerase chain reaction (RT-PCR).
Most studies -- 90% -- had been done in hospital inpatients and only two were done at the point of care. In terms of geography, 70% of studies had been conducted in China, and the rest were performed in the U.K., U.S., Denmark, Spain, Germany, Sweden, and Japan.
An analysis that pooled results together found sensitivity ranged from 66% to 97.8% and specificity ranged from 96.6% to 99.7%, Dr. Faiz Ahmad Khan, assistant professor at McGill University, and colleagues reported. The study also flagged lower performance for commercial kits versus noncommercial tests.
"Overall, the poor performance of existing serological tests for [COVID-19] raises questions about the utility of using such methods for medical decision making, particularly given time and effort required to do these tests and the challenging workloads many clinics are facing," Khan et al wrote.
| Comparison of serology test performance | ||
| True result | False result | |
| Patients with COVID-19 | ||
| ELISA | 84% | 16% |
| LFIA | 66% | 34% |
| CLIA | 98% | 2% |
| Patients without COVID-19 | ||
| ELISA | 98% | 2% |
| LFIA | 97% | 3% |
| CLIA | 98% | 2% |
The researchers wrote that they were particularly concerned to find that sensitivity was consistently lower for the LFIA method compared with the other types, as LFIA point-of-care tests are being considered by governments as a means of providing "immunity passports," that is, a certificate to allow people to resume work and other activities.
"Higher quality clinical studies assessing the diagnostic accuracy of serological tests for [COVID-19] are urgently needed," Khan and colleagues wrote. "Currently, available evidence does not support the continued use of existing point-of-care serological tests."
Serology pros and cons
The study was conducted to evaluate whether the evidence supported growing use of antibody tests, with a proliferation coming to the market, and as governments are considering how such testing could be used to manage reopenings. Compared with RT-PCR tests, serological tests are less expensive, easier to administer, and have potential to pick up past infection and associated immunity.
"The pace of development has, however, exceeded that of rigorous evaluation, and important uncertainty about test accuracy remains," the authors wrote.
In their analysis of randomized trials, cohort or case-control studies, and case series, the authors included preprints because they are commonly being used to inform public policy during the health emergency. There were major limitations to the studies included in the review, such as a high risk of patient selection bias in 98% of assessments and high or unclear risk of bias from test performance or interpretation. The authors also noted the lack of representation of outpatients and point-of-care assessments.
Based on the performance in the studies evaluated, in a setting where the prevalence of SARS-CoV-2 was 10% and 1,000 people were tested, 31 who never had COVID-19 would get a false positive and 34 who were infected would get a false negative, Khan and colleagues wrote. Testing is more accurate at least three weeks after the onset of symptoms than in the first week, they pointed out.
Results stratified by test type showed lower specificity in people with suspected COVID-19 and false positives for the LFIA method. Low sensitivity of LFIA is particularly problematic because the studies considered took steps to improve accuracy that would not be used at the point of care, the authors wrote. Consequently, LFIA serological tests should not be used beyond research and evaluation purposes, they advised.
"While the scientific community should be lauded for the pace at which novel serological tests have been developed, this review underscores the need for high quality clinical studies to evaluate these tools," Khan et al advised.
