Analytical Sensitivity of the Abbott BinaxNOW COVID-19 Ag Card
ABSTRACT
Multiple rapid antigen (Ag) tests for severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) have recently received emergency-use authorization (EUA) from the U.S. Food and Drug Administration (FDA). Although less sensitive than molecular detection methods, rapid antigen testing offers the potential for inexpensive, quick, decentralized testing. Robust analytical sensitivity data in comparison to reverse transcription-quantitative PCR (qRT-PCR) are currently lacking for many rapid antigen tests. Here, we evaluated the analytical sensitivity of the Abbott BinaxNOW COVID-19 Ag card using SARS-CoV-2-positive clinical specimens quantified by reverse transcription-droplet digital PCR (RT-ddPCR) and multiple FDA EUA qRT-PCR platforms using RNA standards. Initial and confirmatory limits of detection for the BinaxNOW COVID-19 Ag card were determined to be equivalent to 4.04 × 104 to 8.06 × 104 copies/swab. We further confirmed this limit of detection with 72 additional clinical samples positive for SARS-CoV-2 in either phosphate-buffered saline or viral transport medium. One hundred percent of samples with viral loads of >40,000 copies/swab were detected by rapid antigen testing. These data indicate that the BinaxNOW COVID-19 Ag card has an analytical sensitivity approximately equivalent to a generic qRT-PCR cycle threshold (CT) value of 29 to 30.
INTRODUCTION
Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) continues to spread in the United States and across the world, causing tens of millions of cases and more than 1 million deaths. Testing for SARS-CoV-2 has predominantly been performed on molecular platforms with exquisite analytical sensitivities that can detect fewer than 100 viral RNA copies per ml of viral transport medium (VTM) (1). These analytical sensitivities result in the detection of low-level shedding in patients for weeks to months after initial infection, often with very low viral loads (2–5). A double-edged sword of the sensitivity offered by real-time reverse transcription-quantitative PCR (qRT-PCR) is the possibility that the molecular methodology detects RNA copies of virus that may not necessarily correlate with an active infection, especially if the result is a low viral load (6–8). However, assay sensitivity is critical for detecting additional cases where low viral loads in the nares are present during active lower respiratory tract infection (9).
Despite the current provision of more than 1 million SARS-CoV-2 tests per day in the United States, molecular testing may not be able to scale much further. Since the beginning of the pandemic, many have looked to antigen (Ag) tests to provide rapid, inexpensive, and decentralized testing that might potentially reduce transmission with the thought that antigen tests could successfully detect infectious cases (10, 11). However, the demand for antigen testing has outstripped the associated data on its performance, most notably with the $750 million outlay by the U.S. government for the Abbott BinaxNOW COVID-19 Ag card based on data from just over 100 specimens in the emergency-use authorization (EUA) application (12–14). Only six samples with a cycle threshold (CT) value of >33 were tested as part of the submission for U.S. Food and Drug Administration (FDA) authorization for the BinaxNOW COVID-19 Ag card. Other antigen tests have reported specificities of 100% based on testing tens of specimens, but real-world performance has not supported these estimates (15–17).
Accurately understanding the trade-off of sensitivity and speed is critical to achieving the right balance of diagnostics used in different settings. While determination of clinical sensitivity in vivo is the gold standard, such studies can be complicated by the ordering of swabs and by differences in patient anatomy, collector experience, and transport (18–20). Here, we evaluated the analytical sensitivity of the Abbott BinaxNOW COVID-19 Ag card in a central laboratory by subjecting the lateral flow assay to known amounts of SARS-CoV-2 quantified by a variety of qRT-PCR platforms.
MATERIALS AND METHODS
Specimen collection and quantification.
This work was approved by the University of Washington institutional review board (IRB) (STUDY00010205). Deidentified samples used for the limit of detection (LoD) experiments were collected from patient specimens sent to the University of Washington Medical Center (UWMC) in Seattle, WA, for clinical diagnostic testing. Two nasopharyngeal swabs (NPSs) transported in phosphate-buffered saline (PBS) with high-viral-load CTs of 16.0 and 21.9 by the Panther Fusion SARS-CoV-2 assay (Hologic, Marlborough, MA, USA) were pooled (21). This pool was quantified on the Roche cobas 6800 SARS-CoV-2 platform (Roche, Basel, Switzerland) using AccuPlex verification panel RNA standards (SeraCare Life Sciences, Milford, MA, USA) as well as by the reverse transcription-droplet digital PCR (RT-ddPCR) system (Bio-Rad, Hercules, CA, USA) using the CDC N1 primer (22, 23).
Four hundred microliters of input material was utilized for the Roche cobas SARS-CoV-2 assay, and 500 μl was used for detection using the Hologic Panther Fusion SARS-CoV-2 assay. For RT-ddPCR, nucleic acids were extracted from 200 μl of the sample using the QIAamp viral RNA minikit (Qiagen, Hilden, Germany) and eluted into 100 μl distilled water (dH2O); 10 μl of this extracted RNA was used as the template in a 25-μl amplification reaction mixture, with samples run in triplicate and evaluated by mean quantitation (23). The cobas 6800 calibration standards reported CTs for the E gene of 27.72, 30.89, and 34.38 for 1 × 105, 1 × 104, and 1 × 103 calibration standards, respectively. The Panther Fusion assay standards reported CTs for the open reading frame 1ab (ORF1ab) gene target of 27.3, 30.8, and 34.5 for 1 × 105, 1 × 104, and 1 × 103 calibrators, respectively. According to Hologic’s FDA package insert, the Panther Fusion SARS-CoV-2 assay 1× LoD averaged from 10 contrived swab specimens was a CT of 35.6, while the CT cutoff is approximately 42 cycles (https://www.hologic.com/sites/default/files/2020-09/AW-21159-001_004_01.pdf). Similarly, Roche’s FDA EUA reported the cobas 6800 LoD at mean CTs of 32.7 for ORF1ab and 35.2 for the E gene, with a dynamic CT cutoff of around 44 cycles, although it runs 50 cycles on the instrument (https://www.fda.gov/media/136049/download).
The Roche cobas assay, RT-ddPCR, and the Abbott BinaxNOW rapid antigen test were all performed in parallel. Additionally, BinaxNOW provided positive swabs that were tested for each shipment lot number to account for reagent and Ag assay quality control. For analytical negative controls, at both initial and confirmatory LoD dilutions, a clinical NPS previously determined to be negative by the Panther Fusion assay was subjected to the rapid Ag test.
Determination of analytical sensitivity.
For the initial LoD, rapid Ag tests were run in triplicate with serial 10-fold PBS dilutions of the virus. First, a range of volumes (dH2O) from 25 to 100 μl was applied to each kit-provided sterile foam-tipped applicator to visually evaluate saturation; after soaking the swab in dH2O, any liquid remaining in the microcentrifuge tube was measured with a pipette. Without oversoaking or underexposing the swab, 50 μl was determined to be appropriate for saturation. Next, 50 μl of the quantified sample was aliquoted into microcentrifuge tubes, where each swab was inserted, gently rotated to absorb liquid, and subjected to the rapid antigen test according to the manufacturer’s protocol.
To account for liquid absorbed by the swab that might dilute the extraction buffer, a separate set of triplicate swabs was also allowed to air dry for 25 min after being saturated. To account for the potential impact of PBS on the antigen assay reaction, the original pool was also diluted in triplicate with dH2O for comparison. No difference was detected in either comparison for dH2O-diluted swab saturation and air-dried swabs. The initial LoD in triplicate was determined at the same dilution level regardless of whether the virus applied to the swab was diluted in PBS, diluted in water, or air dried (data not shown). All COVID-19 Ag cards were read after 15 min, and samples were never frozen and stored at 4°C.
RESULTS
The original positive sample pool yielded CTs of 17.26 for target 1 (ORF1ab) and 17.27 for target 2 (E gene) with the Roche cobas SARS-CoV-2 assay. Based on RNA standard quantification using the E gene primer set, this result corresponded to 1.13 × 108 copies/ml, while RT-ddPCR quantified the sample at 5.65 × 107 copies/ml based on the CDC N1 primer set. Initial LoDs of BinaxNOW were recognized as the last dilution set to detect 3/3 (100%) positive samples. At first, this put the detection range between 10-fold and 100-fold dilutions of the neat, positive pool (or 2.83 × 105 to 2.83 × 104 copies/swab when quantified by RT-ddPCR) (Table 1).
TABLE 1
| Sample | Dilution | No. of initial replicates detected/no. tested | No. of confirmatory replicates detected/no. tested | Initial % positive | Confirmatory % positive | No. of copies/ml (ddPCR) | No. of copies/swab (ddPCR) | No. of copies/ml (cobas) | No. of copies/swab (cobas) |
|---|---|---|---|---|---|---|---|---|---|
| S0a | Neat | 3/3 | 100 | 5.65 × 107 | 2.83 × 106 | 1.13 × 108 | 5.64 × 106 | ||
| S1 | 1:10 | 3/3 | 100 | 5.65 × 106 | 2.83 × 105 | 1.13 × 107 | 5.64 × 105 | ||
| S1-2 | 1:20 | 3/3 | 100 | 2.83 × 106 | 1.42 × 105 | 5.64 × 106 | 2.82 × 105 | ||
| S1-4 | 1:40 | 3/3 | 100 | 1.41 × 106 | 7.05 × 104 | 2.82 × 106 | 1.41 × 105 | ||
| S1-5 | 1:50 | 3/3 | 100 | 1.13 × 106 | 5.65 × 104 | 2.26 × 106 | 1.13 × 105 | ||
| S1-6 | 1:60 | 3/3 | 100 | 9.42 × 105 | 4.71 × 104 | 1.88 × 106 | 9.40 × 104 | ||
| S1-7 | 1:70 | 3/3 | 20/20 | 100 | 100 | 8.07 × 105 | 4.04 × 104 | 1.61 × 106 | 8.06 × 104 |
| S1-8 | 1:80 | 3/3 | 17/20 | 67 | 85 | 7.06 × 105 | 3.53 × 104 | 1.41 × 106 | 7.05 × 104 |
| S2 | 1:100 | 0/3 | 0 | 5.65 × 105 | 2.83 × 104 | 1.13 × 106 | 5.64 × 104 | ||
| S3 | 1:1,000 | 0/3 | 0 | 5.65 × 104 | 2.83 × 103 | 1.13 × 105 | 5.64 × 103 | ||
| Positive control | Neat | 1/1 | 1/1 | 100 | 100 | NA | NA | NA | NA |
| Negative control | Neat | 0/1 | 0/1 | 0 | 0 | NA | NA | NA | NA |
a
S0, the undiluted positive pool, was determined to have a CT of 17.3 for both targets on the Roche cobas 6800 platform.
b
The positive control was a kit-provided swab by Abbott; the negative control was a clinical nasopharyngeal swab previously determined to be negative by Hologic Panther Fusion. Abbreviations: ddPCR, droplet digital PCR; Pos, positive; Neg, negative; NA, not applicable. The two dilutions that confirm the LoD are in boldface.
Further dilution sets in triplicate narrowed this initial LoD range to 1:70 to 1:80 dilutions. Confirmatory LoDs were established with 20 replicates at each 1:70 and 1:80 dilution and characterized by ≥95% (19/20) positive replicates detected. Initial and confirmatory LoDs for Abbott’s BinaxNOW rapid antigen test were both determined at a 1:70 dilution of the original pooled positive samples, corresponding to 8.06 × 104 E gene copies/swab by RNA standards run on Roche cobas 6800 or 4.04 × 104 N gene copies/swab by RT-ddPCR (Table 1).
Based on the RNA standard curve on the cobas 6800 platform and a typical dilution of a swab into 3 ml VTM or PBS, these limits of detection correspond to E gene CTs of approximately 30.5 and 29.5, respectively. We then took the 50-μl volume at the LoD (1:70 dilution of the neat, positive pool), applied it to the kit-provided foam-tipped swab, and diluted it in 3 ml of PBS, similar to how a clinical NPS would be treated for molecular detection. The cobas assay quantified this sample with CTs of 28.68 (ORF1ab) and 29.08 (E gene).
After establishing the LoD range at 4.04 × 104 to 8.06 × 104 copies/swab, we next expanded the sample set to include 72 additional clinical positive specimens. These fresh NPSs were previously positive by Hologic Panther Fusion with ORF1ab CTs ranging from 14.6 to 39.9, corresponding to viral loads of 3.29 × 108 to 2.91 × 101 copies/ml, and stored in either PBS or VTM. For all specimens with >40,000 copies/swab (n = 24), the Abbott BinaxNOW-19 Ag card correctly detected 100% of deidentified samples (Table 2). Notably, multiple samples beyond the LoD were still detected, with some having as few as an estimated 2,400 copies/swab.
TABLE 2
| UW sample ID | Matrix medium | Panther Fusion CT | Original no. of copies/ml | No. of copies/swab | No. of copies/ml after dilution in 3 ml | Theoretical Panther Fusion CT | BinaxNOW antigen detection |
|---|---|---|---|---|---|---|---|
| 8 | VTM | 14.6 | 3.29 × 108 | 1.65 × 107 | 5.49 × 106 | 21.0 | DET |
| 3 | PBS | 17.1 | 6.65 × 107 | 3.33 × 106 | 1.11 × 106 | 23.5 | DET |
| 13 | VTM | 17.1 | 6.65 × 107 | 3.33 × 106 | 1.11 × 106 | 23.5 | DET |
| 9 | VTM | 17.4 | 5.49 × 107 | 2.75 × 106 | 9.15 × 105 | 23.8 | DET |
| 72 | VTM | 17.6 | 4.83 × 107 | 2.42 × 106 | 8.06 × 105 | 24.0 | DET |
| 67 | VTM | 17.8 | 4.25 × 107 | 2.13 × 106 | 7.09 × 105 | 24.2 | DET |
| 19 | VTM | 18 | 3.74 × 107 | 1.87 × 106 | 6.24 × 105 | 24.4 | DET |
| 54 | VTM | 18 | 3.74 × 107 | 1.87 × 106 | 6.24 × 105 | 24.4 | DET |
| 71 | VTM | 18.1 | 3.51 × 107 | 1.76 × 106 | 5.85 × 105 | 24.5 | DET |
| 60 | VTM | 18.2 | 3.29 × 107 | 1.65 × 106 | 5.49 × 105 | 24.6 | DET |
| 16 | VTM | 18.5 | 2.72 × 107 | 1.36 × 106 | 4.53 × 105 | 24.9 | DET |
| 44 | VTM | 18.7 | 2.39 × 107 | 1.20 × 106 | 3.99 × 105 | 25.1 | DET |
| 42 | VTM | 19.3 | 1.63 × 107 | 8.15 × 105 | 2.72 × 105 | 25.7 | DET |
| 47 | VTM | 19.5 | 1.43 × 107 | 7.17 × 105 | 2.39 × 105 | 25.9 | DET |
| 64 | VTM | 19.7 | 1.26 × 107 | 6.31 × 105 | 2.10 × 105 | 26.1 | DET |
| 6 | PBS | 19.9 | 1.11 × 107 | 5.55 × 105 | 1.85 × 105 | 26.3 | DET |
| 10 | VTM | 19.9 | 1.11 × 107 | 5.55 × 105 | 1.85 × 105 | 26.3 | DET |
| 69 | VTM | 21 | 5.50 × 106 | 2.75 × 105 | 9.16 × 104 | 27.4 | DET |
| 17 | VTM | 21.2 | 4.84 × 106 | 2.42 × 105 | 8.06 × 104 | 27.6 | DET |
| 63 | VTM | 21.2 | 4.84 × 106 | 2.42 × 105 | 8.06 × 104 | 27.6 | DET |
| 48 | VTM | 21.5 | 3.99 × 106 | 2.00 × 105 | 6.65 × 104 | 27.9 | DET |
| 61 | VTM | 22 | 2.90 × 106 | 1.45 × 105 | 4.83 × 104 | 28.4 | DET |
| 12 | VTM | 22.1 | 2.72 × 106 | 1.36 × 105 | 4.53 × 104 | 28.5 | DET |
| 52 | VTM | 22.1 | 2.72 × 106 | 1.36 × 105 | 4.53 × 104 | 28.5 | DET |
| 70 | VTM | 22.5 | 2.11 × 106 | 1.05 × 105 | 3.51 × 104 | 28.9 | DET |
| 22 | VTM | 23.1 | 1.44 × 106 | 7.18 × 104 | 2.39 × 104 | 29.5 | DET |
| 51 | VTM | 23.4 | 1.18 × 106 | 5.92 × 104 | 1.97 × 104 | 29.8 | DET |
| 49 | VTM | 23.8 | 9.17 × 105 | 4.59 × 104 | 1.53 × 104 | 30.2 | NOT DET |
| 58 | VTM | 23.9 | 8.60 × 105 | 4.30 × 104 | 1.43 × 104 | 30.3 | NOT DET |
| 65 | VTM | 24.2 | 7.10 × 105 | 3.55 × 104 | 1.18 × 104 | 30.6 | DET |
| 57 | VTM | 24.4 | 6.25 × 105 | 3.12 × 104 | 1.04 × 104 | 30.8 | DET |
| 32 | VTM | 24.7 | 5.16 × 105 | 2.58 × 104 | 8.60 × 103 | 31.1 | NOT DET |
| 23 | VTM | 24.8 | 4.84 × 105 | 2.42 × 104 | 8.06 × 103 | 31.2 | NOT DET |
| 1 | PBS | 25 | 4.26 × 105 | 2.13 × 104 | 7.10 × 103 | 31.4 | DET |
| 68 | VTM | 25.1 | 3.99 × 105 | 2.00 × 104 | 6.66 × 103 | 31.5 | NOT DET |
| 43 | VTM | 25.4 | 3.30 × 105 | 1.65 × 104 | 5.50 × 103 | 31.8 | NOT DET |
| 59 | VTM | 25.4 | 3.30 × 105 | 1.65 × 104 | 5.50 × 103 | 31.8 | DET |
| 29 | VTM | 26.9 | 1.26 × 105 | 6.32 × 103 | 2.11 × 103 | 33.3 | NOT DET |
| 55 | VTM | 27.1 | 1.11 × 105 | 5.56 × 103 | 1.85 × 103 | 33.5 | DET |
| 27 | VTM | 27.2 | 1.04 × 105 | 5.21 × 103 | 1.74 × 103 | 33.6 | DET |
| 33 | VTM | 27.4 | 9.18 × 104 | 4.59 × 103 | 1.53 × 103 | 33.8 | NOT DET |
| 35 | VTM | 27.7 | 7.58 × 104 | 3.79 × 103 | 1.26 × 103 | 34.1 | DET |
| 38 | VTM | 27.7 | 7.58 × 104 | 3.79 × 103 | 1.26 × 103 | 34.1 | DET |
| 21 | VTM | 28.1 | 5.87 × 104 | 2.93 × 103 | 9.78 × 102 | 34.5 | NOT DET |
| 37 | VTM | 28.1 | 5.87 × 104 | 2.93 × 103 | 9.78 × 102 | 34.5 | DET |
| 62 | VTM | 28.1 | 5.87 × 104 | 2.93 × 103 | 9.78 × 102 | 34.5 | NOT DET |
| 53 | VTM | 28.3 | 5.16 × 104 | 2.58 × 103 | 8.60 × 102 | 34.7 | NOT DET |
| 2 | PBS | 28.4 | 4.84 × 104 | 2.42 × 103 | 8.07 × 102 | 34.8 | DET |
| 46 | VTM | 28.4 | 4.84 × 104 | 2.42 × 103 | 8.07 × 102 | 34.8 | NOT DET |
| 45 | VTM | 28.6 | 4.26 × 104 | 2.13 × 103 | 7.10 × 102 | 35.0 | NOT DET |
| 11 | VTM | 29 | 3.30 × 104 | 1.65 × 103 | 5.50 × 102 | 35.4 | NOT DET |
| 26 | VTM | 29 | 3.30 × 104 | 1.65 × 103 | 5.50 × 102 | 35.4 | NOT DET |
| 41 | VTM | 29 | 3.30 × 104 | 1.65 × 103 | 5.50 × 102 | 35.4 | NOT DET |
| 34 | VTM | 29.1 | 3.09 × 104 | 1.55 × 103 | 5.16 × 102 | 35.5 | NOT DET |
| 66 | VTM | 29.4 | 2.55 × 104 | 1.28 × 103 | 4.26 × 102 | 35.8 | NOT DET |
| 56 | VTM | 30.1 | 1.63 × 104 | 8.16 × 102 | 2.72 × 102 | 36.5 | NOT DET |
| 18 | VTM | 30.6 | 1.19 × 104 | 5.93 × 102 | 1.98 × 102 | 37.0 | NOT DET |
| 50 | VTM | 30.8 | 1.04 × 104 | 5.22 × 102 | 1.74 × 102 | 37.2 | NOT DET |
| 39 | VTM | 31 | 9.18 × 103 | 4.59 × 102 | 1.53 × 102 | 37.4 | NOT DET |
| 36 | VTM | 31.3 | 7.58 × 103 | 3.79 × 102 | 1.26 × 102 | 37.7 | NOT DET |
| 25 | VTM | 31.6 | 6.26 × 103 | 3.13 × 102 | 1.04 × 102 | 38.0 | NOT DET |
| 14 | VTM | 31.7 | 5.87 × 103 | 2.93 × 102 | 9.78 × 101 | 38.1 | NOT DET |
| 24 | VTM | 32.1 | 4.54 × 103 | 2.27 × 102 | 7.57 × 101 | 38.5 | NOT DET |
| 30 | VTM | 34.1 | 1.26 × 103 | 6.32 × 101 | 2.11 × 101 | 40.5 | NOT DET |
| 20 | VTM | 35 | 7.11 × 102 | 3.56 × 101 | 1.19 × 101 | >40.5 | NOT DET |
| 31 | VTM | 35.1 | 6.67 × 102 | 3.34 × 101 | 1.11 × 101 | >40.5 | NOT DET |
| 4 | PBS | 36.2 | 3.30 × 102 | 1.65 × 101 | 5.50 | >40.5 | NOT DET |
| 28 | VTM | 36.4 | 2.91 × 102 | 1.45 × 101 | 4.84 | >40.5 | NOT DET |
| 40 | VTM | 36.8 | 2.25 × 102 | 1.13 × 101 | 3.75 | >40.5 | NOT DET |
| 15 | VTM | 38.1 | 9.80 × 101 | 4.9 | 1.63 | >40.5 | NOT DET |
| 5 | PBS | 38.8 | 6.26 × 101 | 3.13 | 1.04 | >40.5 | NOT DET |
| 7 | PBS | 39.9 | 3.10 × 101 | 1.55 | 0.52 | >40.5 | NOT DET |
a
Abbreviations: Ag, antigen; UW, University of Washington; CT, cycle threshold; VTM, viral transport medium; PBS, phosphate-buffered saline; DET, detected; NOT DET, not detected. The conversion factor for copies per swab is 1/20 of copies per milliliter. Theoretical Panther Fusion CTs were calculated with RNA standards factoring in a 1:20 dilution for the copies per swab and then an additional dilution in 3 ml of transport medium. Hologic Panther Fusion CTs of >40.5 are expected to be missed.
DISCUSSION
Here, we describe a detailed examination of the analytical sensitivity of the BinaxNOW COVID-19 Ag card in a central laboratory. We estimate a limit of detection of 4.04 × 104 to 8.06 × 104 viral copies/swab, corresponding to a CT of approximately 30. These data generally agree with Abbott’s package insert, which separated samples based on a CT cutoff of 33. These data suggest a difference of approximately 6 to 7 CTs between the LoD of this antigen test and that of RT-PCR tests, indicating an ∼100-fold difference in sensitivity. Importantly, the difference in the CT is notable as several studies have been able to culture virus at viral loads above a CT of 30 but rarely above a CT of 34 (8, 24, 25). According to work by Singanayagam et al., viable viruses with CTs of >35 were found in 8.3% (5/60) of patient samples (26). Importantly, putting the limit of detection in terms of qRT-PCR CTs can vary by platform based on extraction and elution volumes, viral transcripts targeted by the assay, and the amount of transport medium that the swab is diluted in before it is amplified for molecular detection. To avoid being dependent on one assay for quantitation, we quantified our original pooled positive samples in PBS on multiple platforms using multiple quantitation methods in our clinical laboratory in terms of viral copies per swab. These values of copies per swab correlate the amount of virus that each rapid antigen test sees with molecular detection methods.
This study was chiefly limited by the testing of specimens sent in transport media to a central laboratory. Rapid antigen tests are not generally meant to be tested on specimens in transport media, overcoming their higher limits of detection by sampling a whole swab. We took measures to ensure that our contrived specimens would directly answer the question of limits of detection by first using specimens in PBS for the LoD determination since the compositions of viral transport media and other complex matrices have been shown to inhibit rapid antigen assays (14, 27). We focused on rigorously interrogating the analytical sensitivity of the assay and did not evaluate assay specificity as we presumed that specimens in transport media would not adequately reflect relevant interferences found in direct nasal specimens. It has been well established that molecular detection is not grossly affected by PBS versus VTM (28). Moreover, two studies have used PBS or VTM samples for antigen testing (13, 29). It is possible that additional studies could better inform the equivalency of VTM versus PBS. We also note that the assays performed here do not adjudicate the “infectiousness” of the individuals associated with these specimens or the public health impact of different testing algorithms, which can be measured only by detailed clinical investigation outside the laboratory.
Our work also did not measure the analytical specificity of the antigen test as we believed that we could not adequately model specificity under the contrived conditions in the laboratory. Understanding assay specificity is critical to estimating the number of false-positive results that may occur in a given testing environment. Of note, the FDA recently administered a press release on 3 November 2020, warning clinical laboratory staff and health care providers to be aware of the potential for false-positive test results with rapid antigen tests (30). Finally, in order to compare sensitivities in terms of qRT-PCR, we reported the LoD in viral RNA copies, even though viral RNA is not targeted by rapid antigen tests. Although CTs can vary between platforms, laboratories, analytical parameters, and even technicians and runs, LoD CT values are also reported here for semiquantitative reference.
Having access to affordable, rapid testing is critical to reducing transmission during a pandemic. Although rapid antigen tests can be convenient and inexpensive, here, we confirm that thousands to tens of thousands of viral copies are necessary for detection, which is significantly greater than that required for qRT-PCR (12, 13, 31). These limitations in sensitivity are balanced by the ease of use and short turnaround time of the antigen test (32, 33). Our work more closely approximates the inherent abilities and limitations of antigen cards to detect virus, beyond the myriad of preanalytical variables that may differ between tests. These estimates are critical for evaluating when to use antigen versus qRT-PCR testing and to more accurately model how many cases are detected and missed by antigen tests (34). A limit of detection at ∼40,000 copies/swab was also recently seen in outside work performed in parallel (35). More epidemiological and clinical research work, beyond viral culture, is required to determine whether the viral loads detected by antigen cards correspond to “infectious” virus and whether rapid diagnostics will ultimately help to detect and reduce viral transmission.
ACKNOWLEDGMENTS
We thank Aria Bovell, V. Michele Ladd, and Anna Chao for assistance with data acquisition. We thank Reed Schuler, Romesh Gautom, and the Washington State Department of Health for providing the kits.
A.L.G. reports research support from Abbott Laboratories, independent of this work. Abbott had no role in the design or execution of the study.
REFERENCES
1.
Degli-Angeli E, Dragavon J, Huang M-L, Lucic D, Cloherty G, Jerome KR, Greninger AL, Coombs RW. 2020. Validation and verification of the Abbott RealTime SARS-CoV-2 assay analytical and clinical performance. J Clin Virol 129:104474.
2.
Dou C, Xie X, Peng Z, Tang H, Jiang Z, Zhong Z, Tang J. 2020. A case presentation for positive SARS-CoV-2 RNA recurrence in a patient with a history of type 2 diabetes that had recovered from severe COVID-19. Diabetes Res Clin Pract 166:108300.
3.
Tahamtan A, Ardebili A. 2020. Real-time RT-PCR in COVID-19 detection: issues affecting the results. Expert Rev Mol Diagn 20:453–454.
4.
Wu Y, Guo C, Tang L, Hong Z, Zhou J, Dong X, Yin H, Xiao Q, Tang Y, Qu X, Kuang L, Fang X, Mishra N, Lu J, Shan H, Jiang G, Huang X. 2020. Prolonged presence of SARS-CoV-2 viral RNA in faecal samples. Lancet Gastroenterol Hepatol 5:434–435.
5.
Corcorran MA, Olin S, Rani G, Nasenbeny K, Constantino-Shor C, Holmes C, Quinnan-Hostein L, Solan W, Snoeyenbos Newman G, Roxby AC, Greninger AL, Jerome KR, Neme S, Lynch JB, Dellit TH, Cohen SA. 20 August 2020. Prolonged persistence of PCR-detectable virus during an outbreak of SARS-CoV-2 in an inpatient geriatric psychiatry unit in King County, Washington. Am J Infect Control
6.
Surkova E, Nikolayevskyy V, Drobniewski F. 2020. False-positive COVID-19 results: hidden problems and costs. Lancet Respir Med 8:1167–1168.
7.
Cohen AN, Kessel B, Milgroom MG. 2020. Diagnosing COVID-19 infection: the danger of over-reliance on positive test results. medRxiv 2020.04.26.20080911. https://www.medrxiv.org/content/10.1101/2020.04.26.20080911v4.
8.
La Scola B, Le Bideau M, Andreani J, Hoang VT, Grimaldier C, Colson P, Gautret P, Raoult D. 2020. Viral RNA load as determined by cell culture as a management tool for discharge of SARS-CoV-2 patients from infectious disease wards. Eur J Clin Microbiol Infect Dis 39:1059–1061.
9.
Ramos KJ, Kapnadak SG, Collins BF, Pottinger PS, Wall R, Mays JA, Perchetti GA, Jerome KR, Khot S, Limaye AP, Mathias PC, Greninger A. 2020. Detection of SARS-CoV-2 by bronchoscopy after negative nasopharyngeal testing: stay vigilant for COVID-19. Respir Med Case Rep 30:101120.
10.
Service RF. 22 May 2020. Coronavirus antigen tests: quick and cheap, but too often wrong? Science
11.
Samuel L. 2020. Point-of-care testing in microbiology. Clin Lab Med 40:483–494.
12.
Guglielmi G. 2020. Fast coronavirus tests: what they can and can’t do. Nature 585:496–498.
13.
Mak GC, Cheng PK, Lau SS, Wong KK, Lau C, Lam ET, Chan RC, Tsang DN. 2020. Evaluation of rapid antigen test for detection of SARS-CoV-2 virus. J Clin Virol 129:104500.
14.
Lambert-Niclot S, Cuffel A, Le Pape S, Vauloup-Fellous C, Morand-Joubert L, Roque-Afonso A-M, Le Goff J, Delaugerre C. 2020. Evaluation of a rapid diagnostic assay for detection of SARS-CoV-2 antigen in nasopharyngeal swabs. J Clin Microbiol 58:e00977-20.
15.
Wu KJ. 7 October 2020. Nevada halts use of rapid coronavirus tests in nursing homes, citing inaccuracies. New York Times, New York, NY. https://www.nytimes.com/2020/10/07/world/nevada-halts-use-of-rapid-coronavirus-tests-in-nursing-homes-citing-inaccuracies.html
16.
Cairns E. 26 October 2020. Balancing the accuracy and cost of antigen testing. Evaluate Vantage, London, United Kingdom. https://www.evaluate.com/vantage/articles/news/policy-and-regulation/balancing-accuracy-and-cost-antigen-testing.
17.
Marcy D. 29 August 2020. Health commissioner takes issue with COVID-19 claims. Manchester Journal, Manchester, VT. https://www.manchesterjournal.com/news/local/health-commissioner-takes-issue-with-covid-19-claims/article_b2718273-9089-5da2-a994-2a0476f9c89a.html
18.
Altamirano J, Govindarajan P, Blomkalns AL, Kushner LE, Stevens BA, Pinsky BA, Maldonado Y. 2020. Assessment of sensitivity and specificity of patient-collected lower nasal specimens for severe acute respiratory syndrome coronavirus 2 testing. JAMA Netw Open 3:e2012005.
19.
Long DR, Gombar S, Hogan CA, Greninger AL, O’Reilly-Shah V, Bryson-Cahn C, Stevens B, Rustagi A, Jerome KR, Kong CS, Zehnder J, Shah NH, Weiss NS, Pinsky BA, Sunshine JE. 7 June 2020. Occurrence and timing of subsequent severe acute respiratory syndrome coronavirus 2 reverse-transcription polymerase chain reaction positivity among initially negative patients. Clin Infect Dis
20.
Tu Y-P, Jennings R, Hart B, Cangelosi GA, Wood RC, Wehber K, Verma P, Vojta D, Berke EM. 2020. Swabs collected by patients or health care workers for SARS-CoV-2 testing. N Engl J Med 383:494–496.
21.
Perchetti GA, Huang M-L, Peddu V, Jerome KR, Greninger AL. 2020. Stability of SARS-CoV-2 in phosphate-buffered saline for molecular detection. J Clin Microbiol 58:e01094-20.
22.
Lieberman JA, Pepper G, Naccache SN, Huang M-L, Jerome KR, Greninger AL. 2020. Comparison of commercially available and laboratory developed assays for in vitro detection of SARS-CoV-2 in clinical laboratories. J Clin Microbiol 58:e00821-20.
23.
Perchetti GA, Nalla AK, Huang M-L, Zhu H, Wei Y, Stensland L, Loprieno MA, Jerome KR, Greninger AL. 2020. Validation of SARS-CoV-2 detection across multiple specimen types. J Clin Virol 128:104438.
24.
Wölfel R, Corman VM, Guggemos W, Seilmaier M, Zange S, Müller MA, Niemeyer D, Jones TC, Vollmar P, Rothe C, Hoelscher M, Bleicker T, Brünink S, Schneider J, Ehmann R, Zwirglmaier K, Drosten C, Wendtner C. 2020. Virological assessment of hospitalized patients with COVID-2019. Nature 581:465–469.
25.
Jefferson T, Spencer E, Brassey J, Heneghan C. 2020. Viral cultures for COVID-19 infectivity assessment. Systematic review. medRxiv 2020.08.04.20167932. https://www.medrxiv.org/content/10.1101/2020.08.04.20167932v4.
26.
Singanayagam A, Patel M, Charlett A, Lopez Bernal J, Saliba V, Ellis J, Ladhani S, Zambon M, Gopal R. 2020. Duration of infectiousness and correlation with RT-PCR cycle threshold values in cases of COVID-19, England, January to May 2020. Euro Surveill 25:2001483.
27.
Quach C, Newby D, Daoust G, Rubin E, McDonald J. 2002. QuickVue influenza test for rapid detection of influenza A and B viruses in a pediatric population. Clin Diagn Lab Immunol 9:925–926.
28.
Rodino KG, Espy MJ, Buckwalter SP, Walchak RC, Germer JJ, Fernholz E, Boerger A, Schuetz AN, Yao JD, Binnicker MJ. 2020. Evaluation of saline, phosphate-buffered saline, and minimum essential medium as potential alternatives to viral transport media for SARS-CoV-2 testing. J Clin Microbiol 58:e00590-20.
29.
Corman VM, Haage VC, Bleicker T, Schmidt ML, Mühlemann B, Zuchowski M, Jó Lei WK, Tscheak P, Möncke-Buchner E, Müller MA, Krumbholz A, Drexler JF, Drosten C. 2020. Comparison of seven commercial SARS-CoV-2 rapid point-of-care antigen tests. medRxiv. https://www.medrxiv.org/content/10.1101/2020.11.12.20230292v1.
30.
FDA. 2020. Potential for false positive results with antigen tests for rapid detection of SARS-CoV-2—letter to clinical laboratory staff and health care providers. FDA, Silver Spring, MD. https://www.fda.gov/medical-devices/letters-health-care-providers/potential-false-positive-results-antigen-tests-rapid-detection-sars-cov-2-letter-clinical-laboratory.
31.
Nalla AK, Casto AM, Huang M-LW, Perchetti GA, Sampoleo R, Shrestha L, Wei Y, Zhu H, Jerome KR, Greninger AL. 2020. Comparative performance of SARS-CoV-2 detection assays using seven different primer-probe sets and one assay kit. J Clin Microbiol 58:e00557-20.
32.
Larremore DB, Wilder B, Lester E, Shehata S, Burke JM, Hay JA, Tambe M, Mina MJ, Parker R. 2020. Test sensitivity is secondary to frequency and turnaround time for COVID-19 surveillance. medRxiv 2020.06.22.20136309. https://www.medrxiv.org/content/10.1101/2020.06.22.20136309v3.
33.
Rubin R. 2020. The challenges of expanding rapid tests to curb COVID-19. JAMA 324:1813–1815.
34.
Goyal A, Reeves DB, Cardozo-Ojeda EF, Schiffer JT, Mayer BT. 2020. Wrong person, place and time: viral load and contact network structure predict SARS-CoV-2 transmission and super-spreading events. medRxiv 2020.08.07.20169920. https://www.medrxiv.org/content/10.1101/2020.08.07.20169920v3.
35.
Pilarowski G, Lebel P, Sunshine S, Liu J, Crawford E, Marquez C, Rubio L, Chamie G, Martinez J, Peng J, Black D, Wu W, Pak J, Laurie MT, Jones D, Miller S, Jacobo J, Rojas S, Rojas S, Nakamura R, Tulier-Laiwa V, Petersen M, Havlir DV, CLIAHUB Consortium, DeRisi J. 2020. Performance characteristics of a rapid SARS-CoV-2 antigen detection assay at a public plaza testing site in San Francisco. medRxiv 2020.11.02.20223891. https://www.medrxiv.org/content/10.1101/2020.11.02.20223891v2.
Information & Contributors
Information
Published In
Journal of Clinical Microbiology
Volume 59 • Number 3 • 18 February 2021
eLocator: e02880-20
Editor: Michael J. Loeffelholz, Cepheid
PubMed: 33310764
Copyright
© 2021 American Society for Microbiology. All Rights Reserved.
History
Received: 12 November 2020
Returned for modification: 7 December 2020
Accepted: 10 December 2020
Published online: 18 February 2021
Keywords
Contributors
Editor
Michael J. Loeffelholz
Editor
Cepheid
Metrics & Citations
Metrics
Note: There is a 3- to 4-day delay in article usage, so article usage will not appear immediately after publication.
Citation counts come from the Crossref Cited by service.
Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.