Scientists still debate whether millions of cheap, fast diagnostic kits will help control the pandemic. Here’s why.

Nature, 09 FEBRUARY 2021
Giorgia Guglielmi

As the number of UK coronavirus cases surged in early 2021, the government announced a potential game-changer in the fight against COVID-19: millions of cheap, rapid virus tests. On 10 January, it said it would roll these tests out across the country, to be taken by people even if they have no symptoms. Similar tests will play a crucial part in US President Joe Biden’s plans to tame the raging outbreak in the United States.

These speedy tests, which typically mix nasal or throat swabs with liquid on a paper strip to return results within half an hour, are thought of as tests of infectiousness, not of infection. They can detect only high viral loads, so they will miss many people with lower levels of the SARS-CoV-2 virus. But the hope is that they will help to curb the pandemic by quickly identifying the most contagious people, who might otherwise unknowingly pass on the virus.

Yet, as the government announced its plan, a furious argument broke out. Some scientists were delighted by the United Kingdom’s testing strategy. Others said that the tests would miss so many infections that, if rolled out in their millions, they could cause more harm than good. Many people might be falsely reassured by a negative test result and change their behaviour, argued Jon Deeks, who specializes in test evaluation at the University of Birmingham, UK. And, he said, the tests would miss even more infections if people self-administered them, rather than relying on trained professionals. He and his Birmingham colleague Jac Dinnes are among scientists who want more data on rapid coronavirus tests before they’re used widely.

But other researchers soon fired back, saying that the claim that the tests could cause harm was wrong and “irresponsible” (see go.nature.com/3bcyzfm). They included Michael Mina, an epidemiologist at the Harvard T. H. Chan School of Public Health in Boston, Massachusetts, who says that the arguments are delaying a much-needed solution to the pandemic. “We continue to say we don’t have enough data yet, but we’re in the middle of a war — we really can’t get any worse than we are at the moment in terms of the case counts,” he says.

The only thing the scientists do agree on is that there needs to be clear communication about what rapid tests are for, and what a negative result means. “Throwing tools at people who don’t know how to use them appropriately is a terrible idea,” Mina says.

Comparing rapid tests

It’s hard to get reliable information on rapid tests because — at least in Europe — the products can be sold solely on the basis of manufacturer data, without independent evaluation. There are no standard protocols for measuring performance, making it hard to compare assays and forcing each country to do its own validation.

“It’s the Wild West in diagnostics,” says Catharina Boehme, chief executive of the Foundation for Innovative New Diagnostics (FIND), a non-profit group in Geneva, Switzerland, that has reassessed and compared dozens of COVID-19 assays.

In February 2020, FIND started the ambitious task of evaluating hundreds of COVID-19 test types in a standardized trial. In collaboration with the World Health Organization (WHO) and research institutes across the globe, the foundation runs tests on hundreds of coronavirus samples and compares their performance against those obtained using the highly sensitive technique of polymerase chain reaction (PCR). This technique looks for specific viral genetic sequences in a sample taken from a person’s nose or throat (or sometimes saliva). PCR-based tests make more copies of this genetic material through many amplification cycles, so they can detect what are initially minuscule quantities of virus. But they can be time-consuming, and require trained personnel and expensive lab equipment (see ‘How COVID-19 tests work’).

The cheaper, faster tests tend to work by detecting specific proteins, collectively termed antigens, on the surface of SARS-CoV-2 particles. These ‘rapid antigen tests’ don’t amplify what’s in the sample, so can detect the virus only when it reaches a high level in a person’s body — with perhaps hundreds of thousands or millions of viral copies per millilitre of sample. The virus usually reaches these levels around the time that symptoms start, when people are at their most contagious (see ‘Catching COVID-19’).

Manufacturer data on test sensitivity come mostly from laboratory trials on people with symptoms, who tend to have high viral loads, Dinnes says. In those trials, many of the rapid tests seem very sensitive. (They are also extremely specific: they are unlikely to give a false positive result.) But real-world evaluations have flagged up apparent differential performance on people with lower viral loads.

Viral levels in a sample are typically quantified by reference to the number of PCR amplification cycles needed to detect the virus. Generally, if it takes around 25 or fewer PCR amplification cycles (described as a cycle threshold, or Ct, of 25 or lower) then levels of viable virus are thought to be high, indicating that people are probably infectious — although it’s not known whether there’s a key level at which people are or aren’t contagious.

Last November, the UK government released preliminary results of research done at the Porton Down science park and at the University of Oxford; the full results, which have not yet been peer reviewed, were posted online on 15 January. These stated that although many fast antigen (or ‘lateral flow’) tests “do not perform at a level required for mass population deployment”, 4 separate brands had, in laboratory trials, achieved 91–100% sensitivity on samples with Ct values at or below 25. FIND’s re-evaluations of a number of rapid-test kits also generally suggest sensitivities of 90% or more at these viral levels.

As viral levels dip — that is, as Ct values rise — the rapid tests start to miss infections. The Porton Down scientists paid particular attention to tests from Innova Medical in Pasadena, California; the UK government has already spent more than £800 million (US$1.1 billion) ordering these tests, which form a major plank in its strategy to slow the spread of coronavirus. The tests dropped to 88% sensitivity at Ct levels of 25–28, and to 76% for Ct levels of 28–31 (see ‘Rapid tests spot high viral loads’).

That compares unfavourably with a December assessment of the BinaxNOW rapid test from Abbott Laboratories in Abbott Park, Illinois. The study tested more than 3,300 people in San Francisco, California, and scored 100% sensitivity for samples with Ct levels below 30 (even when infected individuals didn’t show symptoms).

But differently calibrated PCR systems mean that Ct levels can’t easily be compared between labs, and don’t always indicate the same level of virus in a sample. Innova says that the UK and US studies used different PCR systems, and that only direct comparisons on the same system are valid. They pointed to a UK government report, written by Porton Down scientists in late December, which ran the Innova test head-to-head with an Abbott Panbio test (similar to the BinaxNOW kit that Abbott sells in the United States). On just over two dozen samples with Ct levels lower than 27, both returned 93% as positive (see go.nature.com/3at82vm).

This nuance about Ct calibration is crucial when considering a trial of Innova tests on thousands of people in Liverpool, UK, which identified only two-thirds of the cases with Ct levels below 25 (see go.nature.com/3tajhkw). This suggested that the tests missed one-third of cases that were probably infectious. But it’s now thought that at the laboratory that processed the samples, Ct values of 25 equate to much lower viral levels — perhaps equivalent to Ct of 30 or above — at other labs, says public-health and informatics researcher Iain Buchan at the University of Liverpool, who led the trial.

However the details shake out, Deeks says that a December trial at the University of Birmingham is an example of how rapid tests can miss infections. More than 7,000 symptom-free students there took an Innova test; only 2 tested positive. But when the university researchers rechecked 10% of the negative samples using PCR, they found another 6 infected students. Scaling that up across all the samples, the test probably missed 60 infected students.

Mina says that these students had lower levels of virus, so were unlikely to be infectious anyway. Deeks argues that although people with lower virus levels might be in the late phase of a waning infection, they might also be on the way to becoming more infectious. Another factor is that some students might have done a poor job taking swab samples, so that not many viral particles made it to the test. He’s worried that people will falsely think they’re safe with a single negative test — when in fact a rapid test is only a snapshot of likely non-infectiousness at that moment. Remarks that the tests could make workplaces completely safe are not the right way to inform the public about their efficacy, Deeks says. “If people get a false sense of security, they could actually spread the virus around,” he says.

But Mina and others say the Liverpool pilots suggest that people don’t behave like that, and they are told that they might still transmit the virus later on. Mina emphasizes that using the tests frequently — say, twice a week — is key to making them effective at quenching a pandemic.

The interpretation of a test result depends not only on the test’s accuracy, but also on the chance that a person already has COVID-19. That depends on infection rates in their area, and whether they show symptoms. If someone from an area with high levels of COVID-19 has symptoms typical of the illness and gets a negative result, it’s probably a false negative that needs double checking with PCR.

Researchers also debate whether people should administer the tests themselves (at home, school or work). The performance of assays can vary depending on how testers take swabs and handle samples. For instance, laboratory scientists achieved nearly 79% sensitivity on all samples (including those with very low viral loads) using the Innova test, but self-trained members of the public got only 58% (see ‘Rapid tests: suitable at home?’) — which Deeks says is a worrying dip.

Despite this, in December, the United Kingdom’s drug regulator authorized the home use of the Innova test to detect infection in people without symptoms. The tests were branded as coming from the country’s National Health Service, and designed by the Department of Health and Social Care (DHSC), but were bought from Innova and manufactured by Xiamen Biotime Biotechnology in China, a DHSC spokesperson confirmed. “Lateral-flow tests used by the UK government go through a rigorous evaluation by the country’s leading scientists. This means they are accurate, reliable and successfully identify those with COVID-19 who don’t show symptoms,” the spokesperson said in a statement.

A German study suggests that self-administered tests can work as well as those done by professionals. The study, which has not yet been peer reviewed, found that when people swabbed their own noses and completed an unnamed rapid test approved by the WHO, the sensitivities were very similar to those achieved by professionals, even though people often deviated from the instructions.

In the United States, the Food and Drug Administration has granted emergency-use approval for 13 antigen tests, but only one — the Ellume COVID-19 Home Test — can be used in people without symptoms. According to Ellume, a firm in Brisbane, Australia, the test detected coronavirus in 10 of 11 asymptomatic people who had already tested positive with PCR. In February, the US government announced it would buy 8.5 million of the tests.

Rolling out rapid tests

Some countries that don’t have the resources for many PCR tests, such as India, have been using antigen tests for many months, simply to supplement their testing capacity. And some that have PCR tests are only just starting to roll out the rapid alternatives, in a limited way, because of the concerns over accuracy. But governments that have implemented mass rapid testing have called it a success. Slovakia, a country of 5.5 million people, was the first to attempt to test its entire adult population. Widespread testing has helped to cut the infection rate by nearly 60%. But the testing was done in combination with tough restrictions that are not implemented in other countries, as well as government financial support for those who tested positive, to help them stay at home. So although the combination of testing and restrictions seemed to reduce the infection rate more quickly than restrictions alone, it’s unclear whether the approach could work elsewhere, experts say. In other countries, many people might not want to take the rapid tests, and those who test positive might lack incentives to isolate. Still, because commercial rapid tests are so cheap — as low as US$5 — Mina says that cities and states could buy millions of them at a fraction of what the pandemic is costing governments.

Situations in which rapid tests might be particularly suitable for asymptomatic screening include prisons, homeless shelters, schools and universities, where people are likely to be congregating anyway, so that any test that can catch some extra infectious cases is useful. But Deeks cautions against allowing the tests to be used in ways that might change people’s behaviour or prompt them to loosen their precautions. For instance, people could interpret a negative result as encouragement to visit a relative in a nursing home.

In the United States, mass rapid-testing programmes have so far been rolled out in places such as schools, prisons, airports and universities. For example, the University of Arizona in Tucson has been using the Sofia tests developed by Quidel in San Diego, California, since May to test its athletes every day. Since August, it has been testing students at least once a month (some students — especially those who live in dormitories with outbreaks — are tested more frequently, up to once per week). So far, the university has administered almost 150,000 tests, and it has not reported spikes in COVID-19 cases in the past two months.

David Harris, a stem-cell researcher in charge of Arizona’s mass testing programme, says the different types of test have different uses: rapid antigen tests shouldn’t be used to assess the prevalence of a virus in a population, he notes. “If you use it like a PCR, you get a terrible sensitivity,” he says. “But in terms of what we’re trying to do — preventing the spread of infection — the antigen test, particularly when it’s applied multiple times, seems to work great.”

Many research groups around the world are devising faster and cheaper testing methods. Some are tweaking the PCR test to speed up the amplification process, but many of these tests still require specialist equipment. Other approaches rely on a technique called loop-mediated isothermal amplification, or LAMP, which is faster than PCR and requires minimal equipment. But the tests aren’t quite as sensitive as those based on PCR. Last year, researchers at the University of Illinois at Urbana-Champaign developed their own rapid diagnostic assay: a PCR-based test that uses saliva instead of a nasal swab and skips an expensive and slow step. The test costs $10–14 and gives results in less than 24 hours, allowing the university to screen everybody on campus twice a week, although it relies on an on-site laboratory to run the PCR. Last August, the frequent-testing programme allowed the university to detect a spike in campus infections and largely bring them under control. Within a week, new cases had decreased by 65%, and the university has not reported similar spikes since.

Boehme says that there is no one test that fits all needs, but assays that can identify people who are infectious are crucial to keeping the world’s economies open. “Testing at airports, borders, workplaces, in schools, in clinical settings — all those are cases where rapid tests have a lot of power because they are easy to use, low-cost and fast,” she says. However, she adds, large-scale testing programmes should rely on the best tests available.

The European Union’s approval process for COVID-19 diagnostic tests is currently the same as that for other kinds of diagnostic, but concerns around the performance of some assays spurred new guidelines last April. These call on manufacturers to produce testing kits that perform at least as well as state-of-the-art COVID-19 tests. But because tests can perform differently in manufacturers’ trials than in the real world, the guidelines recommend that member states validate tests before rolling them out.

Ideally, Boehme says, individual countries would not have to validate every assay. There would be common protocols — such as those developed by FIND — used by labs and manufacturers across the world. “What we need is standardized approaches to test evaluation,” she says. “That would not be different from what happens in evaluating treatments and vaccines.”

Originally published at https://www.nature.com.