How has AI Contributed to Dealing with the COVID-19 Pandemic?

The Gradient
Maxime Nauwynck
May 24, 2021

The calendar reads May, 2021. Although there is still uncertainty, many are already vaccinated against COVID-19, and hope lies on the horizon. The past year has been an annus horribilis in many ways. However, the past year has also been a year of astonishing scientific accomplishments. Numerous research groups, in combination with industry and government, have reoriented their resources to help contribute in the quest to contain the pandemic. The daily number of articles uploaded to arXiv that mention COVID-19 and their cumulative sum is shown below. It gives an idea of how people around the world, including the AI community, contributed to tackling the pandemic…

This piece reviews where these efforts have been realized as practical solutions, i.e. being used by corporations, governments, or individuals. Practical in this case refers to being used in the field, be it on an individual or collective level. “Experimental results” are largely discarded.

Five general categories are used to distinguish the different contributions. Note that the boundaries are not always that well defined and assigned categories may not fit every contribution equally well.

The categories are:

1. Clinical applications
2. Epidemiological applications
3. Applications for biochemistry
4. Providing information
5. Safety assessment

Based on the examples provided within each category, it is possble to evaluate whether or not AI has reached a maturity that allows it to move out of the theoretical realm and has been able to be part of the toolbox that helps society cope with the pandemic.

We can start with the now-famous case of BlueDot, a health monitoring platform that uses AI, and Healthmap by Boston Children’s Hospital.

The former looks at a bunch of different information sources: shared disease cases, news reports, information on air travel, and others to detect and provide warnings on possible outbreaks.

Similarly, the latter also looks at trends, including social media and search queries to monitor for possible outbreaks. 

In the final days of December 2019 — God I miss those days — both BlueDot and Healthmap warned of suspicious activity in the Wuhan area. 

The AI powering both didn’t miss the start of the COVID-19 pandemic. Let’s see how it fared elsewhere…

1.Clinical Applications

The first category includes applications that serve a clinical purpose and can be viewed as working on an individual level.

• Detecting infection (early on), 
• estimating disease progression, outcome, or 
• determining the appropriate treatment are some of the challenges they face. 

Typically, some combination of medically relevant signals is processed by some set of algorithms to complete the specific tasks.

In the early stages of the pandemic, when testing capacity was not yet fully scaled up due to being too expensive or simply not available, medical imaging was and still is used as a source of information for medical practitioners.

• Chest X-ray (CXR) and 
• computed tomography (CT) are the two main modalities.

They offer faster results compared to a routine RT-PCR test and offer a direct assessment of the impact on the lungs. Additionally, diagnostic findings indicative of COVID-19 may be present earlier on, when an RT-PCR test can still return negative.

By January 2021, hundreds of research papers had been published on the use of machine learning on medical images. Despite the obvious potential for computer vision techniques to be applied in clinical practice and the vast amount of research work being done, implementation is still limited. Based on the review by Roberts et al., we see some reasons why the transfer from theory to practice is lacking. In their review, they take a look at published articles and pre-prints related to diagnosis or prognosis using machine learning on CXR and/or CT images. Initially starting from 305 articles, 45 remained after closer examination.

The latter were considered reproducible and taken into account. The conclusion states that none of the studies was deemed fit for clinical use. Similarly, the running review by Wynants et al. also takes a critical look at the experimental study design of proposed prediction models but includes biomarkers and modalities beyond CXR and CT as well. Their findings are similar. A figure from their running paper is included below.

Lack of clinical validation, statistical analyses, and interpretability are common shortcomings. Public datasets are used in half of the papers, which are not necessarily quality checked, and are potentially biased. Based on these findings, the following recommendations were made:

• Be cautious with public datasets. Risks include, but are not limited to, possible duplicates, wrong labels, community-wide overfitting on the same benchmark-dataset, lack of metadata (age, gender, …), or overfitting to imaging protocol. An example is overfitting to a specific view in CXR when patients are immobile due to severe sickness.
• Include external validation, use confidence intervals and perform sensitivity analysis. An example of the need for external validation can be found here.
• Use interpretable methods. Total black-box approaches are simply unfit for use in practice.
• Version the dataset so that results are reproducible in case of changes.
• Use quality protocols like RQS, CLAIM, TRIPOD, PROBAST, and QUADAS to ensure the study doesn’t show bias, is scientifically valid, clinically applicable, and includes all information that is relevant for reproducibility, but also mention shortcomings and motivates design choices.

Eric Topol, who has been a vocal communicator of new findings during this pandemic, also remarks that the studies were performed in regions where there is a high prevalence of COVID-19 and promising results should be replicable in regions where the virus circulates less. He adds that without randomized clinical trials, reported outcomes of algorithms used in practice should be taken with a grain of salt. In September 2020 two guidelines that deal with clinical trials on interventions with an AI component were published. Both extend upon earlier guidelines. The first, SPIRIT-AI deals with how clinical trial protocols are designed, while the second, CONSORT-AI, deals with how to report results. Both are useful to keep in mind while designing and reporting clinical trials, but also when reviewing such reports.

Finally, as mentioned by Luengo-Oroz et al., the model should also be able to differentiate with respect to conditions that show similar findings in medical images.

The criticism of Roberts and others is mostly directed at research papers. In contrast, several companies have brought their products to the market and have gained clearance by regulating authorities. Alibaba’s healthcare division released such a tool, which is said to be used in multiple hospitals, mostly as a second opinion.

2.Epidemiological applications

To inform policy-makers, and the public in general, there is a need for models that predict the spread of infection through populations.
There is also a need to predict the effectiveness of interventions and how they influence the number of infections, how dangerous some places are for viral transmission, and how effective vaccines are.

Models are developed to predict some particular epidemiological aspects on a non-individual level.

Traditionally, some kind of susceptible-infectious-recovered ( SIR) mode l is used to model the dynamic processes that underlie the transfer from one state to another.
The application of machine learning to these problems seems promising as there are a variety of sources that allow accurate modeling of these dynamic patterns.
Social media, mobility, and search results are some examples of what may be included.

Earlier in the pandemic, data was practically not available.
The virus was new and the world was experiencing the first wave of the COVID-19 pandemic. There was uncertainty in the actual number of infections due to limited testing capacity. This posed a challenge for epidemiological applications.

An ensemble of models is used when government agencies and supporting bodies model epidemiological trends.
The ensemble combines a group of different models in a certain way to increase prediction accuracy. For example, the ensemble used by the American Center for Disease Control (CDC) can be found here.
The ensemble includes AI-based models from research institutions and also included a model by an independent data scientist named Youyang Gu.

Another example of AI being used by a governmental organization to gain epidemiological insight is discussed in Facebook AI’s blogpost.
Reports were periodically provided to the European Commission on the spread of infections and the effectiveness of interventions.

Another solution that is derived from epidemiological insights is predicting the required capacity of a health facility as the number of covid cases and hospitalizations increases. 

In the UK, the National Health Service (NHS) worked together with researchers to develop the COVID-19 Capacity Planning and Analysis System (CPAS), a model that does exactly this. It is now also used in practice in several regions of the UK.

3. Applications for biochemistry

A part of biochemistry covers the discovery of viral protein structures and viral features in general and makes efforts to find or design drugs and vaccines.
It is highly unlikely that AI will replace the whole drug/vaccine development pipeline, as clinical trials will always have to take place (or at least in the foreseeable future).

However, AI contributes to applications that facilitate some parts of the pipeline. Since pharmaceutical companies don’t openly share their development processes this part will only include relevant items that were publicly disclosed.

Let’s start with DeepMind’s announcement on the release of Alphafold 2 with its impressive results that have left many in awe.
The work, similar to other recent breakthroughs like GPT-3, shows the promise of scaling and attention-based networks and is set to deliver exciting new algorithms in the coming years.
In August 2020, the Alphafold team announced that their system had predicted the structure of viral proteins.
Despite the lack of validation, they hoped it could help in finding drug targets (later some of the viral protein structures that can be targeted were verified as being accurate).

Other research groups have also released models for drug target prediction, and even though their added value should not be overstated (SARS-CoV-2 belongs to the same family as early 2000’s SARS-CoV-1 and the industry benefits from insights gained during the epidemic) the benefits for future R&D are obvious.

AI already entered the pharmaceutical arena pre-COVID, as part of the computational stack of existing companies on the one hand.
But also in the shape of new players that are more outspoken on their AI efforts on the other hand. The latter has been attracting a lot of investors as mentioned here.

A first example is the UK’s BenevolentAI. 

As a relatively young company, they discovered a possible drug candidate in early February through the use of a knowledge graph built from scientific literature through ML-based algorithms. 

Later, the drug, together with other discovered candidates went through trials to assess efficacy. The National Institutes of Health in the US have allowed and, in specific cases, even recommends the drug for the treatment of the infection.
Further studies will have to further clarify any additional benefits the drug may have over existing therapies.

Canada-based Abcellera is another company that has leveraged its “full-stack AI-powered antibody discovery platform” with success.

Their platform scans antibodies produced by humans and further develops those that are deemed suitable. After successful trials, the first starting in June 2020, the FDA authorized their products for patients with mild symptoms at high risk of progressing to severe disease.

Another company that is more outspoken on its adoption of AI is San Diego’s Inovio Pharmaceuticals (more senior than the companies mentioned earlier) whose machine learning-based algorithm was able to predict a DNA vaccine candidate hours after receiving the viral sequence.

At the time of writing, the vaccine candidate was being tested in Phase II and plans were made to enter Phase III trials in the second quarter of 2021.

As mentioned earlier, pharmaceutical companies are not exactly transparent about their digital development stack.
At the time of writing this piece, Moderna has its vaccines being administered to the population. As an invited speaker at NeurIPS, Moderna’s co-founder, Noubar Afeyan, gave his general vision on the use of digital and AI in specific but also provided a sneak peek of the applied AI tools they employ and will deploy in the future.

Two applications that are in use at the time are discussed.
Both are related to quality assurance and effectively reduce the number of people needed to inspect the quality of the delivered products.

• In the first use case, he describes how they trained an algorithm to match the performance of humans in detecting mRNA strands with disfunct poly-A tails.
  The algorithm is most likely used to assess the mRNA-based vaccine against SARS-CoV-2 as well.
• Similar to the first use case, the second one aims to reduce manual labor. By using a segmentation algorithm on electron microscopy images, the assembly and downstream performance of the lipid nano-particles under different circumstances are evaluated.
  These particles can be viewed as the package that harbors the actual mRNA strands and will release its content in human cells. It is unclear if the method is routinely used at the time of writing.

These manual-labor intensive applications are low-hanging fruits and are probably automated in other pharmaceutical companies as well. Either way, it’s great seeing a company sharing them publicly. A white paper on Moderna’s digitization strategy mentions the use of AI techniques in the optimization of the mRNA’s genetic sequence and its analysis.

Concerning the use of generative AI in the development pipeline of the larger pharmaceutical companies, less is known.
At the end of his talk, Afeyan lays out some early and promising approaches down this route.

One example comes from Generate Biomedicines, a company that belongs to the same family as Moderna. They successfully used their platform to generate multiple possible therapeutics that have proven effective in practice.

4.Providing information

Information is a fourth category where AI has played a role.

• Both the act of providing qualitative and relevant information, 
• but also filtering out what can be viewed as false or misleading information is included in the term. 

The latter refers to ways of distinguishing between what can be viewed as content that is scientifically grounded and objective reporting on the one hand and items that are mispresented, baseless or factually inaccurate on the other hand.

It is clear that the forum of the first subcategory is rather broad, i.e. the retrieval of relevant and qualitative content can take place on a popular platform like YouTube for instance, but also in specialized portals like Semantic Scholar. While similar, the second subcategory focuses more on verifying and moderating content on platforms consulted by a large group of people where quality control typically doesn’t take place.

A practical example in the first subcategory is the creation of the COVID-19 Open Research Dataset ( CORD-19), which is a machine-readable dataset of published literature on COVID-19.
It was developed by the Allen institute at the request of the US government. By mining it, new insights may be found.

Another set of related applications that have been deployed at scale are chatbots. In China, they have been used to lighten the workload of doctors. But also on governmental websites or medical facilities over the world they were set in place.

Social platforms are the main players in the second subcategory. Preventing scientific inaccuracies, misleading news, or conspiracies to run rampant, while also providing a platform for free speech, was a challenge they faced.
Verifying authorized sources like governments and guiding users to their information were some of the rather straightforward actions taken in this conquest.
AI-specific actions that were implemented specifically for the pandemic are less visible.
Facebook AI provides some insight on their blog. A post in May reveals they still use human experts to flag items that should be removed. The AI helps in retrieving flagged content at scale (similar posts and in other languages for instance).

5.Safety assessment

A final category concerns the use of AI to promote safety precautions and control the degree to which safety measures are followed.
Concerns on principles of ethics and privacy are strongly present here as safety measures are very much dependent on how we act.
Consequently, applications that monitor our behavior have a direct effect on our privacy. Additionally, technologies being used to save lives in this pandemic may very well be used for malicious purposes.

Despite concerns, this category does have quite some implementations in practice. In this regard, it is a bit contrary to the clinical applications where there seems to be a relatively larger academic interest, but less practical realizations come to fruition. The use of established techniques that don’t need to be developed from scratch is a possible explanation.

The first set of examples is the use of computer vision to detect crowding in spaces. Practical examples were seen in public spaces like London. The industry has also seen its use, with Amazon using it to monitor whether or not social distancing was kept to for instance.

Similar to the previous examples, cameras have also been used to detect mask-wearing. The technique is claimed to be used by governmental agencies in India and China using CCTV equipment. More recently, it has also been sparsely deployed at some places in Europe, but not on a governmental scale. A first example is in the UK, where mask detection software has been sold to at least one British supermarket chain. In the city of Cannes, the software has been deployed in three markets.

A final example of safety-promoting AI algorithms can be found in Taiwan, where the government has used big data in conjunction with machine learning. Mobility data as well health data were used to identify risk populations and to personally inform individuals to quarantine. It has to be said that Taiwanese policies and by extension policies of other Asian countries have been very effective at driving the number of infections down. Their widespread surveillance programs show that success may be -to some part- bought with privacy.

Thoughts and conclusion

In this piece, an effort was made to show some examples of AI being used to aid in the fight against the pandemic. The premise was to only include items that were established in practice around early 2020.
Some of the discussed applications were developed specifically for this crisis. Others were adapted to it to be useful.

The common denominator is that they all lead to direct or indirect health benefits.
The scope was limited to those contributions that were deployed from the start of the pandemic onward and, to some extent, were designed specifically for dealing with a health crisis like this one.
The latter thus includes what already existed pre-2020 but was yet to show its full power.

The goal of this work was merely to provide some sense of what is there in the open rather than making some critical analysis on the state of the AI field and its community. People are free to interpret what was described to their liking.
They should take into account that what was listed here doesn’t cover the whole spectrum of AI contributions.
At the time of writing some contributions were in the pipeline, waiting to be deployed, and thus can still prove useful.

Based on what has been found in the search for contributions and supported by some reviews of people brighter than the author of this piece, e.g. 1, 2, 3, 4, 5, 6, 7, 8, 9, and 10, some of the main findings can be described.

A first finding was the earlier deployment of applications that are technically more straightforward and where a decrease in performance is not as problematic for health outcomes.
They can be viewed as tools in the sense that they can typically only provide a net gain (in terms of health outcomes) even if they are doing their job rather poorly. Surveillance of people wearing masks and maintaining crowding are some examples.
If the system works, people can adjust their behavior leading to a potential net gain in health outcome, if it doesn’t work no decrease in health outcome is noticed compared to the situation where no systems are in place.
However, this implies that people do not rely on these systems to know whether or not they are following safety protocols. Otherwise, this statement would not hold.

The provided examples show a larger public deployment in countries that are, relatively speaking, in a further stage of digitization and have higher levels of surveillance.

Some remarks are in order.

A first is an obvious potential to extend what was originally intended as measures to combat spread beyond the scope of a pandemic. Potentially, this has undesirable effects downstream which may be harder to revert once in place. Precautions should be taken to ensure infringements on privacy are restricted in time and requires independent courts of law to assess the measures taken.

Transparency is also important, know the data that is collected and which bodies are using it, and their goals. For instance, private medical registers which are shared under the guise of identifying patterns and identifying risk populations should not be used to disadvantage citizens later on.

The same goes for GPS locations, financial transactions, and other types of data that are being monitored.
There are obvious ways to counteract some of the dangers of these techniques, using decentralized apps and ensuring anonymity for instance. However, even with taking these elements into account, a debate on whether or not measures are proportional is important.

Note that while these rather political remarks focus on public interventions, systems to monitor have also been implemented at companies like Amazon. Even if measures are legal, a critical attitude is still in order.

As discussed, it is clear that diagnostics and prognostics were heavily covered by fundamental AI researchers.
I would say it received a much larger interest compared to other research directions. A plethora of papers was written on using X-ray radiography and tomography or other biomarkers and modalities.
Despite stellar ROCs and ingenious ensembles, some criticism is in order. Echoing other reviews, these studies tend to show major flaws of methodological and statistical nature or on matters of quality control.

Wynants et al., in their review, say that none of the studies they reviewed could be translated to practice.

Typical flaws are among others 

• a lack of transparency on which data was used and the code-base.
• Bias in datasets is also a major concern.
• Knowledge of the inclusion criteria, patient demographics, and health status are important to assess generalizability and fairness. Especially when considering only a relatively small geographical part of the world represents the bulk of public datasets.

By settling on well-defined standards and adhering to them, the conversion of future efforts towards the practical realm may increase.

Next to the aforementioned flaws, AI researchers should be aware of what end-users desire and the specifics of their field of expertise. In other words: collaboration is key. On matters of diagnostics and prognostics the goal is to have medical practitioners use the solutions presented by AI researchers, also referred to as human-in-the-loop. In reality, academia does not always leave their “bubble” to have this discussion, which in turn makes the solutions they come up with just not relevant for use in practice.
A simple example is the use of computer vision algorithms that operate as a black box, while medical professionals require some degree of explainability when using these systems as supporting tools (which is typically the case).

Social media companies have also played a role in the information providing problem. It is perhaps remarkable to see them still using a human in the loop for fake news detection.
It seems like there is not a high enough level of trust in their systems to autonomously detect and remove misleading information yet, or at least to my knowledge.
However, the algorithms do prove their worth in retrieving flagged posts after manual analysis. It is worth mentioning that the big companies have invested a lot of money and research into tackling this pandemic, also beyond the information topic.

Finally, what concerns providing and sharing insight into trends and patterns, through epidemiological models for instance, some real-world, practical use-cases were seen. AI-based epidemiological models were also of great interest in the research community, and have also been used in practice to guide policy-makers. One could argue that it has been more successful compared to applications related to diagnostics and prognostics.

To finish this piece, it should be stressed again that not all applications are discussed. The piece provides a snapshot of what was around in January 2021. This implies that at the time of publishing some more theoretical work and experimental research will have been implemented in practice and are not discussed here. A lot of it is still in the works or at the brink of being ready.

In general, I believe the pandemic has strongly benefited from all the work that has been done on AI and, also important, a future pandemic shall strongly benefit from AI and will do so even more widely and more extensively. However, let’s hope SARS viruses and other pathogens let us live normally for a long time first.

About the author

Maxime Nauwynck
Biomedical engineer and AI enthusiast. Former PhD student at the University of Antwerp, Belgium.

Maxime Nauwynck is a biomedical engineer, AI enthusiast, and Former PhD student at the University of Antwerp, Belgium. After receiving his Master of Science in Biomedical Engineering degree at the University of Ghent,
Maxime started his PhD at the Imec-Vision Lab, a research lab of the Department of Physics of the University of Antwerp, in October 2019. 

His research has focused on the application of deep learning to the fields of X-ray tomography and magnetic resonance imaging.

Originally published at https://thegradient.pub on May 24, 2021.

Citation
For attribution in academic contexts or books, please cite this work as

Maxime Nauwynck, “How has AI contributed to dealing with the COVID-19 pandemic?”, The Gradient, 2021.