It may have played a disproportionate role in creating the perception that a catastrophic second wave is unlikely

With nearly 400,000 cases added each day, various scientists are wondering if a government-backed model called SUTRA for predicting the rise and fall of the coronavirus epidemic may have played a disproportionate role in creating the perception that ‘a catastrophic second wave of the pandemic was unlikely in India.

An official linked to the COVID-19 management exercise said, on condition of anonymity, that the SUTRA model entry was “important, but not unique or defining.”

The SUTRA group presented its views to Dr VK Paul, who chaired a committee that received contributions from several modellers and sources. “The worst predictions in this set were used by the National Empowered Group on Vaccines and the groups led by Dr. Paul to take action. However, the surge was many times greater than any modeler had predicted. “

On May 2, the SUTRA group released a statement, carried by the Press Information Office, that the government had solicited their contributions, claiming that a “second wave” would peak in the third week of April and would remain. at about 1 lakh of cases. “It is clear that the model’s predictions in this case were incorrect,” they noted.

Past its peak

SUTRA (Sensitive, Undetected, Tested (Positive) and Suppressed) approached public attention for the first time when one of its expert members announced in October that India was “past its prime”. After new cases hit 97,000 per day in September, there was a steady decline and one of the scientists associated with the development of the model, Ms. Vidyasagar, told a press conference that the model showed the COVID charge is expected to be capped at 10.6 million. symptomatic infections in early 2021, with fewer than 50,000 active cases as of December. At this time in October, there were 7.4 million confirmed cases, of which about 780,000 were active infections.

Computer biologist, Mukund Thattai, of the National Center for Biological Sciences, Bengaluru, summarized in a Twitter thread of cases where the SUTRA predictions were well outside the limits of actual workload. “The so-called Covid ‘dummy’ commissioned by the Indian government is fundamentally flawed,” he tweeted. “Based on Professor Agrawal’s own messages, it was quite clear that the predictions of the SUTRA model were too variable to guide government policy. Many models have been wrong, but the question is why the government has continued to rely on this model, rather than consulting with epidemiologists and public health experts, ”he said. The Hindu.

In an email to The HinduMr. Agrawal admitted that the model, which had multiple purposes, did not perform well on a “prediction of the future under different scenarios” metric.

He said that unlike many epidemiological models which extrapolated cases based on the number of existing cases, the behavior of the virus and the mode of spread, the SUTRA model chose a “data-centric approach”. The equation which gave estimates of the number of future infections and the probability of a peak occurring required certain “constants”. These numbers kept changing and their values ​​were based on the number of infections reported at various intervals. However, the equation couldn’t tell when a constant changed. A rapid acceleration of cases could not be foreseen in advance.

Too many parameters

Rahul Siddharthan, a computational biologist at the Institute of Mathematical Sciences, said in an email that no model, without external input from real-world data, could have predicted the second wave. However, the SUTRA model was problematic because it relied on too many parameters and recalibrated those parameters every time its predictions “broke”. “The more parameters you have, the more you run the risk of ‘overfitting’. You can fit any curve over a short window of time with 3 or 4 parameters. If you keep resetting these settings, you can literally adjust anything. “

Explaining how the model works, which seems to work best when the number of cases reported each day doesn’t fluctuate too much, the main reasons it doesn’t measure an impending exponential increase was, according to Mr Agrawal, that a number, called beta indicating that contact between people and populations has gone awry. “We assumed he could at best achieve the pre-lock value. However, it has gone way above that due to new strains of the virus. “

In addition, the model was not “calibrated” correctly. The model was based on a serological survey conducted by the ICMR in May that found 0.73% of the Indian population may have been infected by that time. “I have good reason to believe now that the results of the first survey were not correct (the actual infected population was much lower than that reported). This calibration led our model to conclude that more than 50% of the population was immune in January. In addition, it is also possible that a good percentage of the immune population has lost their immunity over time. “

Unlike epidemiologists who needed “precise data”, in the SUTRA approach, the factor by which reported cases differ from actual cases is a model parameter that could be estimated from recently reported data (covid19india.org), according to Mr. Agrawal. “I understand it might sound a bit mysterious, but the math shows how. This is actually one of our central contributions, ”he said. The Hindu. This was described in a preprinted research paper which has been available online since January.

The modeling study called ‘COVID-19 India National Supermodel’ is the result of an analysis by an expert committee made up of mathematicians and epidemiologists – although in a research paper explaining how the model works, there is has three authors: Manindra Agrawal, professor of computer science at the Indian Institute of Technology, Kanpur; Mr. Vidyasagar, Senior Professor of Electrical Engineering at the Indian Institute of Technology in Hyderabad and Madhuri Kanitkar, Pediatric Nephrologist and Deputy Chief of the Integrated Defense (Medical) Staff in the Army.

While many groups of epidemiologists, disease experts and groups of mathematicians had developed several types of models to predict the outcome of the pandemic, this group was facilitated by the Department of Science and Technology and was the only one of several forecasting groups, whose numbers were relayed through government advertising channels.

Until February, the model seemed more or less correct, the curve was decreasing and in mid-February when 10,000 to 12,000 new cases were added daily, the overall figures were around 10 million.

Overall workload

In an interview with the newspaper published on February 27, Mr Agrawal said a “second wave was unlikely” even though a slight recovery – to around 15,000 cases per day – had started. India’s overall workload would not exceed mid-March and only 300,000-500,000 new confirmed infections over the next 10 weeks were expected, bringing the overall load to 11.3 or 11.5 million infections by April 2021. This was partly based on 60% of the population who have been exposed to the virus.

On April 2, as India was in the grip of the second wave, he told the PTI that the new cases would “ peak ” on April 15 and 20 – according to the public statement from the SUTRA team.

On April 23, it again reported a new peak, from May 11 to 15, with 3.3 to 3.5 million “active” cases in total and a sharp drop by the end of May. India currently has around 3.4 million active cases.

Gautam Menon, a modeler and professor, Ashoka University, Sonepat, Haryana, who also worked on estimating the spread of COVID-19 disagreed with Mr Agrawal’s approach, on the grounds that it was “something not very simplistic and insufficiently informed by epidemiological data and expertise. “.

At best, Mr. Agrawal’s model could be used with a “package” – where the results of various scenarios have been aggregated. “The use of machine learning to predict the spread of the epidemic is a relatively recent advance. Some of these models work very well. But the problem with these methods is that you can’t really understand what they do and how sensitive they are to just plain bad data. I would use these models, if we had them, along with a bunch of other models, but I wouldn’t totally trust them.

The SUTRA mode’s elimination of the importance of the actual behavior of the virus: that some people were bigger transmitters of the virus than others (say a hairdresser or receptionist more than someone who worked from home), his lack of consideration of social or geographic heterogeneity and failure to stratify the population by age because it does not take into account contacts between different age groups have also undermined its validity.

New variants

Mr Agrawal – who now regularly tweets about the evolution of the pandemic in states and districts – responded that new variants appeared in the SUTRA model as an increase in the value of the parameter called ‘beta’ (this contact rate valued). “As far as the model is concerned, he observes changes in the values ​​of the parameters. He doesn’t care about the reason for the change. And the calculation of the new beta value is enough for the model to correctly predict the new trajectory. “

He admitted that a combination of good epidemiologists, data-driven models like SUTRA, and time-series models worked best. “Time series-based predictions are effective in detecting changes in data patterns. Thus, they can signal phase changes from the start. SUTRA-type data-centric models can explain the past very well [and in studying what was the effect of policy actions, leading to better knowledge-base for future]. They are also very good at predicting the future trajectory assuming the phase does not change. “

In 2002, Mr. Agrawal and two of his students developed a mathematical test called AKS primality that could effectively determine whether you could say that a large number was prime which earned them world accolades. He used a computer approach to solve a pure math problem. “This is the second time that I have entered a field as a complete foreigner. First of all, I proved the primality theorem. Mathematicians around the world have welcomed a computer scientist into their fold and actually went out of their way to celebrate it. Our article was not written in a standard mathematical style, however, the experts were quick to put an end to anyone who questioned the presentation or minor errors in the article. On the other hand, I feel a hostile reaction from epidemiologists, at least in India. “