{"id":1335,"date":"2023-01-11T16:23:52","date_gmt":"2023-01-12T00:23:52","guid":{"rendered":"https:\/\/depts.washington.edu\/deworm3\/?p=1335"},"modified":"2024-02-09T16:28:06","modified_gmt":"2024-02-10T00:28:06","slug":"uninformative-research-is-the-global-health-crisis-youve-never-heard-of-2","status":"publish","type":"post","link":"https:\/\/depts.washington.edu\/deworm3\/2023\/01\/11\/uninformative-research-is-the-global-health-crisis-youve-never-heard-of-2\/","title":{"rendered":"\u201cUninformative research\u201d is the global health crisis you\u2019ve never heard of"},"content":{"rendered":"

See the original article by the Gates Foundation here<\/a><\/p>\n

\n
\n
\n
\n
\n
Historians like to fight about whether Francis Bacon or Galileo Galilei invented the scientific method. Either way, we\u2019ve been relying on it for going on 500 years. Here\u2019s how it works: You 1) observe the world around you, 2) make a hypothesis based on your observations, 3) run an experiment to test your hypothesis, 4) get a result, and 5) repeat the process until you can explain what you observed in step 1.The global health community relies on the scientific method, in the form of clinical trials, to continually improve health care. As of 2022, at least 329,830 trials were in progress, seeking to learn whether a new drug, vaccine, device, procedure, or behavior is better than the alternatives.<\/p>\n

More than a million researchers are devoting their time and energy to these studies. Many millions of people are participating as subjects. All in the pursuit of knowledge that, we hope, will save lives.<\/p>\n

\n
<\/section>\n<\/section>\n

But there\u2019s a major problem. Most clinical trials neither prove nor disprove the hypothesis. It\u2019s not that the intervention being tested failed. It\u2019s that we don\u2019t know whether it failed or succeeded. These studies are what some scholars call \u201cuninformative.\u201d In lay terms, it\u2019s almost like they never happened. Except that they collectively required many billions of dollars and millions of hours, two resources the global health sector doesn\u2019t have nearly enough of.<\/p>\n

According to a 2021 study of thousands of COVID-19 drug trials, only 5% of the “study arms\u201d included enough people to generate a statistically valid result. Chaos is to be expected during a crisis like a pandemic, but the norm during \u201cpeacetime\u201d is nearly as worrisome. A recent study of U.S. clinical trials related to heart disease, diabetes, and lung cancer found that only 26% ended informatively. To put it bluntly: Three out of four were a waste of time and money.<\/p>\n

\n
\n
\n
\n

To put it bluntly: Three out of four [U.S. clinical trials] were a waste of time and money.<\/p>\n<\/blockquote>\n<\/figure>\n<\/section>\n<\/section>\n

How is it possible that the vast majority of these trials don\u2019t answer the question they set out to investigate? One reason that some trials yield no useful findings is that the people who run them, while experts in their fields, aren\u2019t necessarily experts at designing trials, analyzing the results, and communicating the findings.<\/p>\n

The Gates Foundation has launched a program called\u00a0Design, Analyze, Communicate<\/a>\u00a0(DAC) to understand this challenge and begin to address it. But before we delve into DAC and its work, one piece of context is critical: Clinical trials can be exceedingly complex.<\/p>\n

Consider an ongoing clinical trial called\u00a0DeWorm3<\/a> that our foundation supports with grant funding. It asks a simple question: Is it possible to eliminate the transmission of parasitic worms (called soil-transmitted helminths)? The question may be simple, but answering it in an informative way is anything but, so DeWorm3 was designed to run a gauntlet of potential challenges.<\/p>\n<\/section>\n<\/div>\n<\/section>\n<\/div>\n<\/section>\n<\/section>\n

\n
\n
\n
\n
\n
\n

The scientists working to protect kids from parasites<\/h2>\n

Dr. Sitara Swarna Rao Ajjampur has built her career by subverting other people\u2019s expectations.<\/p>\n

\u201cIn India, when you get into medical school,\u201d she said, \u201cIt\u2019s almost impossible to tell someone that you want to do research. You\u2019re expected to be happy just being a doctor.\u201d But Dr. Ajjampur wasn\u2019t happy, because she was \u201cbitten by the research bug\u201d during a summer fellowship as a medical student.<\/p>\n

When she chose research following her M.D., \u201cvirology was considered the cool science.\u201d But, says Dr. Ajjampur, \u201cI was fascinated by parasites.\u201d After her Ph.D. and research fellowship, Dr. Ajjampur, now at Christian Medical College (CMC) in Vellore, India, became the principal investigator of the Indian site of the DeWorm3 trial.<\/p>\n

The origins of DeWorm3 lie in public health policies to prevent infection by intestinal parasites called soil-transmitted helminths. In India and dozens of other countries, school-aged children are given the drug albendazole twice a year. This relatively recent effort has successfully helped communities get control of hookworms, roundworms, and other worms.<\/p>\n

This is a public health breakthrough. Before school-based deworming started, hundreds of millions of children were infected by worms that stunted their neurological and physiological development. When they received their first doses of albendazole, donated by the pharmaceutical company GlaxoSmithKline, many children shed worms not only in their stool but also through their mouths and noses.<\/p>\n

The days of widespread infection by intestinal parasites are over, but the worms haven\u2019t disappeared entirely. Many countries, including India, have to keep administering albendazole year after year after year to keep them at bay. It\u2019s expensive, it\u2019s labor intensive, and it can be a logistical nightmare.<\/p>\n

So experts in the field of neglected tropical diseases started to ask, What if it were possible to eliminate worms altogether and stop administering the drugs? This would give back two lost school days each year, redirect the work of millions of frontline health workers, repurpose manufacturing capacity and supply chains for other lifesaving medicines, and otherwise free up limited resources. Through mathematical modeling, researchers developed a promising hypothesis: elimination could be accomplished by giving albendazole not just to children at school but also to all adults in the community.<\/p>\n

DeWorm3, led by Dr. Judd Walson and his team at the University of Washington (UW), is now testing that hypothesis\u00a0at three study sites<\/a>: Benin (led by Dr. Moudachirou Ibikounl\u00e9 and Dr. Adrian Luty), Malawi (led by Dr. Khumbo Kalua and Dr. Robin Bailey), and India (led by Dr. Sitara Swarna Rao Ajjampur).<\/p>\n<\/section>\n<\/div>\n<\/section>\n<\/div>\n<\/section>\n<\/section>\n

\n
\n
\n
\n
\n
\n

How to design a rigorous clinical trial<\/h2>\n

In India, the DeWorm3 team is trying to show a statistically significant change from \u201cvery low\u201d incidence of parasites to \u201celiminated.\u201d A substantial sample size\u2014the number of people included in the study\u2014increases the odds that they\u2019ll obtain clear results.<\/p>\n

Inadequate sample size is the leading reason that so many studies end uninformatively. Estimating the required sample sizes is not straightforward. Furthermore, recruiting study participants costs a lot of money and takes a long time. As a result of these two factors, many researchers underestimate\u2014and end up with \u201cunderpowered\u201d and therefore uninformative studies.<\/p>\n

The 140,000 participants in Dr. Ajjampur\u2019s part of the study are spread across 477 square kilometers in the state of Tamil Nadu and are divided into 20 clusters. Before her team could give a single person a deworming tablet, they had to take a detailed census of the whole area\u2014that is, precisely who lives there and precisely where they live.<\/p>\n

Once they enumerated the people they intended to treat, they needed to make sure those people would want to be treated. Although adults in Tamil Nadu are used to children getting dewormed at school, the idea of getting dewormed themselves is new. No one will just swallow a tablet if someone shows up at their door and says it\u2019s for the good of science\u2014and they shouldn\u2019t.<\/p>\n

So, again, before Dr. Ajjampur\u2019s team could give a single person a deworming tablet, they had to conduct a vast awareness-raising and education campaign, in all local languages. They even wrote their own jingle and played it from auto rickshaws driving through the villages. \u201cWe were very prepared to answer any questions that they had,\u201d says Dr. Ajjampur. \u201cIt wasn\u2019t just a banner and giving them some information and then leaving. It was very, very interactive.\u201d<\/p>\n

Finally, they launched the intervention: They gave albendazole to the trial participants\u2014children in schools, adults in homes\u2014twice a year for three years. After each round, the team surveyed the community.<\/p>\n

Running any one of the three sites in the study is the rough equivalent of running a mid-size company. Dr. Ajjampur\u2019s team consists of 75 to 125 field workers, plus about 25 coordinators, managers, supervisors, and laboratory staff.<\/p>\n

\n
<\/section>\n<\/section>\n

Working in close coordination with the Indian government (the Ministry of Health and Family Welfare and the Indian Council of Medical Research, New Delhi) and the state of Tamil Nadu (the Directorate of Public Health and Preventive Medicine), the India team completed all the deworming rounds\u2014even finding ways to operate safely through the COVID-19 pandemic, which threatened to upend their plans.<\/p>\n

After three years of deworming and two years of waiting, the next step is to find out whether any worms remain in the community. That means testing 40,000 stool samples for soil-transmitted helminths. And therein lies another challenge.<\/p>\n

When a sample is full of worms, a lab technician can just slide it under a microscope and start counting. But the whole point of DeWorm3 is getting to a very tiny number of worms. Microscopy isn\u2019t sensitive enough for that work. A PCR test is. But, just as there was no PCR test for COVID-19 when the pandemic began, there was no automated, high-throughput PCR test for detecting intestinal parasites when the trial began.<\/p>\n

\n
<\/section>\n<\/section>\n

So the researchers at UW and Dr. Ajjampur\u2019s team at CMC worked at a furious pace with partners at Smith College and the biotech company Quantigen to develop a high-throughput molecular diagnostic test in time to test their large number of samples. Those samples are being tested now, with the results expected late in 2023.<\/p>\n<\/section>\n

\n

The clinical trial advisory service<\/h2>\n

Since its founding in 2000, the Gates Foundation has funded grantees to support thousands of clinical trials, many of which have hit some kind of obstacle and ended uninformatively.<\/p>\n

In Malawi, for example, the foundation funded grantees running a trial to discover whether eating one egg a day would improve the health of malnourished children. (A previous trial in Ecuador had shown that it would.) However, none of the people associated with the trial adequately considered the ramifications of doing the study next to a lake. The children there ate a steady diet of fish, so they already consumed adequate protein. These were not the specific children who should have been studied.<\/p>\n

To help grantees ensure that their research studies are as informative as possible, the foundation\u2019s\u00a0Integrated Development Team<\/a>\u00a0created the Design, Analyze, and Communicate (DAC) program in 2020 specifically for that purpose. DAC advisors are what most principal investigators and their teams are not: trained and experienced in optimizing different aspects of clinical trials for informativeness.<\/p>\n

\n
\n
\n
\n
Design<\/div>\n
\n

Ask the right question(s) using accepted endpoints and simulation tools to explore the performance of different designs.<\/p>\n<\/div>\n<\/div>\n<\/div>\n

\n
\n
Analyze<\/div>\n
\n

Plan statistical and interim analyses, incorporate decision rules, and apply model-informed drug development.<\/p>\n<\/div>\n<\/div>\n<\/div>\n

\n
\n
Communicate<\/div>\n
\n

Engage and communicate with key stakeholders before, during, and after the study.<\/p>\n<\/div>\n<\/div>\n<\/div>\n<\/section>\n<\/section>\n

Grantees that choose to work with DAC submit a draft of their study protocol\u2014the roadmap that researchers produce before they embark on the trial\u2014to the DAC team for feedback. They receive a list of recommendations based on 16 best practices for informativeness, and they have an opportunity to adjust the protocol before finalizing it.<\/p>\n

The recommendations can be highly technical. Not everyone is up to date on the Kato-Katz testing methodology, for example, and some grantees don\u2019t have the cutting-edge expertise or tools to implement DAC\u2019s ideas. In those cases, DAC can make grants to help the researchers get technical assistance to fill in the gaps, run the trial according to best practices, and be better prepared for future studies.<\/p>\n

Back in 2020, when DAC was just getting started, one of its earliest collaborations was with the DeWorm3 team. The DeWorm3 trial was already past its midpoint, so the University of Washington team was not looking to implement radical changes. But because the trial is big and expensive, everyone agreed that it was worth having the DAC team take a look to see if any adjustments could make the results even more useful.<\/p>\n

\u201cIt can be a difficult process to subject your work to a critical review,\u201d says Dr. Walson. \u201cBut we all have the same goal, and together we can drive maximum impact.\u201d<\/p>\n

\n
<\/section>\n<\/section>\n

The DAC team concluded that the DeWorm3 trial was well designed, but they pinpointed a couple of areas where relatively small changes could make a big difference.<\/p>\n

First, because of the complexities described earlier, the trial was extremely expensive, and the DAC team suggested saving money by adjusting the strategy for analyzing the tens of thousands of samples collected during the study. The UW team decided to test the endline samples (after two years) but not the midline samples (after one year). The midline results are not needed to determine whether transmission has been interrupted, and testing the midline samples would cost US$3 million. The researchers have the option to test the midline samples later, however, depending on the results of the study.<\/p>\n

Second, the DAC team suggested conducting additional research to inform the implementation of deworming programs. The ultimate goal of the study is not to answer an academic question, but rather to inform policy and practice. What policymakers want to know is not just whether elimination is theoretically possible, but what the real-life considerations are, such as how much it would cost to implement community-wide deworming or how to keep deworming coverage high when regular health workers, and not a dedicated team, are doing deworming as part of their job. Typically, answering these questions requires an entirely new trial, which means that it can take 20 years or more to go from someone\u2019s idea to actually saving lives.<\/p>\n

\n
\n
\n
\n

\u201cBut we all have the same goal, and together we can drive maximum impact.\u201d<\/blockquote>

\n
Dr. Judd Walson<\/div>\n<\/figcaption><\/figure>\n