Balancing the Evidence: Animal efficacy studies should have more weight in the risk/benefit calculus ahead of clinical trials

Clinicians, sponsors, ethics review committees, and others are charged with ensuring that risk is in a favourable balance with benefit when patients enrol in trials. Yet how do they make this judgment, when the only evidence available is from preclinical animal studies?

In our recent article published in the Journal of Medical Ethics¹, we offer an answer to this question. We argue, first, that review committees and clinicians should evaluate the clinical promise of a new intervention based on preclinical efficacy evidence. Specifically, they should form judgments about the clinical promise of a new intervention based on how well preclinical studies address common threats to clinical generalization. Second, review committees should adjust their estimates of clinical promise based on clinical trials with related drugs (e.g. within the same class of agents). All else being equal, the stronger the clinical promise, the stronger the moral justification for embarking on an early-phase clinical trial.

Before consuming scarce financial and human resources, including exposing volunteers and patients to potentially ineffective treatments in trials, researchers should capitalize on the knowledge gained from animal efficacy studies. Ethics review committees and clinical investigators will likely protest they lack the expertise, skill or time to evaluate preclinical evidence. Or, that drug regulators like FDA already make such judgments.

However, the FDA itself states that “lack of… potential effectiveness information should not generally be a reason for a Phase 1 IND to be placed on clinical hold²,” and it explicitly delegates judgments about risk/benefit to IRBs³. This means that, if ethics reviewers and investigators really do lack the expertise, skill or time to review preclinical evidence- they are surrendering their mandate to protect patients from undue risk in early phase trials.

1. Kimmelman, J. and Henderson, V. Assessing risk/benefit for trials using preclinical evidence: a proposal. J Med Ethics Published Online First: 13 October 2015 doi:10.1136/medethics-2015-102882 
2. Guidance for Industry: Content and Format of Investigational New Drug Applications (INDs) for Phase 1 Studies of Drugs, Including Well-Characterized, Therapeutic, Biotechnology-derived Products. In: Services USDoH, ed., 1995. 
3. 46 FR 8975, Jan. 27, 1981, as amended at 56 FR 28029, June 18, 1991; 66 FR 20599, Apr. 24, 2001.

2015 November 5 | Leave a comment

@Manual{stream2015-904,
    title = {Balancing the Evidence: Animal efficacy studies should have more weight in the risk/benefit calculus ahead of clinical trials},
    journal = {STREAM research},
    author = {Valerie Henderson},
    address = {Montreal, Canada},
    date = 2015,
    month = nov,
    day = 5,
    url = {https://www.translationalethics.com/2015/11/05/balancing-the-evidence-animal-efficacy-studies-should-have-more-weight-in-the-riskbenefit-calculus-ahead-of-clinical-trials/}
}

Too much of a good thing

https://commons.wikimedia.org/wiki/File%3ANude_mouse.jpg

A novel anti-cancer drug is found to shrink every tumour type tested in experimental animal models. Let’s rejoice and start clinical trials without delay!

Well, not so fast.

Preclinical experiments in animal models are aimed at showing that a new drug will be useful in human beings. However, individual animal experiments are often too small to support inferences about clinical utility. Techniques such as meta-analysis offer an attractive method for pooling individual studies and supporting more confident assertions regarding the potential clinical utility of a new drug.

With this in mind, our team undertook a meta-analysis using the anti-cancer drug sunitinib (Sutent). Since its publication in eLife, the meta-analysis has been covered in the BMJ, Nature, The Guardian, and Retraction Watch. We were primarily interested in whether the properties of sunitinib observed in preclinical studies would correlate with those observed in patients

What we found were many avoidable roadblocks to proper meta-analysis. Firstly, poor reporting quality in many experiments made it impossible to include them in the analysis. Examples include lack of error bars or no sample size given. Secondly, poor methodological practices such as lack of blinding and randomization, or reliance on single models were common, calling the quality of the data we were extracting into question. Furthermore, there was no discernible dose-response relationship connected across experiments in the same malignancy. Finally, trim and fill analysis (a tool to detect the possibility of publication bias) suggested that the overall efficacy of sunitinib across all malignancies could be inflated as much as 45%.

Other investigators, including those just published in PLoS Biology, have shown similar results in other diseases and have called for reforms to the way we perform and report animal experiments meant to influence the decision to move into clinical trials. It may be time that we looked more closely at the way we test new anti-cancer agents in animal models to ensure we are getting the highest quality data to make decisions.

Citations:

• Henderson, V. C. et al. eLife 4, e08351 (2015). http://elifesciences.org/content/4/e08351
• Macleod, M. R. et al. PLoS Biol. http://dx.doi.org/10.1371/journal.pbio.1002273 (2015).

2015 October 16 | Leave a comment

@Manual{stream2015-889,
    title = {Too much of a good thing},
    journal = {STREAM research},
    author = {Valerie Henderson},
    address = {Montreal, Canada},
    date = 2015,
    month = oct,
    day = 16,
    url = {https://www.translationalethics.com/2015/10/16/too-much-of-a-good-thing/}
}

Uncaging Validity in Preclinical Research

High attrition rates in drug development bedevil drug developers, ethicists, health care professionals, and patients alike.  Increasingly, many commentators are suggesting the attrition problem partly relates to prevalent methodological flaws in the conduct and reporting of preclinical studies.

Preclinical efficacy studies involve administering a putative drug to animals (usually mice or rats) that model the disease experienced by humans.  The outcome sought in these laboratory experiments is efficacy, making them analogous to Phase 2 or 3 clinical trials.

However, that’s where the similarities end.  Unlike trials, preclinical efficacy studies employ a limited repertoire of methodological practices aimed at reducing threats to clinical generalization.  These quality-control measures, including randomization, blinding and the performance of a power calculation, are standard in the clinical realm.

This mismatch in scientific rigor hasn’t gone unnoticed, and numerous commentators have urged better design and reporting of preclinical studies.   With this in mind, the STREAM research group sought to systematize current initiatives aimed at improving the conduct of preclinical studies.  The results of this effort are reported in the July issue of PLoS Medicine.

In brief, we identified 26 guideline documents, extracted their recommendations, and classified each according to the particular validity type – internal, construct, or external – that the recommendation was aimed at addressing.   We also identified practices that were most commonly recommended, and used these to create a STREAM checklist for designing and reviewing preclinical studies.

We found that guidelines mainly focused on practices aimed at shoring up internal validity and, to a lesser extent, construct validity.  Relatively few guidelines addressed threats to external validity.  Additionally, we noted a preponderance of guidance on preclinical neurological and cerebrovascular research; oddly, none addressed cancer drug development, an area with perhaps the highest rate of attrition.

So what’s next?  We believe the consensus recommendations identified in our review provide a starting point for developing preclinical guidelines in realms like cancer drug development.  We also think our paper identifies some gaps in the guidance literature – for example, a relative paucity of guidelines on the conduct of preclinical systematic reviews.  Finally, we suggest our checklist may be helpful for investigators, IRB members, and funding bodies charged with designing, executing, and evaluating preclinical evidence.

Commentaries and lay accounts of our findings can be found in PLoS Medicine, CBC News, McGill Newsroom and Genetic Engineering & Biotechnology News.

2013 August 5 | Leave a comment | Tags: cancer efficacy preclinical studies translational research validity

@Manual{stream2013-300,
    title = {Uncaging Validity in Preclinical Research},
    journal = {STREAM research},
    author = {Valerie Henderson},
    address = {Montreal, Canada},
    date = 2013,
    month = aug,
    day = 5,
    url = {https://www.translationalethics.com/2013/08/05/uncaging-validity-in-preclinical-research/}
}