The Landscape of Early Phase Research

As Jonathan is fond of saying: Drugs are poisons. It is only through an arduous process of testing and refinement that a drug is eventually transformed into a therapy. Much of this transformative work falls to the early phases of clinical testing. In early phase studies, researchers are looking to identify the optimal values for the various parameters that make up a medical intervention. These parameters are things like dose, schedule, mode of administration, co-interventions, and so on. Once these have been locked down, the “intervention ensemble” (as we call it) is ready for the second phase of testing, where its clinical utility is either confirmed or disconfirmed in randomized controlled trials.

In our piece from this latest issue of the Kennedy Institute of Ethics Journal, Jonathan and I present a novel conceptual tool for thinking about the early phases of drug testing. As suggested in the image above, we represent this process as an exploration of a 3-dimensional “ensemble space.” Each x-y point on the landscape corresponds to some combination of parameters–a particular dose and delivery site, say. The z-axis is then the risk/benefit profile of that combination. This model allows us to re-frame the goal of early phase testing as an exploration of the intervention landscape–a systematic search through the space of possible parameters, looking for peaks that have promise of clinical utility.

We then go on to show how the concept of ensemble space can also be used to analyze the comparative advantages of alternative research strategies. For example, given that the landscape is initially unknown, where should researchers begin their search? Should they jump out into the deep end, to so speak, in the hopes of hitting the peak on the first try? Or should they proceed more cautiously–methodologically working their way out from the least-risky regions, mapping the overall landscape as they go?

I won’t give away the ending here, because you should go read the article! Although readers familiar with Jonathan’s and my work can probably infer which of those options we would support. (Hint: Early phase research must be justified on the basis of knowledge-value, not direct patient-subject benefit.)

UPDATE: I’m very happy to report that this paper has been selected as the editor’s pick for the KIEJ this quarter!

2014 July 4 | Leave a comment | Tags: bioethics cell therapy first in human trials methodology translational research

@Manual{stream2014-567,
    title = {The Landscape of Early Phase Research},
    journal = {STREAM research},
    author = {Spencer Phillips Hey},
    address = {Montreal, Canada},
    date = 2014,
    month = jul,
    day = 4,
    url = {http://www.translationalethics.com/2014/07/04/the-landscape-of-early-phase-research/}
}

The Literature Isn’t Just Biased, It’s Also Late to the Party

Animal studies of drug efficacy are an important resource for designing and performing clinical trials. They provide evidence of a drug’s potential clinical utility, inform the design of trials, and establish the ethical basis for testing drugs in human. Several recent studies suggest that many preclinical investigations are withheld from publication. Such nonreporting likely reflects that private drug developers have little incentive to publish preclinical studies. However, it potentially deprives stakeholders of complete evidence for making risk/benefit judgments and frustrates the search for explanations when drugs fail to recapitulate the promise shown in animals.

In a future issue of The British Journal of Pharmacology, my co-authors and I investigate how much preclinical evidence is actually available in the published literature, and when it makes an appearance, if at all.

Although we identified a large number of preclinical studies, the vast majority was reported only after publication of the first trial. In fact, for 17% of the drugs in our sample, no efficacy studies were published before the first trial report. And when a similar analysis was performed looking at preclinical studies and clinical trials matched by disease area, the numbers were more dismal. For more than a third of indications tested in trials, we were unable to identify any published efficacy studies in models of the same indication.

There are two possible explanations for this observation, both of which have troubling implications. Research teams might not be performing efficacy studies until after trials are initiated and/or published. Though this would seem surprising and inconsistent with ethics policies, FDA regulations do not emphasize the review of animal efficacy data when approving the conduct of phase 1 trials. Another explanation is that drug developers precede trials with animal studies, but withhold them or publish them only after trials are complete. This interpretation also raises concerns, as delay of publication circumvents mechanisms—like peer review and replication—that promote systematic and valid risk/benefit assessment for trials.

The take home message is this: animal efficacy studies supporting specific trials are often published long after the trial itself is published, if at all. This represents a threat to human protections, animal ethics, and scientific integrity. We suggest that animal care committees, ethics review boards, and biomedical journals should take measures to correct these practices, such as requiring the prospective registration of preclinical studies or by creating publication incentives that are meaningful for private drug developers.

2014 June 30 | Leave a comment | Tags: bioethics efficacy preclinical studies publication bias translational research

@Manual{stream2014-542,
    title = {The Literature Isn’t Just Biased, It’s Also Late to the Party},
    journal = {STREAM research},
    author = {Carole Federico},
    address = {Montreal, Canada},
    date = 2014,
    month = jun,
    day = 30,
    url = {http://www.translationalethics.com/2014/06/30/the-literature-isnt-just-biased-its-also-late-to-the-party/}
}

Uncaging Validity in Preclinical Research

High attrition rates in drug development bedevil drug developers, ethicists, health care professionals, and patients alike.  Increasingly, many commentators are suggesting the attrition problem partly relates to prevalent methodological flaws in the conduct and reporting of preclinical studies.

Preclinical efficacy studies involve administering a putative drug to animals (usually mice or rats) that model the disease experienced by humans.  The outcome sought in these laboratory experiments is efficacy, making them analogous to Phase 2 or 3 clinical trials.

However, that’s where the similarities end.  Unlike trials, preclinical efficacy studies employ a limited repertoire of methodological practices aimed at reducing threats to clinical generalization.  These quality-control measures, including randomization, blinding and the performance of a power calculation, are standard in the clinical realm.

This mismatch in scientific rigor hasn’t gone unnoticed, and numerous commentators have urged better design and reporting of preclinical studies.   With this in mind, the STREAM research group sought to systematize current initiatives aimed at improving the conduct of preclinical studies.  The results of this effort are reported in the July issue of PLoS Medicine.

In brief, we identified 26 guideline documents, extracted their recommendations, and classified each according to the particular validity type – internal, construct, or external – that the recommendation was aimed at addressing.   We also identified practices that were most commonly recommended, and used these to create a STREAM checklist for designing and reviewing preclinical studies.

We found that guidelines mainly focused on practices aimed at shoring up internal validity and, to a lesser extent, construct validity.  Relatively few guidelines addressed threats to external validity.  Additionally, we noted a preponderance of guidance on preclinical neurological and cerebrovascular research; oddly, none addressed cancer drug development, an area with perhaps the highest rate of attrition.

So what’s next?  We believe the consensus recommendations identified in our review provide a starting point for developing preclinical guidelines in realms like cancer drug development.  We also think our paper identifies some gaps in the guidance literature – for example, a relative paucity of guidelines on the conduct of preclinical systematic reviews.  Finally, we suggest our checklist may be helpful for investigators, IRB members, and funding bodies charged with designing, executing, and evaluating preclinical evidence.

Commentaries and lay accounts of our findings can be found in PLoS Medicine, CBC News, McGill Newsroom and Genetic Engineering & Biotechnology News.

2013 August 5 | Leave a comment | Tags: cancer efficacy preclinical studies translational research validity

@Manual{stream2013-300,
    title = {Uncaging Validity in Preclinical Research},
    journal = {STREAM research},
    author = {Valerie Henderson},
    address = {Montreal, Canada},
    date = 2013,
    month = aug,
    day = 5,
    url = {http://www.translationalethics.com/2013/08/05/uncaging-validity-in-preclinical-research/}
}

How Many Negative Trials Do We Need?

There is a growing concern in the clinical research community about the number of negative phase 3 trials. Given that phase 3 trials are incredibly expensive to run, and involve hundreds or sometimes thousands of patient-subjects, many researchers are now calling for more rigorous phase 2 trials, which are more predictive of a phase 3 result, in the hopes of reducing the number of phase 3 negatives.

In a focus piece from this week’s Science Translational Medicine, Jonathan and I argue that more predictive phase 2 trials may actually have undesirable ethical consequences–ratcheting up the patient burdens and study costs at a point of greater uncertainty, without necessarily increasing social utility or benefiting the research enterprise as a whole. We articulate four factors that we think ought to guide the level of positive predictivity sought in a (series of) phase 2 trial(s). These are: (1) the upper and lower bounds on evidence needed to establish clinical equipoise and initiate phase 3 testing; (2) the need to efficiently process the volume of novel intervention candidates in the drug pipeline; (3) the need to limit non-therapeutic risks for vulnerable patient-subjects; and (4) the need for decisive phase 3 evidence–either positive or negative–in order to best inform physician practices.

We are confident that these four factors are valid, but they are certainly not exhaustive of the inputs needed to make a robust judgment about the appropriate levels of predictivity needed in phase 2 for a given domain. What are the total costs and benefits of a negative phase 3? How should we weigh these against the costs and benefits of a more rigorous program of phase 2 testing? How many negatives should we tolerate? And at what stage of the development process? Our piece is a first-step toward developing a more comprehensive framework that could provide researchers, funders, policy-makers, and review boards with much needed answers to these important questions.

2013 May 10 | Leave a comment | Tags: negative results phase 2 phase 3 translational research

@Manual{stream2013-44,
    title = {How Many Negative Trials Do We Need?},
    journal = {STREAM research},
    author = {Spencer Phillips Hey},
    address = {Montreal, Canada},
    date = 2013,
    month = may,
    day = 10,
    url = {http://www.translationalethics.com/2013/05/10/how-many-negative-trials-do-we-need/}
}

Teaching Kills Blogging: Somewhat Recent Developments…

Dear Faithful Readers: Teaching has cut my blogging to a trickle, though the teaching has now begun to taper off. My silence is not for want of major developments in the last two months. Among a few highlights:

2010 April 16 | Leave a comment | Tags: bioethics blogging health care reform patents recent developments translational research

@Manual{stream2010-69,
    title = {Teaching Kills Blogging: Somewhat Recent Developments…},
    journal = {STREAM research},
    author = {Jonathan Kimmelman},
    address = {Montreal, Canada},
    date = 2010,
    month = apr,
    day = 16,
    url = {http://www.translationalethics.com/2010/04/16/teaching-kills-blogging-somewhat-recent-developments/}
}

California Dreamin: CIRM Announces New Stem Cell Awards

California’s Institute for Regenerative Medicine just announced a series of large funding awards to fund translational research initiatives involving (mostly) stem cells. The projects funded are telling with respect to what was funded, and what they will attempt to achieve.

2009 November 5 | Leave a comment | Tags: cancer gene transfer CIRN embryonic stem cells first in human trials translational research

@Manual{stream2009-80,
    title = {California Dreamin: CIRM Announces New Stem Cell Awards},
    journal = {STREAM research},
    author = {Jonathan Kimmelman},
    address = {Montreal, Canada},
    date = 2009,
    month = nov,
    day = 5,
    url = {http://www.translationalethics.com/2009/11/05/california-dreamin-cirm-announces-new-stem-cell-awards/}
}

Mice- Three Different Ones: Towards More Robust Preclinical Experiments

One of the most exciting and intellectually compelling talks thus far at the American Society of Gene Therapy meeting was Pedro Lowenstein’s.  A preclinical researcher who works on gene transfer approaches to brain malignancies (among other things), Lowenstein asked the question: why do so many gene transfer interventions that look promising in the laboratory fail during clinical testing? His answer: preclinical studies lack “robustness.”

2009 May 29 | Leave a comment | Tags: first in human trials gene transfer preclinical studies translational research

@Manual{stream2009-98,
    title = {Mice- Three Different Ones: Towards More Robust Preclinical Experiments},
    journal = {STREAM research},
    author = {Jonathan Kimmelman},
    address = {Montreal, Canada},
    date = 2009,
    month = may,
    day = 29,
    url = {http://www.translationalethics.com/2009/05/29/mice-three-different-ones-towards-more-robust-preclinical-experiments/}
}

Found Figures: Picking up the Pieces after an HIV Vaccine Trial Fails

In the November 29, 2008 issue of Lancet, two reports (plus a commentary) report the famously disappointing outcome of a recent placebo-controlled study testing adenoviral vector-based vaccines against HIV. News reports over a year ago reported that the study was halted after an interim analysis failed to show any prospect of proving effective. More troubling, subgroup analysis suggested that vaccine recipients who had high pre-existing immunity to the adenoviral vectors showed higher rates of sero-conversion compared with placebo. As this vaccine was among the most promising and advanced in terms of development, these results were seen as a major setback.

2009 February 3 | Leave a comment | Tags: adenoviral vectors HIV vaccine STEP trial translational research

@Manual{stream2009-110,
    title = {Found Figures: Picking up the Pieces after an HIV Vaccine Trial Fails},
    journal = {STREAM research},
    author = {Jonathan Kimmelman},
    address = {Montreal, Canada},
    date = 2009,
    month = feb,
    day = 3,
    url = {http://www.translationalethics.com/2009/02/03/found-figures-picking-up-the-pieces-after-an-hiv-vaccine-trial-fails/}
}

Stems and Blossoms (part 2): Really Informed Consent

There is a strain within the clinical and bioethics community that takes a minimal view of informed consent: investigators are supposed to provide requisite information to volunteers; if research subjects fail to comprehend this information, pity for them. This view brings to mind a memorable exchange between Inspector Clouseau and a hotel clerk (Clouseau: “does your dog bite?” Clerk: “No.”  Clouseau then extends a hand; the dog lunges at him.  “I thought you said your dog doesn’t bite.” Clerk: “Zat is not my dog.”)

2008 December 30 | Leave a comment | Tags: cell therapy cell transfer embryonic stem cells ISSCR stem cells translational research

@Manual{stream2008-114,
    title = {Stems and Blossoms (part 2): Really Informed Consent},
    journal = {STREAM research},
    author = {Jonathan Kimmelman},
    address = {Montreal, Canada},
    date = 2008,
    month = dec,
    day = 30,
    url = {http://www.translationalethics.com/2008/12/30/stems-and-blossoms-part-2-really-informed-consent/}
}

Stems and Blossoms (part 1): Justice

Shortly before I left for holiday, the International Society for Stem Cell Research (ISSCR) issued a policy paper, “Guidelines for the Clinical Translation of Stem Cells,” outlining ethical and scientific considerations for researchers designing translational trials involving stem cells (whether stem cell derived, adult, or embryonic).

2008 December 28 | Leave a comment | Tags: cell therapy cell transfer embryonic stem cells first in human trials stem cells translational research

@Manual{stream2008-115,
    title = {Stems and Blossoms (part 1): Justice},
    journal = {STREAM research},
    author = {Jonathan Kimmelman},
    address = {Montreal, Canada},
    date = 2008,
    month = dec,
    day = 28,
    url = {http://www.translationalethics.com/2008/12/28/stems-and-blossoms-part-1-justice/}
}