STAIRing at Method in Preclinical Studies

by

Medical research, we all know, is highly prone to bias. Researchers are, after all, human in their tendencies to mix desire with assessment. So too are trial participants. Since the late 1950s, epidemiologists have introduced a number of practices to clinical research designed to reduce or eliminate sources of bias, including randomization of patients, masking (or “blinding”) of volunteers and physician-investigators, and statistical analysis.


In past entries, I have rallied for extending such methodological rigor to preclinical research. This has three defenses. First, phase 1 human trials predicated on weak preclinical evidence are insufficiently valuable to justify their execution. Second, methodologically weak preclinical research is an abuse of animals. Third, publication of methodologically weak studies is a form of “publication pollution.”

Two recent publications underscore the need for greater rigor in preclinical studies. The first is a paper in the journal Stroke (published online August 14, 2008; also reprinted in Journal of Cerebral Blood Flow and Metabolism). Many of the paper’s authors have doggedly pursued the cause of preclinical methodological rigor in stroke research by publishing a series of meta-analyses of preclinical studies in stroke. In this article, Malcolm Macleod and co-authors outline eight practices that journal editors and referees should look for in reviewing preclinical studies. Many are urged by STAIR (Stroke Therapy Academic Industry Roundtable)– a consortium organized in 1999 to strengthen the quality of stroke research.

Their recommendations are:

1- Animals (precise species, strain, and details should be provided)
2- Sample-size calculation
3- Inclusion and exclusion criteria for animals
4- Randomization of animals
5- Allocation concealment
6- Reporting of animals concealed from analysis
7- Masked outcome assessment
8- Reporting interest conflicts and funding

There’s an interesting, implicit claim in this paper: journal editors and referees partly bear the blame for poor methodological quality in preclinical research. In my next post, I will turn to a related news article about preclinical studies in Amyotrophic Lateral Sclerosis. (photo credit: 4BlueEyes, 2006)

2008 October 6 | Leave a comment | Tags:

BibTeX

@Manual{stream2008-132,
    title = {STAIRing at Method in Preclinical Studies},
    journal = {STREAM research},
    author = {Jonathan Kimmelman},
    address = {Montreal, Canada},
    date = 2008,
    month = oct,
    day = 6,
    url = {http://www.translationalethics.com/2008/10/06/stairing-at-method-in-preclinical-studies/}
}

MLA

Jonathan Kimmelman. "STAIRing at Method in Preclinical Studies" Web blog post. STREAM research. 06 Oct 2008. Web. 10 Jan 2025. <http://www.translationalethics.com/2008/10/06/stairing-at-method-in-preclinical-studies/>

APA

Jonathan Kimmelman. (2008, Oct 06). STAIRing at Method in Preclinical Studies [Web log post]. Retrieved from http://www.translationalethics.com/2008/10/06/stairing-at-method-in-preclinical-studies/

Tweet this post

The Octopus of Reference Standards

by

When gene transfer researchers perform an experiment, how do they measure the dose of their vectors?  For that matter, when investigators perform a study using any novel biologic, how have they characterized their agents?


Few would think the questions are ethically significant. But consider the fact that research teams often use different techniques to determine vector dosage, and as a consequence, results from different human and animal studies cannot be compared with each other.

In the July issue of Molecular Therapy, Richard Snyder and Philippe Moullier outline international efforts to estabish reference standards for AAV vectors (a class of viral vectors widely used in gene transfer studies). After about six years (and nearly twice that of trials using human beings), reference standards for two serotypes (AAV2 and AAV8) will soon be available. The authors describe a number of logistic and funding problems for establishing reference standards. Not mentioned are the numerous sociological challenges with herding feline-like researchers.

In my book, I argue that standards represent a critical vehicle for risk management in novel research arenas: small-scale human safety studies have limited value unless results can be linked up with other human studies to identify trends. Standard setting, and the octopus–like extension of standards into all preclinical and clinical studies–thus have important ethical implications. IRBs reviewing early phase novel agent protocols should be on the alert where study agents are not characterized using reference standards (photo credit: Roadsidepictures, 2007)

2008 September 23 | Leave a comment | Tags:

BibTeX

@Manual{stream2008-133,
    title = {The Octopus of Reference Standards},
    journal = {STREAM research},
    author = {Jonathan Kimmelman},
    address = {Montreal, Canada},
    date = 2008,
    month = sep,
    day = 23,
    url = {http://www.translationalethics.com/2008/09/23/the-octopus-of-reference-standards/}
}

MLA

Jonathan Kimmelman. "The Octopus of Reference Standards" Web blog post. STREAM research. 23 Sep 2008. Web. 10 Jan 2025. <http://www.translationalethics.com/2008/09/23/the-octopus-of-reference-standards/>

APA

Jonathan Kimmelman. (2008, Sep 23). The Octopus of Reference Standards [Web log post]. Retrieved from http://www.translationalethics.com/2008/09/23/the-octopus-of-reference-standards/

Tweet this post

The Octopus of Reference Standards

by

| Leave a comment

BibTeX

@Manual{stream2008-134,
    title = {The Octopus of Reference Standards},
    journal = {STREAM research},
    author = {Jonathan Kimmelman},
    address = {Montreal, Canada},
    date = 2008,
    month = sep,
    day = 23,
    url = {http://www.translationalethics.com/2008/09/23/the-octopus-of-reference-standards-2/}
}

MLA

Jonathan Kimmelman. "The Octopus of Reference Standards" Web blog post. STREAM research. 23 Sep 2008. Web. 10 Jan 2025. <http://www.translationalethics.com/2008/09/23/the-octopus-of-reference-standards-2/>

APA

Jonathan Kimmelman. (2008, Sep 23). The Octopus of Reference Standards [Web log post]. Retrieved from http://www.translationalethics.com/2008/09/23/the-octopus-of-reference-standards-2/

Tweet this post

NOTES from the Underground

by

Surgical innovation has always been a problem for medical ethics.  Surgeries are unregulated, and partly as a result, few are introduced to clinical practice having been validated in randomized controlled trials. Moreover, attempts at novel surgeries typically fly beneath the radar of ethical review, because they are viewed as innovative clinical practice rather than research. There are some good–and not so good– reasons for such “surgical exceptionalism.”


On Sunday, the Washington Post ran a story (“Scarless Surgery Uses Body’s Own Openings,” Rob Stein, Sept 21) on a new type of surgery, Natural Orifice Transluminal Endoscopy (NOTES), which avoids skin incisions by accessing organs through the mouth, anus, or vagina. Various teams are testing and/or using NOTES to remove gall bladder stones, to perform appendectomies and cholecystectomies, and to collect tumor tissue for cancer staging.

NOTES has the potential to reduce pain and scarring, and to hasten recovery. But the technique still requires validation in animal models. One major concern is infection: NOTES requires incisions through flora-rich environments like the rectum, stomach, or vagina.

Several U.S. and European medical societies have established initiatives aimed at guiding development of this technique. One example is NOSCAR (www.noscar.org), which maintains an outcome registry and identifies unmet research needs. The group also urges surgeons to seek independent ethics review before attempting NOTES in humans.

However, many first tries at NOTES have been performed outside traditional centers of medical innovation. For instance, the first team to perform a NOTES appendectomy is based in Hyderabad, India; several surgical teams in Brazil are also publishing a disproportionately large volume of human case reports (the prominence of Latin America and Asia is briefly NOTEd in the Washington Post story). One searches far and wide for any commentary on this phenomenon. Why are Brazil and India at the vanguard of surgical innovation? Is this simply one manifestation of the knowledge economy moving overseas? Or does it have something to do with laxer regulations and safety standards? Are there characteristics of NOTES that make it particularly attractive for surgeons working in resource-limited circumstances? (photo credit: photosan0, Pink Floyd album, 2006)

2008 September 22 | Leave a comment | Tags:

BibTeX

@Manual{stream2008-135,
    title = {NOTES from the Underground},
    journal = {STREAM research},
    author = {Jonathan Kimmelman},
    address = {Montreal, Canada},
    date = 2008,
    month = sep,
    day = 22,
    url = {http://www.translationalethics.com/2008/09/22/notes-from-the-underground/}
}

MLA

Jonathan Kimmelman. "NOTES from the Underground" Web blog post. STREAM research. 22 Sep 2008. Web. 10 Jan 2025. <http://www.translationalethics.com/2008/09/22/notes-from-the-underground/>

APA

Jonathan Kimmelman. (2008, Sep 22). NOTES from the Underground [Web log post]. Retrieved from http://www.translationalethics.com/2008/09/22/notes-from-the-underground/

Tweet this post

The Long and Winding Road(map)

by

The research team led by John Ioannidis has, for my (evaporating) money, done some of the most interesting work looking at the “epidemiology” of translational research: how often are high profile genetic linkage studies refuted? (answer: usually); to what degree are translational studies biased? (answer: a lot); how often are major scientific findings translated into clinical applications? (answer: rarely).


Now, Ioannidis’s teamm (led by Despina Contopoulos-Ioannidis) at University of Ioannina in Ioannina, Greece, has a new report in Science (September 5) looking at “The Life Cycle of Translational Research for Medical Interventions.” Here is what they did.

First, they created a pool of important medical interventions, defining “importance” on the basis of 1000 or more citations in the scientific literature for any study that claims an intervention was effective between 1990 and 2004. From this, they identified 32 “important” interventions. Next, they looked at the lag in time between publication of the “1000 cited article” and various milestones in development: the first article suggesting the intervention was effective; the first article about human use; the first article describing the compound’s isolation.

They found that the median lag between initial discovery of an intervention and publication of a highly cited article claiming efficacy was 24 years. They also found a much longer lag for interventions that, though claimed effective, were subsequently shown to be ineffective in other studies (e.g. Vitamin E for the prevention of heart disease).

The article is dense, as is probably my description of it. I worry that, by depending on a “1000 citation count,” lots of important translational discoveries are excluded, making this study biased towards applications used against high profile or highly prevalent diseases.  In other words, their search strategy is likely to emphasize the high profile of specific diseases rather than the high profile of specific interventions. So, for example, many of the most important “successes” in translational research are excluded from their list.  Bone marrow transplantation didn’t make the cut. Nor did any important monoclonal antibodies used in cancer.  Nor did recombinant protein products. Nor did any gene tests.

The authors conclude by saying that 1- don’t expect major new medical uses from drugs that have been sitting around for a long time; 2- translation tends to be faster when research involves multidisciplinary collaboration involving basic and clinical researchers. I expect we’ll see more elaboration from this team on the latter point in the years to come (photo credit: romeo66, 2008).

2008 September 18 | Leave a comment | Tags:

BibTeX

@Manual{stream2008-136,
    title = {The Long and Winding Road(map)},
    journal = {STREAM research},
    author = {Jonathan Kimmelman},
    address = {Montreal, Canada},
    date = 2008,
    month = sep,
    day = 18,
    url = {http://www.translationalethics.com/2008/09/18/the-long-and-winding-roadmap/}
}

MLA

Jonathan Kimmelman. "The Long and Winding Road(map)" Web blog post. STREAM research. 18 Sep 2008. Web. 10 Jan 2025. <http://www.translationalethics.com/2008/09/18/the-long-and-winding-roadmap/>

APA

Jonathan Kimmelman. (2008, Sep 18). The Long and Winding Road(map) [Web log post]. Retrieved from http://www.translationalethics.com/2008/09/18/the-long-and-winding-roadmap/

Tweet this post

In the Dark?

by

Most cancer patients who enter phase 1 clinical trials are motivated by the prospect of controlling their cancer. Increasingly, however, such studies, in the words of one ethicist, “take without giving in return” by involving biopsy procedures in which tissue is collected before and during the study in order to gauge whether a new drug is having intended biological effects. Such procedures are potentially burdensome, and offer no direct benefits for patients.


So is it ethical for clinical investigators to “stick it to” their research subjects? Depends in part on the level of risk. In the August 20, 2008 issue of Journal of Clinical Oncology, Aaron P. Brown and coworkers at NIH performed a literature review examining the complication rate for research biopsies in the context of studies involving radiotherapy (“Performing Nondiagnostic Research Biopsies in Irradiated Tissue: A Review of Scientific, Clinical, and Ethical Considerations”). One might expect to find higher than usual complication rates in this context, because irradiation can interfere with wound healing.

Here is what they found. Of 29 eligible studies, only 3 (!!) actively evaluated adverse events related to biopsy (another 16 reported adverse events– but not specifically related to biopsy) These 3 studies reported a total of 17 adverse events.

Here is what the authors concluded: “Limited data suggest that biopsies can be performed in irradiated tissues without clinically significant excess risk.”

The authors sagely argue that “all clinical trials that perform biopsies in combination with other therapies should actively study and report complications.” But with their specific study question- the complication rate of tissue biopsies- so underexposed, the their conclusions are dodgy. (photo credit: Moominsean 2006)

2008 September 11 | Leave a comment | Tags:

BibTeX

@Manual{stream2008-137,
    title = {In the Dark?},
    journal = {STREAM research},
    author = {Jonathan Kimmelman},
    address = {Montreal, Canada},
    date = 2008,
    month = sep,
    day = 11,
    url = {http://www.translationalethics.com/2008/09/11/in-the-dark/}
}

MLA

Jonathan Kimmelman. "In the Dark?" Web blog post. STREAM research. 11 Sep 2008. Web. 10 Jan 2025. <http://www.translationalethics.com/2008/09/11/in-the-dark/>

APA

Jonathan Kimmelman. (2008, Sep 11). In the Dark? [Web log post]. Retrieved from http://www.translationalethics.com/2008/09/11/in-the-dark/

Tweet this post

A Thick Frosting of Science…

by

On September 2, the Washington Post ran a story (“Injections of Hope: Doctors Promote Offshore Stem Cell Shots, but Some Patients Cry Foul”) on an emerging global economy of stem cell medical tourism. It described how patients with conditions ranging from ALS to spinal cord injury travel to offshore clinics to receive unvalidated cell “therapies”–embryonic or otherwise– for $15-50K; less if they bring in new customers.


The article portrays many of the doctors performing these procedures as well outside the mainstream. Many are. But a closer inspection shows many of the scientists and physicians associated with these outfits are having their cake of staying in the mainstream of scientific practice, while getting to eat the fees provided by desperately ill patients.

One example I encountered in my research is Thailand based Vescell, which claims to provide “a revolutionary new treatment for heart disease that actually rebuilds heart tissue using the patient’s own stem cells.” Their web site is a rich trove of moving patient testimonials. These treatments are not approved in the U.S. or Europe, and using them would be considered a flagrant breach of medical ethics outside of clinical trials. Who is on the scientific advisory board of this company? Nobel Laureat Aaron Ciechenover, and University of Toronto-based Ren-Ke Li. Who holds the patent on Vescell’s procedures? The Toronto-headquartered biotechnology company, Theravitae. Surely, policy-makers and fellow researchers can do much more to police these medical confectioners. (photocredit: mityrina 2007)

2008 September 4 | 2 comments | Tags:

BibTeX

@Manual{stream2008-138,
    title = {A Thick Frosting of Science…},
    journal = {STREAM research},
    author = {Jonathan Kimmelman},
    address = {Montreal, Canada},
    date = 2008,
    month = sep,
    day = 4,
    url = {http://www.translationalethics.com/2008/09/04/a-thick-frosting-of-science/}
}

MLA

Jonathan Kimmelman. "A Thick Frosting of Science…" Web blog post. STREAM research. 04 Sep 2008. Web. 10 Jan 2025. <http://www.translationalethics.com/2008/09/04/a-thick-frosting-of-science/>

APA

Jonathan Kimmelman. (2008, Sep 04). A Thick Frosting of Science… [Web log post]. Retrieved from http://www.translationalethics.com/2008/09/04/a-thick-frosting-of-science/

Tweet this post

Sell Therapy, European Style

by

Two side-by-side news reports in the August 21 issue of Nature spell more trouble for cell therapy in Europe. The first story follows on previous reports about Austrian urologist Hannes Strasser (see postings on Jul 23 and May 27, 2008). According to an Austrian government report, Strasser “failed to get appropriate approval for the trial from authorities… failed to adequately inform patients… [and used] poor study design.” Strasser’s university has banned him from seeing patients. Somewhat cryptically, the article mentions that “several of the hundreds of patients who have undergone the procedure by Strasser’s team… claim that they have had serious side effects.”  Are these attributable to the intervention?  We shall see.


About 600 kilometers to the Southeast, a Bulgarian deputy minister resigned after the European Union shut down a Sophia-based clinic that provides non-validated bone marrow therapy treatments for victims of spinal-cord trauma, stroke, and neurodegenerative diseases.  The minister reportedly has family members who “own two private companies” that perform the procedures. (photocredit: Eesti, Central Hali Market in Sophia, 2005)

2008 August 28 | Leave a comment | Tags:

BibTeX

@Manual{stream2008-139,
    title = {Sell Therapy, European Style},
    journal = {STREAM research},
    author = {Jonathan Kimmelman},
    address = {Montreal, Canada},
    date = 2008,
    month = aug,
    day = 28,
    url = {http://www.translationalethics.com/2008/08/28/sell-therapy-european-style/}
}

MLA

Jonathan Kimmelman. "Sell Therapy, European Style" Web blog post. STREAM research. 28 Aug 2008. Web. 10 Jan 2025. <http://www.translationalethics.com/2008/08/28/sell-therapy-european-style/>

APA

Jonathan Kimmelman. (2008, Aug 28). Sell Therapy, European Style [Web log post]. Retrieved from http://www.translationalethics.com/2008/08/28/sell-therapy-european-style/

Tweet this post

Seeding Trials: Misled in Translation?

by

Translational research- defined as research aimed at bringing discoveries in the basic sciences into clinical practice, is often divided into two types: T1 and T2. This blog is almost entirely focused on the former, which involves testing laboratory discoveries in human beings. 


Often lost in the glitter of T1, however, is the nettlesome challenge of taking clinical findings and translating them into community practice. Such T2 research in many cases involves ‘phase 4’ studies: trials that test a drug that has already been licensed by regulatory authorities.


A few years ago, I organized a research ethics symposium at McGill on phase 1 and phase 4 studies, titled “Dirty Business?” because such studies are often perceived as the most ethically suspect (the former because of their risk-benefit balance, the latter because they are often drug promotion masquerading as science).

The current issue of Annals of Internal Medicine features an article “The ADVANTAGE Seeding Trial: A Review of Internal Documents” (August 19, 2008) where Kevin P. Hill and co-authors show that Merck ran and published the ADVANTAGE study primarily to promote Vioxx sales(gasp!)  rather than to answer a question about the drug’s performance.

In an editorial accompanying the article, editors Drummond Rennie and Harold Sox condemn such “seeding trials” and offer a series of recommendations for ending this abuse of study volunteers and health care professionals. Specifically, they urge physicians and IRBs to “ask… about the study hypothesis and whether it addresses a settled question… they should look for clues [that it does]: such as a sttudy with an open label design, no control group, a very large projected enrollment relative to the importance of the question, a short-term study of a chronic disease, a study of an already approved drug….”

Rennie and Sox couch the ethical problem with seeding trials as one of risk– “few physicians would knowingly enroll their patients in a study that placed them at risk in order to provide a company with a marketing advantage.” But in my opinion, IRBs are reasonably good at sniffing out risk and harm.  Where they underperform is in assessing value. In my experience, IRBs seem to take the attitude “if nobody is going to get hurt, what the heck?” Phase 4 studies are generally harmless– probably more so than typical medical practice. What is troubling about them is that they are often valueless and– in the case of seeding studies– deceptive as well. And current review procedures at IRBs, focused as they are on risks rather than ends, are ill equipped (and frankly unmotivated) to staunch such unethical conduct. (photo credit: nutmeg66 2007)

2008 August 19 | 2 comments | Tags:

BibTeX

@Manual{stream2008-140,
    title = {Seeding Trials: Misled in Translation?},
    journal = {STREAM research},
    author = {Jonathan Kimmelman},
    address = {Montreal, Canada},
    date = 2008,
    month = aug,
    day = 19,
    url = {http://www.translationalethics.com/2008/08/19/seeding-trials-misled-in-translation/}
}

MLA

Jonathan Kimmelman. "Seeding Trials: Misled in Translation?" Web blog post. STREAM research. 19 Aug 2008. Web. 10 Jan 2025. <http://www.translationalethics.com/2008/08/19/seeding-trials-misled-in-translation/>

APA

Jonathan Kimmelman. (2008, Aug 19). Seeding Trials: Misled in Translation? [Web log post]. Retrieved from http://www.translationalethics.com/2008/08/19/seeding-trials-misled-in-translation/

Tweet this post

Fazing Phase 0?

by

Phase 0 studies are a new class of drug trials that involve delivering tiny doses of a new drug to human volunteers to study how the drug is distributed and metabolized in the human body.  Both FDA and its European counterpart, EMEA, have recently issued statements encouraging drug companies to consider ‘phase 0’ studies before going on to phase 1 trials.


I have written elsewhere (Journal of Law, Medicine, and Ethics, 2007) on the ethics of phase 0 studies, and a recent cluster of new articles on their ethics was published in the journal Clinical Cancer Research.  In general, I have tended to support such studies as a way of reducing risk and uncertainty in phase 1 studies.

But the August issue of Nature Medicine contains a small news report describing ongoing controversies about whether such studies accurately predict drug behavior at higher doses.  One company that is marketing itself as a supplier of equipment for phase 0 studies claims such studies are highly predictive, but according to the report, chemists and others question their validity, and fault the company for not releasing “specific criteria” they use to support their claim that phase 0 studies are predictive.

The upshot seems to be: 1- buyer beware on phase 0 studies, and 2- companies that pursue phase 0 studies should publish results (few probably do– they contain sensitive proprietary information) so that policy makers and others can evaluate their value. (photo credit: e-chan photo of Kusama Yayoi artwork, 2008)

2008 August 8 | Leave a comment | Tags:

BibTeX

@Manual{stream2008-141,
    title = {Fazing Phase 0?},
    journal = {STREAM research},
    author = {Jonathan Kimmelman},
    address = {Montreal, Canada},
    date = 2008,
    month = aug,
    day = 8,
    url = {http://www.translationalethics.com/2008/08/08/fazing-phase-0/}
}

MLA

Jonathan Kimmelman. "Fazing Phase 0?" Web blog post. STREAM research. 08 Aug 2008. Web. 10 Jan 2025. <http://www.translationalethics.com/2008/08/08/fazing-phase-0/>

APA

Jonathan Kimmelman. (2008, Aug 08). Fazing Phase 0? [Web log post]. Retrieved from http://www.translationalethics.com/2008/08/08/fazing-phase-0/

Tweet this post

Search STREAM

All content © STREAM research

admin@translationalethics.com
Twitter: @stream_research
3647 rue Peel
Montreal QC H3A 1X1

Site admin