Putting Trials to the Test

© ImageZoo/Corbis

It was a nondescript setting to convey the drama of life and death—a Food and Drug Administration conference room in Silver Spring, Maryland. Over two days in late June, testimony came from breast-cancer patients, spouses of those who had succumbed to the disease, oncologists, medical administrators, and patient-advocate representatives. They had assembled to debate the fate of Avastin, the best-selling cancer drug in the world, but one whose effectiveness was now in doubt.

One speaker characterized the views of some medical experts, including some of those present, as “undignified, beyond the norms of a civilized society, cruel, and unnecessary.” Another called the decision on a course of therapy “a personal question for each patient” not meant to be decided by others. A third insisted that “for a fairy-tale ending, we need our Avastin.”

When Avastin was first considered for approval in treating metastatic breast cancer in 2007, Gary Lyman, a professor of medicine at Duke, was a member of the FDA’s Oncology Drugs Advisory Committee. “We were pretty enthusiastic about it at that time, and we granted the drug conditional approval,” he says. That’s essentially an accelerated path to market, with the proviso that the drug company must return within a specified time period showing additional data from clinical trials. “So they came back in 2010, and we were, unfortunately, very disappointed. The follow-up on the original study showed no impact on overall survival.”

The panel then took a second vote, which was twelve to one, withdrawing approval of Avastin; Lyman’s vote was one of the twelve. Given the options of either granting or withdrawing unconditional approval—in this case, the FDA wouldn’t allow for an in-between option—Lyman saw no other course. Clinical trials had shown that the drug did not, on average, make patients live longer. They also pointed to side effects, such as increased risk of internal bleeding, high blood pressure, and heart failure.

While emotionally fraught and medically contentious, the panel’s recommendation on Avastin showed a typical reliance on clinical trials, which are a core part of the FDA’s process for approving new drugs. Researchers widely agree the evidence from clinical trials is a better guide to medical care than a doctor’s own experience. But they also acknowledge that clinical trials aren’t always structured to deliver the most medically meaningful results. And that has led some—including some in the medical profession—to question how clinical-trial data are interpreted and deployed in medical practice.

Understanding clinical trials begins with understanding what they are not. And they are not large-population observational studies, such as studies on the risks of smoking. In the 1940s, researchers used a combination of hospital records and interviewing to make the now-familiar link between smoking and lung cancer. Similar observational studies have been done around cell-phone use and brain cancer, so far, with ambiguous results—in part because memories of exposure to cell phones (or anything else) can be biased.

Clinical trials test drugs in four phases: Phase I, the initial phase, is limited to a small group and usually tests whether a drug can be safely delivered and how people will react to it; Phase II looks at more subjects to better gauge the drug’s effects on the particular disease or disorder; Phase III, with preliminary evidence that the drug is effective, enrolls an even larger group and compares the drug with a placebo or with standard treatment; and Phase IV refers to the period after the drug has been licensed and marketed. Researchers aim for “clinical equipoise”: In advance of the trial, they should be uncertain about the preferred treatment, even as they can envision risks and benefits with the control group and the experimental group alike. The trials are usually double-blind, so neither the patients nor their doctors know which treatment regimen they’ve been assigned.

In Lyman’s view, the process worked well with Avastin. “There are a few examples in medicine where randomized trials are either unethical or impractical. But the randomized trial remains the gold standard. We all recognize its value for the drug-approval process, for the end stage of drug development.”

But to Kimberly Blackwell ’89, an associate professor of medicine at Duke and a breast-cancer specialist, the Avastin turndown is hardly something to be celebrated. Blackwell says she has patients who, for years, have been kept alive on the drug.

“We talk a lot about personalized medicine,” says Blackwell. “To me, personalized medicine is getting the right drug to the right patient, or the right procedure to the right patient. But our clinical trials have been designed for population-based outcomes, to improving survival for a population of patients without the requirement that we know exactly who is going to be helped and who isn’t. And then to say that the drug should not be available to the individual patient—well, the individual patient’s benefit was never meant to be studied in these trials.”

If the FDA upholds the decision to withdraw support of Avastin for treating breast cancer, “it sets a new standard,” Blackwell says. “We’re not going to see another first-line drug for another ten years, at least.”

Clinical trials fuel the work of the Duke Clinical Research Institute (DCRI), which, with some 1,200 employees, calls itself the largest academic clinical- research organization in the world. Its director, cardiologist Robert Harrington HS ’93, says, “We’re trying to ask and answer questions that give the practitioner a body of knowledge based on good, quantitative information.” The largest part of the DCRI—which includes Duke faculty members in clinical practice and in areas such as biostatistics and epidemiology—is devoted to designing and carrying out clinical trials.

DCRI associate director Eric Peterson, also a cardiologist, says that what’s on the horizon is mining medical data collected in large clinical registries, potentially for hundreds of thousands of patients, and figuring out which patients did better or worse with a particular treatment—all without an expensive, formal, randomized study. For now, though, he says, “there’s no question that medicine has been made better by the requirement to have these clinical trials, and we have better evidence now about which therapies work and which therapies don’t than at any other time in history.”

Even as they’ve been refined in concept, clinical trials have a long history. Back in the eighteenth century, the British Royal Navy, which sent sailors out to sea for a year or more at a time, looked to rein in the devastating impact of scurvy and conceived what was, in effect, a clinical trial for that purpose. “They randomized ships to receive or not receive extra fruits and vegetables,” Peterson says. “And lo and behold, it seemed that if they could get fruits and vegetables, they wouldn’t develop this horrific disease.”

© Images.com/Corbis

 

For decades, clinical trials were based in university medical centers. Beginning in the 1980s, pharmaceutical companies outsourced many of their trials to larger, private networks, so-called contract research organizations (CROs). And now, trials increasingly are moving outside the U.S. “Some of that is reflective of the fact that many of the common diseases are global diseases,” says the DCRI’s Harrington. “Heart disease is certainly a global disease. Cancer is a global disease. Well, why not study them globally? And particularly from a big pharmaceutical company perspective, they don’t want to just sell their products in the U.S.; they want to sell their products globally. So why not make these trials global?”

In a 2009 essay in The New England Journal of Medicine, Harrington and DCRI colleagues observed that clinical trials abroad allow companies to overcome regulations on compliance, documentation, and training that they may find burdensome. The cheaper costs of overseas trials may be a particularly powerful incentive: This year alone, because of patent expirations, the drug industry will lose control over mega-medicines with combined annual sales of some $50 billion.

This past summer, a study in the Journal of the American College of Cardiology underscored how global trials can challenge conventional health-care wisdom. Duke cardiologist Christopher O’- Connor A.H.C. ’87, M.H.S. ’94 and his colleagues pooled results from clinical trials—with a total of some 9,000 participants—and found that Americans with heart failure may benefit less from standard medications than patients in other countries. Beta blockers cut deaths among non-U.S. patients by 36 percent; there was no statistically reliable drop among U.S. patients. The factors behind those national differences are, for now, mysterious.

As more trials move overseas, ethically minded oversight is a major concern, Harrington and his colleagues wrote in their essay. “Wide disparities in education, economic and social standing, and health-care systems may jeopardize the rights of research participants.… In some places, financial compensation for research participation may exceed participants’ annual wages, and participation in a clinical trial may provide the only access to care for persons with the condition under study.”

Irrespective of a trial’s location, researchers wrestle with how best to protect their subjects. Ross McKinney HS ’82, a professor of pediatrics who directs the Trent Center for Bioethics, Humanities & History of Medicine at Duke, says informed-consent protocols nationally are a work in progress. Most investigators communicate clearly, he says. But sometimes informed-consent documents can reflect the sensibilities of lawyers or insurance companies more than the interests of patients, he adds, with the result that they “obfuscate and confuse.” He’d like to see streamlined language that would give potential subjects “a fair, quick assessment of the risks and benefits of the research so they can make a good decision.”

It can be tough for clinical-trial subjects to get beyond what McKinney calls a “therapeutic misconception.” As he puts it, “Even when you go through a careful informed-consent process, there is still an assumption that the doctor is working on the model of the traditional patient-physician relationship, which is a fundamental caring relationship. The research physician is still working in the patient’s best interest, but that research physician is also working in the interests of the study, because he or she wants to make a contribution to similar patients in the future. So the doctor in this role is something other than just the patient’s advocate.”

One doctor familiar with the role of treating the youngest patients is Daniel Benjamin HS ’01, a pediatrics professor and faculty associate director for the DCRI. He says it’s only been in the past dozen years or so that the FDA has approved drugs with a specific focus on children. Benjamin leads a network for pediatric trials; supported by the National Institutes of Health, it oversees off-patent studies, that is, studies of drugs that are not proprietary to drug companies.

“Typically, most of your surprises in pediatric-drug development are around what dose to use,” he says. “Children’s kidneys develop differently. Their livers develop differently. Their skin integrity, their gut absorption, their body surface—they’re all different from adults. Even when you know what you’re doing in adults, it’s impossible to extrapolate it down to children. What’s that right amount? And is it different for a twelve-year-old going through puberty who weighs two or three times his ideal body weight, when all your original dosing studies were in twentyfive- year-old healthy males of ideal body weight?”

Historically, he says, the view was that children are too fragile for trials. “The consequence here was that every child became an experiment of one. And you never were able to learn from your experience. Ultimately, you were just taking a vulnerable patient population and making them more vulnerable.”

The DCRI’s Eric Peterson makes the point that small samples— whether of small children or adults—are not ideal in clinical trials. “I can do a study on this narrow spectrum of patients with this one therapy versus no therapy, as an example, and find that this therapy is good,” he says. “But when I extrapolate from a narrow band of healthy, young, non-complex-disease patients, and I apply the results to patients in my own practice who are older, sicker, and have lots of other issues, I might find that the therapy works very differently.”

That concern over sample size has been a recurring theme for Robert Califf ’73, M.D. ’78, vice chancellor for clinical research and director of the Duke Translational Medicine Institute. Califf founded the DCRI in 1996 and led it for ten years. (The DCRI grew out of the Duke DataBank for Cardiovascular Diseases, which is still active.) He says, “There are a lot of reasons to do small clinical trials, like understanding biology, or the very early phase of drug development, when you need a sense of whether there’s any evidence that you’re hitting the target. But the vast majority of clinical trials are too small to really give you anything useful, at least with regard to informing about what treatment is best.”

Cardiology, “probably more than any other field, has moved further in using randomized evidence to drive what we do in practice, as opposed to just doing what we thought was the best thing for patients,” Peterson says. He gives an example from studies of patients whose hearts produce extra beats, or premature ventricular contractions (PVCs). A class of drugs effectively blocked those extra beats. But it was only when they were tested in a large randomized trial that the PVC blockers were found to increase mortality.

Califf can recall, as recently as a decade or so ago, “being in these raging debates with people from medical schools who said that if you just understand biology, you’ll know what to do as a doctor. Having a doctor’s opinion is better than nothing. But in deciding on treatment, it is nowhere as good as having the right clinical evidence.”

Last winter, Califf was widely quoted around his work in directing the largest study ever done on heart failure. The study looked at nesiritide, which had been approved after small studies in carefully selected patients. It seemed to soothe a particular symptom of heart failure, a drowning-like sensation in which the lungs fill with fluid. Still, the drug had fallen out of use. That was because small studies seemed to indicate an increased risk of kidney problems and an increased death rate.

Califf’s study enrolled patients at 450 participating hospitals around the world who had heart failure and difficulty breathing. They were randomly assigned to get an infusion of nesiritide or saline. The results contradicted those earlier findings about the drug: that it was effective, and that it had safety issues. As Califf told The New York Times, “To me, the really important message is that the drug got very widely used for reasons that are incorrect, and then it got bashed for reasons that are incorrect. Unless we do these kinds of large clinical trials, we are engaged in a comedy of errors.”

The validity distinction between small and large clinical trials shows how the laws of probability can come into play, says Peterson. “So if I do a trial and I run it in 100 patients, and I find it has a 20 percent benefit, and then I run it in another 100 patients. Will I get exactly that 20 percent, or will I get 10 percent or 30 percent?” He uses the analogy of flipping a coin ten times. On average, you should get five heads and five tails, but the results can vary widely. If you flip it 100 times, you’re much more likely to get closer to that 50 percent mark: An anomalous result is more likely to come from a small-scale exercise.

Duke has hardly been immune to questions surrounding clinical trials. Over the past year, the clinical-trial results that most occupied the university centered on cancer researcher Anil Potti. More than two dozen biostatisticians and cancer researchers had called into question the validity of Potti’s findings on the relationship between genetics and cancer treatment. Last winter, Potti resigned from his positions in the medical school and the Institute for Genome Sciences & Policy. The three clinical trials based on his research were suspended and ultimately terminated.

Among observers of clinical trials, it’s the relationship between the pharmaceutical industry and academic medical centers that has brought some of the most consistent criticism. Last winter, ProPublica, an investigative news website, compiled compensation disclosures from seven drug companies in 2009 and 2010, totaling $258 million. In North Carolina, sixteen doctors were paid more than $100,000 by drug companies; several were identified as practicing with Duke Medicine. One Duke doctor was reportedly paid $240,150 over two years by drug companies. This past May, The Wall Street Journal reported that a Duke thrombosis specialist and faculty member in the Pulmonary, Allergy, and Critical Care Medicine division of Duke Medicine encouraged the FDA to delay approval of generic versions of the anti-clotting drug Lovenox. According to the article, he had received more than $260,000 from the company that manufactures Lovenox.

Ross McKinney, the Trent Center director, says it can be problematic when industry is “the primary driver”—as it typically is—behind the testing of drugs. He mentions the notorious case, from seven years ago, of the anti-arthritic drug Vioxx, whose manufacturer sat on data that showed the drug contributed to heart attacks. “The primary motivation for industry is to build a market for their product. The primary motivation for academic doctors is to take care of patients.” Still, he says, the collaborative relationship between industry and the academic community “can give you good science.”

Drug companies, if they control the data, can choose to withhold results that they’re not happy with. At the DCRI, says Harrington, its director, “we will not sign a contract that does not allow us to have independent access to the data, and I mean the entire set of data. And we will not a sign a contract that does not allow us the full, independent right to publish the information.”

As he sees it, “If one is looking for villains in the relationship with industry, it’s not hard to find them. There are examples of industry behaving egregiously, and there are equally bad examples of clinicians behaving egregiously. But there’s also the opportunity to take the expertise of the individual investigator and couple it with the product-development expertise of a company, to do some good things in collaboration that ultimately would be good for society. The challenge is knowing where the boundaries are.”

But researcher bias can be unconscious, quite apart from any rewards system. Some years ago, a metaanalysis looked at clinical studies of acupuncture. The studies conducted in the U.S. showed very little effect from acupuncture; those conducted in Asia showed appreciable benefits. McKinney makes a comparison to fans of Duke and the University of North Carolina watching a basketball game between the rival teams. “Say you are in a sports bar, and you watch a Duke player put himself in a perfect position and a UNC player run over him. The question is, was it a block or a charge? Half the audience says, ‘It’s a bad call.’ Half the audience says, ‘No, they got it right.’ Same data, same information. It’s all about how you process that information.”

In the clinical-trials arena, processing information often means a tilt toward favoring positive results, says Duke’s Gary Lyman, editor-in-chief of the journal Cancer Investigation as well as an adviser to the FDA.“It’s so much easier to embrace something new that looks exciting than it is to step back and say, ‘You know, we were mistaken, it’s not that good, or it’s not good at all.’ It’s the same with the Avastin issue. Even if the FDA comes out and says, ‘We don’t believe the science is there now, and we can’t approve it for this intervention,’ it’s going to be very hard to convince oncologists. It’s going to be very hard to convince patients who think their life is dependent on the drug, even if the data don’t support that.”

JAMA, the Journal of the American Medical Association, has delved into researcher bias. (Eric Peterson of the DCRI is one of JAMA’s editors.) One early study published in the journal, from the late 1950s, reported that 97 percent of the articles published in a given year found the tested drug was performing safely and effectively. A more recent study showed that scientists presented with identical (mock) clinical trials found one weaker in significance than the other—based solely on the fact that one trial arrived at a positive conclusion and the other at a negative conclusion.

In McKinney’s view, clinical trials will never be definitive in the course of medical treatment; medicine always will be part science, part art. “People are not simple and clean, as clinical trials would have it. They’re complicated,” he says. One complicated human being would accept a nagging cough as a side effect to control blood pressure. Another would find it an unattractive tradeoff. One complicated human being would take antipsychotics even if it produced weight gain. Another would find the thought untenable.

Among those right now feeling the consequences of clinical trials is Marcia Gilbert, a thirty-year resident of Charlotte and an occasional patient of Kimberly Blackwell, the Duke breast-cancer specialist. She received a diagnosis of breast cancer more than sixteen years ago; after a period of remission, her cancer was found to have metastasized just over seven years ago. “When I asked the surgeon what I should expect from this long-term, he said I’ll be lucky to be alive in five years. He was, thank the Lord, way wrong on that.”

Gilbert has been on Avastin, the disputed breast-cancer drug, for more than two years. She’s tolerated Avastin much better, she says, than standard chemotherapy: “It’s allowed me to live a good quality of life. It’s felt like the dangers are minimal, and the side effects can easily be monitored by the patient and the patient’s doctor.” The clinical-trial findings that question the drug’s safety and effectiveness don’t square with her own experiences, she says. She’s concerned that with those findings, insurers will back off paying for Avastin for breast cancer. The yearly costs of $90,000 or so would be impossible for her or most future patients to absorb, she says. She also wonders whether the FDA’s decision against approval, following years of expensive research and clinical trials, will discourage drug makers from pursuing new therapies in the future.