June 14, 2006 | Eli Lilly’s chief medical officer, Alan Breier, describes a problem that is common — if seldom acknowledged. The company was designing a trial for a cardiovascular drug. “We had no idea what the dose was,” Breier told the 2006 Post-Approval Summit at Harvard. “We put in an adaptive design. At the end of this trial, we had a very good sense of what the dose was without missing a beat.”
For Breier, the emergence of an adaptive trial paradigm could mean the merging of Phases I and II, or even Phases II and III. “What is the difference between Phase II and III? Let’s blur that,” Breier says. “We’re not jeopardizing the sanctity of the blind. Let’s be informed by the data in real time.”
Lilly is not the only company exploring adaptive designs. Several of the industry’s giants, not to mention obscure companies such as Asthmatx, are testing the approach. The concept is simple. At predetermined intervals, in a statistically rigorous manner, sponsors are peeking at early data from a trial to change the same trial before it ends. Perhaps 10 percent of device trials already use adaptive techniques. The lobbying group Pharmaceutical Research and Manufacturers Association (PhRMA) is putting out position papers to shape the debate, and European and U.S. regulators vocally support the idea. As a result, many in the industry are asking what they will need — and when they will be ready — to execute adaptive designs. Vendors such as Cytel, Axio Research, Phoenix Data Systems, Almac Clinical Technologies, Tessella, and ClinPhone are stepping forward with software and services.
Agility Above All
“The value of an adaptive trial is really significant,” predicts Gary Lubin, the hype-averse cofounder of Merck Capital Ventures. Lubin says adaptive designs could accelerate go/no-go decisions by five or six months. “The use of modeling will become an important part of designing these protocols. A lot of money will be spent in these [modeling and adaptive] areas.”
The FDA is supportive. Robert Powell is the FDA’s senior pharmacist in the Office of Clinical Pharmacology and Biopharmaceutics at the Center for Drug Evaluation and Research (see “FDA Evolving Towards Adaptive Designs,” p. 30). Powell says it is still uncommon to see adaptive protocols at the FDA but confirms that they are common in the part of the agency reviewing medical device trials. He adds that adaptive designs are well suited for therapeutic areas in which patients are difficult to find. “An adaptive-type design is sort of like what happens in clinical practice,” he notes. “The traditional designs, the parallel designs where patients don’t cross over, are not representative of how physicians actually treat patients.”
Adaptive designs have been blessed by the FDA. Just last month, the agency announced a new draft guidance for such approaches in the medical device arena. And the agency includes adaptive trials in its audacious but unfunded Critical Path initiative. Indeed, the Critical Path “opportunities” list contains both revolutionary scenarios and provocative questions. In some cases, the FDA proposes, different elements of the Critical Path program could have synergistic effects that could save the industry years in development and billions in costs. If biomarkers are developed, for example, the FDA is hoping to sign off on “enrichment” trials that focus on a subgroup of patients that share a biomarker and are highly responsive to a particular drug. “Enriched trials have greater power and could result in therapies targeted at those most likely to benefit,” the FDA notes. “How will the data on the marker status of potential trial enrollees be used in trial design? How much data are needed on the unselected population? What types of retrospective subset analyses are valid (e.g., what can reliably learned from subgroup analyses that were not prespecified in the trial design)?” Such questions, Powell says, may be resolved with Critical Path research, simulation and modeling scenarios, or accumulating FDA expertise over time.
But some in industry are less certain that the FDA has done enough to spur the transition toward adaptive designs for drug-related trials. Mark Chang is technical director of research biostatistics at Millennium Pharmaceuticals, which has been exploring adaptive designs for several years. Chang applauds the FDA for telegraphing its willingness to consider adaptive protocols. He notes that Wyeth and Pfizer are plunging into the arena.
But Chang says other sponsors are reticent. That’s because each adaptive study, for now, is evaluated by the FDA on a case-by-case basis, without formal published regulatory guidelines. The FDA should release more standard, explicit ways to expedite an adaptive design, Chang believes.
Infrastructure Is Key
For Chang, electronic data capture (EDC) is a vital component of any foundation to support adaptive trials. “At Millennium,” he says, “we use it. That’s a crucial piece. You are able to not only capture the data but also do some of the common, key analyses. If we can use EDC or similar technology to have a view into the key efficacy and safety data quickly, in real time, we are able to make a decision earlier.”
Chang warns there may be minefields ahead with vendors assisting in adaptive designs. Some of the decisions are so critical to a company’s future that it could be culturally difficult to farm out such work to a consultancy, technology vendor or contract research organization. “There could be a solution with external people doing the analysis and some internal people involved,” Chang speculates.
Of course, much of the potential in adaptive designs lies in the ability to legally and nimbly change course. Appropriate, preplanned midstream adjustments could have far-reaching ramifications. Two statisticians at the University of Reading in England have estimated that adaptive designs could reduce the needed number of patients by 60 percent or more.
In an article in the Drug Information Association Journal, Susan Todd and Nigel Stallard discuss a real consulting project. The trial might have required 4,000 patients in a conventional Phase III study, and an unspecified number of patients for Phase II. An adaptive approach merging Phases II and III would have required just 1,390 patients. Under an alternative scenario, they estimate that a four-arm Phase II followed by a two-arm Phase III trial would have needed just 2,100 patients. However you slice the sample size, such projections translate into mammoth savings on an industrywide basis.
Why are the savings so large? Donald Berry, a biostatistician at M.D. Anderson Cancer Center, addressed the question in Cancer Medicine, an online text published by the National Center for Biotechnology Information. Berry is perhaps the most respected statistician pushing the adaptive approach, having helped to design more than 100 so-called Bayesian trials, which focus on the probability that a hypothesis is true given limited evidence. He writes: “Among the modifications possible are stopping early, restricting eligibility criteria, expanding accrual to additional sites, extending accrual beyond the trial’s original sample size if its conclusion is still not clear, dropping arms or doses and adding arms or doses.”
Adaptive designs could also refine the industry’s understanding of ambiguous data. In an article published last January in Nature Reviews Drug Discovery, Berry recalculated some of the data on Pravigard Pac, which the FDA approved in January 2003. A combination of Pravachol and aspirin, Pravigard Pac is the only drug known to be approved with an adaptive design. Scientists had doubts about the relative risk and benefits contributed by the Pravachol and the aspirin. Thanks to a heavy dose of Greek symbols and Bayesian math, Berry was able to retrospectively analyze five trials. He showed the combination of the two had a lower risk than either drug separately. With combination therapies of interest throughout the industry, the moment for adaptive designs is here.
The operational underpinnings to launch adaptive trials, sadly, are more debatable than the theoretical benefits. Can trials with paper-based forms be termed adaptive? How quickly must data be cleaned, analyzed, and acted upon? What is the minimal necessary technological infrastructure? Can patients be prevented from learning (from reporters or Wall Street) that one adaptive arm of a trial has been closed — and thus not tempted to leave the trial for some other treatment?
These questions may intimidate some companies. But Novartis appears to have begun to work them out. It doesn’t hurt that almost all of the company’s trials use EDC. John Orloff is VP of development and project lead on Novartis’ Project Delphi — a far-reaching effort to transform every aspect of its research and development. “There has been a lot of talk but not a lot of translation into action,” Orloff says. The real number of ongoing adaptive trials is not yet large, he says.
At Novartis, there is a distinction between an “adaptive” design and a “seamless” approach. A seamless design is one in which one trial segues without interruption into another, removing the need to recruit another batch of investigators or patients. Says Orloff: “We have half a dozen trials starting this year with adaptive or adaptive-seamless designs.”
In some cases, Orloff says, the time needed for recruitment of some trial arms will lengthen. But the key point is that these delays between trials are cut out of the process. “The white space between one trial and another is eliminated,” Orloff says.
Orloff adds that EDC is a crucial enabling tool for adaptive designs: “The paper case report forms (CRFs) are just — they’re so slow. It would be very difficult to do that and make decisions rapidly enough to apply them to the design of the trial that is ongoing.”
Special social “firewalls” may also be necessary to partition key piles of data from the active clinical operations personnel. In some cases, different teams that should not be conversing with each other can be separated geographically. In general, Orloff says, external vendors to help with adaptive designs may not be needed. “We are relying to a large degree on our own statistical group,” he says.
Could the humble telephone help bring adaptive designs to more trials? Another tool, interactive voice response systems (IVRS), is also useful in rapidly assigning new patients to new arms of a trial. The second-biggest IVRS company, Almac Clinical Technologies, asserts itself as one of the implementation and thought leaders in the field. “We are the leader in adaptive trial design theory,” says Jim Murphy, VP of business development, noting the 225-person company was founded in 1995 and was formerly called ICTI. “We’ve published and presented more than anyone in the market.”
After working on 900 trials, Murphy says it is no big deal for his company to help customers turn on a dime. Almac can help a sponsor instantly light up on a new arm of a clinical trial — or shut down an arm where the data has been disappointing. “When a change is needed, we can implement in minutes, not weeks,” he says.
Almac’s approach is a bit like the just-in-time logistical marvels of Procter & Gamble or Wal-Mart, which use sophisticated tracking and prediction algorithms to dispatch a new semi tractor-trailer of Crest toothpaste just as the last remaining inventory is placed on store shelves. The bad news? Just-in-time trials could raise supply budgets: Multiple doses will have to be formulated and shipped to clinical sites. Not all of them may be needed. It’s worth noting that Almac competitor ClinPhone, the leader in IVRS, is touting its system and sophisticated Monte Carlo simulation capabilities to more accurately predict drug supplies in adaptive trials.
Murphy says that interest in adaptive designs has been intensifying since early 2005. More than a dozen Almac clients are planning trials with adaptive approaches. Murphy notes that one aspect of the work, adaptive randomization, has been commonplace for 30 years. But the uptake of EDC is taking the industry to another level. Almac works with the major EDC companies, but it also has its own EDC solution, having bought the remains of a failed EDC vendor a few years back.
“The rise of EDC and the increasing acceptance of it has been a key factor driving forward the ability to use more information to assess the viability or efficacy or safety in more of a real-time situation,” says Murphy. “You don’t have to do a lot to change the EDC strategy. You just have to make sure the data is available.”
There is no escaping a significant amount of statistical planning to implement one adaptive trial, Murphy says. But once that is complete, the technical facilitation of the trial does not have to be the biomedical equivalent of a NASA launch. And the benefits from avoiding several separate trials could be significant. “Adaptive trials cost less than doing the cumulative cost of multiple trials,” he says. “You can get a little bit more information. Net-net, the absolute number of trials you conduct should be lower.”
Murphy concedes that EDC and adaptive trials won’t solve all of the industry’s productivity issues. Still, with so much other real-time data in everyday life, adaptive designs could reassure patients that a trial was being conducted in the smartest, most technologically adroit manner. “If your loved one were in a clinical trial, would you want them to be in a regular clinical trial?” he asks. “Or in an adaptive clinical arm?”
Are Vendors Necessary?
Another sophisticated enabler of adaptive designs is Cytel, a Cambridge, Mass., concern founded to provide industrial-strength computational products for statisticians. Jerald S. Schindler, president, Cytel Pharmaceutical Research, is in the odd position of having been a notorious vendor-basher during his years as a senior biostatistician at Wyeth.
| ||Jerald S. Schindler|
Now, Schindler insists, external suppliers have an indispensable role to play. The stakes in any interim peeking at clinical data are high and the temptations significant. Yes, Schindler says, academic statisticians with no stake in the results of a trial are available as consultants. But they’re in even shorter supply than SAS programmers. “If you look at a trial in progress, it’s not appropriate for the sponsor to look at the data,” says Schindler. “You have a dilemma. It’s a cottage industry of academic consultants. They do a good job, but they’re small. They don’t have the bandwidth for a large pharma.”
At large companies including his former employer, Schindler predicts, it will not be difficult to envision the need to support 100 adaptive trials a year. “These academic groups will have trouble supporting that,” he says.
Does an adaptive approach presume the trial will se EDC? Schindler says the answer is probably no. But to use paper, an adaptive trial would need to be able to drop the data into an electronic system in short order — something that is notoriously iffy in some trials. “You want the data to be in the system as soon as possible after the patient was seen,” says Schindler. “If EDC can help you do that, that would be nice.”
Having said that, it’s clear that adaptive methodologies will work much more smoothly if significant process change and technology has also occurred. That may encompass senior managers cracking the whip over not only clinical operations staff but also data managers, biostatisticians, and others. “It’s a process of reengineering change,” Schindler says. “Technology can help you a lot. Companies that have invested in technology will be far better off than companies that have not. You don’t do this one trial at a time.”
There is some debate about exactly how great the savings could be. Perhaps mindful of other technology vendors who overpromise, Schindler is careful about how grandly he paints a portrait of triumph related to adaptive techniques. “The ability to take advantage of patients in earlier phases [who would remain enrolled] in later phases might mean you have fewer patients overall,” he says.
Still, he says sponsors that develop the ability to do adaptive designs across a portfolio of drug candidates might need 25 or 30 percent fewer patients overall. Other trials might require more patients, Schindler observes, to reach sufficient statistical power. “You might pay more and end up with a more significant result at the end. You’ve taken what would have been a failed trial and turned it into a success.”
In such cases, there might be higher costs, but a return of a different sort. The conventional wisdom is that half of all Phase III trials fail. If adaptive designs at a particular company could slowly move that to more than a 70 percent success rate, it would represent a tectonic shift. “That’s a huge advantage,” says Schindler. “That’s a big deal. You can drive down the risk of your clinical program.”
For now, sadly, there is more visibility into the precise location of a FedEx package than the progress of a multimillion-dollar clinical trial. Says Schindler: “You want to be able to review the data. A patient gets seen, data comes in, you realize a boundary has been crossed. You have to make a decision to drop a dose.” As he describes the process, it’s clear it takes hours, not days, but never happens solely at a computer’s discretion.
Cytel’s East application and internal simulation tools not available in shrink-wrapped versions are not so much replacements for industry bellwether SAS tools as enhancements. “You can re-estimate the sample size as you go,” Schindler says. “You can re-estimate over time.” As such, there is probably no technology company that has more to gain from the adaptive trial phenomenon than Cytel, even though the company also has a services arm.
“We are able to help companies design trials, all trials, but focus on interim analysis and adaptive trials.” As the industry adapts to a new kind of trial, it’s clear tools and personnel from companies like Cytel will play a role.