Resolving Bottlenecks III
By Mark D. Uehling
May 9, 2003 | Pasquale Balachia can tell you exactly how long it takes him to transfer data from one patient chart into a clinical trial database: seven minutes. As special projects coordinator at Gottlieb Memorial Hospital in Melrose Park, Illinois, outside Chicago, he's using the Internet to record information for a clinical trial studying heart medications.
Balachia used to write the same chart data on paper. That took 37 minutes per chart. The process dragged on so long that by the time he figured out what else he needed to ask the doctors, they'd left for the night, leaving him — and the data collection — stuck. Now he's using software from Outcome Sciences of Cambridge, Mass., and he's happy: "Before, I had to spend three-fourths of my day doing the paper. This is more efficient. It's so much faster."
His view is increasingly shared — and scorned. Some companies shepherding drugs through clinical trials believe software is the best way to generate usable clinical data quickly. Other companies, also seeking to shorten the expensive, protracted odyssey of bringing new drugs to market, prefer to focus on process. The rationally designed leukemia medicine Gleevec (after trials and FDA review lasting just three years) is the poster drug for quickening the pace using both far smaller patient populations and the computer-assisted techniques described in parts I and II of this series. While all that is being sorted out, the most crucial question facing the industry is how enthusiastically to embrace the electronic capture of clinical data.
In principle, it seems ludicrous to contemplate a huge pharmacogenomic clinical trial using, conservatively, half a million sheets of paper FedExed from all over the world. In principle, using software to collect clinical data sounds far more sensible, efficient, practical. And futuristic. Over the years, the clinical trials fraternity has discovered the most eye-popping demo doesn't necessarily deliver clinical data as cheaply or efficiently as advertised. Sometimes the code isn't ready. "I've had vendors say, 'Delay the start of this program so we can work the details out,'" said James Kirwin, assistant vice president for global medical affairs at Wyeth Pharmaceuticals, speaking at the recent Bio-ITWorld Conference & Expo in Boston. "This is the wave of the future? When is that future going to get here?"
EDC: Ready for Prime Time?
By necessity, clinical research associates and coordinators have learned to be skeptical of well-intentioned but ill-conceived efforts to streamline their data collection. Jackie Bonner, a clinical research coordinator in Auburn, Maine, is currently using software from Phase Forward — and raves about it. Just don't get her started on another clinical trial program, developed internally by the trial's sponsor. "It was a nightmare to use," Bonner says. "It was just really hard to maneuver and hard to understand where to look for information."
|The Watershed Moment
|Forrester Research's senior healthcare analyst, Michael Barrett, shares some thoughts on electronic data capture.
Bonner says one doctor she worked for in the past participated in exactly one trial. "The paperwork is unbelievable. Everything the sponsor sends you, they want the doctor to sign. After the study was done, he said, 'This isn't worth it,'" Bonner recalls. With the number and complexity of clinical trials rising, drug companies are understandably wary of antagonizing valuable physician-investigators. The abandonment of a clinical trial application or vendor — or punishment of executives who lobby for new technology over the objections of doctors in the clinical trenches — is a chronic topic for off-the-record speculation.
In a way, it's the opposite of a Faustian bargain. The industry has elected to indulge paper-based inefficiency in most of the 80,000 clinical trials conducted each year, rather than alienate valued partners. "The software has failed to serve clinical medicine well," says Bernard Wess, president of Perseid Software. "People don't get what they expect and don't participate in the process to the extent we need them to."
The more auspicious news is that in the past year, some of the largest big pharmas — Novartis, Wyeth, Bayer, and Roche — have finally reported greater confidence in EDC. Details of their projects are as murky as anything else such companies grudgingly release. But the technology is slowly scaling, says Jeffrey Green, CEO of DataTrack, an EDC-centric clinical trial application service provider: "It is happening," he says of enterprisewide solutions. "I don't think we're going to be flying around to pick up paper forever." Indeed, in some cases, large drug companies are announcing companywide mandates that all new clinical trials use EDC.
That's a seismic shift. The rule of thumb has been that just 5 percent of clinical trials used the technology. "Adoption is increasing rapidly," said Kenneth Getz, founder of CenterWatch, a leading clinical trials publication and Web site, during the recent Bio-ITWorld Expo. "EDC is being used extensively in one out of four projects." As he defines EDC, it includes interactive voice response (IVR) systems, in which patients punch in data using the telephone keypad.
|Speaking Clinical Esperanto
|Clinical trial data inhabit a fantasy landscape not unlike J.R.R. Tolkien's Lord of the Rings.
But even Getz, optimist that he is, cannot sugarcoat the ease of adoption or effectiveness of EDC. "A fairly high percentage of respondents are neutral or negative about the experience they've had using EDC to date," he said, citing survey data. Getz ticked off trials vendors that have recently been either delisted from NASDAQ (DataTrack), entered bankruptcy court (CB Technology), laid off staff (e-trials), or slashed salaries (Arracel).
That upheaval — along with persisting doubts about the technology — has made big companies nervous. "I'm seeing a lot of companies that are even more cautious about implementing these solutions than they were 18 months ago," Getz said. "We are seeing companies experiment with other vendors. They are still shopping around. It's an immature environment, and there is limited loyalty between the industry and the vendor. This is a very alarming set of dynamics." Indeed, some sponsors of clinical trials would rather try their own hand at creating software. "Many of them have decided to take everything back in-house," Getz said.
That may not go as smoothly as the companies predict in press releases. Keith Howells, Oracle vice president of pharmaceutical applications, says that writing your own software isn't cheap, either. It's part of his job to point out the formidable costs of updating and supporting an in-house clinical trials application for a decade or two.
As Howells views the key bottlenecks in clinical trials, many are tied to paper. "The mechanics of shuffling that stuff with a mixture of speed and care turns into a major piece of relatively useless work," he says. "How do you get it from the sites, back to the sponsors? Do the data monitors go pick it up? Do they fax it in?"
Queries About Queries
That's not the end of the matter. Once the paper is in the system, "queries" arise (see "Clinical Pipelines
|Clinical Pipelines Clogged with Paper
|The promise of electronic data capture (EDC) is simple: fewer steps to a clean set of clinical data.
Clogged with Paper," right). A query is a potential discrepancy, omission, or error in a patient's data that is typically flagged by a human being or the clinical trial software. There can be tens of thousands of them in any large clinical trial, each costing a few dollars or hundreds of dollars to fix. The correct information must also be investigated, recorded, and signed for on paper, electronically, or both if the trial sponsor wants to be safe.
Not least, Howells adds, is the challenge of ensuring the data are complete and "clean," free of errors that could cause the FDA to reject a drug. Cleaning data takes ages. Howells recalls the lag in the outcome of the recent trial of cancer drug Erbitux by the German Merck in Europe. "That study finished five months ago," Howells notes. "Retail sales figures are ready two days after the month closes. With clinical trials data, it takes six months to figure out whether it works or not?" (The Erbitux data did turn out to be moderately positive.)
For a vendor, and someone proud of Oracle Clinical software, Howells is surprisingly candid about why some sponsors of clinical trials are leery of software in general. "Everything in this industry is harder than it looks," he says. "Unless [consultants or sales reps] go in either already having learned how hard it is, and with a sensitivity about what you see — you will get suckered into making promises you can't keep, and the pharmaceutical companies that invest in your technology will get burned."
Does 100% = 100%?
Howells says not all companies proclaiming that 100 percent of their trials are electronic are actually making the transition away from paper. "One of those companies actually had an investigator mutiny," he says. "The [investigators] said, 'If you make us use this software, we're dropping out of your trial.' They dropped the software."
Clinical sites undoubtedly have inefficiencies, but outsiders should resist breezily offering magical solutions to remove them. The clinical workflow is simply too complex. "There are a lot of things that are intrinsically, exceedingly complicated," Howells says. "You've got to roll in the mud with the database administrator and the data manager and people like that to really understand what's important."
For Richard Gliklich, size is what counts. President and CEO of Outcome Sciences, a Web-based EDC vendor specializing in Phase IV trials, Gliklich says the costs of validation (at least $20,000) mean that EDC is not economical for small clinical trials. He estimates, roughly, that EDC might not make sense for trials with fewer than 1,500 patients — he calls big trials the industry's sweet spot.
"We have nearly 2,000 hospital sites logged on to our system on a daily basis," Gliklich says. "Most of the hospital sites have at least five different people using the program. We currently manage data from 350,000 to 400,000 patients per year on our system. I believe that's the largest out there."
Gliklich sweats the IT details. He can quickly list the mirrored servers, the separate provisions to conform to European privacy laws, the consultants paid to break into the company's systems to make sure they're impregnable. But his pride and joy seems to be the Achilles heel of any clinical trial: the case report form (CRF). There may be 100 different forms for each patient in a clinical trial.
"People are not going to want to do data entry right off," Gliklich says. "A big part of designing an effective study on the Web is designing the CRF to work as a Web CRF, not just be an electronic representation of the paper form." He continues: "Depending on the diagnosis, the form will expand when it's important to expand. We work with CROs [contract research organizations] and sponsors to get forms to appear to be one page, even when they're longer."
There is stiff resistance at some clinical sites to trying new technology. "When people are first faced with EDC," Gliklich says, "they see the work is being shifted to them." But most of those in the medical trenches learn to love a good EDC solution. "Once sites start doing Web EDC, they do prefer it," he says. "They won't want to go back to the paper nightmare." The appeal for sponsors is even larger: cleaner data, inspected sooner.
EDC also seems to be working at PRA International, a CRO in Charlottesville, Va. Bucky Walsh, senior vice president of business services, has been with the company 18 years, long enough to remember when it had six people. Today it has more than 2,000 employees and $235 million in annual revenue. One of the company's trials, on diabetes, recently expanded from 7,000 patients to at least 10,000 without difficulty, thanks to PRA's centralized databases and global corporate structure.
While Walsh likes EDC, he argues that it's important not to get too excited about it. "EDC does the entry part of data management," he says. "I could accomplish all your de-bottlenecking goals if you faxed me all your CRFs and I entered them in the same day you faxed me. That little bit bought me something, but it didn't buy me a lot. The truth is, no one is entering data live, while the patient is right there. That's simply impractical."
Walsh describes the fragmentation in the $140-million clinical software industry with equanimity. "We have experience with a number of EDC systems," he says, noting that sponsors of trials may request that he use a certain technology. "The truth is, in one way or another, they are all more or less the same. Some work a little better than others. It's a very immature market. In three to four years, there will be one or two stable, market-leading vendors that have products that haven't changed much in the past 12 months. That is not the situation today."
His enthusiasm rises, however, in talking about his company's people. "Any time you capture clinical data," Walsh notes, "you have the opportunity for it to be incomplete or inconsistent. Why was that patient taking aspirin? If I'm running a clinical trial, I need to know. EDC may or may not help me determine that. We've got human beings who do clinical data coordination. They're actually looking at the data and trying to make intelligent assessments. It's not something that I can easily write a program for. There is some clinical judgment involved."
Thinking Outside the Database
For all the focus on the trial database, it's only a fraction of the information required to make a decision about a drug inside a sponsor — and file the paperwork with regulators. Document management in clinical trials is becoming a subspecialty all its own, with Documentum one of the leading suppliers to pharma.
Another is Open Text, a Canadian software company. Martin Sumner-Smith, Open Text's vice president of pharmaceutical and life sciences solutions, estimates 80 percent of the data in clinical trials is "unstructured" — that is, never destined for a database. Much of that is tied up with the management of the trials. "There will be clinical reports, summary reports, and a submission report that contains all these things," Sumner-Smith says. "There are huge amounts of documents."
Even when many clinical sites participating in a trial gather their data electronically, it is not uncommon to have one paper-loving holdout. Sumner-Smith says that means the reality is hybrid paper-electronic solutions: "For the foreseeable future, you're going to have paper." His company scans in everything and indexes it. "Simply being able to get access to those paper records instantaneously is a huge advantage," he says. "In the paper world, it's a nightmare. Where is the paper? Who's sitting on it? Did they put it back where they were supposed to?"
At Pfizer, he reports, Open Text's Livelink software is able to keep track of both electronic and paper documents, with 13 million "objects" in the system. On the fly, it can create case books with every form about a particular patient. That's helpful if the FDA asks a question — or the patient develops complications that the investigators need to inspect more closely. "Every time a case form arrives, the book is regenerated automatically," Sumner-Smith says.
That's possible because of a vast amount of information about the information that's added to the system. For any CRF, the system tracks which protocol, which patient, which site, and which physician — about 40 metadata fields for each object in the system.
Sumner-Smith is neither pro- nor anti-EDC, recognizing it may one day displace paper. "The EDC stuff has missed the point that there is a lot of process management that happens with the sponsor as the forms come in," he says. "There are literally hundreds of people involved." One of the grandiose mistakes of failed EDC vendors, he suggests, was assuming that so much precious, mission-critical data could be wrestled away from the sponsors. Says Sumner-Smith: "The original business plans said, 'We will manage your data for you.' I said, 'You've got to be kidding! Third-party management of my data? No thank you.'"
Protocol Design Software
Other companies are working on redesigning clinical trials entirely. There may be any number of goals: less time until
|Where the Millions Go
|Estimating the costs of clinical trials — and the price of paper — is difficult.
the database is locked, lower cost, greater simplicity. Fast-Track Systems offers its Trial Space Protocol Designer tool. In part, the software relies on the company's exhaustive, anonymized database of clinical trial data costs (see "Where the Millions Go,"
right). CEO James McCord says one customer canceled a $40-million trial when his software showed it could not be completed as quickly as the sponsor hoped.
The company helps clinical trials professionals to quickly see that their trial costs may be twice as much as expected, or about to exclude a large percentage of the adult population in the United States. "As you're designing the study," McCord says, "and put in a parameter, the model will tell you something about the consequences of that parameter. If you spend a little more time and money on designing the study, you get a tenfold return downstream."
Others are getting results now. Timo Ahopelto, vice president of operations and co-founder at CRF Box, predicts the Finnish company may more than triple last year's $6-million revenue. The company is running 50 studies in 23 countries, all using Palm and Handspring PDAs to collect data from patients. Because patients cannot, say, enter an "11" for their pain on a 1-10 scale, any PDA substantially reduces the numbers of queries. There is no paper to file, copy, fax, FedEx — or spill coffee on.
"Ninety percent of EDC is actually capturing information on paper," Ahopelto notes, referring to nurses' and physicians' crib sheets known as "source" documents in the trade. Starting a trial or changing a study protocol, a time-consuming process on paper, can unfold quickly with his devices; and that means the data reach statisticians more quickly than when tabulated from garden-variety EDC approaches that have paper documents along the way. "We are able to make decisions faster," Ahopelto says.
|A Vendor's Lament
|Paul Bleicher, co-founder of clinical trial software vendor Phase Forward, loves digital cameras.
Ironically, one of CRF Box's most cutting-edge devices for patient data in trials is also very familiar. Some patients in European clinical trials are using their own GSM-enabled cell phones to key in answers to simple questions. Like a PDA, the GSM phones can remind patients to take their pills or record their experiences. "It's the device they already have," Ahopelto says. "They know how to use it."
As a result, fewer patients drop out of trials and fewer patients need to be recruited in the first place. "In the future," Ahopelto says, "this has the potential to revolutionize drug discovery and pharmaceutical marketing." Imagine, he suggests, millions of users tapping into their phones with reports of illnesses associated with a particular medicine, or requests for more information about it.
If it was painless enough, or there was something in it for them, some patients might actually be badgering the drug companies to participate in trials, and not the other way around.
ILLUSTRATION BY BRIAN STAUFFER