By David Fishbach
July 20, 2005 | What kind of insight do you have into your clinical trial operations? How are patient recruitment efforts and visits progressing, for example? What about investigator recruitment -- is it moving forward? What issues, if any, are arising with data collection and query management? Can you tell if one site is taking longer than others to enter data?
Questions like these have driven the development of operational metrics modules in clinical trial management systems (CTMSs). These modules provide visibility into trial operational metrics so that clinical managers can take timely and appropriate action to keep trials running as smoothly and as quickly as possible. CTMSs deliver crucial trial operational metrics, but at a price. Staff must enter operational data as an additional step into the CTMS for tracking, which takes time away from their core tasks.
State-of-the-art, Web-based electronic data capture (EDC) systems are an alternative for obtaining trial operational metrics. Such systems generate trial operational metrics as by-products, accelerating the collection and visibility of clinical data. Because EDC systems capture trial operational metrics as by-products of entering and managing patient data, they require no extra effort for the staff. As a result, more and more sponsor organizations are choosing to extract operational metrics directly from their Web-based EDC systems, as opposed to their CTMSs, as such EDC systems can quickly and cost-effectively deliver trial operational metrics without added effort.
If state-of-the-art EDC is a faster and more economical alternative, does it herald the end of CTMS?
Start at the Beginning
Part of the answer to this question rests in the respective histories of CTMS and EDC systems, both of which have roots in clinical data management systems (CDMSs). Now a common means of capturing, managing, and cleaning clinical data, a CDMS allows sponsor staff to enter paper case report form (CRF) data into computer repositories. While a CDMS provides trial operational metrics as by-products of collecting clinical data, it often does so too slowly for clinical staff to intervene meaningfully. By the time patient visits occur and paper CRFs are assembled and sent to sponsors for heads-down data entry into the CDMS, typically one to three months have passed. A great deal can go wrong in that period before the operational data are visible from the CDMS.
Both clinical and sponsor staff needed more immediate visibility into clinical trial data. Clinical managers need nearly immediate visibility into trial operational metrics -- metrics such as the number of queries being issued against a CRF and the percentage completion of source data verification -- so that they can keep their trials moving expediently. Sponsor organizations need more rapid access to clinical data because they want to launch safety monitoring and preliminary analyses as soon as possible.
What arose in response to the CDMS dilemma were two parallel initiatives, both aimed at increasing visibility into clinical trial data. To address the visibility lag time inherent in CDMS clinical data, clinical organizations first worked to optimize their paper processes. They then began to explore “remote” or “electronic” data capture options as means of gathering clinical data more quickly and making it more immediately visible.
While clinical organizations worked to enhance visibility into clinical data, sponsor organizations worked to make trial operational metrics more immediately visible. Aiming to measure and track trial conduct independently of clinical data collection, these organizations assembled a conglomeration of homegrown efforts, including manual spreadsheets and tracking systems, as well as formal and informal reports. To deliver near-time visibility into trial operational data, some vendors developed what came to be known as CTMSs.
While CTMSs could provide the trial operational metrics clinical managers needed, they required extra effort from monitors and data management staff. In addition to their core responsibilities -- collecting and verifying clinical data and resolving data discrepancies -- these staff were charged with entering the operational data into the CTMS for tracking. Typically, staff would only do so after addressing their primary directive. In addition, the data that organizations sought for their CTMS often required staff to “self-report” -- to spend time logging how well they were doing their jobs rather than simply doing their jobs. This information would then be used to analyze staff performance. Many found this dynamic less than motivating. Rather than reporting on their performance, most staff preferred to simply be performing. As a result, staff often entered operational data into the CTMS “later.” Sometimes, though, “later” would never come.
The challenge of motivating staff to keep operational metric data current is one reason CTMS adoption has been sluggish. In addition, there are several drawbacks to having staff manually enter duplicate information into multiple, discrete business and/or clinical systems. For example, aside from the obvious duplication of effort, staff may spend significant time resolving discrepancies across systems and ensuring that updates propagate properly.
While CTMS was ramping up, a corollary technology was taking hold. Evolving from cumbersome systems on investigators’ desks, robust, Web-based EDC began to eliminate the main defect of paper data capture and in-house CDMS: It provided near-instant visibility into clinical data, as well as associated trial operational metrics such as data entry cycle times, query management statuses and cycle times, and source verification statuses, to name a few. Unlike CDMS, EDC systems render data like these visible in minutes rather than months. And because EDC, like CDMS, captures trial operational metrics as by-products of entering and managing patient data, EDC surmounts the main limitation of CTMS operational metric modules: No one has to spend time entering operational data separately.
As EDC becomes more prevalent -- EDC adoption is accelerating on the strength of the technology’s ability to provide rapid access and visibility to clinical data -- it becomes clear why organizations are grappling with the options for obtaining trial operational metrics. When an EDC system is in place, it is the de facto source for nearly all operational data about the trial. Given this, will EDC systems then replace CTMS entirely?
The answer is, in short, no. EDC is unlikely to touch many CTMS capabilities, including budgeting, contract management, investigator payment, contact management for investigators and patients, and monitor visit scheduling. But will EDC systems replace the operational metric modules of CTMS systems, the modules that provide data such as data entry cycle times? Over time, they nearly certainly will.
With two options for acquiring operational metrics, the challenge for an organization then becomes from which system -- EDC or CTMS -- to capture those metrics -- and how. Should a company integrate EDC and CTMS applications to exchange operational metrics? If so, what’s the best integration strategy? If integration isn’t the best option, should a company bypass the operational metrics modules of the CTMS altogether and work directly from the EDC application’s listings and reports?
Companies can leverage clinical trial management reports and activities deployed against the EDC system either through the EDC application directly or through its integration with the appropriate CTMS modules. Companies that have invested significantly in CTMS will be interested in how to quickly leverage information from the EDC system into the CTMS. If, however, companies have been working from homegrown spreadsheets and manual reports, or are facing major reinvestments in CTMS that are failing to deliver value, they may be more than ready to work directly from an EDC system’s reporting output, presuming the output is appropriately flexible and complete.
Consider the table (included here), which outlines several different scenarios for obtaining operational metrics, as well as a recommended approach for each.
Organizations weighing options for acquiring and reporting on operational metrics may conclude that they have invested too much time establishing processes that are CTMS-centric -- too much time, that is, to consider changing their approaches and possibly adopting EDC systems. The argument may be compelling, but companies should rely on more than “gut feel” to make such decisions.
At the least, companies should consider that the processes they have established for CTMS can apply almost across the board to EDC. There is little additional effort needed to use EDC with the same processes. How so? Consider this. During a typical CTMS deployment, companies must, for example, identify and spec out the reports they need, program those reports from their CTMS, develop standard operation procedures for using the data those reports deliver, and, finally, train their staff appropriately. For companies implementing EDC systems, only one of those processes must be redone: programming reports from the system providing the metrics, in this case, the EDC system. When the EDC system has straightforward report-writing capabilities, or even predefined reports that meet the company’s requirements, the programming of reports may be somewhat painless.
Of course, the decision to deploy EDC generally is driven more by the desire for accelerated access to clean clinical data than it is by a desire to obtain trial operational metrics. Companies deploy CTMS for non-trial-specific administrative functionality, things like contact and financial management and site administration, in addition to access to trial operational metrics. The influences outlined here are just a few of the factors impacting decisions about clinical systems and potential integrations. Clearly, some of the options outlined in the fourth and fifth columns of the accompanying table will be more costly and/or more effective than others. Existing infrastructure, legacy systems, staff composition and skill sets, and other factors will all come into play. Regardless, when companies are deciding whether and how to use EDC and CTMS, they should include clear operational metrics strategies as parts of the comprehensive analysis that is needed to weigh options accurately.
The market’s need for real-time, accurate operational metrics to monitor and influence clinical trial conduct, combined with the accelerating adoption of EDC, is compelling organizations to reassess how and where to obtain, aggregate, and report on trial operational metrics. CTMS can deliver and have delivered these metrics, but ultimately it may be faster and more economical to work directly from EDC systems. Even when CTMS is the chosen solution, effective integration with an EDC system can reduce organizational effort while accelerating operational metric visibility.
Will EDC systems replace CTMS entirely? No, they won’t. EDC will not replicate all CTMS capabilities. As EDC adoption grows and EDC systems continue to deliver improved access to trial operational metrics, however, the operational metrics modules of traditional CTMS may become an endangered species.
Choosing to Integrate: Surmounting the Challenge
If it is feasible to manage trials efficiently and effectively using information from the EDC system without deploying a CTMS, then the potential cost and effort savings are obvious. If CTMS and EDC are to both deliver clinical trial data, the advantages of well-planned and well-executed system integrations are numerous. They include eliminating redundant data entry across discrete systems, eliminating discrepancy resolution across these systems, establishing a system of record in which changes are to be made, and propagating updates in near real time.
When considering integration, an organization first must determine which operational metrics are available via the CTMS modules. These data generally include such items as enrollment rates, visit completions, cycle times for data entry and cleaning, trial-based investigator payment triggers, and drug usage and inventory calculations. Because this type of information generally comes directly from the trial’s EDC system or CDMS, direct integration is one way to eliminate double data entry -- and discrepancy resolution -- for these types of operational metrics. Of course, it remains true that reporting and follow-up directly from the EDC system also could eliminate this redundant effort.
Companies planning to integrate EDC and CTMS can glean added benefits by exchanging information bidirectionally between the CTMS and the EDC system. Why? Because when the EDC system is not the source of operational data, it is often the destination. For example, if a CTMS is tracking Institutional Review Board approval statuses, site contact information, and monitor workload distributions, companies may be able to leverage this information directly into the EDC system as appropriate.
Identifying Integration Basics
Once you decide to integrate EDC and CTMS, you will have several factors to consider when selecting applications and vendors, when deploying applications, and when managing the organizational change that must occur around the integrations.
Choosing Applications and Vendors
Look for vendors with robust product suites of both integrated and integratable applications, with validated and configurable import/export, reporting, and integration capabilities. Also, ensure that the vendor has adequate regulatory awareness, sufficient service and support organizations, and long-term viability.
Conducting a Deployment Program
Adhere to good project management fundamentals by defining the program’s goals, scope, and responsibilities in detail; by establishing measurable intermediate milestones; by planning for implementation and testing; and by communicating often with the extended project team.
Managing the Integration’s Organizational Change
Set clear organizational objectives, redesign processes as appropriate to leverage the new technologies, ensure that management is prepared to have the organization stay the course when things get difficult, and communicate twice as much as you think is necessary.
David Fishbach is consulting manager of Phase Forward. E-mail: firstname.lastname@example.org.