By Ann Neuer
The question of who and where images are read.
January 20, 2010 | As clinical trial protocols increasingly call for the use of imaging technology, a controversy is brewing as to whether efficacy can be based on site evaluation of images, or if the images should be read centrally by trained and independent professionals. The problem is particularly acute in oncology trials and whether sites are able to adequately identify progression-free survival (PFS), an image-based surrogate endpoint that indicates whether a patient’s disease has progressed.
“Making this kind of determination requires training and consistent methodology in acquiring and interpreting the images,” says radiologist Rick Patt, principal of RadMD, a provider of blinded reader services and imaging consultants. Blinded read is shorthand for blinded independent central review (BICR), the form of review advocated by the Food and Drug Administration (FDA) for radiographic exams used in oncology studies when PFS is the primary endpoint.
Applying consistent and unbiased methodology to BICR is essential because image interpretation helps determine the efficacy of investigational compounds. Too often, this critical work is left in inadequately trained hands. “There are instances in which the non-radiologist site investigator or even the study coordinator is completing the imaging case report form. It would be better to have the images read by a radiologist … but this costs money,” Patt says. Even if the site does engage a radiologist, many are not trained in clinical trials and are unfamiliar with RECIST—Response Evaluation Criteria in Solid Tumors— rules for objectively determining PFS in solid tumors.
The challenges don’t stop there. Site reads may be acceptable and image interpretation may be similar to findings made by a BICR, but a faulty methodology at the site level or inconsistencies in reading can hinder result comparisons. To address these issues, RadMD has a cadre of more than 600 radiologists, oncologists, medical imaging physicists and other professionals. Anyone who reads images for a trial goes through the Blinded Reader and Investigator Training Institute (BRITI), RadMD’s sister company, which teaches standardized reading of medical images for clinical trials.
Despite the apparent advantages of using a BICR provider, some experts maintain that there may not be significant differences between BICR interpretation of images and those made locally at the site level. Lori Dodd, mathematical statistician at the National Institute of Allergy and Infectious Disease (NIAID), says that research supports this notion. Some who are concerned that evaluations at the local site may be influenced by the clinician’s knowledge of treatment assignment, in other words, whether the subject is on the experimental or control arm. “For example, consider a crossover trial, in which patients on the control arm receive the experimental therapy at the time of disease progression. “It is conceivable that, in some borderline cases, clinicians may tend to call disease progression earlier in patients on the control arm. If the clinician calls progression, say, one cycle earlier for the patient in the control arm, this is a bias. That’s the concern, but there just isn’t evidence that this type of bias exists,” Dodd comments.
Dodd co-authored an article in the Journal of Clinical Oncology that reviewed seven oncology trials that used retrospective BICR as compared to local evaluation, and says “results were not very different between the two.” In addition, the Pharmaceutical Research and Manufacturers of America (PhRMA) Progression-Free Survival Working Group has presented results from more than 20 clinical trials. “Their findings are consistent with what we found,” says Dodd. “There isn’t evidence of such a bias. I have been asking for researchers to bring forward cases that contradict this, because it would be critical for the community to know of such cases.”
Indeed, Dodd suggests that BICR may actually introduce bias because of a practice known as “informative censoring.” The term “censoring” refers to the concept that if a subject is removed from a study because the site determines that his or her disease has progressed, the data linked to that subject is removed from the clinical trial, and no further images are collected. Informative censoring is complex, but in brief, it suggests that if censoring has occurred but cannot be confirmed through a central read, censoring will make the survival course of those still on treatment appear more favorable than it actually is. To minimize biased endpoint evaluation, Dodd believes that the best approach is to have a double-blinded local read. “The local investigator has a lot of information about the patient’s state. If the read is double-blinded, it is not influenced by knowledge of the treatment assignment because they don’t know the treatment assignment.”
Oncologist Eric Chen recently co-authored an article in the Annals of Oncology that supports Patt’s perspective. Chen looked at the variability in assessment between local investigators and independent review committees for response rate and PFS. His team reviewed 21 published clinical trials and concluded that local investigators overestimate response rate as compared to central review. Of the two trials that directly compared frequency of disagreement between local review to central review, the differences in determining disease progression were substantial, between 22-25%. Chen says that when many investigators measure scans for disease progression, measurement error is unavoidable. “For the majority of the trials, most will have a concordant conclusion, but 10% to 20% of trials could reach different conclusions.”
Some of the discordance between a site determining disease progression and receiving confirmation from an independent radiologist stems from the fact that independent confirmation is often not performed in real time. Patt contends that much of this gap can be substantially reduced. “Current technology enables fast turnaround on confirmation of disease progression: 24-48 hours. If the site feels that the patient has progressed, those images can be sent to us immediately for independent review before removing a patient for suspected progression,” he says.
It is apparent that there is divergent opinion on the subject of who should be reading images when progression-free survival is the endpoint, and when those images should be read.
This article also appeared in the January-February 2010 issue of Bio-IT World Magazine.
Subscriptions are free for qualifying individuals. Apply today.