By Karl E. Peace
Georgia Southern University
July 21, 2008 | OPINION | The recent eCliniqua article McClellan Envisions Lifecycle Approach to Drug Surveillance reminded me of some thoughts I’ve held regarding the safety of pharmaceutical compounds for quite some time [see ref. 1]. Whereas establishing efficacy of a drug for a particular indication may end with the submission and FDA approval of the new drug application (NDA), monitoring drug safety is, indeed, a lifecycle exercise.
Safety and efficacy data—pre- and post-market approval—on pharmaceutical compounds are among the most valuable resources of the pharmaceutical industry. Drugs in development proceed according to two axioms of drug development [2]: A drug that has been elevated to clinical development status continues in development until the null hypothesis of safety is contradicted, or until the null hypothesis of inefficacy is contradicted.
Axiom 1: Drugs in development are considered safe until proven otherwise.
Hos: Drug is safe
Has: Drug is not safe
Axiom 2: Drugs in development are considered inefficacious until proven otherwise.
Hoe: Drug is not efficacious
Hae: Drug is efficacious
Regarding efficacy, Hoe must be contradicted (else the NDA would not be compiled or submitted). The Type I error associated with Hoe is synonymous with the regulatory decision risk regarding approval, and becomes the consumer’s risk as well. The FDA has operated under a maximum Type I error risk of 5 percent per trial for quite some time. Assuming that two independent, adequate and well-controlled pivotal proof-of-efficacy trials bear the weight of approval, the regulatory approval risk of inefficacy would be 0.25 percent maximum. Under the view that efficacy is a one-sided alternative, the approval risk reduces to 0.0625 percent [3].
The Type II decision error associated with Hoe is synonymous with the drug developer’s risk. Type II risk is reduced as the sample size increases. Drug developers may decide to abandon a truly efficacious drug if the sample size is too small.
Regarding safety, Hos must be not contradicted (else development of the drug would be discontinued). The Type I error associated with Hos is synonymous with the drug developer’s risk. The Type II error associated with Hos is synonymous with the regulatory risk. So it should not be surprising that large sample sizes are needed in the assessment of safety.
The safety profile of a compound at the time of NDA submission is merely a snapshot of the true safety profile upon wider and more general conditions of use. Since there is no Kefauver Harris Amendment for safety, and it is impossible to prove (with 0 risk) that a compound is safe [4], drugs need to be aggressively monitored for safety throughout their lives. This argues for the creation of safety databases over the life of a compound and dedicated biostatisticians and clinicians whose job it is to continually monitor and explore the accumulation of safety data in an attempt to know at the earliest time when there may be a signal that the compound is unsafe [1, 2].
Basically, the assessment of safety of a new drug before market approval is an exercise in estimation [5]. We should profile safety data as best we can, and provide point and interval estimates. The statistician is essential in this regard, but the responsibility for deciding about the safety of the compound and the interpretation rests with the substantive scientific and medical experts.
We know, if only from the sheer volume of the number of possible hypothesis tests on safety data, that the false positive rate will be high. This may lead erroneously to termination of a new drug in development. Certainly this has negative consequences of an obvious nature for the drug developer, but there is a greater concern: namely, when a decision is made that the drug is safe because the p-value is greater than 0.05 (due to low power and small numbers). For the drug developer to go on record that there is sufficient evidence via hypothesis-testing that the drug is safe carries a high potential for litigation if it turns out from greater experience with the compound that it is not safe.
The assessment of safety of a compound in humans should be an ongoing process. Every batch of additional safety data we collect may give us additional comfort that the compound is safe, but the totality of such data is only a snapshot at that particular time of the safety of the compound. If probability values are computed at various times in the life of the compound based upon the total safety data collected by those times, then they should be viewed in a sequential sense [6]. However it’s very unclear as to what should be the overall significance level. Another criticism against probability analyses of safety data is that the analyses reflect group comparisons—usually based on averages. We know that two groups may have similar averages, yet differ in the proportions of patients with higher or lower values. Therefore, to conclude that a compound is safe based upon group averages may fail to identify individual patients who are at risk.
Twenty years ago, I stressed the importance of company or corporate worldwide database for all clinical trials conducted on a particular compound [1]. It is perhaps even more important today. This should be a high-priority project at every pharmaceutical company. Information about each compound, particularly clinical in nature, should represent the greatest asset a company has. Successful companies will have to manage that information well. The key to successful management lies in standardization, planning and imaginative forward thinking. Statisticians within their companies should take a proactive role in articulating the types of questions investigation of such databases is expected to answer, so that the appropriate data will be accessible, and data specifications can evolve. The ideal situation would be one where the development of the database begins with the formulation of the clinical development plan.
Finally, in my last position within the pharmaceutical industry, I created a Drug Efficacy and Safety Implementation (DESI) group within the Biometrics department. The primary purpose of the DESI group was to explore and summarize safety data in each trial and across trials to detect possible signals as early as possible, and provide results to physicians and data safety monitoring committees. DESI also facilitated interim analyses of efficacy trials. Organizationally, such a group should not report to Clinical Development, its primary client. Perhaps it should report to Regulatory Affairs or to a new position: Chief Safety Information Officer.
References:
1. Peace, KE (1989): "Some Thoughts on the Biopharmaceutical Section and Statistics"; ASA Joint Sesquicentennial Meetings, Publication of the American Statistical Association: 98-105, Arlington, VA, August.
2. Peace KE (1987): "Design, Monitoring and Analysis Issues Relative to Adverse Events." Drug Information Journal; 21: 21-28.
3. Peace KE (1991): “One-sided or Two-sided P-values: Which Most Appropriately Address the Question of Drug Efficacy?” J. of Biopharmaceutical Statistics, 1(1): 133-138.
4. Bross ID (1985): “Why Proof of Safety Is Much More Difficult than Proof of Hazard.” Biometrics; 41(3): 785-793.
5. Peace KE (1991): "Sample Size Considerations in Clinical Trials Premarket Approval" Invited Presentation: Annual Meeting of the Drug Information Association, Washington, DC, June.
6. Jennison C, Turnbull B (1984): “Repeated Confidence Intervals for Group Sequential Trials.” Controlled Clinical Trials, 5: 33-45.
______________________________________
Karl E. Peace, Ph.D., is the Georgia Cancer Coalition Distinguished Cancer Scholar, Senior Research Scientist and Professor of Biostatistic in the Jiann-Ping Hsu College of Public Health, Georgia Southern University.
Previous Expert Commentary by this author:
Is Informed Consent as Ethical as It Could Be?
________________________________
This story first appeared in eCliniqua,one of Bio-IT World’s free e-newsletters. Subscribe here.