Splash of Cold Water for Clinical Genomics at Cold Spring Harbor Personal Genomes Meeting
By Aaron Krol
November 19, 2014 | Last week, the Cold Spring Harbor Laboratory in Laurel Hollow, NY, convened its annual meeting on the subject of Personal Genomes: Discovery, Treatment & Outcomes. The meeting dates back to October 2008, a time when there were perhaps only a dozen “personal genomes” in the entire world — one of which was in attendance, in the body of recently retired CSHL Director James Watson, who watched presenters run through interesting features of his genome in a slideshow.
Six years later, whole human genomes are being sequenced at a rate of tens of thousands a month. Yet attendees at the Personal Genomes meeting are still asking some of the same questions they have pondered from the beginning: is this information useful? And if so, why can’t we seem to convince doctors to use it?
Those questions were at the heart of Friday night’s ethics discussion, which as at previous Personal Genomes meetings was the centerpiece of the event. This year’s discussion was framed as a debate, between Munir Pirmohamed, a noted pharmacogenomics researcher from the University of Liverpool, and Robert Nussbaum, Chief of the Division of Medical Genetics at UC San Francisco. Pirmohamed had come to Cold Spring Harbor to argue that the evidence bar for incorporating genomics into clinical practice has been set too high, denying patients access to genetic tests that have been well validated in research studies. Nussbaum was of the opposing view, that a spirit of “genetic exceptionalism” has become entrenched among geneticists, leading them to believe that their tests deserve special consideration. In his opening remarks, Nussbaum mischievously defined genetic exceptionalism as the view that “genomics is so cool, we don’t need to gather effectiveness data… and furthermore, the patients and providers of genomic testing get to decide what’s useful, while insurance and the government are just delighted to pay for it.”
Both speakers paid particular attention to pharmacogenetic tests as an important gauge for the acceptance of genomic medicine. The best-known drug-gene associations, like the connection between CYP2C19 and clopidogrel or VKORC1 and warfarin, have been the subject of multiple large retrospective studies, leading certain academic groups to recommend genetic testing before prescribing these drugs. The tests are also relatively inexpensive and immediately actionable, removing two major objections to other genomic tests. Nevertheless, as Alan Shuldiner, head of the University of Maryland School of Medicine’s Personalized and Genomic Medicine Program, and an author of several early clopidogrel studies, said in a presentation on the first day of the Personal Genomes meeting, “if you ask most cardiologists whether they’re doing CYP2C19 genetic testing for their patients, the vast majority of them will say, no, they’re not.” Physician groups have been waiting on prospective, randomized clinical trials before accepting pharmacogenomics as part of their routine practice.
To Pirmohamed, this is proof of an unreasonable evidence bar for genomic medicine. He compared testing for CYP2C19 variants to taking blood draws to measure patients’ cholesterol levels, a test so standard that primary physicians order it in annual checkups. This test, said Pirmohamed, has become ingrained in medical practice despite never being subjected to a randomized trial to prove its value. In fact, he continued, “If you look at cholesterol in terms of screening, you can see that the difference between those who are unaffected [by heart conditions] and those who are affected is quite small. Only 15% who die from heart disease will be identified, for a false positive rate of 5%... But we not only do it once… we do it multiple times.”
Yet Nussbaum had a very different perspective on the slow uptake of CYP2C19 tests. His belief was that pharmacogeneticists have been entranced by the power of their science to predict variable effects of drugs like clopidogrel, and lost sight of more important measures of their tests’ value. “I think it’s actually beautiful science,” said Nussbaum. “But when the statement becomes that it’s a no-brainer to use this, because the validity is so good… that doesn’t necessarily mean that it’s useful when implemented in the clinic.”
To Nussbaum, it was important to plainly define the evidence bar genomic medicine is being asked to clear. He named four hurdles for a genetic test, the first three of which are the usual metrics the FDA considers for all tests: analytical validity, clinical validity, and clinical utility. The fourth is a cost-benefit analysis between the positive outcomes of a test, and the potential harms of incidental findings.
“I think the analytical validity is not equivalent to other medical tests. We miss very important variants, like indels and unstable repeats,” he said, adding later, “The false positive rate is not under control. We have terrible trouble interpreting a lot of variants.”
This echoed a problem raised by Richard Lifton, Chair of the Yale Department of Genetics, in the meeting’s opening address on Wednesday. “Today, when we see missense variants, we can classify them as being at conserved positions or not conserved positions, of different functional groups or similar functional groups,” said Lifton. “But we really are not very good at predicting which variants are of clinical importance and which are not. When we go into the clinic, this is a critical issue, because we need to make better estimates of risk than simply saying we’ve got variants of unknown significance.” Lifton, whose talk focused on rare and serious hereditary diseases, did not see this as reason enough to shy away from testing, but outside of these high-stakes diagnostic odysseys, Nussbaum finds the amount of uncertainty very worrying.
Pirmohamed’s most urgent point, shared by many speakers and attendees at the Personal Genomes meeting, was that the economic model of the healthcare industry is stacked against genetic tests, if they are held to the standard route for demonstrating clinical utility. Prospective clinical trials are hugely expensive, while pharmacogenetic testing, to take one example, is not only unlikely to make any single drug or device maker a significant profit, but could actually eat into pharmaceutical companies’ profits by flagging certain patients as poor responders. Alan Shuldiner made the same point when he said, “These are the kinds of trials that clearly pharmaceutical companies are not interested in conducting, especially for a drug already off patent.” Meanwhile, the rate at which new, medically relevant genetic variants are discovered far outpaces anyone’s ability to conduct clinical trials.
“Can we really afford to undertake a randomized control trial for every variant or group of variants that we find within the human genome?” Pirmohamed asked in summary. “Can we afford to ignore everything that we find in the human genome?”
Nussbaum agreed that clinical trials are too much to ask for every possible genetic test, but continued to stress that a model has to be found that takes into account patient outcomes, not just the accuracy of the test. In fact, there was likely more agreement than disagreement between the debaters. Both thought, for example, that in rare disease, whole exome or genome sequencing should be used much earlier and more often than it is, perhaps as soon as doctors have made one failed diagnosis. Nonetheless, while Pirmohamed thought that whole genome sequencing’s continued “last resort” status illustrates the barriers to adoption that even the best genomic tests face, Nussbaum saw reimbursement for these tests as proof that the current evidence bar can already accommodate good medicine.
In the end, Nussbaum appeared to win over the Cold Spring Harbor attendees. A highly informal audience poll, taken before and after the debate, showed a big swing toward the position that the evidence bar for genomic medicine is not too high. This is perhaps surprising at a meeting where, earlier the same day, Russ Altman, Director of the Biomedical Informatics Training Program at Stanford, could say of pharmacogenetic testing that “the final three hundred yards is actually putting this information in decision support systems in the hospital, so that physicians in the 12 minutes when they’re seeing their patients can quickly get the information, triage it, and make a clinical decision.” Everyone in attendance was surely excited about the prospect of using personal genomes to directly change clinical practice, in ways large and small.
The debate, however, threw some needed perspective on that vision for medicine. While there may be a great deal of clinical power in the genome, there is also a great deal of noise, and thousands of variants of small effect that, however intriguing on paper, may never make a real difference for patients. The skepticism of many physicians toward genetic testing may be frustrating at times, but it is, at its best, rooted in a sense of responsibility to patients to keep their real-life outcomes at the fore.
Richard Lifton may have said it best in his opening address. After running through some of the fantastic advances rapid genome sequencing has opened up in the field of rare disease, he turned to the question of personal genomes for healthy individuals. “When it becomes cheap enough, should we be sequencing everybody at birth?” he asked. “Today I think that would be a very difficult proposition, because we’ll find many more variants that aren’t going to cause disease than will. We need to make sure that we’re improving people’s health, rather than simply increasing the neurosis of the population.”
The next Personal Genomes meeting is slated for fall of 2016, as Cold Spring Harbor Laboratory switches to a biennial schedule.