The study looked at recently approved major changes to critical devices such as blood-sugar monitors, stents, and defibrillators meant to restart the heart.
In May of 2015, the Food and Drug Administration approved a software update for a blood-sugar-monitoring system for children with diabetes, made by a California-based company called Dexcom. The system has a sensor that goes under kids’ skin, and it measures their blood-sugar levels every five minutes. If that level is ever too high or low, an alarm is supposed to go off.
A little more than a year later, the FDA recalled the Dexcom G4 Platinum Pediatric Receiver because the software was flawed—sometimes, the alarm wouldn’t sound when it was supposed to. The FDA issued what it calls a “Class 1” recall, which means agency officials thought there was a “reasonable probability” that using the Dexcom system would “cause serious adverse health consequences or death.” By that time, more than 19,000 Dexcom devices were sitting on hospital shelves and being used by kids at home.
Could the FDA have done more to prevent all those defective monitors from being shipped in the first place? A new study suggests so. The FDA originally green-lit Dexcom’s 2015 software update based on a study, submitted by Dexcom scientists, that followed kids with G4 Platinum Receivers for just seven days. In fact, the FDA often approves major software and hardware updates to critical medical devices—things like stents, hip and knee replacements, and implanted defibrillators—with similarly meager evidence, finds the study, which was published Tuesday in the Journal of the American Medical Association.
“Our goal was to define the strength of the evidence, and I guess what we found is that there may be opportunities to improve that strength,” says Sanket Dhruva, a Yale University cardiologist who worked on the study. “Ideally, there would be some enhanced rigor.”
THE FDA UNDERSTANDS THAT IT NEEDS TO CHANGE
Over the past decade, the FDA has issued several high-profile recalls of medical devices. Besides the Dexcom glucose monitor, it has recalled two heart devices that turned out to have faulty wires and that have been linked to 35 deaths in total. “We can’t go back in the past and say this would have been prevented,” Dhruva says. “But we do think that it is possible that some pre-market clinical studies may have detected these complications before they were so widespread.”
Dhruva and his team analyzed studies that the FDA recently relied on to allow major updates to 78 critical medical devices. (The technical name for the kind of approval the researchers examined is a “panel-track supplement.”) The team members looked for important markers of rigor in the studies. They looked for randomization: Studies generally compare groups of people—Do people who get a new version of a device fare better than people who get an older version of it?—and it’s important that the groups are chosen randomly. They looked for blinding: Neither doctors nor study volunteers should know what group they’re in, which could bias the results. And they looked for how long the studies tracked their volunteers.
The results: Fewer than half of the device-update approvals were based on randomized studies. Fewer than a third were blinded. And the studies lasted a median of six months.
No one expects 100 percent of these studies to be randomized and blinded. Sometimes it’s just not ethical: You can’t perform surgery on somebody and implant a placebo device in them, for example. Still, Dhruva thinks those figures ought to be higher. “I can’t put a number on that, but I would have hoped for higher than both of those numbers being less than 50 percent,” he says. As for how long studies should follow people, that depends on the device, but: “In general, I do think there are opportunities for longer follow-up before approval, which would give us more confidence.”
“In the case of the glucose monitoring system, there may have been an opportunity to follow the patients for longer than one week,” Dhruva says.
The FDA understands that it needs to change, as the agency’s former commissioner, Robert Califf, writes in an essay published alongside Dhruva’s study. Officials are considering how to use big data, gleaned from the electronic health records that have replaced paper ones over the past decade, to help them more quickly spot problems with FDA-approved drugs and devices after they’ve entered the market. There will always be such problems because no study can perfectly replicate the number and diversity of people who will use a device in the real world, Dhruva says. But maybe they can do a better job at reflecting what’s to come.