Changes to High-Risk Medical Devices Often Backed by Subpar Studies

Although their initial FDA approval might have been based on high-quality studies, less than half of studies to support changes to those devices were randomized, blinded, or controlled. 

Randomized, blinded, or controlled trial designs are the "gold standards" for clinical trials, but when it comes to trials that test changes in how high-risk medical devices are designed or used, these standards often aren’t met.

That’s according to a new study by researchers at the University of California, San Francisco and Yale School of Medicine and published today in JAMA.

The FDA defines high-risk devices as ones that "that support or sustain human life, are of substantial importance in preventing impairment of human health, or which present a potential, unreasonable risk of illness or injury."

Such devices require premarket approval (PMA), which according to the FDA is the “is the most stringent type of device marketing application required by FDA.”

When device makers update their devices, such as by modifying the design or providing for different uses than the original purpose, they must submit PMA "supplements" to the FDA for approval. Although their initial FDA approval might have been based on high-quality studies, standards for updating the devices aren’t as high, the JAMA study shows.

"Many devices can have hundreds of different supplements attached to them," Sarah Zheng, MD, a resident physician in psychiatry at UCSF, and a co-lead author on the paper, said in a statement. "So we wanted to check the quality of evidence supporting these important changes as well."

The authors reviewed clinical trials supporting FDA approval of changes to high-risk devices over the last decade. They found that fewer than half of these studies were randomized, blinded, or controlled. In nearly a quarter of the supplement studies, the analysis procedures were changed after the study began, and many failed to present data from all the participants, potentially biasing the studies' results.

In addition, most studies lasted less than six months, and many presented incomplete data on age and sex, making it harder for physicians to evaluate the risk of a device in particular patients, the researchers found.

"There's a lot of pressure on the FDA to speed things up, to let innovative or life-changing treatments go to market," Rita Redberg, MD, MSc, a professor of medicine at UCSF and the study's senior author, said in a statement. "But you don't know if they're innovative or life-changing until you've done a careful study. These were short studies, with low-quality data, and little follow-up after approval."