Donavyn Coffey
February 17, 2023
UPDATED February 23, 2023 // Editor’s note: The headline of this article was changed from “New Cancer Screen, Same Issues: Physicians Confront Galleri Test” to better reflect the content of the article.
In January 2022, Anthony Arenz, a 51-year-old living in Mesa, Arizona, breathed a small sigh of relief.
The Galleri blood test, which screens for 50 types of cancer, hadn’t detected any positive signs.
It would be welcome news to anyone but especially to a firefighter with a 9% greater risk of developing cancer and a 14% greater risk of dying from it than the average person. The Mesa unit had lost two servicemen to cancer in the past 3 years. Both were more than a decade younger than Arenz.
When the city of Mesa offered additional free screening — including a full-body MRI — to firefighters over 50, Arenz initially shrugged it off. With a negative Galleri test in hand, he didn’t want to spend more time dwelling on it.
Still, he began to feel a creeping guilt for skipping a test that many of his fallen colleagues hadn’t been offered. He tried to soothe his anxiety with research. A look through the company’s website didn’t set him at ease. According to Grail Bio, a test result of “no cancer signal detected” does not rule out cancer.
Arenz booked his free MRI.
The results left him heavy: stage I kidney cancer. The Galleri test had missed it.
Arenz received his free Galleri test through a cancer screening program funded by the city of Mesa. The program is housed at Vincere Cancer Center in Scottsdale, Arizona. Under the leadership of radiation oncologist and Vincere co-owner Vershalee Shukla, MD, the program currently screens first responders in more than 10 Arizona cities at no cost to them.
Vincere began using Galleri shortly after the test launched for consumers in June 2021. Since then, the first responder program has become an avid user of the test.
But the ability of the Galleri test to identify cancer and, perhaps more importantly, the fallout from erroneous results have been under scrutiny since the test’s launch. The Galleri test, which has not yet been approved by the US Food and Drug Administration, is so new that few know what incorrect results look like in practice and how often they might occur.
After running the test on about 2000 servicemen and servicewomen, Shukla can offer some insight about the test’s real-world value in a high-risk population.
“Cancer screening is a very complicated issue,” Shukla told Medscape. “Being honest, the tests are good but are not ready yet [for wider use].”
“Good” but “Not Ready Yet”
Arenz was not the only firefighter who got a surprise after taking a Galleri test.
In nearby Phoenix, 51-year-old firefighter Mike Curtis knew his risk for cancer was high, but he wasn’t that worried. Curtis had been running into fires since he was 17. His dad, also a firefighter, had died of cancer at age 58.
Curtis had taken the Vincere Cancer Center up on every free screening service since the program began in late 2018 — well before Shukla started using Galleri in 2021. His most recent lung CT was clear. But he underwent the Galleri test just to stay vigilant.
His result was a shock. The test detected signs of cancer.
Curtis decided to tell no one, not even his wife. He’d bear the bad news alone until he was certain.
Shukla, however, immediately doubted the blood test result. She expedited several follow-up tests. One week, a PET and CT of the abdomen and pelvis later, her hunch was confirmed. The Galleri test result was wrong, Curtis did not have cancer.
The price of his peace of mind: an extensive workup with a $4000 price tag. Fortunately, the bill was covered by the screening program.
Overall, in just over 18 months of using the blood test, Shukla has only encountered one other false positive out of about 2000 Galleri results.
She also discovered two positive signals for cancer using Galleri that were confirmed with follow-up tests. One was a chordoma, a rare type of bone cancer, and the other was a squamous cell carcinoma of the head and neck. The Galleri test caught both remarkably early, in time for treatment.
For Shukla, however, false negatives were particularly “horrible.” Arenz’s was just one of 28 cancers that the blood test missed. And because 500 negative tests are yet to be validated, the 28 false negatives may be an underestimate.
In her experience, the binary test result — a simple positive or negative cancer signal — is an oversimplification of risk, she said. It “gives a false perception that you have cancer or you don’t,” although the test itself is not definitive.
Grail Senior Medical Director Whitney Jones, MD, agreed that the test is not meant to be a stand-alone screening test for cancer. The purpose of the Galleri test is to “complement other screenings, not replace them,” Jones told Medscape Medical News.
According to an analysis of Galleri data and Shukla’s experience, the test’s specificity was over 99%. That means the test successfully minimizes false positives.
But the test’s sensitivity was much lower. From data from first responders, Shukla determined the sensitivity to be 6.7%. That means the test misses about 93 of every 100 cancers. According to Grail’s latest data from more than 6300 people older than 50, the test’s sensitivity was 29%.
Specificity and sensitivity are metrics used to credential a test and establish confidence in its ability to detect the target disease. A test with high specificity can correctly identify patients who do not have the condition in question, while a test with high sensitivity can correctly identify patients who do have the disease. But there are trade-offs between sensitivity and specificity. One value is increased at the expense of the other.
It’s normal for a cancer screening test to prioritize specificity, according to Aparna Parikh, MD, an oncologist at Massachusetts General Cancer Center Hospital in Boston. In a test like Galleri, which is meant to be an adjunct to other screening modalities, “at least we are seeing a good specificity, which is important, because we don’t want false positives, where the downstream impact on the patient can be high.”
Overall, Jones said, Grail Bio’s aim is to build a test that’s sensitive enough to catch the most dangerous cancers without inundating the healthcare system with false positives. In addition, Jones explained, sensitivity varies by cancer type. It tends to be lower for cancers for which other screening modalities are available, as well as for earlier-stage disease.
However, the Galleri sensitivity values are “a little bit scary,” said Ji-Hyun Lee, DrPH, professor of biostatistics at the University of Florida and director of the Division of Quantitative Sciences at the University of Florida Health Cancer Center. Lee, who is not affiliated with Grail, reviewed the company’s publicly available data as well as Shukla’s data at Medscape’s request.
While there’s no definitive threshold for sensitivity, miss rates as high as 93% and 71% “provide little confidence in the [accuracy of the] test,” Lee said.
Positive and negative predictive values, however, are more clinically relevant measures of a screening test. These numbers indicate how likely it is that a patient’s results are true and therefore how worried they should be about a positive result and how much they should trust a negative result.
Galleri’s data in the over-50 population and Shukla’s in first responders suggest the test’s negative predictive value is very high — 98.6% and 98.1%, respectively — which means most people can trust a negative test result.
The positive predictive value, however, was less straightforward. In first responders, Shukla found that only half of positive Galleri tests were confirmed cases of cancer. And an analysis of Grail’s data found that only 38% of positive Galleri tests — 35 of 92 tests — represented a validated cancer diagnosis.
Grail noted that using a refined version of their assay in their analysis yielded a slightly better positive predictive value of 43% — specifically, of 58 positive signals for cancer, 25 were confirmed cancer cases.
“In a clinical setting, positive predictive value is more useable for decision-making for the patient,” said Lee. “Positive predictive value isn’t always high, because everything doesn’t always transfer perfectly to the clinic.” But in the general population, if only 38% of patients with positive Galleri results truly have cancer, the test is “not quite useful to make a decision for the patient or the providers.”
Galleri may also be a costly prospect for patients, no matter the result, cautioned Electra Paskett, PhD, an epidemiologist and cancer screening expert at the Ohio State University in Columbus. A positive Galleri test leads to a cascade of follow-up diagnostic tests, which payers may not cover. For a negative result, Galleri recommends that the patient undergo screening again in a year, at an annual cost of $950 plus the cost of any follow-up testing when Galleri does pick something up.
“If a provider wants to offer the Galleri test, all those things need to be made abundantly clear, in my opinion,” Paskett said.
Following the negative Galleri test, Arenz’s cancer didn’t slip through the cracks because he received other advanced imaging free of charge. But whether all doctors will go to such lengths to back up Galleri results, even for patients with negative results, is unknown.
A negative result can give patients “a huge false sense of security,” said Shukla. And if a test is positive, the workup isn’t simple, she added. Chasing cancer, especially one that’s not really there, can be nerve-wracking and expensive.
The question, then, is why perform the Galleri test at all if results require so much validation?
Parikh explained that a high-risk group such as firefighters represents an ideal-use case for Galleri and other liquid biopsy tests. But she noted that she would be “wary of the ability of the system to manage this test en masse” were the test to be used more widely in the general population.
Shukla said it’s less about the results she’s getting today and more about making the test more effective for her patients in the future. First responders need a test such as this that can quickly identify multiple cancers, she said. However, to improve the test, Grail needs more data from this high-risk population. That’s what she’s after.
Curtis doesn’t regret taking the Galleri test. The emotional toll of thinking he had cancer for a few days wasn’t too high a price, in his opinion. It’s part of cancer screening.
But he acknowledged that it would have been a much more burdensome experience had he’d been financially responsible for the workup or if he hadn’t had Shukla to manage his case from start to finish.
Because it was free, Arenz doesn’t regret undergoing the Galleri test either. But he tells his co-workers to check the site, do their research, and get more screening.
“Any medical center that’s just doing this one test, you just have to be careful,” Shukla said. “It’s not that easy.”
Medscape will explore Shukla’s cancer screening program and the public health implications of using the Galleri cancer test in this high-risk population and more broadly in a follow-up piece.
Donavyn Coffey is a Kentucky-based journalist reporting on healthcare, the environment, and anything that affects the way we eat. She has a master’s degree from NYU’s Arthur L. Carter Journalism Institute and a master’s in molecular nutrition from Aarhus University in Denmark. You can see more of her work in Wired, Scientific American, Popular Science, and elsewhere.
Leave a Reply