When I’m asked my opinion about tests like Galleri from Grail, Inc., I look at my watch to determine if there’s enough time left in the day. These tests are generically referred to as MCED (multicancer early detection tests) or “liquid biopsy” or ctDNA (circulating tumor DNA). The science is sound. Understanding the test is frightful.

On the surface, it looks great. A single blood sample can detect more than 50 types of cancer at an early stage. Already, we’re in trouble. There is a huge amount of data for each of the 50 cancers. We are used to at least 7 “performance characteristics” “as we discuss diagnostic tests one cancer at a time. But blood test performance is different for each type of cancer. Multiply 7 performance characteristics by 50 cancer types, and we have about 350 data points to start the review if we want to know how good the test is for each type of cancer. It’s a statistical mess and one needs tens of thousands of patients, maybe 100,000, to draw conclusions. AND, we need long-term follow-up (a so-called false-positive might end up being a true positive).

Recently, the PATHFINDER STUDY was conducted at 7 major health care facilities using the MCED test (Galleri) to screen prospectively for 50+ cancer types. The results? It would take a book filled with data to explain everything. In fact, the authors gave us Supplementary Data online (15 Tables and 5 Figures) because there wasn’t enough room in the main body of the article.

One thing we already know from past articles in the development of Galleri (and other similar tests) — it DOES NOT WORK WELL FOR BREAST CANCER. For some reason, breast cancer cells don’t shed detectable ctDNA as reliably as other types of cancer (more on this later). And, for full disclosure, I’ve been in the breast cancer blood test business for 30 years, so I’m looking at the ctDNA approach with a bias that indicates we should stick with one type of cancer unless future developments indicate otherwise.

Where the test has its most promising role is for those cancer types where screening is not routinely performed or recommended. Take pancreatic cancer, for instance. Finding pancreatic cancer earlier does not automatically save a life. The biology is aggressive and might not lend itself to early detection. Then again, it might prove to be the answer for pancreatic as well as other cancer types not being screened. However, the data is being presented to clinicians in a confusing way that puts a positive spin on the test. The marketing department is not fabricating the facts, rather they are simply molding the perspective.

For instance, we are told that Specificity (one of the “performance characteristics” is 99% for multiple types of cancer. That’s great, but it’s misleading. Specificity says, “Given no cancers in a population, what are the chances that the test will be positive (a.k.a. a false-positive). Because only a small percentage of patients are carrying occult cancer, the large number of “negatives” in a study dilute the formula and make Specificity look really, really good at 99% (only 1% chance of a false-positive). In the Pathfinder study, there were over 6,500+ patients and that 6,500+ is part of both the numerator and denominator of the formula, generating a tiny 1% chance of getting a false-positive.

But ask the question differently, and in a more clinically relevant fashion — “If I get a POSITIVE result on the Galleri test, what are the odds that I’ll actually have one of the 50 types of cancer? This is a different performance characteristic called Positive Predictive Value (PPV), and it is this number that is critical for a good, informed consent when discussing the test with laymen and health care providers. Here, the huge number of 6,500 patients is not part of the PPV formula, so one might get a PPV of 43%, leaving a 57% chance of the test being a false-positive. The false-positives are reasonable with Galleri, but their focus is on the 99% Specificity. Potential users must understand the 99% means that, prior to testing, there is only a 1% chance of a false-positive. Yet, given a positive test, there is a 43% chance of some type of cancer being present upon further investigation, usually radiologic, yet ending up with a 57% false-positive rate.

Now, each cancer type has its own Specificity, Positive Predictive Value (PPV), and Negative Predictive Value (NPV). The NPV is hugely diluted by the 6,500 negatives, so it’s not very helpful here). Additional performance characteristics are mentioned below. Then, it gets more complicated as, depending on the cancer type, some cancers are not early at all. So, when divided into stages, one can get frustrated when told “we find 60% of a certain type of cancer, only to learn that 50% are advanced stage where screening doesn’t help. Leaving the larger numbers in place, however, can lead to misleading acceptance of the test, even though the higher number is technically valid.

Even when a test is finding early (smaller) cancers reliably, eventually it will be necessary to document a mortality reduction. We might think we’ve solved the problem of aggressive cancers by finding them smaller, but they could prove to be just as deadly. That’s why a mortality reduction, or an indirect surrogate for mortality, must eventually be confirmed.

Hang on…here’s the punchline. I’ve not mentioned the most important performance characteristic of all — SENSITIVITY. This is the percentage of cancers that will be detected by a diagnostic test. Stated alternatively, given a certain number of cancers in a cohort, what percentage will be found by the diagnostic test. Sensitivity is how we generate our mortality reductions. This is where lives are saved, finding more cancers earlier. Specificity is very important, but it doesn’t save lives. One has to detect as many cancers as possible, and this is entirely through Sensitivity. The other way to state Sensitivity is through a “false-negative rate.” If mammography is detecting 70% of breast cancers, then it’s missing 30%, giving a false-negative rate of 30% (where the mammogram is negative, but cancer is actually present).

For those of us working on a blood test that “specializes” in one cancer type — breast cancer — Sensitivity vastly exceeds what is being seen in the MCED tests. Scientists are approaching 80 to 90% Sensitivity with breast-only blood testing, most all early detection. And this is where it turns bizarre when it comes to breast cancer in particular. In nearly all the MCED studies, the Sensitivity for breast cancer early detection is low. Instead of acknowledging an issue here, amazingly, in the PATHFINDER paper, the most important word in breast cancer screening — SENSITIVITY — is not even mentioned (at least I couldn’t find the most important numbere in screening). I can calculate overall sensitivity for all cancer types from an algorithm in Figure S3 — 21% (fyi — that’s terrible — only finding 22% of cancers), and that’s all types of cancer, and all stages. Limited to smaller Stage I & 2, the number would be even lower.

The shocking point is not simply the most important word is left out of the Discussion, but also that the breast cancer data, once identified deep in the Supplementary materials, reveals 14 breast cancers identified through routine screening (exam and mammography) and ZERO were identified with the MCED blood test. Repeat: 14 breast cancers were identified per the usual screening methodology, and not a single one of the 14 had a positive Galleri blood test. There were 5 breast cancer recurrences detected, but that’s not what a screening blood is for. The medical oncologists might be able to use the information gleaned from recurrences, but I’m totally focused on asymptomatic screening. We’ve lost our way when marketing departments tell us that MCED testing has 99% Specificity (technically true), yet the practical ability of the test to find early, occult cancer was ZERO in the PATHFINDER STUDY.

I’ve barely scratched the surface here, but try to imagine explaining all of the above to a patient or clinician. I didn’t even get into the other performance characteristics — Accuracy (a combo of specificity and sensitivity), Cancer Detection Rate (CDRs), or Number Needed to Screen to Save One Life (NNS). Multiply the 7 items we discussed by 50 to get data 350 data points from which to draw an informed consent. If you’re trying to tackle undiagnosed cancer, the Galleri test (and other MCEDs) are going to help more in those cancers that are not routinely screened. As for breast cancer, it’s not going to help unless major improvements are made.

“One Ring to Rule Them All” is a famous phrase from JRR Tolkein and his fantasty world. But when it comes to breast cancer, that’s what we need — a single test that generates Sensitivity of 80-90%, with Specificity of 90%, and good results in the remaining performance characteristics. Furthermore, Sensitivity results should be applied to tumor sizes as well, given that Sensitivity declines with smaller tumor sizes.

So, here’s a breast cancer screening approach for the future:

3D mammography, and if 3D is negative, proceed to a blood test. If the blood test is positive, then proceed to contrast-enhance mammography, breast MRI, or molecular imaging. A positive blood test turns screening into a diagnostic work up. Implementation should start with women having dense breast tissue since the miss rate is much higher in dense breasts, and so far, blood tests being developed are not affected by density levels.

If shouldn’t be hard to implement a blood test. After all, when used primarily for women with dense tissue on negative mammography, then we are talking about a cohort of women for whom, by definition, have demonstrated ZERO Sensitivity (mammos negative), yet we know that if we proceed with MRI, we will find 10 to 20 occult cancers in the general population. With this knowledge, one can move directly to MRI — but this is where the blood test comes in — sparing many from having the MRI, while still finding most of the hidden cancers.

You can see now why I look at my watch whenever I’m asked what I think about MCED blood testing. The Powerpoint version lasts an hour.

END