Assessing the Real World Accuracy of ctDNA Diagnostics

Sensitivity has become the go-to metric for assessing circulating tumor DNA (ctDNA) diagnostics. But while high sensitivity, or true positive rate, is important, it must be contextualized with other key measurements.

Relying on high sensitivity alone can generate dangerous false positives. If the ctDNA diagnostic says a tumor has an actionable mutation that’s not really there, the patient could be overtreated, a costly error on many levels.

To understand how often a ctDNA diagnostic provides correct results – positive if there is a mutation, negative if there isn’t – we must consider other measures, including true positive, true negative, false positive and false negative. Judging a ctDNA test by sensitivity alone is like having a GPS tracker that only displays longitude – it won’t get you to the correct destination.

What is Sensitivity?

Sensitivity is the number of true positives among all positives results. In other words, it measures an assay’s ability to detect mutations, inform treatment decisions, and ultimately help patients. Poor sensitivity means an assay is unlikely to detect any variants and doesn’t have the diagnostic firepower to influence care.

However, sensitivity is a double-edged sword – the more sensitive an assay, the higher its false positive rate. On a clinical level, there’s a trade-off between missing an actionable mutation (false negative) and overtreating the patient (false positive).

If high sensitivity were the only metric, we would have difficulty knowing whether any detected mutations are true positives or false ones. We would essentially be flying blind, which is no way to care for cancer patients.

This is an old debate. For many years, the prostate specific antigen (PSA) test was a workhorse diagnostic for prostate cancer. However, the test produced many false positives, and people received unnecessary biopsies and other interventions. The PSA also had its share of false negatives, with actual disease going undetected. This is a nightmare scenario that nobody wants to revisit.

Positive Predictive Value

To better understand an assay’s effectiveness, we need to add other measures, such as positive predictive value (PPV), which examines the relationship between the number of true positives and the total number of positives. PPV tells us how often a patient who tests positive actually has that mutation.

If an assay shows 100% sensitivity detecting ctDNA mutations, but the positive predictive value is 10%, that means the test will catch all of the true mutations; however, 90% of the results will be false positives. Needless to say, we must aim for much higher PPV.

Dynamic Range

In an ideal scenario, every sample would have an ample ctDNA concentration, making it easy to measure mutations. But in the real world, mutation frequency can vary quite dramatically, depending on the cancer type and other factors. Dynamic range is crucial, impacting whether the assay can provide accurate information under varied circumstances.

For example, a ctDNA assay might show 100% PPV when mutation frequencies are 1% or above. However, below 1%, PPV may sink. If it’s reduced to 50%, that means the test will make a false prediction 50% of the time.

As a result, PPV as a function of dynamic range is a critically important issue. An assay may detect mutations at .0001%, but how often does it succeed, and how often does it produce false positives? If we start drawing clinical conclusions at the lower range, where performance characteristics are weaker, then we’re going to make some incorrect judgments.

This is where understanding an assay’s limitations can actually be a powerful tool. If we’re working at the lower range, and we know there’s a higher likelihood the assay will provide inaccurate results, it is important to factor the increased uncertainty. When using any test, its important to know the PPV as well as the sensitivity or lower limit of detection in the range of values where the test result is being considered for clinical decision making.

Conclusion

Combining different approaches to measure assay accuracy gives us a much better read than sensitivity alone. This is understandably important for patient care – and to reduce unnecessary interventions and delays, as well as costs for overall care.

Back