【特別演講】

Beyond ROC Curves: Modern Statistical Tools to Quantify the Clinical Utility of Biomarkers and AI Classifiers

石瑜教授

Professor, 美國范德堡大學生物統計系

Abstract
Biomarkers and risk models increasingly drive early decisions in oncology and other fields, from accelerated drug approval to deployment of artificial intelligence tools at the bedside. Yet two fundamental questions remain difficult to answer with traditional methods: (1) does a biomarker truly stand in for a later hard endpoint such as overall survival, and (2) does a biomarker-based classifier actually deliver useful predictions in the real clinical prevalence setting. Conventional operating characteristics, especially the ROC curve and its area under the curve (AUC), describe discrimination but not clinical utility. In particular, an AUC of 0.95 can still correspond to very low positive or negative predictive value when the event is rare, and AUC does not tell us whether shifting a biomarker changes patient outcomes across trials.

This talk will present two complementary, modern frameworks to address these gaps.
First, we introduce a bivariate random effects meta-analytic model to quantify trial level association (T- association) between treatment effects on an early biomarker endpoint and a true clinical endpoint such as survival. The key measure is a correlation between treatment effects across studies, estimated with both frequentist and Bayesian methods, with guidance on the minimum number of trials needed and illustrated in non-small cell lung cancer immunotherapy trials. This provides an interpretable, US FDA aligned metric for deciding when a biomarker can support accelerated approval as a surrogate endpoint.
Second, we focus on F1 score, a machine learning metric that combines precision (positive predictive value) and sensitivity and is especially informative for rare events. 
We present psF1, a unified framework that delivers exact and large sample confidence intervals, hypothesis tests, and power and sample size calculations for single and comparative F1 scores. This allows investigators to design biomarker based classifiers around clinically meaningful F1 targets rather than relying on informal ROC comparisons or ad hoc simulations.

Together, these approaches move biomarker development beyond ROC curves, toward statistically rigorous and clinically interpretable measures of utility for both surrogate endpoints and AI enabled diagnostic tools.

• 講  題  : Beyond ROC Curves: Modern Statistical Tools to Quantify the Clinical Utility of Biomarkers and AI Classifiers
• 講  者  : 石瑜教授
               Professor and Chair, 美國范德堡大學生物統計系
• 時  間  : 114年12月17日（三）14:00-15:00
• 地  點  : 臺北醫學大學信義校區 EMBA個案教室  與 線上會議 (於會議前一天寄送連結)
• 報名連結：https://forms.gle/TfMyKNgwjHSn1ewv9