Statistical Issues in the Interpretation of Risk Prediction Markers
There are two popular statistical approaches to biomarker evaluation. One models the risk of disease (or disease outcome) using, for example, logistic regression. A marker is useful if it has a strong effect on risk. The second evaluates classification performance using the ROC curve. There is controversy about which approach is most appropriate. Moreover, the two approaches often give contradictory results on the same data. A marker that has a strong effect on risk may not improve the ROC.
We present a new graphic, the predictiveness curve, that complements the risk modeling approach. It assesses the usefulness of a risk model when applied to the population. In addition, the predictiveness curve relates directly to classification performance measures. We show that it provides a more coherent and cohesive assessment of a risk marker or model than either the risk modeling or ROC approaches alone.
We demonstrate first using data on PSA and risk factors for prostate cancer. We then apply the methods to two datasets on CRP and risk factors, the Framingham Heart Study and the Women’s Health Study.
- Margaret S. Pepe, Ziding Feng, Ying Huang, Gary M. Longton, Ross Prentice, Ian M. Thompson, and Yingye Zheng, "Integrating the Predictiveness of a Marker with its Performance as a Classifier" (June 1, 2006). UW Biostatistics Working Paper Series, #289 . http://www.bepress.com/uwbiostat/paper289
- Ying Huag, Margaret S. Pepe, and Ziding Feng, "Evaluating the Predictiveness of a Continuous Marker" (March 1, 2006). UW Biostatistics Working Paper Series, #282. http://www.bepress.com/uwbiostat/paper282
- Pepe MS, Janes H, Longton G, Leisenring W and Newcomb P. Limitations of the odds ratio in gauging the performance of a diagnostic or prognostic marker. Am J of Epidemiology 159(9): 882-890, 2004.
- Cook NR, Buring JE and Ridker PM The effect of including C-recative Protein in cardiovascular risk prediction for women. Annals of Internal Medicine 145:21-29, 2006
- Wilson PWF, Nam B-H, Pencina M, D’Agostino R, Benjamin E and O’Donnell CJ. C-reactive protein and risk of cardiovascular disease in men and women from the Framingham heart study. Archives of Internal Medicine 165:2473-2478, 2005
Back to Agenda