Original Article

Testing the significance of crossover receiver operating characteristic curves in the presence of multiple markers


Background & Aim: In multivariate receiver operating characteristic (MROC) curve analysis, comparing two tests is usually done by means of area under the curve (AUC’s) and sensitivities. However, the existing procedures have not addressed the issue of comparing two MROC curves when they cross each other.
Methods & Materials: A modified version of AUC (mAUC) under MROC setup is proposed to address the above-mentioned problem. It is also shown that mAUC performs better than AUC. The performance of mAUC in the aspect of crossover curves is supported by a real dataset and simulation studies at different sample sizes.
Results: Two real datasets, namely, Intra Uterine Growth Restricted Fetal Doppler Study (IUGRFDS) and Indian liver patient (ILP) datasets are used and apart from these simulation studies are also carried out to observe the effect of sample size. These mAUC’s are then compared with each other to show that difference exists between two curves while comparing AUC’s cannot identify the true difference existing between them. With respect to IUGRFDS dataset, MROC curves of the diagnostic procedures middle cerebral artery and cerebroplacental ratio cross each other and are found to be similar when their AUC’s and mAUC’s are compared. In ILP dataset, the extent of correct classification achieved in the case of males is shown to be better than that of females when mAUC’s at 0.5 and 0.8 are compared.
Conclusion: It is observed that the mAUC’s are competent in identifying the true difference between the crossover MROC curves when the sample size is adequate, and the λ values are 0.5 and 0.8 but not 0.3.

Liu A, Schisterman EF, Zhu Y. On linear combinations of biomarkers to improve diagnostic accuracy. Stat Med 2005; 24(1): 37-47.

Su JQ, Liu JS. Linear combinations of multiple diagnostic markers. J Am Stat Assoc 1993; 88(424): 1350-5.

Johnson RA, Wichern DW. Applied multivariate statistical analysis. Upper Saddle River, NJ: Pearson Prentice Hall; 2007.

Pepe MS, Thompson ML. Combining diagnostic test results to increase accuracy. Biostatistics 2000; 1(2): 123-40.

Sameera G, Vardhan RV, Sarma KV. Binary classification using multivariate receiver operating characteristic curve for continuous data. J Biopharm Stat 2016; 26(3): 421-31.

Vardhan RV, Sameera G, Chandrasekharan PA, Beere T. Inferential procedures for comparing the accuracy and intrinsic measures of multivariate receiver operating characteristic (MROC) Curve. Int J Stat Med Res 2015; 4(1): 87-93.

Yu W, Chang YcI, Park E. A modified area under the ROC curve and its application to marker selection and classification. J Korean Stat Soc 2014; 43(2): 161-75.

Ramana BV, Babu SP, Venkateswarlu NB. ILPD (Indian Liver Patient Dataset) Data Set (Online). (cited 2012); Available from: URL: https://archive.ics.uci.edu/ml/datasets/ILPD+ Indian+Liver+Patient+Dataset)

IssueVol 2 No 4 (2016) QRcode
SectionOriginal Article(s)
Multivariate Area under the curve Crossing-over Multivariate receiver operating characteristic curve

Rights and permissions
Creative Commons License This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
How to Cite
Govindaraju SG, Vardhan Rudravaram V. Testing the significance of crossover receiver operating characteristic curves in the presence of multiple markers. JBE. 2017;2(4):164-172.