A study on the use of bootstrap aggregation methods in estimation of stable parameters
Abstract
Background & Aim: In many medical studies, one data set is used to construct the model, and to test its performance. This approach is prone to over optimization, and leads to statistics with low chance of external validity. Data splitting can be used to create training and test sets but the cost is reduction in power. The aim of this study was to demonstrate the ability of bootstrap aggregating (bagging) in improving performance of classification and regression tree (CART) models.
Methods & Materials: CART was applied on a sample of 404 subjects, to identify the factors that encourage people to change their body shape by cosmetic surgeries. Comparing known status of subjects with predicted group, sensitivity and specificity of models were compared. Firstly, all data was used to construct the tree and to test its performance. Secondly, model was fitted on half of data and tested on the second half. Thirdly, bagging was applied in which we drew 100 bootstrap samples. Using each bootstrap data, a tree was constr cted and its performance was tested on the unselected subjects. Final group prediction for each subject was determined following majority voting.
Results: When the whole data was used the overall accuracy was 59%. In the test data set and bagging, accuracy reduced to 53% and 56%. Corresponding figures in terms of sensitivity were 60%, 52%, and 55%, respectively.
Conclusion: Bagging corrected performance estimates for over optimization. Bagging method produces statistics which has higher chance for external validity.
Altman DG, Lyman GH. Methodological challenges in the evaluation of prognostic factors in breast cancer. Breast Cancer Res Treat 1998; 52(1-3): 289-303.
Haybittle JL, Blamey RW, Elston CW, Johnson J, Doyle PJ, Campbell FC, et al. A prognostic index in primary breast cancer. Br J Cancer 1982; 45(3): 361-6.
Todd JH, Dowle C, Williams MR, Elston CW, Ellis IO, Hinton CP, et al. Confirmation of a prognostic index in primary breast cancer. Br J Cancer 1987; 56(4): 489-92.
Galea MH, Blamey RW, Elston CE, Ellis IO. The Nottingham Prognostic Index in primary breast cancer. Breast Cancer Res Treat 1992; 22(3): 207-19.
Balslev I, Axelsson CK, Zedeler K, Rasmussen BB, Carstensen B, Mouridsen HT. The nottingham prognostic index applied to 9,149 patients from the studies of the Danish Breast Cancer Cooperative Group (DBCG). Breast Cancer Res Treat 1994; 32(3): 281-90.
Baneshi MR, Warner P, Anderson N, Tovey S, Edwards J, Bartlett JM. Can biomarkers improve ability of NPI in risk prediction? A decision tree model analysis. Iran J Cancer Prev 2010; 3(2): 62-74.
Therneau TM, Atkinson EJ, Foundation M. an introduction to recursive partitioning using the RPART routines [Online]. [cited 2015 Jun 29]; Available from: URL: https://cran.r- project.org/web/packages/rpart/vignettes/long intro.pdf
Dannegger F. Tree stability diagnostics and some remedies for instability. Stat Med 2000; 19(4): 475-91.
Steyerberg EW, Harrell FE, Jr., Borsboom GJ, Eijkemans MJ, Vergouwe Y, Habbema JD. Internal validation of predictive models: efficiency of some procedures for logistic regression analysis. J Clin Epidemiol 2001; 54(8): 774-81.
Sutton CD. Classification and regression trees, bagging and boosting. In: Rao CR, Wegman EJ, Solka JL, Editors. Handbook of statistics: data mining and data visualization. Philadelphia, PA: Elsevier; 2005. p. 303-29.
Garrusi B, Garousi S, Baneshi MR. Body image and body change: Predictive factors in an Iranian population. Int J Prev Med 2013; 4(8): 940-8.
Al-Sendi AM, Shetty P, Musaiger AO. Prevalence of overweight and obesity among Bahraini adolescents: a comparison between three different sets of criteria. Eur J Clin Nutr 2003; 57(3): 471-4.
Thompson JK, Heinberg L, Tantleff S. The Physical Appearance Comparison Scale (PACS). Behav Ther 1991; 14: 174.
Keery H, Boutelle K, van den Berg P, Thompson JK. The impact of appearance- related teasing by family members. J Adolesc Health 2005; 37(2): 120-7.
Shapurian R, Hojat M, Nayerahmadi H. Psychometric characteristics and dimensionality of a Persian version of Rosenberg Self-esteem Scale. Percept Mot Skills 1987; 65(1): 27-34.
Mendelson BK, Mendelson MJ, White DR. Body-esteem scale for adolescents and adults. J Pers Assess 2001; 76(1): 90-106.
Stice E, Bearman SK. Body-image and eating disturbances prospectively predict increases in depressive symptoms in adolescent girls: a growth curve analysis. Dev Psychol 2001; 37(5): 597-607.
Stunkard AJ, Sorensen T, Schulsinger F. Use of the Danish Adoption Register for the study of obesity and thinness. Res Publ Assoc Res Nerv Ment Dis 1983; 60: 115-20.
Fallon AE, Rozin P. Sex differences in perceptions of desirable body shape. J Abnorm Psychol 1985; 94(1): 102-5.
Zanjani Z, Kheradmand A. Comorbidity of fetishism and pedophilia with obsessive compulsive disorder: A case report. J Fundam Ment Health 2008; 10(38): 149-55.
Baneshi MR, Talei AR. Multiple imputation in survival models: applied on breast cancer data. Iran Red Crescent Med J 2011; 13(8): 544-9.
Wilkinson L. Tree structured data analysis: AID, CHAID and CART [Online]. [cited 1992]; Available from: URL: http://qpsy.snu.ac.kr/teaching/bigdata/ref1.pdf
Frank E. Pruning decision trees and lists. Hamilton, New Zealand: University of Waikato; 2000.
Edeki CH, Pandya SH. Comparative study of data mining and statistical learning techniques for prediction of cancer survivability. Mediterr J Soc Sci 2012; 3(14): 49-56.
Sabzevari H, Soleymani M, Noorbakhsh E. A comparison between statistical and Data Mining methods for credit scoring in case of limited available data. Proceedings of the 3rd CRC Credit Scoring Conference; 2007 Edinburgh, UK.
Mochizuki S, Murakami T. Accuracy comparison of land cover mapping using the objectoriented image classification with machine learning algorithms. Proceedings of the 33rd Asian Conference On Remote Sensing; 2012 Nov 26-30; Pattaya, Thailand.
Tan AC, Gilbert D. Ensemble machine learning on gene expression data for cancer classification. Appl Bioinformatics 2003; 2(3 Suppl): S75-S83.
Hu H, Li J, Wang H, Daggard G., Shi M. A maximally diversified multiple decision tree algorithm for microarray data classification. Proceedings of the workshop on Intelligent systems for bioinformatics; 2006 Dec 4; Hobart, Australia.
Sujatha G, Usha Rani K. An experimental study on ensemble of decision tree classifiers. International Journal of Application or Innovation in Engineering & Management 2013; 2(8): 300-6.
Asha.T, Natarajan S, Murthy KN. A data mining approach to the diagnosis of tuberculosis by cascading clustering and classification. J Comput 2011; 3(4).
Jelinek HF, Abawajy JH, Kelarev AV, Chowdhury MU, Stranieri A. Decision trees and multi-level ensemble classifiers for neurological diagnostics. AIMS Medical Science 2014; 1(1): 1-12.
Salari N, Shohaimi S, Najafi F, Nallappan M, Karishnarajah I. Application of pattern recognition tools for classifying acute coronary syndrome: an integrated medical modeling. Theor Biol Med Model 2013; 10: 57.
Alizadehsani R, Habibi J, Alizadeh SZ, Mashayekhi H, Boghrati R, Ghandeharioun A, et al. Diagnosing coronary artery disease via data mining algorithms by considering laboratory and echocardiography features. Res Cardiovasc Med 2013; 2(3): 133-9.
| Files | ||
| Issue | Vol 2 No 2 (2016) | |
| Section | Original Article(s) | |
| Keywords | ||
| Classification and regression tree (CART) External validity Bootstrap aggregating Data splitting Bagging | ||
| Rights and permissions | |
|
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License. |
