Longitudinal relationship between academic staffs’ evaluation score by students and their characteristics: Does the choice of correlation structure matter?
Abstract
Each semester, students are asked to evaluate the academic staff through an online questionnaire. Generalized estimating equations model (GEE), taking into account the correlation between scores, is the established tool to analyze longitudinal data. The aim of this manuscript is to identify characteristics that influence staff score and to address the importance of selection of appropriate correlation structure. We analyzed scores of 336 staff in six consecutive semesters applying GEE with three correlation structures: exchangeable, autoregressive, and unstructured. We also compared the performance of these correlation structures via simulation study. Three normally distributed outcomes with exchangeable correlation structure were simulated. Four independent variables (two continuous and two binary) of which only one was related to the outcome were generated. In the empirical data set, time and academic degree were positively correlated with staffs’ score. Our simulation study showed that the probability that autoregressive and unstructured correlation structures select wrong predictors as being significant is 1.3% and 3.7%. We concluded tha evaluation of staff by students improved the quality of education. In addition, selection of inappropriate correlation structure affects the significance of variables.
Resifar A, Khaghanizadeh M, Ebadi A, Masoumeh M. An overview of the current evaluation methods in universities. With emphasis on the evaluation of teachers by students. Iranian Bimonthly of Education Strategies in Medical Sciences 2008; 1(1):10-8. [In Persian].
Rajabi S, Papzan A. Combination of qualitative and quantitative research methods used to design a tool for evaluation of facultymmembers' performance case study: agricultural college, Razi University of Kermanshah province. Higher Education 2010; 3(1): 151-69. [In Persian].
Siamian H, Bala Ghafari A, Aligolbandi K, Rezanezhad SF, Sharifi Nick M, Shahrabi A, et al. Characteristics of a good university lecturer according to students. J Mazandaran Univ Med Sci 2013; 22(96): 106-13. [In Persian].
Aliasgharpour M, Monjamed Z, Bahrani N. Factors affecting students' evaluation of teachers: comparing viewpoints of teachers and students. Iran J Med Educ 2010; 10(2): 186-95. [In Persian].
Mianroodi AA, Asgari Baravat Z, Khanjani N. Explaining the pros and cons of different sources of faculty evaluation from the viewpoints of Medical University Academics. Strides Dev Med Educ 2012; 9(1): 65-77. [In Persian].
Shakurnia A, Motlagh ME, Malayeri AR, Jouhanmardi A, Komaili Sani H. Students' opinion on factors affecting faculty evaluation in Jondishapour Medical University. Iran J Med Educ 2005; 5(2): 101- 10. [In Persian].
Carle AC. Evaluating College Students' Evaluations of a Professor's Teaching Effectiveness across Time and Instruction Mode (Online vs. Face-to-Face) Using a Multilevel Growth Modeling Approach.mComputers & Education 2009; 53(2): 429-35.
Rafiei M, Mosayebi G. Results of six years] professors' evaluation in Arak University of Medical Sciences. J Arak Univ Med Sci 2010; 12(4): 52-62. [In Persian].
Nakai M, Ke W. Statistical models for longitudinal data analysis. Applied Mathematical Sciences, 2009; 3(40): 1979-89.
Twisk JWR. Applied longitudinal data analysis for epidemiology: a practical guide. 2nd ed. Cambridge, UK: Cambridge University Press; 2013.
Cui J, Qian G. Selection of working correlation structure and best model in gee analyses of longitudinal data. Communications in Statistics - Simulation and Computation 2007; 36(5): 987-96.
Burton P, Gurrin L, Sly P. Extending the simple linear regression model to account for correlated responses: an introduction to generalized estimating equations and multilevel mixed modelling. Statist Med 1998; 17: 1261-91.
Ghisletta P, Spini D. An introduction to generalized estimating equations and an application to assess selectivity effects in a longitudinal study on very old individuals. Journal of Educational and BehavioralStatistics 2004; 29(4): 421-37.
Hin LY, Wang YG. Working-correlationstructure identification in generalized estimating equations. Stat Med 2009; 28(4):642-58.
Jang MJ. Working correlation selection in generalized estimating equations [Thesis]. Iowa, IA: University of Iowa; 2011.
Shakournia A, Elhampour H, Mozafari A, Dashtbozorgi B. Ten year trends in facultymembers' evaluation results in Jondishapour University of Medical Sciences. Iran J Med Educ 2008; 7(2): 309-16. [In Persian].
Lesani M, Mohamadi V. Assess trend of teaching quality for faculty from the mathematics and computer school of Shahid Bahonar University of Kerman. Proceedings of the 5th Conference on Quality Assessment in University Systems; 2011 Apr 27; Tehran, Iran. [In Persian].
Ghafourian Boroujerdnia M, Shakurnia AH, Elhampour H. The opinions of academic members of Ahvaz University of MedicalSciences about the effective factors on their evaluation score variations. Strides Dev Med Educ 2006; 3(1): 19-25. [In Persian].
McPherson MA, Jewell RT, Kim M. Whatdetermines student evaluation scores? A random effects analysis of undergraduateeconomics classes. Eastern Economic Journal 2009; 35(1): 37-51.
Fattahi Z, Mousapour N, Haghdoost A. The trend of alterations in the quality of educational performance in faculty members of Kerman University of Medical Sciences. Strides Dev Med Educ 2005; 2(2): 63-71. [In Persian].
Ziegler A. Generalized estimating equations. New York, NY: Springer Science & Business Media; 2011.
Pan W. Akaikes information criterion in generalized estimating equations [Online]. [cited 2000]; Available from: URL: http://sph.umn.edu/faculty1/wpcontent/uploads/2012/11/rr2000-013.pdf
Wang YG, Hin LY. Modeling strategies in longitudinal data analysis: covariate, variance function and correlation structure selection. Computational Statistics & Data Analysis 2010; 54(12): 3359-70.
| Files | ||
| Issue | Vol 2 No 1 (2016) | |
| Section | Original Article(s) | |
| Keywords | ||
| longitudinal data generalized estimating equations correlation structure evaluation simulation study | ||
| Rights and permissions | |
|
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License. |
