Evaluating lecturer's capability over time. Some evidence from surveys on university course quality

The attention towards the evaluation of the Italian university system prompted to an increasing interest in collecting and analyzing longitudinal data on students’ assessments of courses, degree programs and faculties. This study focuses on students’ opinions gathered in three contiguous academic years. The main aim is to test a suitable method to evaluate lecturer’s performance over time considering students’ assessments on several features of the lecturer’s capabilities. The use of the same measurement instrument allows us to shed some light on changes that occur over time and to attribute them to specific characteristics. Multilevel analysis is combined with Item Response Theory in order to build up specific trajectories of performance of lecturer’s capability. The result is a random-effects ordinal regression model for four-level data that assumes an ordinal logistic regression function. It allows us to take into account several factors which may influence the variability in the assessed quality over time.