Taking into account the students’ evaluation of the quality of degree programs this paper presents a proposal for building up an adjusted performance indicator based on Latent Class Regression Analysis. The method enables us (i) to summarize in a single indicator statement multiple facets evaluated by students through a survey questionnaire and (ii) to control the variability in the evaluations that is mainly attributable to the characteristics (often referred as the Potential Confounding Factors) of the evaluators (students) rather than to real differences in the performances of the degree programs under evaluation. A simulation study is implemented in order to test the method and assess its potential when the composition of the degree programs as regards to students’ characteristics is sensibly different between one another. Results suggest that when the evaluations are strongly affected by the students’ covariates, the assessment based on the value of an unadjusted indicator can lead to bias and unreliable conclusions about the differences in performance. An application to real data is also provided.

Comparing degree programs from students’ assessments: A LCRA-based adjusted composite indicator

SULIS, ISABELLA;PORCU, MARIANO
2012

Abstract

Taking into account the students’ evaluation of the quality of degree programs this paper presents a proposal for building up an adjusted performance indicator based on Latent Class Regression Analysis. The method enables us (i) to summarize in a single indicator statement multiple facets evaluated by students through a survey questionnaire and (ii) to control the variability in the evaluations that is mainly attributable to the characteristics (often referred as the Potential Confounding Factors) of the evaluators (students) rather than to real differences in the performances of the degree programs under evaluation. A simulation study is implemented in order to test the method and assess its potential when the composition of the degree programs as regards to students’ characteristics is sensibly different between one another. Results suggest that when the evaluations are strongly affected by the students’ covariates, the assessment based on the value of an unadjusted indicator can lead to bias and unreliable conclusions about the differences in performance. An application to real data is also provided.
University evaluation; Adjusted indicators; Latent class regression analysis; Potential confounding factors
File in questo prodotto:
File Dimensione Formato  
2012a.pdf

Solo gestori archivio

Dimensione 293.01 kB
Formato Adobe PDF
293.01 kB Adobe PDF   Visualizza/Apri   Richiedi una copia

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: http://hdl.handle.net/11584/30880
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 2
  • ???jsp.display-item.citation.isi??? 2
social impact