INDICATORS FOR RANKING ONLINE UNIVERSITIES: THE STUDENTS’ PERSPECTIVE
Article Sidebar
Published:
Dec 30, 2019
Keywords:
Online Universities, Ranking, Evaluation, Quality in Online Learning, Educational Technology, Technology Enhanced Learning (TEL)
Main Article Content
Abstract
University ranking systems are frequently used to assess and compare universities, and are of particular interest to prospective students. Current ranking systems, however, do not assess the currently growing online dimension of universities. To overcome this limitation, the European Project Creating an Online Dimension for University Rankings (CODUR) engaged online teaching experts to identify criteria and indicators for assessing this online dimension. This paper compares these evaluations with those provided by 55 university students attending courses online. The comparison shows that the expert and student perspectives are largely similar, although with some important differences. Specifically, compared to the experts, the students regard quality of student support as more important, and quality of the learning experience as less important. These results represent a first step towards integrating students’ perspective in the online dimension of university rankings.
Article Details
Section
Articles - Dossier
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons CC BY 4.0 Attribution 4.0 International License.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access)
References
Abramo, G., & D'Angelo, C. A. (2015). The VQR, Italy's second national research assessment: Methodological failures and ranking distortions. Journal of the Association for Information Science and Technology, 66(11), 2202-2214. doi: 10.1002/asi.2332
Amsler, S. S., & Bolsmann, C. (2012). University ranking as social exclusion. British Journal of Sociology of Education, 33(2), 283-301. doi: 10.1080/01425692.2011.649835
Aversano, N., Manes-Rossi, F., & Tartaglia-Polcini, P. (2018). Performance measurement systems in universities: a critical review of the Italian system. In E. Borgonovi, E. Anessi-Pessina, C. Bianchi, (Eds.), Outcome-Based Performance Management in the Public Sector. System Dynamics for Performance Management, vol. 2. Cham, CH: Springer. doi: 10.1007/978-3-319-57018-1_14
Barron, G. R. (2017). The Berlin Principles on ranking higher education institutions: Limitations, legitimacy, and value conflict. Higher Education, 73(2), 317-333. doi: 10.1007/s10734-016-0022-z
Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society. Series B (Methodological), 57(1), 289-300. doi: 10.1111/j.2517-6161.1995.tb02031.x
Billaut, J. C., Bouyssou, D., & Vincke, P. (2009). Should you believe in the Shanghai ranking? An MCDM view. Scientometrics, 84(1), 237-263. doi: 10.1007/s11192-009-0115-x
Bonaccorsi, A., & Cicero, T. (2016). Nondeterministic ranking of university departments. Journal of Informetrics, 10(1), 224-237. doi: 10.1016/j.joi.2016.01.007
Bougnol, M. L., & Dulá, J. H. (2015). Technical pitfalls in university rankings. Higher Education, 69(5), 859-866. doi: 10.1007/s10734-014-9809-y
Bowden, R. (2000). Fantasy higher education: University and college league tables. Quality in Higher Education, 6(1), 41-60. doi: 10.1080/13538320050001063Q
Bowman, N. A., & Bastedo, M. N. (2011). Anchoring effects in world university rankings: Exploring biases in reputation scores. Higher Education, 61(4), 431-444. doi: 10.1007/s10734-010-9339-1
Brasher, A., Holmes, & Whitelock, D. (2017). A means for systemic comparisons of current online education quality assurance tools and systems. CODUR Intellectual Output IO1.A2. Retrieved from http://edulab.uoc.edu/wp-content/uploads/2018/06/CODUR-deliverable-IO1-A2.pdf
Çakır, M. P., Acartürk, C., Alaşehir, O., & Çilingir, C. (2015). A comparative analysis of global and national university ranking systems. Scientometrics, 103(3), 813-848. doi: 10.1007/s11192-015-1586-6
Cook-Sather, A. (2006). Sound, presence, and power: “Student voice” in educational research and reform. Curriculum Inquiry, 36(4), 359-390. doi: 10.1111/j.1467-873X.2006.00363.x
Cremonini, L., Westerheijden, D. F., Benneworth, P., & Dauncey, H. (2014). In the shadow of celebrity? World-class university policies and public value in higher education. Higher Education Policy, 27(3), 341-361. doi: 10.1057/hep.2013.33
Cremonini, L., Westerheijden, D., & Enders, J. (2008). Disseminating the right information to the right audience: Cultural determinants in the use (and misuse) of rankings. Higher Education, 55(3), 373–85. doi: 10.1007/s10734-007-9062-8
Ehlers, U.-D. (2006). Learners as active stakeholders of eLearning quality, EFQUEL Green paper Series. doi: 10.13140/RG.2.2.17720.14086
Ghislandi, P.M.M., & Raffaghelli, J. (2013). La voce degli studenti per la qualità dell’eLearning nella formazione universitaria: un approccio partecipativo. In V. Grion & A. Cook- Sather (Eds.), Joining the movement: bringing student voice to educational theory and practice in Italy (pp. 273–286). Milano, IT: Guerini & Associati.
Johnson, J. H. (1991). Student voice motivating students through empowerment (No. ED337875). Eugene, OR, USA: Oregon School Study Council.
King, B. (2012). Distance education and dual-mode universities: An Australian perspective. Open Learning: The Journal of Open, Distance and e-Learning, 27(1), 9-22. doi: 10.1080/02680513.2012.640781
Kroth, A., & Daniel, H. D. (2008). Internationale Hochschulrankings. Zeitschrift für Erziehungswissenschaft, 11(4), 542. doi: 10.1007/s11618-008-0052-0
Kurre, F. L., Ladd, L., Foster, M. F., Monahan, M. J., & Romano, D. (2012). The state of higher education in 2012. Contemporary Issues in Education Research (CIER), 5(4), 233-256. doi: 10.19030/cier.v5i4.7268
Li, K. C. (2018). The evolution of open learning: A review of the transition from pre-e-learning to the era of e-learning. Knowledge Management & E-Learning: An International Journal, 10(4), 408-425. doi: 10.34105/j.kmel.2018.10.025
Lynch, K. (2015). Control by numbers: New managerialism and ranking in higher education. Critical Studies in Education, 56(2), 190-207. doi: 10.1080/17508487.2014.949811
Marginson, S. (2007). The public/private divide in higher education: A global revision. Higher Education, 53(3), 307-333. doi: 10.1007/s10734-005-8230-y
McAleese, M., Bladh, A., Bode, C., Muehlfeit, J., Berger, V., & Petrin, T., ... Tsoukalis, L. (2013). Improving the quality of teaching and learning in Europe’s higher education institutions. High level group on the modernisation of higher education [Report to the European Commission] (p. 84). Luxembourg, LU: Publication Office of the European Union.
Pipitone, V., Fulantelli, G., & Allegra, M. (2004). Students perception on e-learning: a case-study. Academic Exchange Quarterly, 8(1), 261-267.
Pozzi, F., Manganello, F. Passarelli, M., & Persico, D. (2017). Develop test and refine representative performance online quality education indicators based on common criteria. CODUR Intellectual Output IO1.A3. Retrieved from http://edulab.uoc.edu/wp-content/uploads/2018/06/CODUR-deliverable-IO1-A3.pdf
Salmi, J., & Saroyan, A. (2007). League tables as policy instruments. Higher Education Management and Policy, 19(2), 1-38.doi: 10.1787/17269822
Schnelle, E. (1979). The MetaPlan method: Communication tools for planning and learning groups. Hamburg, DE: Quickborn.
Sponsler, B. (2009). The role and relevance of rankings in higher education policymaking. Washington, DC: Institute for Higher Education Policy.
Thurstone, L. L. (1927). A law of comparative judgment. Psychological Review, 34(4), 273. doi: 10.1037/h0070288
Turner, D. A. (2014). World class universities and international rankings. Ethics in Science and Environmental Politics, 13(2), 167-176.doi: 10.3354/esep00132
Usher, A., & Savino, M. (2006). A world of difference: A global survey of University league tables. Toronto, ON: Educational Policy Institute.
Van Dyke, N. (2005). Twenty years of university report cards. Higher Education in Europe, 30(2), 103-125. doi: 10.1080/03797720500260173
Vlăsceanu, L., Grünberg, L., & Pârlea, D. (2004). Quality assurance and accreditation: A glossary of basic terms and definitions. Bucharest, RO: Unesco-Cepes.
Amsler, S. S., & Bolsmann, C. (2012). University ranking as social exclusion. British Journal of Sociology of Education, 33(2), 283-301. doi: 10.1080/01425692.2011.649835
Aversano, N., Manes-Rossi, F., & Tartaglia-Polcini, P. (2018). Performance measurement systems in universities: a critical review of the Italian system. In E. Borgonovi, E. Anessi-Pessina, C. Bianchi, (Eds.), Outcome-Based Performance Management in the Public Sector. System Dynamics for Performance Management, vol. 2. Cham, CH: Springer. doi: 10.1007/978-3-319-57018-1_14
Barron, G. R. (2017). The Berlin Principles on ranking higher education institutions: Limitations, legitimacy, and value conflict. Higher Education, 73(2), 317-333. doi: 10.1007/s10734-016-0022-z
Benjamini, Y., & Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society. Series B (Methodological), 57(1), 289-300. doi: 10.1111/j.2517-6161.1995.tb02031.x
Billaut, J. C., Bouyssou, D., & Vincke, P. (2009). Should you believe in the Shanghai ranking? An MCDM view. Scientometrics, 84(1), 237-263. doi: 10.1007/s11192-009-0115-x
Bonaccorsi, A., & Cicero, T. (2016). Nondeterministic ranking of university departments. Journal of Informetrics, 10(1), 224-237. doi: 10.1016/j.joi.2016.01.007
Bougnol, M. L., & Dulá, J. H. (2015). Technical pitfalls in university rankings. Higher Education, 69(5), 859-866. doi: 10.1007/s10734-014-9809-y
Bowden, R. (2000). Fantasy higher education: University and college league tables. Quality in Higher Education, 6(1), 41-60. doi: 10.1080/13538320050001063Q
Bowman, N. A., & Bastedo, M. N. (2011). Anchoring effects in world university rankings: Exploring biases in reputation scores. Higher Education, 61(4), 431-444. doi: 10.1007/s10734-010-9339-1
Brasher, A., Holmes, & Whitelock, D. (2017). A means for systemic comparisons of current online education quality assurance tools and systems. CODUR Intellectual Output IO1.A2. Retrieved from http://edulab.uoc.edu/wp-content/uploads/2018/06/CODUR-deliverable-IO1-A2.pdf
Çakır, M. P., Acartürk, C., Alaşehir, O., & Çilingir, C. (2015). A comparative analysis of global and national university ranking systems. Scientometrics, 103(3), 813-848. doi: 10.1007/s11192-015-1586-6
Cook-Sather, A. (2006). Sound, presence, and power: “Student voice” in educational research and reform. Curriculum Inquiry, 36(4), 359-390. doi: 10.1111/j.1467-873X.2006.00363.x
Cremonini, L., Westerheijden, D. F., Benneworth, P., & Dauncey, H. (2014). In the shadow of celebrity? World-class university policies and public value in higher education. Higher Education Policy, 27(3), 341-361. doi: 10.1057/hep.2013.33
Cremonini, L., Westerheijden, D., & Enders, J. (2008). Disseminating the right information to the right audience: Cultural determinants in the use (and misuse) of rankings. Higher Education, 55(3), 373–85. doi: 10.1007/s10734-007-9062-8
Ehlers, U.-D. (2006). Learners as active stakeholders of eLearning quality, EFQUEL Green paper Series. doi: 10.13140/RG.2.2.17720.14086
Ghislandi, P.M.M., & Raffaghelli, J. (2013). La voce degli studenti per la qualità dell’eLearning nella formazione universitaria: un approccio partecipativo. In V. Grion & A. Cook- Sather (Eds.), Joining the movement: bringing student voice to educational theory and practice in Italy (pp. 273–286). Milano, IT: Guerini & Associati.
Johnson, J. H. (1991). Student voice motivating students through empowerment (No. ED337875). Eugene, OR, USA: Oregon School Study Council.
King, B. (2012). Distance education and dual-mode universities: An Australian perspective. Open Learning: The Journal of Open, Distance and e-Learning, 27(1), 9-22. doi: 10.1080/02680513.2012.640781
Kroth, A., & Daniel, H. D. (2008). Internationale Hochschulrankings. Zeitschrift für Erziehungswissenschaft, 11(4), 542. doi: 10.1007/s11618-008-0052-0
Kurre, F. L., Ladd, L., Foster, M. F., Monahan, M. J., & Romano, D. (2012). The state of higher education in 2012. Contemporary Issues in Education Research (CIER), 5(4), 233-256. doi: 10.19030/cier.v5i4.7268
Li, K. C. (2018). The evolution of open learning: A review of the transition from pre-e-learning to the era of e-learning. Knowledge Management & E-Learning: An International Journal, 10(4), 408-425. doi: 10.34105/j.kmel.2018.10.025
Lynch, K. (2015). Control by numbers: New managerialism and ranking in higher education. Critical Studies in Education, 56(2), 190-207. doi: 10.1080/17508487.2014.949811
Marginson, S. (2007). The public/private divide in higher education: A global revision. Higher Education, 53(3), 307-333. doi: 10.1007/s10734-005-8230-y
McAleese, M., Bladh, A., Bode, C., Muehlfeit, J., Berger, V., & Petrin, T., ... Tsoukalis, L. (2013). Improving the quality of teaching and learning in Europe’s higher education institutions. High level group on the modernisation of higher education [Report to the European Commission] (p. 84). Luxembourg, LU: Publication Office of the European Union.
Pipitone, V., Fulantelli, G., & Allegra, M. (2004). Students perception on e-learning: a case-study. Academic Exchange Quarterly, 8(1), 261-267.
Pozzi, F., Manganello, F. Passarelli, M., & Persico, D. (2017). Develop test and refine representative performance online quality education indicators based on common criteria. CODUR Intellectual Output IO1.A3. Retrieved from http://edulab.uoc.edu/wp-content/uploads/2018/06/CODUR-deliverable-IO1-A3.pdf
Salmi, J., & Saroyan, A. (2007). League tables as policy instruments. Higher Education Management and Policy, 19(2), 1-38.doi: 10.1787/17269822
Schnelle, E. (1979). The MetaPlan method: Communication tools for planning and learning groups. Hamburg, DE: Quickborn.
Sponsler, B. (2009). The role and relevance of rankings in higher education policymaking. Washington, DC: Institute for Higher Education Policy.
Thurstone, L. L. (1927). A law of comparative judgment. Psychological Review, 34(4), 273. doi: 10.1037/h0070288
Turner, D. A. (2014). World class universities and international rankings. Ethics in Science and Environmental Politics, 13(2), 167-176.doi: 10.3354/esep00132
Usher, A., & Savino, M. (2006). A world of difference: A global survey of University league tables. Toronto, ON: Educational Policy Institute.
Van Dyke, N. (2005). Twenty years of university report cards. Higher Education in Europe, 30(2), 103-125. doi: 10.1080/03797720500260173
Vlăsceanu, L., Grünberg, L., & Pârlea, D. (2004). Quality assurance and accreditation: A glossary of basic terms and definitions. Bucharest, RO: Unesco-Cepes.