THE EFFECTIVE INTERACTION BETWEEN ASSESSMENT AND TECHNOLOGIES: FINDINGS FROM A SYSTEMATIC REVIEW

Main Article Content

Denise Tonelli
Valentina Grion
Anna Serbati

Abstract

Assessment is a widely discussed topic in educational research, due to its key role in learning processes. An emerging theme in the current debate is the need to adopt approaches more consistent with the pedagogical, cultural and technological developments concerning and determining teaching and learning activities. In this context, the interest in the relationship between assessment and technologies keeps growing. In order to investigate this relationship, this paper reviews the literature, researching ways in which digital technologies are used in schools and higher education contexts, trying to find elements of novelty, in comparison with traditional assessment practices. The findings highlight some important affordances of digital technologies in the assessment processes at different educational levels as well as in different subjects. Furthermore, the results showcase a large number of significant elements and new assessment scenarios, which are difficult to implement without the use of technologies.

Article Details

Section
Articles - Special Issue

References

Barrett, M.E., Swan, A.B., Mamikonian, A., Ghajoyan, I., Kramarova, O., & Youmans, R.J. (2014). Technology in Note Taking and Assessment: The Effects of Congruence on Student Performance. International Journal of Instruction, 7(1), 49-58.

Boubouka, M., & Papanikolau, K.A. (2013). Alternative assessment methods in technology enhanced project-based learning. International Journal of Learning Technology, 8(3), 263-296. doi: 10.1504/IJLT.2013.057063

Boud, D. (2000). Sustainable assessment: rethinking assessment for the learning society. Studies in Continuing Education, 22(2), 151-167. doi: 10.1080/713695728

Boud, D., & Soler, R. (2015). Sustainable assessment revisited. Assessment & Evaluation in Higher Education, 41(3), 400-413. doi: 10.1080/02602938.2015.1018133

Brantmeier, C., Vanderplank, R., & Strube, M. (2012). What about me? Individual Self-Assessment by Skill and Level of Language Instruction. System: An International Journal of Educational Technology and Applied Linguistics, 40(1) 144-160. doi:10.1016/j.system.2012.01.003

Cantillon, P., Irish, B., & Sales, D. (2004). Using computers for assessment in medicine. British Medical Journal, 329(7466), 606–609. doi: 10.1136/bmj.329.7466.606

Cisterna, D., Williams, M., & Merritt, J. (2013). Students’ Understanding of Cells & Heredity: Patterns of Understanding in the Context of a Curriculum Implementation in Fifth & Seventh Grades. American Biology Teacher, 75(3) 178-184. doi: 10.1525/abt.2013.75.3.6

Chigne, H.S., Labra Gayo, J.M., Alva Obeso, M.E., Ordonez De Pablo., P., & Cueva Lovelle, J.E. (2016). Towards the Implementation of the Learning Analytics in the Social Learning Environments for the Technology-Enhanced Assessment in Computer Engineering Education. International Journal of Engineering Education, 32(4), 1637-1646.

Conejo, R., Garcia-Vinas, J., Gastón, A., & Barros, B. (2016). Technology-Enhanced Formative Assessment of Plant Identification. Journal of Science Education & Technology, 25(2), 203-221. doi: 10.1007/s10956-015-9586-0

Conole, G., & Warburton, B. (2005) A review of computer-assisted assessment. ALT-J, 13(1), 17-31. doi: 10.1080/0968776042000339772

de Klerk, S., Veldkamp, B. P., & Eggen, T. J. (2015). Psychometric analysis of the performance data of simulation-based assessment: A systematic review and a Bayesian network example. Computers & Education, 85, 23-34. doi: 10.1016/j.compedu.2014.12.020

Debuse, J. C. W., & Lawley, M. (2016). Benefits and drawbacks of computer-based assessment and feedback systems: Student and educator perspectives. British Journal of Educational Technology, 47(2), 294-301. doi: 10.1111/bjet.12232

Deutsch, T., Hermann, K., Frese, T., & Sandholzer, H. (2011). Implementing computer-based assessment – A web-based mock examination changes attitudes. Computers and Education, 58(4), 1068-1075. doi: 10.1016/j.compedu.2011.11.013

Ferguson, R. (2012). Learning analytics: fattori trainanti, sviluppi e sfide. TD Tecnologie Didattiche, 22(3), 138-147. doi: 10.17471/2499-4324/183

Flanagan, E., & Hall. T. (2017). Digital Ensemble: The ENa CT design-based research framework for technology-enhanced embodied assessment in English education. English in Education, 51(1), 76-99. doi: 10.1111/eie.12136

Greiff, S., Wüstenberg, S., Holt, D., Goldhammer, F., & Funke, J. (2013). Computer-based assessment of Complex Problem Solving: concept, implementation, and application. Educational Technology Research & Development, 61(3), 407-421. doi: 10.1007/s11423-013-9301-x

Grion V., & Cesareni, D. (2016). Multiplicity, fluidity, dialogue and sharing: keywords to understand the complex dynamics between human learning and technology [Editorial]. QWERTY, 11(1), 5-10. Retrieved from http://www.ckbg.org/qwerty/index.php/qwerty/issue/view/34

Grion V., & Serbati A. (2017). Assessment for Learning in Higher Education. Nuove prospettive e pratiche di valutazione all’università. Lecce, IT: Pensa Multimedia.

Grion, V., Serbati, A. Tino, C., & Nicol, D. (2017). Ripensare la teoria della valutazione e dell’apprendimento all’università: un modello per implementare pratiche di peer review. Giornale Italiano della Ricerca Educativa, 10(19), 1-17.

Herborn, K., Mustafíc, M., & Greiff, S. (2017). Mapping an Experiment-Based Assessment of Collaborative Behavior onto Collaborative Problem Solving in PISA 2015: A Cluster Analysis Approach for Collaborator Profiles. Journal of Educational Measurement, 54(1) 103-122. doi: 10.1111/jedm.12135

Hettiatachchi, K.H.L.W., Huertas, M.A., & Mor, E. (2015). E-Assessment System for Skill and Knowledge Assessment in Computer Engineering Education. International Journal of Engineering Education, 31(2), 529-540.

Hols-Elders, W., Bloemendaal, P., Bos, N., Quaak, M., Sijstermans, R., & Jong, P. (2008). Twelve tips for computer-based assessment in medical education. Medical Teacher, 30, 673-678. doi: 10.1080/01421590802279595

Ifenthaler, D., (2014). Toward Automated Computer-Based Visualization and Assessment of Team-Based Performance. Journal of Educational Psychology, 106(3), 651-66. doi: 10.1037/a0035505

Jurado, F., Redondo, M., & Ortega, M. (2014). Elearning standards and automatic assessment in a distributed eclipse based environment for learning computer programming. Computer Applications in Engineering Education, 22(4), 774-787. doi: 10.1002/cae.21569

Kalogeropoulos, N., Tzigounakis, I., Pavlatou, E. A., & Boudouvis, A.G. (2013). Computer-based assessment of student performance in programing courses. Computer Applications in Engineering Education, 21(4), 671-683. doi: 10.1002/cae.20512

Kemp, J., Mellor, A., Kotter, R., & Oosthoek, J.W. (2012). Student-Produced Podcasts as an Assessment Tool: An Example from Geomorphology. Journal of Geography in Higher Education, 36(1), 117-130. doi: 10.1080/03098265.2011.576754

Kim, J. (2015). A Study of Perceptional Typologies on Computer Based Assessment (CBA): Instructor and Student Perspectives. Journal of Educational Technology & Society, 18(2), 80-96.

Lin, J.W. (2016). Development and Evaluation of the Diagnostic Power for a Computer-Based Two-tier Assessment. Journal of Science Education and Technology, 25(3), 497-511. doi: 10.1007/s10956-016-9609-5

McKenna, C. (2000). Using computers to assess humanities: Some results from the national survey into the use of Computer Assisted Assessment (CAA). Computers & Texts. Web Supplement, 18/19, A6-A7.

Neumann, D.L., Hood, M., & Neumann, M.M. (2012). Evaluation of Computer-Based Interactive Simulations in the Assessment of Statistical Concepts. International Journal for Technology in Mathematics Education, 19(1), 17-24.

Newhouse, C.P. (2013). Computer-Based Exams in Schools: Freedom from the Limitations of Paper? Research and Practice in Technology Enhanced Learning, 8(3) 431-447.

Nguyen, Q., Rienties, B., Toetenel, L., Ferguson, R., & Whitelock, D. (2017). Examining the designs of computer-based assessment and its impact on student engagement, satisfaction, and pass rates. Computers in Human Behavior, 76, 703-714. doi: 10.1016/j.chb.2017.03.028

Nikou, S.A, & Economides, A.A. (2016). The impact of paper-based, computer-based and mobile-based self-assessment on students’ science motivation and achievement. Computers in Human Behavior, 55, 1241-1248. doi: 10.1016/j.chb.2015.09.025

Nouri, J., Åkerfeldt, A., Fors, U., & Selander, S. (2017). Assessing Collaborative Problem Solving Skills in Technology-Enhanced Learning Environments – The PISA Framework and Modes of Communication. International Journal of Emerging Technologies in Learning, 12(4), 163-174. https://doi.org/10.3991/ijet.v12i04.6737

Odo, D.M. (2012). Computer Familiarity and Test Performance on a Computer-Based Cloze ESL reading Assessment. Teaching English with Technology, 12(3), 18-35.

Oldfield, A., Broadfoot, P., Sutherland, R., & Timmis, S. (2012). Assessment in a Digital Age: A research review. Retrieved from http://www.bristol.ac.uk/education/research/sites/tea/publications/index.html

Osborne, R., Dunne, E., & Farrand, P. (2013). Integrating technologies into “authentic” assessment design: an affordances approach. Research in Learning Technology, 21, 1-18. doi: 10.3402/rlt.v21i0.21986

Perrotta, C. (2014). Innovation in technology-enhanced assessment in the UK and the USA: Future scenarios and critical considerations. Technology, Pedagogy and Education, 23(1), 103-119. doi: 10.1080/1475939X.2013.838453

Race, P., Brown, S. & Smith, B. (2005). 500 tips on assessment (2nd edition). New York, NY, USA: Routledge Falmer.

Quellmalz, E.S., Davenport, J., Timms, M.J., Buckley, B.C., De Boer, G., Jordan, K.A., & Huang, C. (2013). Next-Generation Environments for Assessing and Promoting Complex Science Learning. Journal of Educational Psychology, 105(4), 110-114. doi: 10.1037/a0032220

Robles, D.C., Angulo, J.S., & De La Serna, M. C. (2014). Federated eRubric Service to Facilitate Self-Regulated Learning in the University Model. European Educational Research Journal, 13(5), 575-584.

Rosen, Y. (2017). Assessing Students in Human-to-Agent Settings to Inform Collaborative Problem-Solving Learning. Journal of Educational Measurement, 54(1), 36-53. doi: 10.1111/jedm.12131

Rosen, Y., & Foltz, P.W. (2014). Assessing Collaborative Problem Solving through Automated Technologies. Research & Practice in Technology Enhanced Learning, 9(3), 389-410.

Sheard, M.K., & Chambers, B. (2014). A case of technology-enhanced formative assessment and achievement in primary grammar: How is quality assurance of formative assessment assured? Studies in Educational Evaluation, 43, 14-23. doi: 10.1016/j.stueduc.2014.02.001

Shute, V. J., & Rahimi S. (2016). Review of computer‐based assessment for learning in elementary and secondary education. Journal of Computer Assisted Learning, 33(1), 1-19. doi: 10.1111/jcal.12172

Siddiq, F., & Scherer, R. (2017). Revealing the process of students’ interaction with a novel collaborative problem solving task: An in-depht analysis of think-aloud protocols. Computers in Human Behavior, 76, 509-525. doi: 10.1016/j.chb.2017.08.007

Sim, G., Holifield, P., & Brown, M. (2004). Implementation of computer assisted assessment: Lessons from the literature. Research in Learning Technology, 12(3), 215-229. doi: 10.3402/rlt.v12i3.11255

Stobart, G. (2008). Testing times, the uses and abuses of assessment. Abingdon, UK: Routledge.

Struyven K., Dochy F., & Janssens, S. (2005). Students’ perceptions about evaluation and assessment in higher education: a review. Assessment & Evaluation in Higher Education, 30(4), 325-341. doi: 10.1080/02602930500099102

Sweeney, T., West, D., Groessler, A., Haynie, A., Higgs, B., Macaulay, J., … Yeo, M. (2017). Where’s the transformation? Unlocking the potential of technology-enhanced assessment. Teaching & Learning Inquiry, 5(1). doi: 10.20343/5.1.5

Tang, J., Rich, C.S., & Wang, Y. (2012). Technology-enhanced English Language Writing Assessment in the Classroom. Chinese Journal of Applied Linguistics (De Gruyter), 35(4), 385-399. doi: 10.1515/cjal-2012-0029

Timmis, S., Broadfoot, P., Sutherland, R., & Oldfield, A. (2016), Rethinking assessment in a digital age: opportunities, challenges and risks. British Journal of Educational Technology, 42(3), 454–476. doi: 10.1002/berj.3215

Trinchero, R. (in press). Valutazione formante per l’attivazione cognitiva. Spunti per un uso efficace delle tecnologie per apprendere in classe. Italian Journal of Educational Technology. doi: 10.17471/2499-4324/1013.

van der Kleij, F., Feskens, R. C., & Eggen, T. J. (2015). Effects of feedback in a computer based learning environment on students’ learning outcomes: A meta-analysis. Review of Educational Research, 85(4), 475-511. doi: 10.3102/0034654314564881

van der Kleij, F., Timmers, C., & Eggen, T. (2011). The effectiveness of methods for providing written feedback through a computer-based assessment for learning: A systematic review. CADMO, 19(1), 21-38. doi: 10.3280/CAD2011-001004

Wang, Y. (2014). Use of Interactive Web-Based Exercises for English as a Foreign Language Learning: Learners’ Perceptions. Teaching English with Technology, 14(3) 16-29.

Wilson, J., Olinghouse, N.G., & Andrada, G.N. (2014). Does Automated Feedback Improve Writing Quality. Learning Disabilities – A Contemporary Journal, 12(1), 93-118.

Word Economic Forum. (2016a). The future of jobs: Employment, skills and workforce strategy for the fourth industrial revolution. Geneva, CH: World Economic Forum.

Word Economic Forum. (2016b). New Vision for Education: Fostering Social and Emotional Learning through Technology. Geneva, CH: World Economic Forum.

Yurdabakan, I., & Uzunkavak, C. (2012). Primary School Students’ Attitudes towards Computer based Testing and Assessment in Turkey. Turkish Online Journal of Distance Education, 13(3), 177-188.