Lecturer in Sociology and Social Policy
The digital collaboration I undertook was the use of Turnitin to electronically grade and provide audio feedback to students on their assessments. It was with Damien Raftery of our Teaching and Learning Centre in the academic year 2017/2018. Over the years I found that despite the fact, I had spent hours grading and providing feedback on assignments many students failed to attend for feedback. Those who did attend appeared to have difficulty fully understanding the feedback. I felt that a number of factors may have contributed to this such as illegible handwriting, lack of detail and/or the short period of time they had to review the feedback in class. (The assignment with the feedback and marking sheet are retained by the lecturer as they must be made available to the external examiner at the end of the academic year. I did note that increasingly students were taking photographs of the feedback on their assignment in an attempt to overcome this issue). As a result, I observed a significant number of students submitting assignments year in year out with many of the same errors, and many saw little or no improvement in their grade from one assignment to the next. On some occasions when students did attend for feedback, a small minority of students became emotional (upset, disappointment and/or anger) and this appeared to impede their ability to take the feedback on board. Sometimes this impacted on me emotionally and as a result this had the potential to influence my objectiveness when grading their future assignments. Therefore, I was looking for a way to encourage students to become involved in the feedback process, and to take ownership and responsibility for their own learning. I also wished to take some of the emotion out of the feedback process both for the student and for myself as the lecturer. My current teaching includes year 1, year 2 and year 4 on the Applied Social Care Degree, students may exit at year 3 with a Level 7 degree or Year 4 with a Level 8 degree. I also teach a module on the MA in Child, Youth and Family. For the purpose of this collaboration I decided to pilot electronic grading and audio feedback with year 1, with year 4 and with the MA students. I chose the year 1 as I have a very large number of students in the class (over 100) and I wanted to see if grading assignments electronically and providing the audio feedback would be time efficient or more time consuming than my traditional method of marking and providing feedback. I also found that year 1 students require more in-depth feedback. I wanted my students’ to become engaged in the feedback process from the outset of their higher education and for them to see the value of feedback. I wanted to encourage them to place greater emphasis on the feedback rather than the grade value. I chose year 4 as they have had three years of receiving feedback in a more traditional manner and therefore they were best placed to provide me with a comparative overview of their experience of receiving traditional feedback versus the electronic and audio feedback. Finally I chose the MA students as this is the smallest cohort of students I teach (six to eight students on average each year) and I wanted to give them an opportunity to submit a draft piece of work, receive detailed formative feedback and then submit the final submission. By assessing their work electronically, giving the audio feedback on the draft piece, and then getting the students to submit their final submission having had the benefit of the electronic and audio feedback, I could measure if the students had engaged with the feedback and implemented it into their final submission. Damien and I began by discussing the use of audio feedback and he directed me toward the audio function which is available within Turnitin. Turnitin also provided me with an opportunity to give written electronic feedback as well as up to three minutes of audio feedback. Students were already familiar with submitting their work into Turnitin which is embedded into the VLE. Damien provided me with practical support and advice throughout the year, from the initial stages where he showed me how to use Turnitin to provide electronic and audio feedback, to ensuring that I had the equipment I needed to provide the audio feedback such as a headset and an iPad. Throughout the “settling in” period of trying to get students to engage with the electronic and audio feedback, Damien and I had numerous conversations regarding how it was working. Toward the end of the last academic year (2017/2018), we collaborated on the evaluation of using the electronic and audio feedback as a teaching aid; we applied for, and were granted, ethical approval. Specifically Damien advised around survey design and the use of Qualtrics for data collection and analysis. Once the data was collected we also collaborated on the presentation of this information at the EdTech 2018 conference held here in IT Carlow. I presented on my rational for using electronic and audio feedback as well as providing an overview of my experience and the students’ evaluation of the process from their perspective. Participating at EdTech2018 also provided Damien and myself with an opportunity for discussion with colleagues in other academic settings. This analysis was also presented at the Dariah Ireland 2018 conference, where I sat on a panel with a number of others involved in the SPEEDS project to discuss the findings with other interested parties.
As I had not previously used Turnitin feedback studio, I attended a workshop which was facilitated by Damien, in this workshop he demonstrated how to use the Turnitin feedback studio. We had an opportunity to ask questions and practice using the different functions within the Turnitin feedback studio. When I began to use Turnitin feedback studio, I discovered that there were a few practicalities I needed to consider. For example when grading the work in the office I needed headphones and microphone, to cancel out noise and to record the audio. I also discovered it was less time consuming and more efficient if I had the Turnitin feedback studio open on two screens when I was grading an assignment. For example I would read the students work and insert my electronic comments/feedback within the Turnitin feedback studio on my PC, and simultaneously I had Turnitin feedback studio open on the iPad. This allowed me to review my comments as I recorded the audio feedback. I felt this help me give clearer and more comprehensive feedback when recording the audio feedback as it allowed me to point out particular issues and examples. It should also be noted to access the Turnitin Feedback Studio you must have access to the VLE and an internet connection. To encourage and motivate my students to engage with the feedback process, I did some preparatory work with the students’ in the classroom. I discussed with them the importance of feedback and the value of engaging and learning from their feedback. With the first year group I provided a demonstration on where they would find the Turnitin on the VLE, how to upload their work. After their work had been graded I provided a demonstration to all of the groups on how to access and listen to the feedback. I did this by uploading a sample assignment with electronic and audio feedback, which we viewed and listened to in class so students were clear as to where and how to access their feedback. I advised all students to access the Turnitin Feedback Studio via a PC or tablet/iPad with internet connection, some students indicated that they were also able to access the audio feedback via the VLE app on their phones, others were not.
From the outset of this project, I began examining some of the literature available on the implementation and use of audio feedback. According to McCarthy (2015, p. 153) assessment and feedback is central to student learning in higher education and effective feedback is essential “as the scaffolding that enhances learning”. He stresses that despite current literature outlining the importance of high quality and timely feedback, there are many obstacles to the delivery of such feedback. The literature in this area indicates that one of the major obstacles is engaging students in the assessment feedback process. Students often find that the feedback they receive is, too late, too vague and unclear due to illegible handwriting. This results in the students misunderstanding the feedback and or failing to engage with it and as a result student simply ignore the feedback provided (ibid, 2015). This is in direct contrast to audio feedback where research under taken by Lunt and Curran (2010) found that students were ten times more likely to download an audio file online than collect their written feedback and students on average accessed their audio up to 3- 4 times on average. The literature I examined was overwhelmingly positive about the use and impact of audio feedback. Research conducted by Ice et al. (2007) found that students perceive audio feedback as more supportive and caring than the more traditional approach of written feedback, reporting a greater understanding of the feedback. Further research conducted by Merry and Osmond (2007), King et al. (2008), and Hennessy and Foster (2014) also found this to be the case. In addition, Northcliffe and Middleton (2008) indicate that the tone and expression of the lecturer providing the feedback both to the depth of communication between the lecturer and the student. Simply referring to students by their name during feedback facilitates a deeper connection between the student and the lecturer (Ice et al., 2007). Värlander (2008) argues that the social interaction between the lecturer and the student has power at its core. She argues, that as a result, giving and receiving feedback can often awake emotions such as pride or shame as well as guilt and anxiety in one or both parties. Students’ emotions greatly influence the way in which they are able to receive and process feedback. Feedback can influence how a students’ feel about their about their academic ability (positively and negatively) and what and how they can learn (Dweck, 1999), and sometimes the value of the feedback may be ‘eclipsed by learner’s reactions’ to it (Race, 1995 as cited in Värlander, 2008, p. 145). However, Voelkel and Mello (2014) indicate that it’s easier for students to hear critical feedback than to read it. “Judging one’s own work accurately and dispassionately requires substantial personal detachment from it “(Sadler 2013, p. 55). Audio feedback gives lecturers a platform to provide a comprehensive and sensitive approach to providing feedback. It should be noted that when using audio feedback the lecturer must be mindful of their tone of voice as frustration (at a poor attempt) or weariness (having corrected large numbers) can be demotivating for a student. Sadler (2013) argues that the role of the lecturer is not to critique and offer advise on how to improve but rather it is to teach students how to assess the quality of their work and modify it as they do it. Nicol & Macfarlane-Dick (2007) also stress the importance of formative assessment and feedback in supporting students take control of their own learning and becoming self-regulated learners.
As well as local videos and handouts on using Turnitin Feedback Studio, I found helpful resources from National Forum projects such as Transformation through Collaboration (http://www.digitalchampions.ie), Yr1Feedback (http://y1feedback.ie/), TELU (http://telu.me), and All Aboard (http://www.allaboardhe.ie).
Evaluating and assessing student course work is a fundamental responsibility of lecturers. This process of providing and receiving feedback can be an emotionally charged experience for both the student and the lecturer alike. For the past eight years I have been teaching large groups of between 60-150 students on average. One of the major challenges I face each year is providing timely and comprehensive feedback to my students. As previously mentioned, traditionally I have done this via marking sheets and providing student with an opportunity to view their assignment feedback and ask any questions they may have regarding the same. However it has been my experience that large numbers of students failed to avail of this opportunity and usually only students who had done very well or students who wished to question their grade show up to the feedback sessions. Therefore I was searching for a way to reach out directly to each student in a more positive and personalised way, and to try and engage as many of them as possible in the feedback process. I found that by using audio feedback I was able to speak directly to students. I was able to provide them with clear and comprehensive feedback that they could access time and time again. Also via the audio feedback I encouraged students to place more emphasis on engaging with the feedback rather than the grade. I attempted to encourage this by releasing the audio and electronic feedback to the students two days before I made their grades available. I was also able to monitor via Turnitin who had reviewed the feedback. Electronic and audio feedback also provided me with an opportunity to provide my students with feed forward. Prior to engaging with this project previous the MA students wrote an assignment weighted at 60%. I would grade it and provide feedback, however the feedback was redundant as this programme runs for one year, so they had no opportunity to use the feedback I provided to improve and enhance their grade. Therefore by providing these students with an opportunity to submit a draft assignment, the feedforward provided each student with an opportunity to engage with, process and implement the feedback into their final submission which would then be graded. This gave the students an opportunity to improve and enhance their submission and ultimately their grades. This did in fact happen, the majority of the final submission were substantially better than the original draft.
To gauge student feedback on the use of audio and electronic feedback, Damien and I developed an online questionnaire through Qualtrics. Qualtrics was new software being piloted, so Damien began by explaining how it worked, and how I could use it to collect and analyse data. The questionnaire was sent to 1st and 4th year students through a link in the VLE in April 2018. Over a 12-day period 55 students responded. The feedback was easy to find in Turnitin/Blackboard (n=55) The feedback was useful in helping me know how to improve on future assignments (n=55) Audio feedback is more personal than written feedback (n=50) For future assignments, I would like to get audio feedback again (n=50) The audio feedback was as good as an individual face-to-face feedback session with my lecturer (n=50) When asked for a comment on what students found to be the benefits of audio and electronic feedback, positive student responses included: • “Audio feedback is highly beneficial and allows students to feel more satisfied in understanding what they need to work on to improve in the future and why they got the given mark” • “I feel when students are given their assignments for 10-15 minutes during class we may feel more under pressure to accomplish a better grade as our work is compared to the other students in the classroom and then if we see that we may not be doing as well as other friends or classmates it may cause a more negative feeling as students could feel embarrassed that they are not doing as well as they had hoped and therefore have no sense of achievement” • “It helped me understand to where I went wrong in the assignment” • “You get comments specifically related to your work. I found this very helpful” • “That it was personal and you can access the assignment as many times as you wished” • “It is easier to process feedback as you can go back over it in your own time” • “Definitely so helpful and I’ve seen my other assignments improve since this” I was pleased to see that students could see the value in audio feedback and that they were engaging with the feedback that was provided over and over again, and could see these practical benefits of engaging in feedback to improve and enhance their work and future grades. Students were also asked for suggestions on how they would improve the audio and electronic feedback however they were overwhelming unable to provide any suggestion other than • “Would love if every lecturer would do this” A small number also indicated that they felt it would be beneficial if the audio could be extend to five minutes rather than the three minutes of audio that is currently available and also it would be beneficial if that could download the audio and have it for the following academic year. I have passed on these suggestions to the Turnitin representatives I met at the EdTech conference.
Some practical advice that I can offer in relation to the electronic and audio feedback is as follows: 1) Ensure you have the necessary software and equipment: I found it useful to do the electronic marking on my PC and also have Turnitin open on the iPad simultaneously – therefore I could review my comments on the PC while I recorded the audio on the IPad. A set of noise cancelling headphones with a microphone is a must when recording the audio feedback on a PC. The set I used cost less than €50. When recording on the iPad I just spoke directly into the iPad. When recording the audio it needs to be done in a quite location where you will have minimum disruption as I found that once you begin recording its best to talk for the three minutes and not to pause. 2) Take your time at the start: The first 10 to 15 assignments are time consuming to correct and mark as you will need to build your reserve of comments. However once you have done this it’s only a matter of dragging them across to the relevant section of the assignment. I have found that many of the students make the same errors and omissions. My advice would be to provide detailed information in your comment, however keep the comments generic so you can use them over and over again. I also found it useful to write a brief script to explain to a student why their assignment is within a particular grade band. This is helpful as a prompt at the start and prevent stammering, stuttering and pauses while you speak. However as you become more familiar and confident doing the recording you will need this less and less. 3) Think about your audio feedback: You need to think about your pitch and tone when doing the audio feedback as well as the speed of your speech. The students indicated that they really felt the tone of voice when listening to the audio feedback was very important. Students who were disappointed with their grades or who struggle writing academic assignments indicated that found that the tone and the languages/ choice of words used were very important. They outlined that they liked the soft tone and the motivational language I used in the audio. They also highlighted the fact that this encouraged them to take on board the feedback rather than feeling ashamed or disappointed with the feedback. They indicated that this approach eliminated some of the shame and hopelessness they feel when they receive feedback and instead helped them to put the feedback within a context and also to identify that assessment and feedback is part of their learning journey. The feedback I received from my students in relation to pitch and tone, mirrored the findings of other researchers who have conducted research on audio feedback. 4) Try it out with a small group to begin with: This provides an opportunity to become familiar with using Turnitin Feedback Studio. Be warned that it is time consuming initially; however once you are familiar and comfortable recording and you have a suite of comments, I have found it to be as quick if not a quicker way of grading work and providing feedback, particularly when dealing with large numbers. 5) Read the literature: There is a significant body of literature in the area of feedback, and specifically audio feedback, which I found beneficial. In conclusion, I have found that using audio and electronic feedback has allowed me to engage my students in a more meaningful way with their feedback. I now provide audio and electronic feedback to all of my students across all of the programmes that I am teaching. Since the beginning of this academic year 2018/2019, all of my assessments are via the use of digital technology and as a result I have gone paperless which also has a positive impact environmentally. Now I cannot imagine retuning to the more traditional methods of doing assessments and providing feedback.
Dweck, C. (1999) Self theories: their role in motivation, personality and development (Philadelphia, PA, Psychology Press). Hennessy, C and Forrester, G., (2014). Developing a framework for effective audio feedback: a case study. Assessment & Evaluation in Higher Education. 39(7), pp. 777-789 Ice, P., Curtis R., Philips, P. & Wells, J. (2007). Using asynchronous audio feedback to enhance teaching presence and students’ sense of community. Journal of Asynchronous Learning Networks. Vol 11, No 2 pp 3-25. King, D., McGugan, S., & Bunyan, N. (2008). Does it make a difference? Replacing text with audio feedback. Practice and Evidence of Scholarship of Teaching and Learning in Higher Education. 3 (2). pp-145-163. Lunt, T., & Curran, J. (2010). Are you listening please? The advantages of electronic audio feedback compared to written feedback. Assessment & evaluation in higher education, 35 (7). Pp- 759-769. McCarthy, J (2015) Evaluating written, audio and video feedback in higher education summative assessment tasks. Issues in Educational Research, 25(2), 2015, pp 153-163. Merry, S., & Osmond, P. (2007) Student’s responses to academic feedback in Higher Education summative assessment tasks. Issues in Educational Research, 25(2), pp. 153-169. Nicol, D., & Macfarlane, D., (2007). Formative assessment and self – regulated learning: a model and seven principles of good feedback practice. Studies in Higher Education, 31(2). Pp 199-218. Northcliffe, A. & Middleton, A. (2008). A three year case study of using audio to blend the engineer’s learning environment. Engineering Education. Vol 3, No 2 pp 45-57. Sadler, D. R. (2013) Opening up feedback: Teaching learners to see. In Merry, S., Price, M., Carless, D., & Taras, M. (Eds). Reconceptualising Feedback in Higher Education: developing dialogue with students. (Ch. 5, 54-63). London: Routledge. Värlander, S. (2008).The role of students’ emotions in formal feedback situations. Journal Teaching and learning in Higher Education. Vol 12, No.2, pp 145-156. Voelkel, S. and Mello, L.V. (2014). Audio feedback – Better feedback? Bioscience Education, 22(1), pp. 16-30.