Feedback best practices for Teaching Support staff.
Return of all coursework is now being monitored by the Informatics Teaching Organisation (ITO). This means that coursework marks should be returned to the ITO. In general this should be within 10 working days of the submission deadline for the students. If you regard this as an inappropriately short turnaround time, an extension can be requested. Extension requests should go to Ian Stark . These should be made as soon as possible and no later than the submission deadline.
Students complaints are mostly related to two separate kinds of feedback. Firstly students complain bitterly where the reason behind their grade is not made clear. Secondly students would like to know how they could have improved. The university calls these two separate kinds of feedback "summative" and "formative".
Summative feedback is the feedback which gives some explanation as to the grade awarded. This should hopefully reference, or at least be related to, the specification of the coursework in question. To avoid a large amount of writing it is possible to set up a page on your course web page explaining common mistakes and point students to that. Please make sure each student understands which portions of it are relevant to them, in some cases this may be self-evident, in others not.
Formative feedback is feedback which is intended to improve the students' performance both later in the same course, in future courses and indeed beyond their studies. This can take many forms. It can simply be a part of the feedback returned with each coursework's grade, together with the summative feedback. Alternatively, or in addition, it can take the form of a lecture detailing the common errors the class made. Importantly make clear what lessons the students can take from these errors, whether or not they themselves made each particular mistake.
Without very detailed data, there is some evidence that the students are willing to wait for good feedback. The dissatisfaction appears to be coming from courses that provide low quality feedback late. One piece of evidence to suggest this concerns feedback teaching award nominations. At least two nominees have been late with their feedback, but because of the quality have still garnered a nomination from their students.
A constant theme that arises when discussing feedback with staff is that they often feel that their efforts are wasted. I have many times heard stories of detailed feedback being left in the ITO for students to collect and the coursework never being collected by the students. To date, I've never managed to isolate a concrete case of this happening. It may be a case of one incident being repeated many times. Whether it is common or not is beside the point, clearly the story resonates with many staff. Clearly, many are concerned about their efforts going to waste.
There are at least two ways to mitigate this to some degree. The first is to provide detailed but generic feedback. The second is to insist that the students make some effort to solicit feedback.
This works well when the class size is large, and many of the students make similar errors. So there is only a small set of points to provide feedback on. The lecturer publishes a generic document which the students can read, and then for each individual student explains or specifies which points from the generic document are applicable to them.
This has the additional benefit that students can learn from the mistakes of others. Even if a particular student has not made a particular mistake that may have been more through luck than judgement, so reading about the common errors can be helpful. In particular it means that a student who gets everything correct might still feel like they are getting some feedback.
The idea here is to insist that the students make some effort to explain that they would like some feedback and more importantly exactly what they would like some feedback on.
The negative side of this approach is that students who do not explicitly ask for specific feedback may, nevertheless, feel that they are not provided with adequate feedback. This is a point that I feel unequipped to address. I'm not sure if we, as a school, care more about our scores in student surveys, whether or not we feel those scores are justified.
The following is an example provided by Iain Murray who received excellent feedback scores in his course survey along with some specific comments commending his efforts on feedback:
Every year I've offered to give detailed written feedback on any work that anyone wants to give me. Tutorial questions, textbook questions, past exam questions, a restatement of the material in their own words, whatever.
This policy has been minimal work for me, because less than one student a year has taken me up on it. My guess is that my award nomination was from the one student that asked for feedback this year.
I'll continue to offer, because I like engaging with the students that take some responsibility for their own learning, and put some effort in themselves. But it will only benefit ~2% of students.
I put a fair amount of effort into trying to explain how compulsory assignment work could be done better in future. I'll continue to try, but I'm not convinced it has much impact.
Without detailed data, anecdotally it seems that office hours are not used by the students. It may be that the Informatics Forum is seen as something of an unapproachable ivory tower. It may also be that email has largely reduced the necessity for an office hour. In any case I have two examples of the use of an alternative to office hours with some positive response.
Here is what Iain Murray had to say about his approach to this: "My IT lectures are just after lunch, and the lecture theatre is usually free beforehand. I'd often show up 20 mins early for a general informal chat with whomever wanted to show up early. I got much better engagement with that than office hours. Sadly lectures are usually sandwiched together and we have to get in and out of the theatre in quick time."
Charles Sutton also tried this, with the approach of having coffee in the Appleton Tower cafe prior to the lecture:
"Overall I would do this again, but I would also look for ways to tweak this.
These were not on the same day as the lectures. The lectures were 10am TF, so I did 10am MW for the office hours.
I decided to do this because I wanted to reduce the friction involved in a student coming to see me, i.e., they knew that I was there solely to answer their questions. Also, I like coffee.
Typically I got between 2-4 students attending, out of a class of 100. There were some regulars, but it was not the same ones each time. This is not exactly earth shattering uptake, but still much better than when I used my office.
I chose the AT coffee shop because (a) everyone knew where it was, and (b) the students never bought anything, so at a Uni coffee shop, they could take up a table without feeling guilty. At 10am there are enough free tables in AT that this is possible. We would not have had space to do this at lunch time.
I think that the students who attended found it useful, although I will be curious to see if anyone mentions this in the class surveys.
For the future I would like to think of ways to improve uptake, but I am not sure how best to start."
Knowing about this, I have attempted this for my course this year as well. Instead of showing up early, I stated that I would be available in a specific common area near the lecture theatre for around 20 minutes after the lecture. Whilst I had several students ask individual questions after the lecture, still in the lecture theatre, I had only one in the whole course ask something in the common area. So perhaps it is crucial to be there early, or perhaps the students simply had somewhere else to be directly after the lecture. In either case, perhaps some consultation with the students is required to find the correct formula here.
In my course I had the students write a series of essays as their coursework. Rather than have the students simply send their essays to me, they instead posted them to a class blog. Each blog post/essay could then be discussed by the students in the comments section.
In addition to writing two essays the students were also asked to write a response essay. The response essay was intended to be a response to one of their fellow students’ essays. I find that reading other peoples’ essays is a very good way to learn what does and does not work well. Hence the response essay was an attempt to force the students into reading at least some of their fellow students’ essays.
One simple motivation was that students write essays and it seems a shame if I am the only one who ever reads what they have written. One point that I try to teach is that the effort spent in making an article readable is hopefully re-paid by the time saved by many readers. This is of course entirely lost if I am the only person ever to read their essay.
A further point that I try to teach them is regarding scope. Their essays are typically far too broad in scope. My hope in having them post to a blog was to show that the broad nature of their scope meant that they are not contributing much to the wider discussion. A post with a large scope is making a point that has been made before. A much narrower scope is contributing towards a point that has been made before.
In addition, many of the students feel that the point of the essay is to “demonstrate what they know” which is entirely not what I am hoping to teach them. The point is that being right and convincing others that you are right are two different things and usually one is not much use without the other. Having them submit essays to me emphasises too much the “demonstrate what you you know” and I hope the wider audience places more emphasis on the “convince the reader”.
I think in general the students did enjoy posting to the blog rather than handing an essay in to one reader. Somehow the time spent writing their essay is a little more worthwhile. But I do not have much data to back up this claim.
I did try rewarding the students for the discussion generated by their essays in comments section and also for contributing to the discussion in other essays. I basically found this did not work terribly well. Students seem to regard this as a chore, rather than more of a group activity. The second year in which there were no marks for comments still actually had a number of comments on some of the more popular posts. Arguably the discussion was more interesting as well.
I believe that plagiarism is also further discouraged by this approach. Since it is much more likely that someone other than me notices plagiarism and reports it, hence it seems like a bigger risk to plagiarise and post to a blog than to plagiarise and submit to one marker. Again though I’m not sure I have much data to back this up.
In their response articles I emphasised that they should respond to the point being made, not to the manner in which it was made. Unfortunately having received the feedback I gave them to their first essay, many students noticed the same kinds of flaws in the other students’ essays. But that was not supposed to be the point. However, if I were running this again, I might very well deliberately ask the students to ‘grade’ each others’ essays on the quality of the argument.
This practice is as scalable as marking a set of essays given only to the marker. It only involves setting up the blog in the first place and without that you need some mechanism for the students to submit their essays.
A couple of quotes from the course survey:
“Great course, the lecturer introduced a new style of homework (blog posts), which was awesome”
“Coursework was an awesome idea, really encouraged lots of reading on topics that you may not have read about but students had written”
and the negative: