Technologies for Feedback & Advice in PGR Supervision

Dr Stacy Gillis (Senior Lecturer) and Dr Chiara Pellegrini (RA)

English Literature, Language and Linguistics

Faculty of Humanities and Social Sciences

What did you do? 

We received funding from the Newcastle University Technology Enhanced Learning Advocates (NUTELA) group to identify which technologies are used to give feedback and advice to postgraduate students as part of the supervision process at Newcastle University.

Who was involved? 

The initiative was funded by NUTELA and was led by Dr Stacy Gillis and Dr Chiara Pelligrini on behalf of the School of English Literature, Language and Linguistics.

How did you do it? 

We conducted a literature review and mapped institutional resources on ‘feedback in postgraduate supervision’ from a variety of sources:

  • Journal articles
  • Advance HE
  • Research Councils
  • University handbooks

This information was used to inform the design of a survey distributed to postgraduate students across all faculties at Newcastle University.

Participants were asked about gender, race and ethnicity, disabilities, caring responsibilities, funding status and EAL in order to map any significant correlations between variables indicating personal circumstances and preferred mode of feedback/advice (no correlations emerged).

The survey provided twelve possible ways of sharing feedback and advice. Respondents were asked to indicate whether they receive feedback in any/all ways and on which one of three aspects of their work:

  • Thesis Drafts
  • Research Work & Development
  • Career Development
A small screenshot showing the structure of a question in the survey consisting of feedback options down the left hand side and tick boxes across the page for the categories of Thesis drafts, Research work and Career Development

Figure 1 – a screenshot of the survey

‘Feedback and advice’ were defined as any way(s) in which supervisor(s) can offer comments, support, or direction as they engage with a postgraduate student’s research, writing practices, career development, and more.

Why did you do it? 

This project was developed to understand the forms of technology being used to provide feedback on PhD/MRes/Mlitt/MPhil draft work, and to transform the postgraduate supervisor process.

It gave postgraduate students the opportunity to reflect on how they receive feedback/advice from their supervisor(s), with a focus on the forms that this feedback takes, the different technologies used, and what works most efficiently and productively for students.

The data was used to identify best practice, areas for improvement and final recommendations.

What did the survey show? 

There were 127 responses to the survey from 24 different schools. The breakdown between Faculties is as follows:

  • HaSS – 29.9%
  • FMS – 22.8%
  • SAgE – 48%

(Note: 2 respondents were in both SAgE and FMS)

The survey provided twelve possible ways of sharing feedback and advice. Respondents were asked to indicate whether they receive feedback in any/all ways and on which one of three aspects of their work:

  • Thesis Drafts
  • Research Work & Development
  • Career Development

The survey provided useful data on students’ experience of receiving feedback, including the most popular forms of feedback, shown in Figure 2.

Bar chart showing the most popular forms of feedback for the categories of Thesis drafts, Research work and Career Development. The four types of feedback are Oral discussion via video, oral discussion in person, tracked changes on electronic document sent via email and comments on an electronic document sent via email. All types of feedback were popular for Thesis drafts and research work with over 70 respondents for each, but for career development there were less than 50 respondents for all feedback types.

Figure 2 – the most popular forms of feedback for postgraduate students

The least popular forms of feedback were:

  • Oral discussion over the phone
  • Comments on hard copies or other manually written feedback
  • Recorded oral feedback

Sitting between the most and least popular forms of feedback are the other options provided:

  • Comments through an official feedback form or feedback platform such as ePortfolio
  • Comments on a [shared] electronic online document
  • Tracked changes on a shared electronic online document
  • Written paragraphs of feedback at the end of a document or in the body of an email
  • Group meeting with other researchers

The open comment section asked about other forms of feedback and mention was made of Slack, Teams, and WhatsApp, as well as mind maps and creative work on Zoom.

The survey also asked students to select from a list the ways in which they experience multiple supervisors providing feedback/advice (Figure 3).

A bar chart showing how frequent different types of feedback were. Starting from the least frequent to the most frequent they are: In-text comments on a shared hard copy document, Feedback from different supervisors through different technologies, written feedback from different supervisors on separate documents, feedback and advice from different supervisors in separate meetings, in-text comments from multiple supervisors on a shared electronic document and finally feedback from multiple supervisors in the same meeting.

Figure 3 – Multiple Supervisors’ Feedback

Other points raised:

  • Career development, across all ways of receiving feedback/advice, is the aspect of the PhD on which respondents receive the least amount of feedback.
  • The majority of respondents have 2- 3 supervisors, and 76.6% reported that all their supervisors give them feedback and advice more than once a year.
  • While there are some standard practices adopted across the university (e.g. comments on thesis drafts and regular meetings with oral discussions), from the PGR students’ perspective, there is no formalized discussion of supervisors, supervisees and the most efficient and productive forms of feedback.

What does Good Practice look like? 

The majority of respondents prefer a mix of oral feedback and written comments/tracked changes on an electronic text (some clarified that the former is better for general research discussions and career development; others that they like to receive written feedback first and then discuss it). No single overall preference emerged. Recording both in-person and online meetings was mentioned for situations in which it is difficult to take notes are highlighted (e.g. English is their second language; a disability).

What are the areas for improvement? 

Written paragraphs in the body of an email are the least preferred ways of communicating. Some respondents indicated that there could be better integration with NU digital platforms:

  • ‘It would be good to receive feedback against Reflection and Meeting posts on NU Reflect. This would enable more regular formative feedback to be given and taken on board.’
  • ‘My supervisors submit comments on meeting records and they submit forms for progression reports on NU Reflect, but otherwise we don’t use it.’

What are the recommendations? 

The majority of respondents reported that in 2020, meetings moved to online platforms like Zoom, and some that hard copies of work were no longer exchanged. A handful of respondents commented that they receive feedback and advice less frequently post-COVID.

There is a fairly even split between respondents who prefer online to face-to-face meetings, and those that do not. Disparate reasons such as flexibility of scheduling, ease of communication, ways of sharing documents or practices, formality of the situation, length of meetings, and more, were cited in support of one or the other preference, without significant correlations emerging.

A few respondents used the open comments sections to remark that they get very little feedback and advice from supervisors, or that they are finding it difficult to manage the differences in communication (both in content and frequency) between different supervisors.

Avenues for resolving problems around feedback and advice – and how this should involve career advice, as well as comments on submitted work – should be made more visible to PGRs. 

Figure 4 shows the final project suggestions for improving feedback for postgraduate students.

Image showing the 4 final suggestions of the project, which are: 1) Technologies (including NUReflect) for receiving feedback and advice should be an explicit item for discussion for supervisors and supervisees, at both the beginning of the research degree, and ad regular intervals (including APRs). 2) PGRS should be provided with multiple spaces and opportunities to speak up about what works for them, rather than deferring to supervisors. 3) Post-COVID-19, supervisors and supervisees should be asked to reassess the efficacy of changes in technologies for communication that were adopted out of necessity and 4) supervisors should receive support in reflecting on their log-standing practices around feedback and advice, and whether these are the most productive and efficient modes of doing so.

Figure 4 – the project final suggestions

Graduate Framework

The project develops the following attributes:

  • Collaborative
  • Critical Thinker
  • Digitally capable
  • Engaged
  • Future-focused
  • Reflective and self-aware

Leave a Reply

Your email address will not be published. Required fields are marked *