The Dialogue option offers an alternative to a written application, enabling participants to demonstrate their evidence for Fellowship recognition in a mode that best suits their preferences and needs.
Our challenge / opportunity
The PSF CPD Scheme is accredited by Advance HE to award Fellowship recognition against the Professional Standards Framework for teaching and supporting learning in higher education 2023 (the PSF). The scheme is open to colleagues and postgraduate research students who teach and/or support learning, and supports them to make a claim for recognition as Associate Fellow, Fellow, or Senior Fellow of the HEA. This includes applicants from the Evidencing Learning and Teaching Skills (ELTS) route for PGRs who teach, Newcastle Educational Practice Skills (NEPS) for colleagues new to HE teaching/learning support, and the PSF Experiential Route for colleagues with existing HE teaching/learning support experience.
Our participants are very diverse, both in terms of their backgrounds and experiences, and their individual characteristics and needs. They include colleagues and PGR students from academic disciplines and professional services across the institution, with wide-ranging prior academic and professional experiences. Among them are participants who are non-native speakers of English, participants who have disabilities, and participants who are neurodivergent. The PSF CPD Scheme is very flexible, giving participants the freedom to decide on the time period when they will work on their applications, and when they will submit them.
Given this diversity, to evaluate claims for Fellowship recognition equitably we need to give participants the opportunity to evidence how they meet the relevant PSF descriptor criteria in ways that suits their diverse experiences, needs and preferences.
Our solution (approach)
We offer participants a choice of two options to make their Fellowship application: they can either write a Documentary application, or they can engage in a Professional Dialogue.
The Professional Dialogue is a discussion between the participant and two reviewers. The duration ranges from 20-25 minutes for Associate Fellowship, to 25-35 minutes for Fellowship, and 35-45 minutes for Senior Fellowship, designed to allow time for participants to reflect on the relevant breadth and depth of experience needed to for each category.
Before their dialogue, participants who have chosen this option prepare supporting documents, including a short overview of their professional practice in HE teaching/learning support, and a summary table of the evidence they plan to reflect on in the dialogue, together with the relevant supporting references for the category of Fellowship that they are applying for. Preparing these documents supports the participant to prepare for the dialogue by gathering their evidence. The documents are submitted to a Canvas assignment in advance, allowing the reviewers to read and familiarise themselves with the participant’s context and the evidence they will elaborate on during the dialogue.
All participants have support to prepare their Fellowship application from either the Academic Practice Team (ELTS participants) or a PSF mentor (NEPS and Experiential Route participants). Mentors review draft application documents and provide formative feedback, and offer participants the opportunity for a mock dialogue. Written guidance for dialogue participants is available in the Canvas course for each route, as well as recordings of example successful dialogues (made available with permission from the participant in each case).
The Academic Practice Team schedule each dialogue, based on availability of the participant and two reviewers. Dialogue reviewers are all members of the 70 strong PSF mentoring team, including colleagues from across academic disciplines and professional services who have Fellowship or Senior Fellowship themselves. Mentors complete initial training and engage with regular mentor development and calibration workshops to keep their reviewing practice up to date, and they are supported with structured guidance and resources in a dedicated Canvas course.
The majority of Dialogues take place online in MS Teams, while some participants choose an in person dialogue. This flexibility enables participants to choose an environment and mode of participating in the dialogue that suits their needs and preferences. For online dialogues it is not compulsory for the participant to have their camera on; for in person dialogues adjustments to accommodate any accessibility needs can be made when arranging the venue.
Although the application documents are submitted in advance, the dialogue itself is the primary means by which they are reviewed against their chosen PSF Descriptor. To help participants know what to expect and to feel at ease at the start of the dialogue, we use a standard starting prompt question for each category, which is provided to the participant in advance.
Reviewers ask further questions during the dialogue to prompt the participant and give them the opportunity to demonstrate that they meet the relevant Descriptor criteria. Example prompt questions relevant to each category are provided for reviewers, and a list of example areas of focus for the discussion is provided for participants in advance. The further questions are not prescribed, to give the reviewers scope to prompt the participant in ways that are relevant to their individual experience, informed by the information in the application documents.
To make dialogues as inclusive as possible participants are welcome to write down the question, to pause to think before they answer, and to bring documents or notes into the dialogue which they can refer to. Reviewers are provided with guidance about asking accessible questions (e.g. keeping them short or making clear where a question has multiple parts, being open to repeating, explaining or rewording questions when needed).
To support quality assurance of Fellowship applications all dialogues are recorded using MS Teams (whether they take place online or in person). After each dialogue the recording files are automatically made available in the MS Teams meeting space and are accessible to everyone who attended the meeting. The team administrator manually moves the recording to a secure Sharepoint folder, and edits the settings to remove the default expiry date. Sample recordings are shared with the moderators and External Reviewer. All dialogue recordings are saved securely for 3 years, and then manually deleted.
The impact (results)
Typically between 20-30% of participants choose the Professional Dialogue option. The proportion of Associate Fellowship applicants who choose a dialogue is relatively small (usually fewer than 5%), while between 15-25% of Fellowship applicants choose a dialogue, and the proportion of Senior Fellowship applicants choosing dialogue is much higher (generally between 70-80%).
The success rate on first submission is slightly higher for dialogues than for documentary applications. This is attributed to the fact that the ‘claim’ for recognition is shaped during the dialogue itself in contrast to the static nature of a written submission. In the context of a dialogue, if evidence for an aspect of the PSF criteria is not fully evidenced initially the reviewers are able to ask a question that can prompt the participant to elaborate on their evidence.
Reviewers and participants both report that they enjoy the discursive nature of Professional Dialogues, and find participating in them a rewarding experience. For example one participant commented
“I found my dialogue to be constructive and a friendly experience that was an assessment but didn’t feel like it at the time.”
Another explained:
“I felt this approach was more interactive and gave me the opportunity to clarify and highlight my work effectively, rather than relying solely on written evidence. It also felt more personal and engaging, making it easier to showcase the impact of my teaching and leadership”.
Feedback from several neurodivergent participants highlights that they found the option of a dialogue as an alternative to a written submission was more accessible to them. As one participant put it:
“I found it much more comfortable, as someone with a hidden disability, to present myself in oral formats.”
Another participant who has dyslexia commented that the dialogue option had enabled them to apply for Senior Fellowship recognition, which they had previously been put off from doing by the amount of writing required for a Documentary application.
Lessons learned
Some participants talk at length in response to the starting prompt question, making it difficult for the reviewers to interject and ask questions on other aspects of the criteria. Encouraging all participants to make use of the example dialogue recordings and take up the opportunity of a practice dialogue helps to address this.
There is significant cognitive load on reviewers during a dialogue, as they have multiple documents to refer to (including the applicant’s submission documents, and the PSF descriptor criteria and dimensions), whilst also listening to the participant’s answers, and considering what further prompt questions to ask. To help with this as well as the reviewer guidance with example prompt questions, the 2 reviewers are advised to set up a short pre-meeting to decide on potential areas for discussion based on the criteria and the information in the application documents. There is also an induction process for new mentors to begin reviewing dialogues, to help them develop the skills and confidence to review dialogues effectively.
Tips for colleagues
To help new mentors prepare to start reviewing dialogues they have the opportunity to shadow one or two dialogues before starting to review them, and for at least the first 3 dialogues they review they will act as second reviewer paired with a more experienced first reviewer. All reviewers also engage with regular marking standardisation for both documentary and dialogue applications, and have access to example dialogue recordings (with consent of the participants).
The practicalities of managing dialogue assessments are complex, and careful consideration and planning is needed before implementing this type of assessment.
In terms of scheduling, between 20-40 dialogue applications are reviewed over a 2 week period following each of 3 submission deadlines every year. To manage scheduling we plan a series of dialogue slots (up to 60 minutes each) during the dialogue period, and use MS Forms for reviewers and participants to provide their availability. The team manually plan the schedule using this data.
Getting to grips with the functionality for recording dialogues in Teams, and then for storing them securely, took some time. We sought advice from the Digital Adoption team in NUIT to help get things working smoothly, and did some testing before using this approach for the first time. Their advice and support with queries has been invaluable – and our team are also happy to answer questions or discuss with colleagues considering how to implement similar approaches.
Education for Life Strategy
This case study reflects the following aims of the Education for Life strategy:
- Fit for the future: To ensure our students are fit for their future, our teaching is fit for the future of our offer, and our colleagues are fit for the future of HE
Authors
 |
Dr Rebecca Gill Professional Development Adviser Learning and Teaching Development Servicerebecca.gill@newcastle.ac.uk |