{"id":2549,"date":"2021-07-23T16:07:13","date_gmt":"2021-07-23T15:07:13","guid":{"rendered":"https:\/\/microsites.ncl.ac.uk\/casestudies\/?p=2549"},"modified":"2025-07-11T14:33:36","modified_gmt":"2025-07-11T13:33:36","slug":"equitable-assessment-framework-for-todays-digital-learners","status":"publish","type":"post","link":"https:\/\/microsites.ncl.ac.uk\/casestudies\/2021\/07\/23\/equitable-assessment-framework-for-todays-digital-learners\/","title":{"rendered":"Equitable Assessment Framework for today\u2019s Digital Learners"},"content":{"rendered":"<h4 class=\"entry-content\"><strong style=\"color: #003f80\">Cagri Yildirim<\/strong><\/h4>\n<h4><strong style=\"color: #003f80\">Learning Enhancement and Technology Projects Adviser for an Institute of Coding Project<\/strong><\/h4>\n<hr \/>\n<h4><strong>What did you do?<\/strong><\/h4>\n<p>Introduce and support instructor\u2019s adoption of the <a href=\"https:\/\/tilthighered.com\/\"><em>Transparency in Learning and Teaching (TILT)<\/em><\/a> framework to help structure and communicate their assessments for adult learners returning to Higher Education. This framework ensures that learners have a fairer and more inclusive starting point, by embedding meta-cognitive thinking strategies (thinking about their thinking process).<\/p>\n<hr \/>\n<h4><strong>Who is involved?<\/strong><\/h4>\n<p>Cagri Yildirim, the Learning Enhancement and Technology Projects Advisor for the Institute of Coding project collaborated with <a href=\"https:\/\/www.linkedin.com\/in\/gareth-james-aa401ab\/?originalSubdomain=uk\">Mr. Gareth James<\/a> and <a href=\"https:\/\/www.ethicalhealthcare.org.uk\/\">Mr. Thomas Webb<\/a>, practitioners and part-time instructors at Newcastle University\u2019s Business School to ensure their modules aligned with the MSc and Apprenticeship standards for assessment and content, Module Outline Form requirements for teaching and learning activities, and adopted Canvas VLE according to the standards of Quality Matters, TILT framework, and branding of the programmes.<\/p>\n<p><a href=\"http:\/\/microsites.ncl.ac.uk\/casestudies\/files\/2021\/07\/NBS8416.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-2557 size-full\" src=\"http:\/\/microsites.ncl.ac.uk\/casestudies\/files\/2021\/07\/NBS8416.png\" alt=\"NBS8416 Digital Innovation Technology Management\" width=\"405\" height=\"225\" srcset=\"https:\/\/microsites.ncl.ac.uk\/casestudies\/files\/2021\/07\/NBS8416.png 405w, https:\/\/microsites.ncl.ac.uk\/casestudies\/files\/2021\/07\/NBS8416-300x167.png 300w\" sizes=\"auto, (max-width: 405px) 100vw, 405px\" \/><\/a><\/p>\n<p>However none of this would have been possible without the timely and valuable input of different stakeholders overseeing the programmes, schools and the day to day experiences of the learners. Thank you to my colleagues:<\/p>\n<p>Institute of Coding<\/p>\n<ul>\n<li>Project Manager, Melanie Dunnett<\/li>\n<li>Apprenticeship Tutors, Dr Tejal Shah and Dr Tong Xin<\/li>\n<li>Principal Project Investigator, Professor Aad von Moorsel<\/li>\n<li>Business Partners Manager, Xander Brouwer<\/li>\n<\/ul>\n<p>School of Computing<\/p>\n<ul>\n<li>Degree Programme Director (DPD), Senior Lecturer, Dr Ahmed Kharrufa<\/li>\n<li>Learning and Teaching Team, Alex Barfield and Maddie Kinnair<\/li>\n<\/ul>\n<p>Newcastle University Business School (NUBS)<\/p>\n<ul>\n<li>Degree Programme Director, Senior Lecturer, Dr Joanne James<\/li>\n<li>NBS8416 Instructor, Thomas Webb<\/li>\n<li>NBS8417 Instructor, Gareth James<\/li>\n<li>Learning and Teaching Team, David Ormston-Simpson<\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n<ul>\n<li>2020 MSc Degree Apprentices in the Cyber Security, Data Analytics, and Software Engineering programmes (45 in total)<\/li>\n<\/ul>\n<hr \/>\n<p><strong>How did you do it?<\/strong><\/p>\n<p><strong>(1) Collaborative Perspectives<\/strong><\/p>\n<p>We had the luxury of designing these modules and their assessments from scratch, but we also had to ensure that the assessment formats would support both the MSc and Apprenticeship requirements.<\/p>\n<p><span class=\"TrackChangeTextInsertion TrackedChange SCXW34096079 BCX0\"><span class=\"TextRun SCXW34096079 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\"><span class=\"NormalTextRun SCXW34096079 BCX0\">Conversations with different stakeholders\u00a0<\/span><\/span><\/span><span class=\"TrackChangeTextInsertion TrackedChange SCXW34096079 BCX0\"><span class=\"TextRun SCXW34096079 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\"><span class=\"NormalTextRun SCXW34096079 BCX0\">involved in the programme design\u00a0<\/span><\/span><\/span><span class=\"TrackChangeTextInsertion TrackedChange SCXW34096079 BCX0\"><span class=\"TextRun SCXW34096079 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\"><span class=\"NormalTextRun SCXW34096079 BCX0\">helped to understand the perspective of the instructor, student needs, schools, and business partners.<\/span><\/span><\/span><span class=\"EOP TrackedChange SCXW34096079 BCX0\">\u00a0<\/span><\/p>\n<p><strong>(2) Understanding Learner Needs<\/strong><\/p>\n<div>As a group of staff and academics involved in the apprentice&#8217;s programme of studies we were able to confirm that our MSc Apprentices came with an inherit access to technology (hardware and software,) and wifi courtesy of their workplaces. But as adult learners returning to Higher Education some felt nervous about the Business modules that focused on scholarly reading, writing and reflection. They had chosen professions that played to their technical strengths. Therefore our challenge would be to provide clear assessment instructions and expectations, embed academic skills (that our full time students might have developed naturally), and increase their self-efficacy to succeed.<\/div>\n<p><strong>(3)\u00a0 Our Challenge<\/strong><\/p>\n<p><span class=\"TrackChangeTextInsertion TrackedChange SCXW165010504 BCX0\"><span class=\"TrackedChange SCXW165010504 BCX0\"><span class=\"TextRun SCXW165010504 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\"><span class=\"NormalTextRun SCXW165010504 BCX0\">Our instructors could appreciate adult learners\u2019 struggles, but they were not hired to teach academic reading and writing skills explicitly, or dilute (scaffold) their materials to make them mo<\/span><\/span><\/span><\/span><span class=\"TrackChangeTextInsertion TrackedChange SCXW165010504 BCX0\"><span class=\"TextRun SCXW165010504 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\"><span class=\"NormalTextRun SCXW165010504 BCX0\">re accessible.<\/span><\/span><\/span><\/p>\n<div>Arguably as MSc Apprentices they have access to the University&#8217;s Library Services that provide both individual and group study skills support. But unlike regular MSc students, they are also apprentices who work full time. Their workplaces give them a single day off to take classes which are carefully stacked back to back on that day. Therefore another solution was needed. We couldn&#8217;t assume that they could all budget time for this support.<\/div>\n<p>&nbsp;<\/p>\n<p><strong>(4) The Solution<\/strong><\/p>\n<div>As the learning designer, my role is to listen, observe, and make selective recommendations that can help solve multiple stakeholder\u2019s needs. With the instructor and learner in mind, I recommended that we adopt the\u00a0 <a href=\"https:\/\/tilthighered.com\/\">Transparency in Learning and Teaching (TILT) framework<\/a>\u00a0a research informed framework that embeds <a tabindex=\"-1\" title=\"https:\/\/cft.vanderbilt.edu\/guides-sub-pages\/metacognition\/\" href=\"https:\/\/cft.vanderbilt.edu\/guides-sub-pages\/metacognition\/\" target=\"_blank\" rel=\"noreferrer noopener\">meta-cognitive thinking skills<\/a> into the assessment instructions to help learners understand why an instructor has set the assessment, how they can breakdown the assessment into smaller more manageable tasks, and clear criteria for what looks good. The combination of these three parts allows the learner to self\u2013regulate their learning<a tabindex=\"-1\" title=\"https:\/\/cft.vanderbilt.edu\/guides-sub-pages\/metacognition\/\" href=\"https:\/\/cft.vanderbilt.edu\/guides-sub-pages\/metacognition\/\" target=\"_blank\" rel=\"noreferrer noopener\">.<\/a> They know what they are aiming for, and they are given a fair and inclusive (equitable) opportunity to approach the assessment.<\/div>\n<p>The instructors and I collaborated for three weeks, discussing and building out the different sections of the assessment. The toughest and most rewarding aspect being the grading criteria. But we were glad to complete all three parts, because instructors found that apprentices asked less logistical questions and more specifics about the content criteria.<\/p>\n<hr \/>\n<h4><strong>Challenge Yourself<\/strong><\/h4>\n<p>Find a summative assessment you or a colleague has written, and see if you can identify the following.<\/p>\n<p>Assessment instructions must include<\/p>\n<ol>\n<li style=\"list-style-type: none\">\n<ol>\n<li>Explanation of the <strong>purpose<\/strong> of the assignment.<\/li>\n<li>Description of the <strong>task<\/strong> in some detail.<\/li>\n<li>Explanation of the <strong>criteria<\/strong> for grading or success.<\/li>\n<\/ol>\n<\/li>\n<\/ol>\n<div>\n<p>Questions to ask yourself<\/p>\n<ol>\n<li>Given the current instructions, do you think you would be able to produce a response that meets the grading requirements?<\/li>\n<li>What aspects of the instruction could be improved so that the learner isn&#8217;t left guessing?<\/li>\n<li>What would you assume learners would need to know or do to approach this assessment (e.g. task analysis, project management, collaboration)? See our example <a href=\"https:\/\/newcastle.sharepoint.com\/sites\/LTDS\/DigitalLearningSite\/Forms\/AllItems.aspx?id=%2Fsites%2FLTDS%2FDigitalLearningSite%2Fweb%2FTILT%20Example%20Assessment%5Fby%20C%2EYildirim%20Abaci%2Epdf&amp;parent=%2Fsites%2FLTDS%2FDigitalLearningSite%2Fweb\">NBS8416 assessment<\/a> for inspiration.<\/li>\n<li>How do you consider the learner&#8217;s time and ability to complete the task? Based on your own experience trying out the assessment, do you double or triple that time?<\/li>\n<\/ol>\n<\/div>\n<div>\n<p><strong>Challenge your Students<\/strong><\/p>\n<ol class=\"copy-paste-block\">\n<li>Alternatively you could provide your students with Meta-cognitive skills, by asking them questions about the assessment to help them think about how they would approach the assessment. <a tabindex=\"-1\" title=\"https:\/\/tilthighered.com\/assets\/pdffiles\/transparent%20equitable%20learning%20framework%20for%20students.pdf\" href=\"https:\/\/tilthighered.com\/assets\/pdffiles\/Transparent%20Equitable%20Learning%20Framework%20for%20Students.pdf\" target=\"_blank\" rel=\"noreferrer noopener\">TILT provides sample questions to ask.<\/a><\/li>\n<\/ol>\n<\/div>\n<p><strong>Limitations of TILT<\/strong><\/p>\n<p><span class=\"TrackChangeTextInsertion TrackedChange SCXW88657101 BCX0\"><span class=\"TrackedChange SCXW88657101 BCX0\"><span class=\"TextRun SCXW88657101 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\"><span class=\"NormalTextRun SCXW88657101 BCX0\">TILT is just a framework.<\/span><\/span><\/span><\/span><span class=\"TrackChangeTextInsertion TrackedChange SCXW88657101 BCX0\"><span class=\"TextRun SCXW88657101 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\"><span class=\"NormalTextRun SCXW88657101 BCX0\">\u00a0You can adapt as needed<\/span><\/span><\/span><span class=\"TrackChangeTextInsertion TrackedChange SCXW88657101 BCX0\"><span class=\"TextRun SCXW88657101 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\"><span class=\"NormalTextRun SCXW88657101 BCX0\">\u00a0or opt out<\/span><\/span><\/span><span class=\"TrackChangeTextInsertion TrackedChange SCXW88657101 BCX0\"><span class=\"TextRun SCXW88657101 BCX0\" lang=\"EN-GB\" xml:lang=\"EN-GB\"><span class=\"NormalTextRun SCXW88657101 BCX0\">.<\/span><\/span><\/span><span class=\"EOP SCXW88657101 BCX0\">\u00a0<\/span><\/p>\n<ol>\n<li><strong>Formative Assessments<\/strong> that seek to generate open ended discussion the criteria could be merged with the task details as overarching expectations. E.g. in 250 words express your question, and try to respond to at least two other peers and continue the conversation by answering or asking additional questions.<\/li>\n<li><strong>Pedagogical strategies<\/strong> like Problem Based Learning, can adopt this framework and co-generate the descriptions for purpose, tasks, and criteria in collaboration with learners.<\/li>\n<li><strong>Not all learners are equal learners, <\/strong>those who are advanced in meta-cognitive study skills may find that this type of instructional detail oversimplifies, but arguably it gives them a clear baseline for what is expected so that they can measurably excel.<\/li>\n<\/ol>\n<p>For those who would like a more robust framework, <a href=\"https:\/\/tilthighered.com\/assets\/pdffiles\/Checklist%20for%20Designing%20a%20Transparent%20Assignment%20copy.pdf\">TILT has produced a free checklist.<\/a><\/p>\n<h4><strong>Who benefits from TILT?<\/strong><\/h4>\n<p><strong>Instructor Wins!<\/strong><\/p>\n<ol>\n<li>Assessment documents ensure that if the instructor falls sick a DPD or instructor can help oversee the implementation and grading<\/li>\n<li>Easy to scale and moderate grading across multiple instructors<\/li>\n<li>Easy to adjust or adapt for future iterations<\/li>\n<\/ol>\n<p><strong>External Examiner Wins!<\/strong><\/p>\n<ol>\n<li>Transparent expectations indicating fair grading criteria<\/li>\n<li>Expedites reviewing and grading<\/li>\n<\/ol>\n<p><strong>Learner Wins!<\/strong><\/p>\n<ol>\n<li>Transparent instructions and criteria ensuring fairness in grading<\/li>\n<li>Helps learners to monitor their own progress towards the criteria<\/li>\n<li>Allows learners to judge for themselves how much more or less time they need to complete activities based on the baseline recommendation from the instructor<\/li>\n<li>Helps learners to express what they do\/don\u2019t know and when\/where they need help<\/li>\n<li>Builds trust between instructor and learners because the details show that the instructor has thought through the assessment steps from the learner perspective and ensures that the work is achievable in the timeframe<\/li>\n<li>They feel optimistic or less intimidated to try and succeed<\/li>\n<\/ol>\n<hr \/>\n<h4><strong>Why did you do it?<\/strong><\/h4>\n<p>As the learning designer, my role is to listen, observe, and make selective recommendations that can help solve multiple stakeholder\u2019s needs. Given that our apprentices came with equal access to technology, software, and hardware through their places of work, our focus was to ensure that the instruction and particularly assessment instructions were fair, inclusive (equitable), and empowering for adult learners from diverse backgrounds. The TILT framework allows learners to self-regulate their learning, a skill that might not be inherit or taught previously, but helps decode a post graduate higher education assessment without removing the thinking.<\/p>\n<hr \/>\n<h4><strong>Graduate Framework<\/strong><\/h4>\n<p>This approach develops the following attributes:<\/p>\n<ul>\n<li aria-setsize=\"-1\">Critical Thinkers<\/li>\n<li aria-setsize=\"-1\">Confident<\/li>\n<li aria-setsize=\"-1\">Digitally Capable<\/li>\n<li aria-setsize=\"-1\">Engaged<\/li>\n<\/ul>\n<p>Find out more about the\u00a0<a href=\"https:\/\/www.ncl.ac.uk\/careers\/graduateframework\/\">Graduate Framework.<\/a><\/p>\n<hr \/>\n<h4><strong>Interested in finding out more?<\/strong><\/h4>\n<p>Take a look at these resources:<\/p>\n<ul>\n<li><a href=\"https:\/\/newcastle.sharepoint.com\/:b:\/r\/sites\/LTDS\/DigitalLearningSite\/web\/TILT%20Example%20Assessment_by%20C.Yildirim%20Abaci.pdf?csf=1&amp;web=1&amp;e=7CfFG4\">NBS8416\u00a0 Assessment using TILT<\/a><\/li>\n<li><a href=\"https:\/\/eur03.safelinks.protection.outlook.com\/?url=https%3A%2F%2Ftilthighered.com%2Fassets%2Fpdffiles%2FTemplate.pdf&amp;data=04%7C01%7CClaire.Burnham%40newcastle.ac.uk%7Cf64cfdaeb3074136b25e08d94cf41f91%7C9c5012c9b61644c2a91766814fbe3e87%7C1%7C0%7C637625434210963272%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=Sdgzci0FQepTZ1LToNHIh7IXJbK3hqxgbb6TCVFy4bU%3D&amp;reserved=0\">TILT Template Handout for Instructors<\/a><\/li>\n<li><a href=\"https:\/\/eur03.safelinks.protection.outlook.com\/?url=https%3A%2F%2Ftilthighered.com%2Fassets%2Fpdffiles%2FTransparent%2520Assignment%2520Template%2520for%2520Students_v2.pdf&amp;data=04%7C01%7CClaire.Burnham%40newcastle.ac.uk%7Cf64cfdaeb3074136b25e08d94cf41f91%7C9c5012c9b61644c2a91766814fbe3e87%7C1%7C0%7C637625434210963272%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=bzVbtfC2v7BsvceThbTGSL97YSpuKqh8AgU37IKEGkA%3D&amp;reserved=0\">TILT Handout for Students (Reflective question prompts)\u00a0 <\/a><\/li>\n<li><a href=\"https:\/\/eur03.safelinks.protection.outlook.com\/?url=https%3A%2F%2Ftilthighered.com%2Fassets%2Fpdffiles%2FChecklist%2520for%2520Transparent%2520Assignments.pdf&amp;data=04%7C01%7CClaire.Burnham%40newcastle.ac.uk%7Cf64cfdaeb3074136b25e08d94cf41f91%7C9c5012c9b61644c2a91766814fbe3e87%7C1%7C0%7C637625434210973258%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=aQofeVee8cbcMnxhIJKQxec1SGVTRox6glzwX0%2FQ%2FO4%3D&amp;reserved=0\">TILT Checklist<\/a><\/li>\n<li><a href=\"https:\/\/eur03.safelinks.protection.outlook.com\/?url=https%3A%2F%2Fcft.vanderbilt.edu%2Fguides-sub-pages%2Fmetacognition%2F&amp;data=04%7C01%7CClaire.Burnham%40newcastle.ac.uk%7Cf64cfdaeb3074136b25e08d94cf41f91%7C9c5012c9b61644c2a91766814fbe3e87%7C1%7C0%7C637625434210983255%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=Pib67XBUGv%2BxXaqRRaCRwFRkZzMLMMGZxGCl9NAqCvM%3D&amp;reserved=0\">Metacognitive Learning explained<\/a><\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>This collaborative project supported instructor\u2019s adoption of the Transparency in Learning and Teaching (TILT) framework to help structure and communicate their assessments.<\/p>\n","protected":false},"author":3007,"featured_media":2565,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1,280,306,314,272,360,273],"tags":[605,608],"class_list":["post-2549","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-all","category-assessment-and-feedback","category-business-school","category-computing","category-hass","category-learning-and-teaching-development-service","category-sage","tag-critical-thinking","tag-digital-skills"],"_links":{"self":[{"href":"https:\/\/microsites.ncl.ac.uk\/casestudies\/wp-json\/wp\/v2\/posts\/2549","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/microsites.ncl.ac.uk\/casestudies\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/microsites.ncl.ac.uk\/casestudies\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/microsites.ncl.ac.uk\/casestudies\/wp-json\/wp\/v2\/users\/3007"}],"replies":[{"embeddable":true,"href":"https:\/\/microsites.ncl.ac.uk\/casestudies\/wp-json\/wp\/v2\/comments?post=2549"}],"version-history":[{"count":25,"href":"https:\/\/microsites.ncl.ac.uk\/casestudies\/wp-json\/wp\/v2\/posts\/2549\/revisions"}],"predecessor-version":[{"id":3928,"href":"https:\/\/microsites.ncl.ac.uk\/casestudies\/wp-json\/wp\/v2\/posts\/2549\/revisions\/3928"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/microsites.ncl.ac.uk\/casestudies\/wp-json\/wp\/v2\/media\/2565"}],"wp:attachment":[{"href":"https:\/\/microsites.ncl.ac.uk\/casestudies\/wp-json\/wp\/v2\/media?parent=2549"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/microsites.ncl.ac.uk\/casestudies\/wp-json\/wp\/v2\/categories?post=2549"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/microsites.ncl.ac.uk\/casestudies\/wp-json\/wp\/v2\/tags?post=2549"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}