{"id":22452,"date":"2024-11-26T00:51:48","date_gmt":"2024-11-25T13:51:48","guid":{"rendered":"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/?p=22452"},"modified":"2025-03-04T10:15:28","modified_gmt":"2025-03-03T23:15:28","slug":"aligning-our-assessments-to-the-age-of-generative-ai","status":"publish","type":"post","link":"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/aligning-our-assessments-to-the-age-of-generative-ai\/","title":{"rendered":"Aligning our assessments to the age of generative AI"},"content":{"rendered":"<h2>TL;DR: The reality of assessment in the age of generative AI<\/h2>\n<ul>\n<li>Generative AI tools exist that can accurately create content (including written work, images, music, video, and the spoken word) and answer questions.<\/li>\n<li>Although current AI tools can and do produce errors, the technology is improving at a very fast pace.<\/li>\n<li>AI detection relies on the user not knowing how to use it properly and can circumvented by AI tools designed to do so or by tactics advertised to students available widely on social media.<\/li>\n<li>Only some students can afford to pay for state-of-the-art AI tools which can complete our assessments and avoid detection even more rigorously.<\/li>\n<li>We need to equip our students to be leaders in a world where AI is everywhere.<\/li>\n<\/ul>\n<p>Read about the <a href=\"#novice\">ability of generative AI to complete assessments<\/a>, <a href=\"#2-lane\">our two-lane approach to assessment<\/a>, <a href=\"#Sem1\">changes to the Academic Integrity Policy for Semester 1<\/a>, and how it affects coordinators, and the <a href=\"#Sem2\">plan for larger changes to assessment categories and types for Semester 2<\/a> to fully align them with the reality of assessment in the age of generative AI.<\/p>\n<h2>Background<\/h2>\n<p>The release of ChatGPT in November 2022 represented a huge leap in the power and availability of generative AI. In the past two years, generative AI tools have multiplied in number and advanced considerably. There is no indication that investment and development will slow in the near future. Generative AI tools can take text, image, audio, video, and code inputs and generate new content into any of these formats. <a href=\"https:\/\/www.tandfonline.com\/doi\/full\/10.1080\/22054952.2024.2372154\">Numerous<\/a> <a href=\"https:\/\/journals.plos.org\/plosone\/article?id=10.1371\/journal.pone.0305354\">reports<\/a> show that these tools can mimic human outputs well enough to pass or excel in the types of assessments commonly used to validate learning in higher education and elsewhere. These reports also demonstrate that new versions show a marked improvement in the grades obtained. Generative AI is becoming baked into everyday productivity tools and devices including wearables, and it will become increasingly difficult, expensive, and undesirable to control their use outside highly supervised environments.<\/p>\n<blockquote><p>If we do not change our assessments to align them with the capabilities of generative AI, our degrees will be easier to pass <em>and<\/em> we will have failed to prepare our students for their future.<\/p><\/blockquote>\n<p>Whilst the effects on our teaching, and assessment and program design are immense, the implications for our graduates is profound. Described by Silicon Valley pioneer Reid Hoffman as a \u201csteam engine of the mind\u201d, generative AI tools are already replacing and changing the white-collar jobs traditionally sought by our students. Our graduates are already being asked to demonstrate their AI skills in interviews and expected to be skilled users. A <a href=\"https:\/\/www.digitaleducationcouncil.com\/post\/digital-education-council-global-ai-student-survey-2024\">recent report<\/a> indicated that more than half of students feel they have sufficient AI knowledge and skills.<\/p>\n<p>Our assessment design thus needs to both be able to assure learning <em>and<\/em> to develop students\u2019 contemporary capabilities in an ever-changing world. If we do not change our assessments to align them with the capabilities of generative AI, our degrees will be easier to pass <em>and<\/em> we will have failed to prepare our students for their future.<br \/>\n<a name=\"novice\"><\/a><\/p>\n<h2>What assessments can a novice use generative AI to complete without reliable detection?<\/h2>\n<p>TEQSA recently commissioned two academics from the University of Sydney, Danny Liu and Ben Miller, to make a <a href=\"http:\/\/bit.ly\/teqsa-sydney\" target=\"_blank\" rel=\"noopener\">series of short videos<\/a> demonstrating the capabilities of modern generative AI tools:<\/p>\n<ul>\n<li><a href=\"https:\/\/youtu.be\/8L_HvZClSnA?si=C4ylSUdpAD1zOCGK\" target=\"_blank\" rel=\"noopener\">Gen AI and student learning<\/a><\/li>\n<li><a href=\"https:\/\/youtu.be\/e_PMfpcvlpQ?si=MIiPxHQmerMoUfDp\" target=\"_blank\" rel=\"noopener\">Generative AI and reflective writing<\/a><\/li>\n<li><a href=\"https:\/\/youtu.be\/FUpoRTyNvVE?si=-0kbq7o6N-ixkVJ8\" target=\"_blank\" rel=\"noopener\">Multiple modalities and generative AI<\/a><\/li>\n<\/ul>\n<p>As shown in these videos, students can use these tools to support or to subvert their learning, with <strong>AI able to potentially complete <em>all<\/em> take-home or remotely supervised assessment at a passable level without detection<\/strong>. In the videos, multiple assessment forms are shown to be completed by AI, including (i) MCQs and short answer questions, (ii) essays and literature reviews with real references, (ii) data analysis and scientific reports, including graphs, (iv) course discussion and discussion board posts using our resources, (v) presentations including slides, script and audio, (vi) reflective tasks and writing in any genre, (vii) reflective analysis including its own biases, (viii) online oral exams and (ix) podcasts and videos using the student&#8217;s face and voice. Other assessments or versions of these are also at risk from existing or soon-to-be-produced AI tools.<br \/>\n<a name=\"2-lane\"><\/a><\/p>\n<h2>The two-lane approach to assessment<\/h2>\n<p>The University&#8217;s <strong><a href=\"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/what-to-do-about-assessments-if-we-cant-out-design-or-out-run-ai\/\" target=\"_blank\" rel=\"noopener\">&#8216;two-lane approach&#8217; to assessment<\/a>\u00a0is a realistic and forward-looking approach<\/strong> to assessment design for a world where generative AI is ubiquitous and can mimic many of the products of &#8216;traditional&#8217; assessments (Table 1). It is deliberately binary, because we need to ensure that:<\/p>\n<ol>\n<li>Our graduates are able to demonstrate the knowledge, skills, and dispositions detailed in the course learning outcomes through \u201csecure\u201d assessments (lane 1)<\/li>\n<li>Our graduates have the ability to learn and prosper in the contemporary world through appropriately-designed \u201copen\u201d assessments that support and scaffold the use of all available and relevant tools (lane 2)<\/li>\n<\/ol>\n<p><a name=\"table1\"><\/a><strong>Table 1: <\/strong>Summary of the two-lane approach.<\/p>\n<table border=\"1\" width=\"100%\" cellpadding=\"5\">\n<tbody>\n<tr>\n<td><\/td>\n<td><b>Secure (Lane 1)<\/b><\/td>\n<td><b>Open (Lane 2)<\/b><\/td>\n<\/tr>\n<tr>\n<td><b>Role of assessment<\/b><\/td>\n<td>Assessment <em>of<\/em> learning<\/td>\n<td>Assessment <em>for<\/em> and <em>as<\/em> learning<\/td>\n<\/tr>\n<tr>\n<td><b>Level of operation<\/b><\/td>\n<td>Mainly at program level<\/td>\n<td>Mainly at unit level<\/td>\n<\/tr>\n<tr>\n<td><b>Assessment security<\/b><\/td>\n<td>Secured, in person<\/td>\n<td>\u2018Open\u2019 \/ unsecured<\/td>\n<\/tr>\n<tr>\n<td><b>Role of generative AI<\/b><\/td>\n<td>May or may not be allowed by examiner<\/td>\n<td>As relevant, use of AI scaffolded &amp; supported<\/td>\n<\/tr>\n<tr>\n<td><a href=\"https:\/\/www.teqsa.gov.au\/guides-resources\/resources\/corporate-publications\/assessment-reform-age-artificial-intelligence\">TEQSA alignment<\/a><\/td>\n<td>Principle 2 \u2013 forming trustworthy judgements of student learning<\/td>\n<td>Principle 1 \u2013 equip students to participate ethically and actively in a society pervaded with AI<\/td>\n<\/tr>\n<tr>\n<td><b>Examples<\/b><\/td>\n<td>In person interactive oral assessments; viva voces; contemporaneous in-class assessments and skill development; tests and exams.<\/td>\n<td>AI to provoke reflection, suggest structure, brainstorm ideas, summarise literature, make content, suggest counterarguments, improve clarity, provide formative feedback, etc<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>&nbsp;<\/p>\n<h2>The two-lane approach, our legislative requirements, and the Coursework Policy<\/h2>\n<p>The <a href=\"https:\/\/www.legislation.gov.au\/F2021L00488\/latest\/text\">Higher Education Standards Framework (Threshold Standards) (HESF), requires:<\/a><\/p>\n<p style=\"padding-left: 30px;\">1.4.3 Methods of assessment are consistent with the learning outcomes being assessed, are capable of confirming that all specified learning outcomes are achieved and that grades awarded reflect the level of student attainment.<\/p>\n<p style=\"padding-left: 30px;\">1.4.4 On completion of a course of study, students have demonstrated the learning outcomes specified for the course of study.<\/p>\n<p>The accreditation functions of our professional bodies all have similar statements. For example:<\/p>\n<ul>\n<li>The learning and assessment design must rigorously confirm delivery of the graduate outcomes specification for the program as a whole (<a href=\"https:\/\/www.engineersaustralia.org.au\/sites\/default\/files\/content-files\/2016-12\/110214_G02EA_Curr_Accred_Criteria_Guidelines__Rev_0.PDF\" target=\"_blank\" rel=\"noopener\">Engineers Australia<\/a>).<\/li>\n<li>Graduates of the program demonstrate achievement of all the required performance outcomes (<a href=\"https:\/\/www.pharmacycouncil.org.au\/resources\/pharmacy-program-standards\/\" target=\"_blank\" rel=\"noopener\">Australian Pharmacy Council<\/a>).<\/li>\n<\/ul>\n<p>Both <strong>Threshold Standards 1.4.3 and 1.4.4 require the inclusion of secure (lane 1) assessments in our <em>courses<\/em>\u00a0<\/strong>so that we can demonstrate that <em>all<\/em>\u00a0students have achieved <em>all<\/em> the learning outcomes, irrespective of the pathway they choose. Our course learning outcomes, aligning with our overarching <a href=\"https:\/\/www.sydney.edu.au\/students\/graduate-qualities.html\">Graduate Qualities<\/a>, also require we develop students with contemporary capabilities including cultural competence, ethical identity, information and digital literacy, and communication skills, as well as disciplinary knowledge, skills, and dispositions, through open (lane 2) assessment and related learning activities.<\/p>\n<p>Although TEQSA have not yet published regulations on how we need to demonstrate how our approach to generative AI will allow us to meet the Threshold Standards, the two-lane approach aligns closely with the <a href=\"https:\/\/www.teqsa.gov.au\/guides-resources\/resources\/corporate-publications\/assessment-reform-age-artificial-intelligence\">TEQSA \u2018Assessment reform for the age of artificial intelligence\u2019<\/a> (see Table 1) and the <a href=\"https:\/\/go8.edu.au\/group-of-eight-principles-on-the-use-of-generative-artificial-intelligence\">Group of Eight principles on the use of generative artificial intelligence<\/a>. Our <a href=\"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/rules-access-familiarity-and-trust-a-practical-approach-to-addressing-generative-ai-in-education\/\">recent submission<\/a> to TEQSA\u2019s \u2018Request for Information\u2019 on our response to generative AI, outlined below, has the two-lane approach at the centre.<\/p>\n<p>In November 2023, two new assessment principles were added to the <a href=\"https:\/\/www.sydney.edu.au\/policies\/showdoc.aspx?recnum=PDOC2014\/378&amp;RendNum=0\">Coursework Policy<\/a> to align with the two-lane approach. Assessment principle 6 states that \u201cAssessment practices develop contemporary capabilities in a trustworthy way\u201d. This principle requires that:<\/p>\n<ol style=\"list-style-type: lower-alpha;\">\n<li style=\"list-style-type: lower-alpha;\">assessment practices enable students to demonstrate disciplinary and graduate skills and the ability to work ethically with technologies (including artificial intelligence)<\/li>\n<li style=\"list-style-type: lower-alpha;\">supervised assessments are designed to assure learning in a program;<\/li>\n<li style=\"list-style-type: lower-alpha;\">unsupervised assessments are designed to motivate and drive the process of learning;<\/li>\n<li style=\"list-style-type: lower-alpha;\">where possible, authentic assessments will involve using innovative and contemporary technologies;<\/li>\n<\/ol>\n<p>Requirement (a) relates to the overall approach whilst requirement (b) corresponds to secure (lane 1) and requirements (c) and (d) to open (lane 2) assessments.<\/p>\n<h3>Program-level assessment design<\/h3>\n<p>As noted in Table 1, <strong>it is expected that, in time, most secure (lane 1) assessments will operate at program level<\/strong>. This is also recommended in the <a href=\"https:\/\/www.teqsa.gov.au\/guides-resources\/resources\/corporate-publications\/assessment-reform-age-artificial-intelligence\">TEQSA \u2018Assessment reform for the age of artificial intelligence\u2019<\/a> guidelines. The HESF standards 1.4.3 and 1.4.4 quoted above require us only to assure learning at the <em>course<\/em> or <em>program<\/em> level, with program here referring to a collection of units in a course component such as a major or specialisation. Whilst this could be achieved through assurance in every unit of study, the increasing difficulty of controlling use of generative AI in assessments means that <strong>it is likely to be considerably easier and more reliable to consider assurance at the program level<\/strong>.<\/p>\n<p>Given the complexity and flexibility of many of our courses and their components, assessment principle 5 has also been added to the <a href=\"https:\/\/www.sydney.edu.au\/policies\/showdoc.aspx?recnum=PDOC2014\/378&amp;RendNum=0\">Coursework Policy<\/a>. This states that \u201cAssessment practices must be integrated into program design\u201d. Amongst its requirements is a need that:<\/p>\n<ol style=\"list-style-type: lower-alpha;\">\n<li style=\"list-style-type: lower-alpha;\">students\u2019 learning attainment can be validated across their program at relevant progression points and before graduation;<\/li>\n<li style=\"list-style-type: lower-alpha;\">assessments equip students for success in their studies and in their future, using appropriate technologies; and<\/li>\n<li style=\"list-style-type: lower-alpha;\">academic judgement of student achievement occurs over time through multiple, coherent, and trustworthy assessment tasks.<\/li>\n<\/ol>\n<p>Requirements (a) and (c) require planning of secure (lane 1) assessments at key stages of every program. Requirement (b) builds on assessment principle 5 in requiring students are prepared to use generative AI, and other technologies, well through appropriate open (lane 2) assessments by requiring these are also planned at the program level.<\/p>\n<p>The article \u2018<a href=\"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/program-level-assessment-two-lane\/\">Program level assessment design and the two-lane approach<\/a>\u2019 details the literature, evidence and example typologies for how this might be achieved in our courses.<\/p>\n<h2>Our assessments and academic integrity settings<\/h2>\n<p>In 2024, we ran about 30,000 separate assessments with over 2.2 million student submissions. The labels for the\u00a0<a href=\"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/how-much-assessment-is-enough\/\" target=\"_blank\" rel=\"noopener\">assessment categories and types<\/a>\u00a0attempt to describe to students what to expect as well as to inform special consideration and academic plans. They were not chosen to reflect the two-lane approach described above, but it is estimated that around 15% of the assessments could be considered secure. Around 80% of units (corresponding to ~60% of enrolments) do not have a formal exam, although there are other forms of assessments that may be considered secure outside of examinations. As the difficulty in securing assessments increases, including from the threat of contract cheating, the development of connected devices such as wearables, and the ever-improving ability of generative AI tools, it will become more and more important to plan the placement of secure assessment above the level of the unit of study, at the program level.<\/p>\n<p>For now, the <a href=\"https:\/\/www.sydney.edu.au\/policies\/showdoc.aspx?recnum=PDOC2012\/254&amp;RendNum=0\" target=\"_blank\" rel=\"noopener\">Academic Integrity Policy<\/a> in 2024 states that &#8220;Students <strong>must not<\/strong> &#8230;. generative artificial intelligence to complete assignments unless expressly permitted&#8221; by the unit coordinator. In 2024, coordinators have permitted AI use in around 15% of our assessments. AI is not permitted in around 75% of assessments considered unsecure or where security has not been considered. This poses a clear problem to the validity of these assessments.<br \/>\n<a name=\"Sem1\"><\/a><\/p>\n<h2>Semester 1 2025<\/h2>\n<h3>\u2018Allowable assistance\u2019 in the Academic Integrity Policy in Semester 1 2025<\/h3>\n<p>From Semester 1 2025, the default position in the Academic Integrity Policy has been <em>reversed<\/em> for assessments which are not formal examinations or in-semester tests:<\/p>\n<ol>\n<li>Except for supervised examinations and supervised in-semester tests, students <b>may use<\/b> automated writing tools or generative artificial intelligence to complete assessments, <b>unless expressly prohibited <\/b>by the unit of study coordinator.<\/li>\n<li>Students <b>must not<\/b> use automated writing tools or generative artificial intelligence for examination and in-semester tests, <b>unless expressly permitted<\/b> by the unit of study coordinator.<\/li>\n<\/ol>\n<h3>Unit outlines in Semester 1 2025<\/h3>\n<p>To reflect the change in the Academic Integrity Policy, coordinators will be able to choose from the options shown in the image below, with the default being &#8220;Unsecured assessment full AI&#8221;. For Semester 1, a coordinator can still prohibit the use of generative AI in take-home assessments, but this should be accompanied by an appropriate risk assessment and an elaboration in the unit outline and Canvas. These choices will be reflected in the unit outline, as illustrated in the second image below.<\/p>\n<div class=\"sc-columns half clearfix\">\n<div class=\"col\">\n<p><a href=\"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-content\/uploads\/2024\/11\/Akari-choices-for-semester-1-2025.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter wp-image-22477 size-full\" src=\"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-content\/uploads\/2024\/11\/Akari-choices-for-semester-1-2025.png\" alt=\"Screenshot from Sydney Curriculum showing the choices available to a coordinator to allow or permit the use of generative AI in an assessment.\" width=\"2175\" height=\"454\" srcset=\"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-content\/uploads\/2024\/11\/Akari-choices-for-semester-1-2025.png 2175w, https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-content\/uploads\/2024\/11\/Akari-choices-for-semester-1-2025-300x63.png 300w, https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-content\/uploads\/2024\/11\/Akari-choices-for-semester-1-2025-768x160.png 768w, https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-content\/uploads\/2024\/11\/Akari-choices-for-semester-1-2025-1024x214.png 1024w, https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-content\/uploads\/2024\/11\/Akari-choices-for-semester-1-2025-370x77.png 370w, https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-content\/uploads\/2024\/11\/Akari-choices-for-semester-1-2025-570x119.png 570w, https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-content\/uploads\/2024\/11\/Akari-choices-for-semester-1-2025-770x161.png 770w, https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-content\/uploads\/2024\/11\/Akari-choices-for-semester-1-2025-1170x244.png 1170w\" sizes=\"auto, (max-width: 2175px) 100vw, 2175px\" \/><\/a><\/p>\n<p><small>Choices available in Sydney Curriculum for Semester 1 2025 with &#8220;full AI&#8221; use selected by default.<\/small><\/p>\n<\/div>\n<div class=\"col\">\n<p><a href=\"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-content\/uploads\/2024\/11\/Unit-outline-for-semester-1-2025.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter size-full wp-image-22476\" src=\"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-content\/uploads\/2024\/11\/Unit-outline-for-semester-1-2025.png\" alt=\"Mock up of a unit outline for semester 1 2025 showing an &quot;AI allowed&quot; icon on one assessment.\" width=\"2554\" height=\"841\" srcset=\"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-content\/uploads\/2024\/11\/Unit-outline-for-semester-1-2025.png 2554w, https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-content\/uploads\/2024\/11\/Unit-outline-for-semester-1-2025-300x99.png 300w, https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-content\/uploads\/2024\/11\/Unit-outline-for-semester-1-2025-768x253.png 768w, https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-content\/uploads\/2024\/11\/Unit-outline-for-semester-1-2025-1024x337.png 1024w, https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-content\/uploads\/2024\/11\/Unit-outline-for-semester-1-2025-370x122.png 370w, https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-content\/uploads\/2024\/11\/Unit-outline-for-semester-1-2025-570x188.png 570w, https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-content\/uploads\/2024\/11\/Unit-outline-for-semester-1-2025-770x254.png 770w, https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-content\/uploads\/2024\/11\/Unit-outline-for-semester-1-2025-1170x385.png 1170w, https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-content\/uploads\/2024\/11\/Unit-outline-for-semester-1-2025-1761x580.png 1761w\" sizes=\"auto, (max-width: 2554px) 100vw, 2554px\" \/><\/a><\/p>\n<p><small>Mock-up of a unit outline for Semester 1 2025 showing an icon next to a task where generative AI is allowed.<\/small><\/p>\n<\/div>\n<\/div>\n<a name=\"Sem2\"><\/a><\/p>\n<h2>Semester 2 2025<\/h2>\n<h3>Proposed assessment categories and types<\/h3>\n<p>The labels below have been developed through consultations with staff and students across the University. Further feedback is very welcome.<\/p>\n<h4>Secured<\/h4>\n<p>Secure (lane 1) covers assessment <em>of<\/em> learning. It must be appropriately secured, through in-person supervised assessment. The use of AI and other technologies is controlled by the assessor and may be restricted completely.<\/p>\n<p><a name=\"table2\"><\/a><strong>Table 2: <\/strong>Proposed assessment categories and types for secured assessments for Semester 2 2025.<\/p>\n<table border=\"1\" width=\"100%\" cellpadding=\"5\">\n<tbody>\n<tr>\n<td><b>Category<\/b><\/td>\n<td><b>Assessment types<\/b><\/td>\n<\/tr>\n<tr>\n<td rowspan=\"3\"><b>Final exam \u2013 secured<\/b><\/td>\n<td>Written exam<\/td>\n<\/tr>\n<tr>\n<td>Practical exam<\/td>\n<\/tr>\n<tr>\n<td>Oral exam<\/td>\n<\/tr>\n<tr>\n<td rowspan=\"3\"><b>In-semester test \u2013 secured<\/b><\/td>\n<td>Written test<\/td>\n<\/tr>\n<tr>\n<td>Practical test<\/td>\n<\/tr>\n<tr>\n<td>Oral test<\/td>\n<\/tr>\n<tr>\n<td rowspan=\"5\"><b>In class &#8211; secured<\/b><\/td>\n<td>Interactive oral<\/td>\n<\/tr>\n<tr>\n<td>Practical or skills test<\/td>\n<\/tr>\n<tr>\n<td>In person practical, skills, or performance task or test<\/td>\n<\/tr>\n<tr>\n<td>In person written or creative task<\/td>\n<\/tr>\n<tr>\n<td>Q&amp;A following presentation, submission or placement<\/td>\n<\/tr>\n<tr>\n<td rowspan=\"3\"><b>Placement, internship, or supervision<\/b><\/td>\n<td>Peer or expert observation or supervision<\/td>\n<\/tr>\n<tr>\n<td>In person practical or creative task<\/td>\n<\/tr>\n<tr>\n<td>Clinical exam<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>Coordinators will be able to control the use of generative AI, and other technologies, in secured assessments, including completely restricting their use.<br \/>\n<a name=\"online-assessment\"><\/a><\/p>\n<h4>Note on online courses and units of study<\/h4>\n<p>Our experience of remote assessment in recent years indicates that securing examination style assessments through proctoring is already difficult and costly. The availability of easy to access (and increasingly difficult to detect) powerful generative AI tools on devices and wearables indicates that future-proofing remote assessment is likely to be near impossible. Program level assessment design is therefore paramount to maximise the assurance of learning in secure settings. For example, online postgraduate courses should make use of opportunities for in-person experiential activities including on placements and in intensive sessions. The use of capstone units should also be considered with in-person assessments using exam centres, agreements with other providers etc.<\/p>\n<p>Non-in-person assessments designed to assure learning will be evaluated case by case, based on risk level, and approved by the relevant delegate.<\/p>\n<h4>Open<\/h4>\n<p>Open (lane 2) covers assessment <em>for<\/em> learning and <em>as<\/em> learning. It includes the scaffolded use of appropriate technologies where relevant. Students are assisted in how they use AI and other technologies, with their use fully allowed.<\/p>\n<p>The choice of the category labels has been made to focus on the types of activities known to be effective in learning and the development of evaluative judgement.\u00a0The task types below align with those used by the <a href=\"https:\/\/educationstandards.nsw.edu.au\/wps\/portal\/nesa\/11-12\/Understanding-the-curriculum\/assessment\/assessment-in-practice\/formal-assessment\">NSW Education Authority<\/a>.<\/p>\n<p><a name=\"table3\"><\/a><strong>Table 3: <\/strong>Proposed assessment categories and types for open assessments for Semester 2 2025.<\/p>\n<p>&nbsp;<\/p>\n<table border=\"1\" width=\"100%\" cellpadding=\"5\">\n<tbody>\n<tr>\n<td><b>Category<\/b><\/td>\n<td><b>Assessment types<\/b><\/td>\n<\/tr>\n<tr>\n<td colspan=\"1\" rowspan=\"3\"><strong>Practice or application &#8211; open<\/strong><\/td>\n<td>In-class quiz<\/td>\n<\/tr>\n<tr>\n<td>Out of class quiz<\/td>\n<\/tr>\n<tr>\n<td>Practical skill<\/td>\n<\/tr>\n<tr>\n<td rowspan=\"4\"><b>Inquiry or investigation &#8211; open<\/b><\/td>\n<td>Experimental design<\/td>\n<\/tr>\n<tr>\n<td>Data analysis<\/td>\n<\/tr>\n<tr>\n<td>Case studies<\/td>\n<\/tr>\n<tr>\n<td>Research analysis<\/td>\n<\/tr>\n<tr>\n<td rowspan=\"6\"><strong>Production and creation &#8211; open<\/strong><\/td>\n<td>Portfolio or journal<\/td>\n<\/tr>\n<tr>\n<td>Performance<\/td>\n<\/tr>\n<tr>\n<td>Presentation<\/td>\n<\/tr>\n<tr>\n<td>Creative work<\/td>\n<\/tr>\n<tr>\n<td>Written work<\/td>\n<\/tr>\n<tr>\n<td>Dissertation or thesis<\/td>\n<\/tr>\n<tr>\n<td rowspan=\"4\"><b>Discussion &#8211; open<\/b><\/td>\n<td>Debate<\/td>\n<\/tr>\n<tr>\n<td>Contribution<\/td>\n<\/tr>\n<tr>\n<td>Conversation<\/td>\n<\/tr>\n<tr>\n<td>Evaluation<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>Coordinators will <strong>not<\/strong> be able to control or restrict the use of generative AI, and other technologies, in open (lane 2) assessments. They should, though, scaffold and guide students in how to use them effectively.<\/p>\n<h3><span style=\"color: #000000;\">Special consideration including simple extensions<\/span><\/h3>\n<p><span style=\"color: #000000;\">The introduction of the new approach, categories, and types is an opportunity to rethink special consideration, including simple extension.<\/span><\/p>\n<p><span style=\"color: #000000;\"><b>Secured<\/b> assessments measure learning in the program, and many are likely to become hurdle or \u201cmust pass\u201d tasks. As generative AI tools become ubiquitous in applications and even wearables, controlling their use will become more difficult and potentially intrusive. Special consideration responses need to reflect this.<\/span><\/p>\n<p><span style=\"color: #000000;\"><b>Open <\/b>assessments are for and as learning, so should be universally designed with opportunities for feedback, classroom discussion, and collaboration optimised. This may influence the nature of the special consideration that a task receives.<\/span><\/p>\n<h3><span style=\"color: #000000;\">Academic adjustments<\/span><\/h3>\n<p><span style=\"color: #000000;\">Assessment data, including assessment types, weightings, and due dates, are passed electronically from Sydney Curriculum to Sydney Assist for students who have a disability, medical condition, or carer\u202fresponsibilities. The Inclusion and Disability Services Unit produce <em>personalised<\/em> academic plans with reasonable adjustments based on each student\u2019s circumstances and the assessments. There is no equivalent of the Decisions Matrix and adjustments for non-examination assessments may include additional time, alternative formats or a different assessment. For examinations, students may get additional time, different venues, additional breaks or a scribe.<\/span><\/p>\n<p><span style=\"color: #000000;\"><b>Secured<\/b> assessments measure learning in the program, so academic adjustments must be prioritised.<\/span><\/p>\n<p><span style=\"color: #000000;\"><b>Open<\/b> assessments are for and as learning, so should be universally designed. Although personalised adjustments will always be available, ideally universal design will mean there is little or no need for additional adjustments to be made for most students. For example, restricting time or allowing only one attempt in online quizzes now have little or no educational value, since AI plugins exist which can complete the quiz with good accuracy in little time; m<\/span><span style=\"color: #000000;\">aking such a quiz open for a week with students allowed multiple attempts means that no adjustment is needed for most students.<\/span><\/p>\n<h2>Support and next steps in planning for Semester 2 2025<\/h2>\n<p>Support will be available to faculties, schools and directly to coordinators to make the required changes ahead of Semester 2, including:<\/p>\n<ul>\n<li>1:1 consultations with each coordinator to map existing assessment types to new assessment types.<\/li>\n<li>The selection of new assessment types in Sydney Curriculum based on agreed mapping.<\/li>\n<li>Workshops on new approach and assessment types.<\/li>\n<li>Consultations and educational design support for uplift<\/li>\n<\/ul>\n<p>Academic Standards and Policy Committee (ASPC) will need to approve a number of changes early in 2025, including:<\/p>\n<ul>\n<li>The new assessment categories and types in the Assessment Procedures.<\/li>\n<li>The associated special consideration matrix in the Assessment Procedures.<\/li>\n<li>The settings in the\u00a0Academic Integrity Policy so that generative AI use can only be restricted for Secure (Lane 1) assessments.<\/li>\n<\/ul>\n<h2>Tell me more<\/h2>\n<ul>\n<li><a href=\"https:\/\/www.youtube.com\/playlist?list=PLts-Ei4tTTZ5HOGi9RFixZcNVzml9XcLT\" target=\"_blank\" rel=\"noopener\">TEQSA videos on generative AI and student learning<\/a><\/li>\n<li><a href=\"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/frequently-asked-questions-about-the-two-lane-approach-to-assessment-in-the-age-of-ai\/\">FAQs about the two lane approach<\/a><\/li>\n<li><a href=\"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/program-level-assessment-two-lane\/\" target=\"_blank\" rel=\"noopener\">Program level assessment design and the two-lane approach<\/a><\/li>\n<li><a href=\"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/frequently-asked-questions-about-generative-ai-at-sydney\/\" target=\"_blank\" rel=\"noopener\">Frequently asked questions about generative AI at Sydney<\/a><\/li>\n<li><a href=\"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/rules-access-familiarity-and-trust-a-practical-approach-to-addressing-generative-ai-in-education\/\" target=\"_blank\" rel=\"noopener\">Rules, access, familiarity, and trust \u2013 A practical approach to addressing generative AI in education<\/a> (includes the University of Sydney\u2019s response to the TEQSA\u2019s request for information)<\/li>\n<li><a href=\"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/menus-not-traffic-lights-a-different-way-to-think-about-ai-and-assessments\/\" target=\"_blank\" rel=\"noopener\">Menus, not traffic lights: A different way to think about AI and assessments<\/a><\/li>\n<li><a href=\"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/embracing-the-future-of-assessment-at-the-university-of-sydney\/\" target=\"_blank\" rel=\"noopener\">Embracing the future of assessment at the University of Sydney<\/a><\/li>\n<\/ul>\n","protected":false},"excerpt":{"rendered":"<p>TL;DR: The reality of assessment in the age of generative AI Generative AI tools exist that can accurately create content (including written work, images,&#8230;<\/p>\n","protected":false},"author":18,"featured_media":22503,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1368,1367,56],"tags":[],"coauthors":[462,488,463],"class_list":["post-22452","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-assessment","category-educational-integrity","category-news-events","post-item","post-even"],"_links":{"self":[{"href":"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-json\/wp\/v2\/posts\/22452","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-json\/wp\/v2\/users\/18"}],"replies":[{"embeddable":true,"href":"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-json\/wp\/v2\/comments?post=22452"}],"version-history":[{"count":45,"href":"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-json\/wp\/v2\/posts\/22452\/revisions"}],"predecessor-version":[{"id":22469,"href":"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-json\/wp\/v2\/posts\/22452\/revisions\/22469"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-json\/wp\/v2\/media\/22503"}],"wp:attachment":[{"href":"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-json\/wp\/v2\/media?parent=22452"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-json\/wp\/v2\/categories?post=22452"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-json\/wp\/v2\/tags?post=22452"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-json\/wp\/v2\/coauthors?post=22452"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}