{"id":23178,"date":"2025-05-08T07:45:28","date_gmt":"2025-05-07T21:45:28","guid":{"rendered":"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/?p=23178"},"modified":"2025-05-08T23:29:16","modified_gmt":"2025-05-08T13:29:16","slug":"making-peace-with-ai-chatgpt-integrated-assessments-in-economics","status":"publish","type":"post","link":"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/making-peace-with-ai-chatgpt-integrated-assessments-in-economics\/","title":{"rendered":"Making Peace with AI &#8211; ChatGPT-Integrated Assessments in Economics"},"content":{"rendered":"<p>The University of Sydney is leading the way in embracing Generative Artificial Intelligence (GAI) in education, particularly in assessment. Starting Semester 2, 2025, a\u00a0<a href=\"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/the-sydney-assessment-framework\/\" target=\"_blank\" rel=\"noopener\">two-lane approach<\/a>\u00a0will be introduced: Lane 1 (&#8220;secure&#8221;) assessments requiring in-person attendance with GAI prohibited, and Lane 2 (&#8220;open&#8221;) assessments where students can use GAI tools. This approach aims to balance assessment integrity concerns while preparing students with the GAI literacy that employers increasingly expect.<\/p>\n<p>Motivated by these rapid developments in GAI and the pedagogical challenges they present, we share our experience implementing two GAI-integrated assessments in the School of Economics during 2023 and 2024.<\/p>\n<p><em>This article was a collaboration of academics from the School of Economics: A\/Prof. Mark Melatos (Assessment Program Director, Faculty of Arts and Social Sciences, Unit Coordinator, ECON1002, S1-2023), Dr. Chandana Maitra (Unit Coordinator, ECON7030) and Dr. Jordi Vidal Robert (Unit Coordinator, ECON1002, S1-2024)<\/em><\/p>\n<div class=\"sc-separator type-thin\"><\/div>\n<p>We designed our assessments for two distinctly different student cohorts, including large numbers of international students: ECON1002 (a first-year undergraduate economics unit with approximately 1,000 students per semester) and ECON7030 (a Master of Economics postgraduate capstone with around 200 students per semester). Despite the different academic levels, <strong>our approach in both units centred on\u00a0<a href=\"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/what-to-do-about-assessments-if-we-cant-out-design-or-out-run-ai\/\">AI-Human collaboration<\/a><\/strong>\u00a0&#8211; requiring students to critically evaluate GAI-generated content before enhancing it with their own insights. Students learned to craft thoughtful prompts, critically assess GAI outputs, and recognise the tool&#8217;s limitations, applying their academic knowledge to overcome these constraints.<\/p>\n<h2>Responding to educational challenges<\/h2>\n<p>In ECON1002, one of the key assessments has traditionally been a structured essay (worth 30%) run as an in-person exam. The move to remote learning during the COVID-19 pandemic, as well as the emergence of ChatGPT, created insurmountable challenges for this assessment. For example, even post-Pandemic (S1 2023) in-person mid-term exams were not permitted for some classes, with many students still studying remotely.<\/p>\n<p>In response to this new reality, an alternative writing task was needed that would:<\/p>\n<ul>\n<li>develop research, analytical and writing skills,<\/li>\n<li>be relatively &#8220;ChatGPT-proof&#8221; despite being \u2018take-home\u2019, and<\/li>\n<li>minimise the risk of contract teaching,<\/li>\n<li class=\"whitespace-normal break-words\">teach students to use AI appropriately as a research aid rather than a substitute.<\/li>\n<\/ul>\n<p>The new challenges posed by GAI technology actually created a new pedagogical opportunity to develop an assessment that could address the crucial and often overlooked professional skills of \u201casking relevant questions to elicit information\u201d and \u201cevaluating the accuracy of information\u201d. Such skills are not formally taught, let alone practised, in economics programs despite being crucial in the workplace.<\/p>\n<p>In ECON7030, students are required to submit a research proposal (weight 25%) via Canvas assignment in Week 4 of the semester, which incorporates a detailed literature review on the topic of their choice. With the increasing accessibility and popularity of ChatGPT, ensuring academic integrity became challenging. As with ECON1002, we also wanted to guide students toward meaningful, efficient, and responsible GAI use to enhance their learning.<\/p>\n<h2>What did we do?<\/h2>\n<figure id=\"attachment_23180\" aria-describedby=\"caption-attachment-23180\" style=\"width: 397px\" class=\"wp-caption alignright\"><a href=\"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-content\/uploads\/2025\/05\/ECON1002-Assessment-Structure.png\"><img loading=\"lazy\" decoding=\"async\" class=\"wp-image-23180 \" src=\"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-content\/uploads\/2025\/05\/ECON1002-Assessment-Structure-300x229.png\" alt=\"\" width=\"397\" height=\"303\" srcset=\"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-content\/uploads\/2025\/05\/ECON1002-Assessment-Structure-300x229.png 300w, https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-content\/uploads\/2025\/05\/ECON1002-Assessment-Structure-768x586.png 768w, https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-content\/uploads\/2025\/05\/ECON1002-Assessment-Structure-1024x781.png 1024w, https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-content\/uploads\/2025\/05\/ECON1002-Assessment-Structure-370x282.png 370w, https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-content\/uploads\/2025\/05\/ECON1002-Assessment-Structure-570x435.png 570w, https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-content\/uploads\/2025\/05\/ECON1002-Assessment-Structure-770x587.png 770w, https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-content\/uploads\/2025\/05\/ECON1002-Assessment-Structure-1170x892.png 1170w, https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-content\/uploads\/2025\/05\/ECON1002-Assessment-Structure-760x580.png 760w, https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-content\/uploads\/2025\/05\/ECON1002-Assessment-Structure.png 1960w\" sizes=\"auto, (max-width: 397px) 100vw, 397px\" \/><\/a><figcaption id=\"caption-attachment-23180\" class=\"wp-caption-text\"><em><span style=\"color: #000000;\">How AI-Human collaboration works in ECON1002 assessment (click to enlarge)<\/span><\/em><\/figcaption><\/figure>\n<p><strong>In ECON1002<\/strong>, the GAI assignment consists of two parts: (i) engaging ChatGPT in a conversation on the essay topic, and (ii) critically analysing ChatGPT&#8217;s responses.<\/p>\n<p>When we ran this assignment again in Semester 2, 2024, we implemented several enhancements to increase its pedagogical value. These included running GAI practice exercises \u2013 mirroring the assessment \u2013 in tutorials.\u00a0 This clarified the assignment requirements and consolidated lessons on effective AI use.\u00a0 Moreover, the marking rubric was adjusted to: (i) link ChatGPT dialogue and critique more directly to knowledge acquired in class and (ii) place greater emphasis on critical thinking.<\/p>\n<p class=\"whitespace-pre-wrap break-words\"><strong>In ECON7030<\/strong> (Semesters 1 &amp; 2, 2024), we formally authorised students to use ChatGPT for their literature reviews while adding a new task focused on critical evaluation of AI-generated content. Key features of this revised assessment included:<\/p>\n<ul class=\"[&amp;:not(:last-child)_ul]:pb-1 [&amp;:not(:last-child)_ol]:pb-1 list-disc space-y-1.5 pl-7\">\n<li class=\"whitespace-normal break-words\">Emphasis on effective prompt design<\/li>\n<li class=\"whitespace-normal break-words\">Critical evaluation of AI-generated outputs<\/li>\n<li class=\"whitespace-normal break-words\">Requirement to clearly identify ChatGPT-generated sources in reference lists<\/li>\n<\/ul>\n<p class=\"whitespace-pre-wrap break-words\">The marking rubric was updated with four additional criteria to evaluate the &#8216;Co-creation with AI&#8217; section, following guidelines from the <a href=\"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/what-to-do-about-assessments-if-we-cant-out-design-or-out-run-ai\/\">University&#8217;s recent AI policy discussions<\/a>. <span style=\"color: #000000;\">We have included an example of how we evaluated co-creation with AI in the table below.<\/span><\/p>\n<table style=\"width: 100%;\" border=\"1\" cellspacing=\"0\" cellpadding=\"10\">\n<tbody>\n<tr style=\"color: white; text-align: center;\" bgcolor=\"#4a90e2\">\n<td colspan=\"2\"><strong>Evaluating co-creation with AI<\/strong><\/td>\n<\/tr>\n<tr>\n<td valign=\"top\" bgcolor=\"#cce0ff\" width=\"50%\"><strong>Include a brief evaluation of your interaction with ChatGPT versus your own search of the scholarly literature.<\/strong><\/p>\n<p>\u2713 What was your prompt?<\/p>\n<p>\u2713 Do you think you could solely rely on AI generated output? Why or why not?<\/p>\n<p>\u2713 How do the scholarly citations that you included improve upon ChatGPT generated output?<\/td>\n<td valign=\"top\" bgcolor=\"#f0f8ff\" width=\"50%\"><strong>Marking Criteria<\/strong><\/p>\n<p>\u2713 <strong>Prompt design demonstrates disciplinary expertise:<\/strong> How thoughtfully students design the prompt(s) &#8211; consider complexity and clarity.<\/p>\n<p>\u2713 <strong>Critical evaluation:<\/strong> How effectively student evaluates and utilises AI suggestions &#8211; simply adopting AI-generated content versus making conscious choices about what to include.<\/p>\n<p>\u2713<strong> Information and digital literacy:<\/strong> How innovatively student complements AI-generated content by relevant scholarly sources to enhance the rigour and reliability of the output.<\/p>\n<p>\u2713 <strong>Ethical consideration:<\/strong> Student demonstrates awareness of the reliability, biases, and other limitations of AI generated content.<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>&nbsp;<\/p>\n<h2>What did the students say?<\/h2>\n<p>In both units, many students appreciated our positive (not punitive) approach to AI integration, which aligned with the University&#8217;s subsequent move to permit AI use in open assessments. In ECON7030, students (Semester 2, 2024) rated the initiative 4.5 out of 5. Among survey participants, 75% favoured continuing with the new design. Interestingly, some students had no prior experience with ChatGPT, and this assessment provided their first opportunity to learn about the tool.<\/p>\n<table style=\"width: 100%;\" border=\"1\" cellspacing=\"0\" cellpadding=\"10\">\n<tbody>\n<tr>\n<th style=\"text-align: center; background-color: #f2f2f2; padding: 8px;\"><strong>ECON1002 2024<\/strong><\/th>\n<th style=\"text-align: center; background-color: #f2f2f2; padding: 8px;\"><strong>ECON7030 2024<\/strong><\/th>\n<\/tr>\n<tr>\n<td style=\"padding: 8px; vertical-align: top;\"><em>&#8220;I found the assessment task both challenging and rewarding as it called for your own consideration and understanding.&#8221;<\/em><\/p>\n<p><em>&#8220;I liked the assessment based on ChatGPT, that was a good way to show us how AI can be a helpful tool. I like that this course integrated how to use AI&#8221;<\/em><\/p>\n<p style=\"text-align: center;\"><em>&#8220;The essay task using ChatGPT is ridiculous&#8221;<\/em><\/p>\n<\/td>\n<td style=\"padding: 8px; vertical-align: top;\"><em>&#8220;We got to see why it should not be used as a standalone tool and instead as an aid to paraphrase at best.&#8221;<\/em><\/p>\n<p><em>&#8220;The chatGPT in A3 is also worth keeping, as its presence gives me a sense that AI cannot replace our own thinking.&#8221;<\/em><\/p>\n<p><em>&#8220;I&#8217;m indifferent about this. Actually none of my friends use gpt to generate citations, because we know it makes up citations.&#8221;<\/em><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>The feedback reveals both enthusiasm and some scepticism, highlighting the importance of continuous refinement in our GAI integration approach. Students particularly valued learning when and how to critically evaluate AI outputs &#8211; a skill increasingly relevant in their academic and professional futures.<\/p>\n<h2>What next?<\/h2>\n<h3 class=\"text-lg font-bold text-text-100 mt-1 -mb-1.5\">Building on our success<\/h3>\n<p class=\"whitespace-pre-wrap break-words\">In <strong>ECON1002<\/strong>, future iterations of the AI assignment will focus on:<\/p>\n<ul class=\"[&amp;:not(:last-child)_ul]:pb-1 [&amp;:not(:last-child)_ol]:pb-1 list-disc space-y-1.5 pl-7\">\n<li class=\"whitespace-normal break-words\">Providing more guidance during tutorials, especially for formulating effective prompts<\/li>\n<li class=\"whitespace-normal break-words\">Teaching students how to ask appropriate questions of AI tools<\/li>\n<li class=\"whitespace-normal break-words\">Improving students&#8217; ability to enhance AI-generated answers using unit content<\/li>\n<\/ul>\n<p class=\"whitespace-pre-wrap break-words\">For <strong>ECON7030<\/strong>, based on student feedback, we plan to:<\/p>\n<ul class=\"[&amp;:not(:last-child)_ul]:pb-1 [&amp;:not(:last-child)_ol]:pb-1 list-disc space-y-1.5 pl-7\">\n<li class=\"whitespace-normal break-words\">Provide clearer context and instructions<\/li>\n<li class=\"whitespace-normal break-words\">Share exemplars of successful AI integration<\/li>\n<li class=\"whitespace-normal break-words\">Follow the ECON1002 model by dedicating tutorial time for students to practice and discuss the task<\/li>\n<\/ul>\n<h3>Broader implications for teaching practice<\/h3>\n<p class=\"whitespace-pre-wrap break-words\">As economists, we understand the complexities of decision-making when tradeoffs are involved. When incorporating GAI tools into student learning, both teachers and students need to recognise the limitations of these tools while leveraging their potential to deliver enhanced and more equitable learning experiences.<\/p>\n<p class=\"whitespace-pre-wrap break-words\">Our approach embraces a &#8220;<a href=\"https:\/\/www.unesco.org\/en\/digital-education\/artificial-intelligence\" target=\"_blank\" rel=\"noopener\">human-centered approach to AI education<\/a>&#8221; that:<\/p>\n<ul class=\"[&amp;:not(:last-child)_ul]:pb-1 [&amp;:not(:last-child)_ol]:pb-1 list-disc space-y-1.5 pl-7\">\n<li class=\"whitespace-normal break-words\">Promotes critical thinking and ethical considerations &#8211; traditional focuses of university education<\/li>\n<li class=\"whitespace-normal break-words\">Encourages responsible use of new technology<\/li>\n<li class=\"whitespace-normal break-words\">Prepares students for workplaces where AI collaboration is increasingly common<\/li>\n<\/ul>\n<p class=\"whitespace-pre-wrap break-words\">These assessments offer a model that other educators can adapt across disciplines, focusing on the critical evaluation of AI outputs rather than attempting to prohibit AI use. By teaching students to recognise when and how to use AI effectively, we&#8217;re equipping them with professional skills that extend well beyond our economics context.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>The University of Sydney is leading the way in embracing Generative Artificial Intelligence (GAI) in education, particularly in assessment. Starting Semester 2, 2025, a\u00a0two-lane&#8230;<\/p>\n","protected":false},"author":3252,"featured_media":23248,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[31,1368,1367],"tags":[],"coauthors":[1846,3799,3729],"class_list":["post-23178","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-arts","category-assessment","category-educational-integrity","post-item","post-even"],"_links":{"self":[{"href":"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-json\/wp\/v2\/posts\/23178","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-json\/wp\/v2\/users\/3252"}],"replies":[{"embeddable":true,"href":"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-json\/wp\/v2\/comments?post=23178"}],"version-history":[{"count":45,"href":"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-json\/wp\/v2\/posts\/23178\/revisions"}],"predecessor-version":[{"id":23280,"href":"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-json\/wp\/v2\/posts\/23178\/revisions\/23280"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-json\/wp\/v2\/media\/23248"}],"wp:attachment":[{"href":"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-json\/wp\/v2\/media?parent=23178"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-json\/wp\/v2\/categories?post=23178"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-json\/wp\/v2\/tags?post=23178"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-json\/wp\/v2\/coauthors?post=23178"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}