The University of Sydney is leading the way in embracing Generative Artificial Intelligence (GAI) in education, particularly in assessment. Starting Semester 2, 2025, a two-lane approach will be introduced: Lane 1 (“secure”) assessments requiring in-person attendance with GAI prohibited, and Lane 2 (“open”) assessments where students can use GAI tools. This approach aims to balance assessment integrity concerns while preparing students with the GAI literacy that employers increasingly expect.
Motivated by these rapid developments in GAI and the pedagogical challenges they present, we share our experience implementing two GAI-integrated assessments in the School of Economics during 2023 and 2024.
This article was a collaboration of academics from the School of Economics: A/Prof. Mark Melatos (Assessment Program Director, Faculty of Arts and Social Sciences, Unit Coordinator, ECON1002, S1-2023), Dr. Chandana Maitra (Unit Coordinator, ECON7030) and Dr. Jordi Vidal Robert (Unit Coordinator, ECON1002, S1-2024)
We designed our assessments for two distinctly different student cohorts, including large numbers of international students: ECON1002 (a first-year undergraduate economics unit with approximately 1,000 students per semester) and ECON7030 (a Master of Economics postgraduate capstone with around 200 students per semester). Despite the different academic levels, our approach in both units centred on AI-Human collaboration – requiring students to critically evaluate GAI-generated content before enhancing it with their own insights. Students learned to craft thoughtful prompts, critically assess GAI outputs, and recognise the tool’s limitations, applying their academic knowledge to overcome these constraints.
Responding to educational challenges
In ECON1002, one of the key assessments has traditionally been a structured essay (worth 30%) run as an in-person exam. The move to remote learning during the COVID-19 pandemic, as well as the emergence of ChatGPT, created insurmountable challenges for this assessment. For example, even post-Pandemic (S1 2023) in-person mid-term exams were not permitted for some classes, with many students still studying remotely.
In response to this new reality, an alternative writing task was needed that would:
- develop research, analytical and writing skills,
- be relatively “ChatGPT-proof” despite being ‘take-home’, and
- minimise the risk of contract teaching,
- teach students to use AI appropriately as a research aid rather than a substitute.
The new challenges posed by GAI technology actually created a new pedagogical opportunity to develop an assessment that could address the crucial and often overlooked professional skills of “asking relevant questions to elicit information” and “evaluating the accuracy of information”. Such skills are not formally taught, let alone practised, in economics programs despite being crucial in the workplace.
In ECON7030, students are required to submit a research proposal (weight 25%) via Canvas assignment in Week 4 of the semester, which incorporates a detailed literature review on the topic of their choice. With the increasing accessibility and popularity of ChatGPT, ensuring academic integrity became challenging. As with ECON1002, we also wanted to guide students toward meaningful, efficient, and responsible GAI use to enhance their learning.
What did we do?
In ECON1002, the GAI assignment consists of two parts: (i) engaging ChatGPT in a conversation on the essay topic, and (ii) critically analysing ChatGPT’s responses.
When we ran this assignment again in Semester 2, 2024, we implemented several enhancements to increase its pedagogical value. These included running GAI practice exercises – mirroring the assessment – in tutorials. This clarified the assignment requirements and consolidated lessons on effective AI use. Moreover, the marking rubric was adjusted to: (i) link ChatGPT dialogue and critique more directly to knowledge acquired in class and (ii) place greater emphasis on critical thinking.
In ECON7030 (Semesters 1 & 2, 2024), we formally authorised students to use ChatGPT for their literature reviews while adding a new task focused on critical evaluation of AI-generated content. Key features of this revised assessment included:
- Emphasis on effective prompt design
- Critical evaluation of AI-generated outputs
- Requirement to clearly identify ChatGPT-generated sources in reference lists
The marking rubric was updated with four additional criteria to evaluate the ‘Co-creation with AI’ section, following guidelines from the University’s recent AI policy discussions. We have included an example of how we evaluated co-creation with AI in the table below.
| Evaluating co-creation with AI | |
| Include a brief evaluation of your interaction with ChatGPT versus your own search of the scholarly literature.
✓ What was your prompt? ✓ Do you think you could solely rely on AI generated output? Why or why not? ✓ How do the scholarly citations that you included improve upon ChatGPT generated output? |
Marking Criteria
✓ Prompt design demonstrates disciplinary expertise: How thoughtfully students design the prompt(s) – consider complexity and clarity. ✓ Critical evaluation: How effectively student evaluates and utilises AI suggestions – simply adopting AI-generated content versus making conscious choices about what to include. ✓ Information and digital literacy: How innovatively student complements AI-generated content by relevant scholarly sources to enhance the rigour and reliability of the output. ✓ Ethical consideration: Student demonstrates awareness of the reliability, biases, and other limitations of AI generated content. |
What did the students say?
In both units, many students appreciated our positive (not punitive) approach to AI integration, which aligned with the University’s subsequent move to permit AI use in open assessments. In ECON7030, students (Semester 2, 2024) rated the initiative 4.5 out of 5. Among survey participants, 75% favoured continuing with the new design. Interestingly, some students had no prior experience with ChatGPT, and this assessment provided their first opportunity to learn about the tool.
| ECON1002 2024 | ECON7030 2024 |
|---|---|
| “I found the assessment task both challenging and rewarding as it called for your own consideration and understanding.”
“I liked the assessment based on ChatGPT, that was a good way to show us how AI can be a helpful tool. I like that this course integrated how to use AI” “The essay task using ChatGPT is ridiculous” |
“We got to see why it should not be used as a standalone tool and instead as an aid to paraphrase at best.”
“The chatGPT in A3 is also worth keeping, as its presence gives me a sense that AI cannot replace our own thinking.” “I’m indifferent about this. Actually none of my friends use gpt to generate citations, because we know it makes up citations.” |
The feedback reveals both enthusiasm and some scepticism, highlighting the importance of continuous refinement in our GAI integration approach. Students particularly valued learning when and how to critically evaluate AI outputs – a skill increasingly relevant in their academic and professional futures.
What next?
Building on our success
In ECON1002, future iterations of the AI assignment will focus on:
- Providing more guidance during tutorials, especially for formulating effective prompts
- Teaching students how to ask appropriate questions of AI tools
- Improving students’ ability to enhance AI-generated answers using unit content
For ECON7030, based on student feedback, we plan to:
- Provide clearer context and instructions
- Share exemplars of successful AI integration
- Follow the ECON1002 model by dedicating tutorial time for students to practice and discuss the task
Broader implications for teaching practice
As economists, we understand the complexities of decision-making when tradeoffs are involved. When incorporating GAI tools into student learning, both teachers and students need to recognise the limitations of these tools while leveraging their potential to deliver enhanced and more equitable learning experiences.
Our approach embraces a “human-centered approach to AI education” that:
- Promotes critical thinking and ethical considerations – traditional focuses of university education
- Encourages responsible use of new technology
- Prepares students for workplaces where AI collaboration is increasingly common
These assessments offer a model that other educators can adapt across disciplines, focusing on the critical evaluation of AI outputs rather than attempting to prohibit AI use. By teaching students to recognise when and how to use AI effectively, we’re equipping them with professional skills that extend well beyond our economics context.
