{"id":22061,"date":"2024-09-23T07:19:53","date_gmt":"2024-09-22T21:19:53","guid":{"rendered":"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/?p=22061"},"modified":"2024-09-23T07:19:53","modified_gmt":"2024-09-22T21:19:53","slug":"using-generative-ai-as-part-of-assessments-cheating-or-opportunity","status":"publish","type":"post","link":"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/using-generative-ai-as-part-of-assessments-cheating-or-opportunity\/","title":{"rendered":"Using generative AI as part of assessments: Cheating or opportunity?"},"content":{"rendered":"

What do you want students to learn and achieve when they complete assessments?<\/p>\n

Are there opportunities for AI to improve how students achieve these learning outcomes?<\/p>\n

I coordinate a second-year unit of study, Genetics and Genomics, with ~400 students, and like many science subjects, students write a practical report about their experiments. While these are important skills to develop in science, working with research literature and writing scientifically is more difficult for students and researchers from linguistically diverse backgrounds<\/a>. This semester, 49% of students enrolled in Genetics and Genomics speak a language other than English at home.<\/p>\n

Speaking a language other than English at home and feeling a lack of support in their learning have been associated with contract cheating<\/a>. Part of the appeal are the quick responses<\/a>, which are difficult to match in large units of study.<\/p>\n

Reminding students of the available support, including generative AI<\/h2>\n
\"Signposting<\/a>
Reminder to students about legitimate support for their assessments. Click to expand.<\/figcaption><\/figure>\n

This semester, I tried to address this by reminding students of the support available to them for their assessments, including generative AI<\/strong>. In my Canvas instructions for the practical report, the first thing students see is a warning about contract cheating, and a reminder about legitimate sources of support, including \u2018GeneAI\u2019 (a Socratic tutor that I built in Cogniti<\/a> that is available to students 24\/7), the discussion board, and the Learning Hub<\/a>.<\/p>\n

I built \u2018GeneAI\u2019 to be a Socratic tutor for students to discuss concepts related to genetics and genomics. Students learn at different paces and even with the option of anonymity on discussion boards, some students may be reluctant to ask what they think to be \u2018basic\u2019 questions. Some science subjects, genetics and genomics included, introduce a lot of new vocabulary that can be difficult for students to learn. I have seen questions about vocabulary asked of GeneAI and in other similar Cogniti agents, so these AI agents can provide students opportunities to ask questions that they would otherwise be too shy to get help for.<\/p>\n

\n
\n
\"Part<\/a>
Part of the agent prompt of \u2019GeneAI\u2018, a Socratic tutor that I built myself that discusses material in Genetics and Genomics. Click to expand.<\/figcaption><\/figure>\n<\/div>\n
\n
\"Example<\/a>
Example of a conversation with GeneAI.<\/figcaption><\/figure>\n<\/div>\n<\/div>\n

As students often ask for practice questions in preparation for quizzes and exams, Bloom\u2019s Taxonomy has also been included in the agent\u2019s design so that students can generate as many practice questions as they need. As a bonus, this approach has also been helpful for me to generate questions that I could select from and edit for quizzes and tutorials.<\/p>\n

I embedded this agent in Canvas, which enabled me to scaffold with additional resources to support students\u2019 interactions with AI (Cogniti and Copilot).<\/p>\n

\"List<\/a>
Additional resources provided to students above the Agent.<\/figcaption><\/figure>\n

Being clear about how to engage with AI: Helpful vs less helpful<\/h2>\n
\"Guidance<\/a>
Assignment instructions suggesting examples of how AI may be helpful for the task. Click to expand.<\/figcaption><\/figure>\n

The assignment instructions encourage students to work with AI, with examples of where I believe AI may be helpful for this task.<\/strong> Given these suggestions, students are allowed to use AI however they choose, but are warned about properly reading the literature, referencing, and privacy concerns with any data they submit. I made these suggestions based on the assessment instructions, observations of how some (not all) AI tools may make up references, and the understanding that these are language models, so they may not be so helpful with numerical data. However, I am aware that students have different experiences with AI tools and so I avoid suggestions of how to or how not to use it.<\/p>\n

At the end of the assessment, students are asked to reflect on how they used AI, or their choice not to use AI. I also provide some real examples of my reflections using it to help write rubrics and quiz questions.<\/p>\n

What is stopping students from writing the whole report with AI? Nothing. However, there are parts that may require understanding of the experiments and theory to know what to ask. This is not a perfect solution, but this task is an opportunity to learn these concepts in preparation for the secured final exam.<\/p>\n

I have found that giving students this guidance and asking them to reflect on AI use has given me insight into what they need help with, how they are interacting with AI to achieve this, and their proficiency in working with AI<\/strong>. It was interesting to read about what tasks they felt AI could be more helpful for, and their understanding of what they need to be particularly careful of checking. Besides improving their writing, I was pleased to see many students describing a collaborative approach where they used AI to help them understand the concepts, which they would then apply to their report discussions and literature search.<\/p>\n

\"Guidance<\/a>
Questions for students to reflect on their use of AI. Click to expand.<\/figcaption><\/figure>\n

AI tools, such as Cogniti<\/a>, will improve how we support students \u2013 we have already seen this in semester 1 with Dr MattTabolism<\/a> and Dr NucleicAlice. We are also still learning how to use these tools and we can support students to do the same by guiding them to experiment as we do and observe where the benefits and limitations are.<\/p>\n

Things you could try in your units<\/h2>\n

Things to try:<\/p>\n