There has been a lot of hype over the last few weeks about students using ChatGPT, a conversational artificial intelligence (AI) that can write human-sounding responses to prompts. Some of the main concerns are that students might use ChatGPT to answer assignment or exam questions, without developing their own writing or critical thinking skills, or disciplinary knowledge. In a previous article, we covered how GPT works, and some important concepts that help us appreciate what this AI does and doesn’t do. Importantly, we emphasised that ChatGPT is not a knowledge model but a language model, albeit a powerful one. Here, we share some ideas about how we might productively engage with ChatGPT, a form of ‘generative’ AI, to make higher education learning, teaching, and assessment better in many ways – for both teachers and students.
Check out the other resources in our AI and Education at Sydney collection.
For teachers
We’re all thinking the same thing – not another disruption that we need to deal with. First emergency remote teaching because of COVID, then myriad systems changes, and now artificial intelligence? But let us share a few ways that AI can actually make your teaching easier and improve students’ learning.
Writing quiz questions and feedback
A perennial challenge is providing students with meaningful formative questions on which they can practise, and with accompanying feedback that helps them learn from mistakes. We’ve promoted Peerwise before for its ability to engage students in deeper understanding by generating their own questions and feedback, whilst creating a question bank for practice. But if there’s little space in your curriculum for that, consider using ChatGPT to generate practice quiz questions with feedback, and load these into a Canvas quiz. Given the inaccuracies and tendency for verbosity we highlighted in the earlier article, ensure that you check the accuracy and difficulty of the questions and feedback that ChatGPT generates, and the alignment to your learning outcomes. Your students will thank you for the extra practice.
Example prompt for ChatGPT: Write three college-level multiple choice questions that target key student misconceptions around how natural selection works. Include feedback for students.
Designing a draft marking rubric
A marking rubric is a powerful table that helps you, your markers, and your students better understand what is expected against marking criteria at different standards of performance. A good rubric will reduce the workload associated with marking and engage your students with the feedback they receive. Each marking criterion is a row in the table, and standards (high distinction to fail) are presented as columns, with a descriptor for each criterion/standard. Rubrics, however, can be difficult to write from scratch. If you provide ChatGPT with a short description of your assessment, and what you’d like students to be assessed on, it can generate a draft rubric that you can then finesse.
Example prompt for ChatGPT: Design a marking rubric for a postgraduate assessment that asks students to apply their knowledge of the global financial crisis to a more contemporary economic challenge. The rubric needs to assess students on their use of literature, their analysis of the underlying causes of the GFC, and apply it creatively to a contemporary challenge. Please provide standards for each criterion from high distinction, distinction, credit, pass, and fail.
Composing exemplars for critique
In addition to rubrics, another powerful tool for helping students better understand expected standards is to provide exemplars. Exemplars are samples of work, often presented with commentary at different levels of performance, to demonstrate and explain the expectations. Exemplars can be hard to come by, especially for a new assessment. Instead of composing an exemplar from scratch, consider asking ChatGPT to generate a few for you, and then amend these drafts based on your expectations and standards. You might want to be up front that the exemplars were produced by ChatGPT to highlight its current linguistic and conceptual limitations, and to model the transparency we expect of students when they use it.
Example prompt for ChatGPT: Create a sample student paper that analyses the underlying causes of the GFC. This paper should achieve a credit standard, demonstrating only a basic understanding and identifying only one or two key causes.
(It’s worth noting that when ChatGPT responded to the above text, it also commented that “It’s worth to notice [sic] that this sample student paper demonstrates a basic understanding of the causes of GFC, it identifies only one or two key causes and lacks a critical evaluation of them, it also doesn’t connect the causes to the consequences of the crisis.”).
Generating discussion prompts for use in class
Because ChatGPT is trained on a huge diet of written content available on the internet, it has linguistic coverage of a vast array of topic areas, opinions, and other text. Based on the examples it ‘learned’ during training, it can give statistically-likely responses to any textual prompt. Although this can sometimes mean that the AI generates inaccurate or biased content, it can provide text that represents human-like opinions and perspectives fairly well. You can leverage this to spark discussion in class, without spending too much time racking your brain for these. Depending on your discipline, you might ask ChatGPT to generate some discussion prompts, edit them based on your expertise and experience, and use them in class. Again, you should consider being up front about the source of the prompts, to illustrate that ChatGPT reflects the biases present in the human-written sources upon which it was trained.
Example prompt for ChatGPT: Bill is a 21 year old football player with an active social life and busy study calendar. He lives on campus in student accommodation. What are five reasons that Bill may give to their physiotherapist for not following a treatment plan for a tendon injury?
Writing lesson plans
If you’ve inherited a new class and haven’t been given a plan for the lessons, or perhaps you’d like some ideas on how you might include more active learning in your classes, consider using ChatGPT to draft a lesson plan for you. Because ChatGPT was trained on dialogue so that it can answer follow-up questions in a conversational way, once it generates an initial lesson plan, you could ask it to refine it, such as by explaining how a particular activity may work, or suggesting how you might make an activity more motivating for students. Importantly, remember to apply your own pedagogical and content knowledge to evaluate and finesse the AI-generated suggestions.
Example prompt for ChatGPT: Write a lesson plan for a 50 minute college tutorial on the implications of artificial intelligence for future work. Include activities where students work in groups to critique different perspectives. Student groups should also investigate different industries that might be impacted by AI.
For students
Most of the hysteria around ChatGPT has been focused around its potential for misuse by students to complete written assignments and exams. This has led many academics and universities to refocus on good assessment design and consider how to productively engage their students with AI to help them prepare for the inevitable future where AI augments human intelligence. There may also be important roles for ChatGPT to support students living with communication disabilities. Here are a few ideas about how you might use AI in your units to help students learn.
Learning by teaching
One of the strengths of language models is their ability to respond to prompts given by users in natural language and generate human-like text in response. As both the prompt (input from users) and response (output from AI) are in natural language, the user interface is friendly and models trained on dialogue such as ChatGPT can be potentially used as conversational agents that enable students to act as a tutor and learn by teaching. When learning a new concept, students can explain the concept to ChatGPT like explaining to another student. And ChatGPT is likely to be a friendly peer because it is fine-tuned to behave in a kind and gentle way. Although sometimes it may still provide less-helpful outputs, the AI has been improving and adapting its behaviour through learning from human feedback.
Students can also ask ChatGPT questions and the potentially inaccurate answers generated by the AI may offer opportunities for students’ learning. When interacting with ChatGPT, students should be reminded to always critically review its answers and explain why they think the answers provided by the AI are correct, inaccurate, or incorrect. Critical thinking is an ever-important skill in today’s knowledge society and in the face of misinformation and fake news on the Internet. Through reviewing the AI’s answers, challenging its answers via follow-up prompts, judging the correctness by looking for supporting evidence or explanations, and experiencing the incorrect or inaccurate answers presented in a confident and persuasive way (like the simple mathematical problem demonstrated in the earlier article), students may gain a deeper understanding of the importance of critical thinking and be more sensitive when viewing plausible information.
In addition, since ChatGPT is able to admit its mistakes, students can inform and teach the AI after noticing its incorrect or inaccurate responses (as demonstrated in the screenshot to the right). Using ChatGPT could be seen as an interactive way of learning, and the processes of identifying and explaining the errors (similar to studying erroneous examples) and providing correct answers with explanations could facilitate learning.
Overcoming writer’s block
Like us, students might struggle to start a written submission because they are not used to a particular style of writing, or are uncertain about expectations, or are anxious about their knowledge and abilities. You might consider explicitly encouraging your students to use ChatGPT to draft some introductory lines, topic sentences, or other parts of the written work to get them started whilst explicitly highlighting its limitations. (Because, let’s face it, your students will probably be doing this anyway). In this way, and where appropriate, the AI may also be used to address language inequality and help those who are equally creative but with less-developed writing or language skills (e.g. non-native speakers) to express their ideas. It’s important to set boundaries and expectations, and perhaps spend some class time demonstrating how they can engage with AI to help catalyse writing – more on this a bit later.
Discussing with a co-programmer
ChatGPT is also able to take in and generate computer code. Professional programmers have suggested that the AI can be a great companion and explains snippets of code, suggests improvements to code, and helps to explore alternative approaches. In and out of the classroom, you might consider suggesting to students to use ChatGPT to suggest explanations for unknown code snippets or generate a boilerplate function for performing a given task. Again, it’s important to remind students that GPT is a large language model, not a knowledge model, so the explanations and code that it generates may contain inaccuracies or other issues. AI won’t replace programmers or programming classes (yet), so it’s important that students understand that they still need to develop their own coding knowledge and skills in order to check ChatGPT’s output including its biases.
Exploring diverse perspectives
In a similar approach to the suggestion to teachers above about generating discussion prompts for class, you could consider asking students to use ChatGPT to generate a set of different opinions or perspectives on a topic. Because ChatGPT has been trained on so much human text, it is likely able to generate text that closely mimics real human perspectives on a range of topics. Students could then use this as stimulus material for discussion, critique, and analysis – and every student could generate and use their own unique set of perspectives, which has benefits for individualising assessments.
For assessments
A discussion of ChatGPT in higher education wouldn’t be complete without directly considering its impacts on assessment. We have covered some uses above where AI could support assessments, such as drafting rubrics, creating exemplars, and individualising assessments. It’s clear that some students could, and indeed probably already are, use it to complete assignments and exams. There are already examples of where this has been detected by markers mostly due to the poor quality of the answers (we’d advise against AI-based detection of AI writing). As teachers, our role is to consider how assessment processes may motivate students to learn, instead of focusing on blocking and subversion and detection of AI. As the inimitable Maha Bali expressed on Twitter:
So what are the characteristics of an assignment that AI cannot fully succeed in writing?
Is that the right question? Or is the question we should be asking:
How do I design an assessment that makes my students want to truly learn? That motivates inquiry and expression?
— Maha Bali, PhD مها بالي 🌵 (@Bali_Maha) January 15, 2023
Sector leaders from Deakin University, which has an active research group on higher education assessment and learning, have asked whether “we [can] really afford to expend precious assessment space on narrow tasks” in a race back to the ostensible safety of invigilated exams. Instead, what are authentic and contemporary approaches to assessment in a context where AI will only become more intelligent and prevalent?
Get creative and incorporate AI into assessment
Ryan Watkins from George Washington University has proposed a compelling range of suggestions for assessments that not only mitigate the risk of AI but also encourage students to develop their AI literacy. These include having students generate ChatGPT responses and critiquing, evaluating, and visibly revising them. For example, AI could generate a series of short paragraphs in response to an assessment prompt, and students need to present the generated text and evaluate the ‘facts’ and perspectives contained therein, providing an accurate commentary supported with key references.
Set boundaries and expectations
The Sentient Syllabus Project has published an open-access set of ‘syllabus resources’ that teachers can adapt for their own classes, which aim to emphasise to students three principles: (i) AI alone cannot pass a unit; (ii) AI-generated material needs to be attributed; and (iii) AI use should be transparent. Their document contains a wide selection of well-considered text that you can adapt and use, that will help to set boundaries and expectations for your students around how they can and should productively engage with AI.
Consider a range of assessment types that are chunked
A common response to guarding against ChatGPT is to make students refer to in-class materials or discussions, or contemporary sources. While well-intended, students could still generate large swathes of the assessment using ChatGPT and just amend it to include these references. ChatGPT can even generate convincing personal reflections, so even reflective writing is not a guarantee of protection. Instead, consider breaking up larger assessments into chunks that need students to visibly iterate on their work through outlines and drafts before a final submission, and include multimodal (e.g. audio, video, pictorial, or other multimedia) outputs. (That said, upcoming AI technology such as VALL-E can generate terrifyingly realistic spoken audio just using a three second sample of someone’s voice). Oral assessments such as vivas are also a possibility, and are considered one of the best approaches to ensure assessment security.
Be open with students, and each other
As we start semester 1, ChatGPT will likely be one of the main elephants in the (class)room. Anna Mills from the College of Marin has curated a set of questions that teachers should be asking about generative AI. In fact, these are great questions to pose to students as well. The questions encourage us to consider how these AIs work and their benefits and limitations, what we value in learning, the implications for equity and inclusion and bias, and more. Having these open conversations with students and each other will encourage us all to be transparent (and honestly, it’s OK to be a little terrified) and work out how to appropriately engage with AI together. We are educating students to be leaders in the world and so we need to acknowledge openly that AI is part of the world now and will live and evolve with us. Be open with students that we want them to use but also lead/disrespect the technology. It doesn’t short circuit their learning. It won’t replace them or us as long as we value critical thinking and education as something beyond a transaction.
As teachers and a university that are committed to developing students’ deep disciplinary expertise as well as key skills such as critical thinking, communication, and leadership, we have a responsibility to develop their (and our) AI literacy, to productively engage and work alongside this powerful technology that has the potential to make our work, study, and leisure more effective, efficient, and inclusive.
Tell me more!
- Come to the 2023 Sydney Education Unconference on Friday 3 February 2023, where there will be sessions dedicated to discussing practical responses to ChatGPT
- Try out ChatGPT for yourself – it is free to access, for now
- A curated and annotated set of links to different AI tools, articles, and other resources relevant for educators at Sydney
- ChatGPT & Education slidedeck from the University of Massachusetts Amherst that provides a succinct and practical summary of ChatGPT for educators
- The street finds its own uses for things – an article from the Wharton School that outlines how students have productively engaged with ChatGPT
4 Comments