It’s almost 12 months since OpenAI released ChatGPT and the internet (and Teaching@Sydney!) is awash with examples of how students and teachers can use it and other general-purpose generative AI tools to enhance learning, teaching, and assessment. These ‘general-purpose generative AI tools’ like ChatGPT, Bing Chat, Claude, and Bard are open-ended chat interfaces that anyone can access and start a conversation.
ChatGPT is great, but…
These general-purpose generative AI tools are great but present some issues for education.
- Access and equity – GPT-4 arrived in March 2023 and was heads above GPT-3.5. But it’s only available to people who can and want to pay USD20 per month for this. We’ve also heard many students being reluctant to sign up for their own free accounts with AI companies.
- Control and safety – because these tools are open-ended, users can ask the AI to do anything they’d like. For example, it will happily re-write a paragraph for you, instead of giving you feedback that you can work into your own writing. Teachers also have no way to know how students are using the AI or to catch issues that arise.
- Hallucination and lack of context – whilst GPT-4 makes things up at a much lower rate compared to GPT-3.5, large language models can still be prone to hallucination, especially when they don’t have as much contextual knowledge to work with.
- Privacy and intellectual property – some generative AI tools allow you to prevent your data from being used for future training of AI. Not all AI tools have this option, though.
How can these be overcome?
Educational Innovation at the University of Sydney have been working for the past few months on solving these challenges, and others, by building and piloting a new AI platform called Cogniti. Cogniti leverages the powerful GPT-4 large language model (but hosted on the secure and private Microsoft Azure platform) and allows teachers to create their own AI chatbot ‘agents’ that can be steered with specific instructions and resourced with specific contextual information from units of study. These agents can be embedded into Canvas, providing a seamless experience for students — meaning that students don’t need to sign up for a separate account and their access is provisioned by the institution. Teachers have full visibility over the conversations with Cogniti agents, and students are able to flag and give feedback on AI messages.
In an educational context, it’s important to see that AI does not replace teachers. Instead, it may be helpful to see Cogniti agents as an ‘AI double’ of yourself, who can be given specific instructions on how to behave, and available to students 24/7.
Imagine you have one generally well-read teaching assistant — one for each student. What would you ask the assistant to do? Perhaps it is to help students come up with ideas for improving their essay but not write it for them. Perhaps the assistant would roleplay different tricky client conversations and provide feedback on how students could improve. Perhaps it might be to coach students towards better study habits that are specific for your discipline. Perhaps your AI double might talk students through a case study by asking them thought-provoking questions but not providing the answers directly.
Now imagine that you didn’t need to give your students a specialised prompt to copy and paste into ChatGPT, and they didn’t need to pay USD20 a month for ChatGPT Plus to get access to GPT-4. And imagine that your ‘AI double’ had available to it specific resources from your unit — perhaps it’s a set of notes from your lectures, or an assignment rubric, or the assessment instructions, or examples of past work, or the key readings — meaning that the AI could respond with higher accuracy and relevance.
This is what we have designed Cogniti to do for you. Not to replace you, but to be an AI double that you control and give to your students so that they can interact with it in specific, educationally meaningful ways.
What are some examples?
We have trialled Cogniti in a few units this semester, and it has recently been approved for use University-wide. Here are a few real examples of use, both student-facing and staff-facing.
‘Mrs S’
In an occupational therapy unit, Joanne Hinitt and Kria Coleman made a Cogniti agent called ‘Mrs S’ to roleplay as a busy kindergarten teacher. Student groups in Joanne’s tutorial would develop a support plan for a mock kindergarten student, and usually the activity would end here with little ability to genuinely apply their knowledge. This semester, students had conversations with Mrs S, presenting their support plans and iterating their ideas with Mrs S. This Cogniti agent was provided with specific instructions to challenge the ideas that Joanne’s students presented, push back if dissatisfied, and had available to it examples of inclusive practice for school classrooms. Students — occupational therapists in training — were able to apply their knowledge to a realistic scenario, a year before they would be exposed to real-life clients. Joanne noted that students were much more engaged with the topic and activity than previous years, and that the class was much more interactive and collaborative.
Read more about the educational design around Mrs S, and the process for developing this agent.
24/7 reflective writing feedback
In a postgraduate education unit, Jessica Frawley, Robyn Martin, and I made a Cogniti agent to provide feedback and ideas on our students’ reflective writing pieces. Our students need to write a reflective account of their professional practice for two of their assessments, in the style of a Fellowship application to AdvanceHE UK. This can be a difficult task because it is an unfamiliar framework and reflective writing about teaching is an unfamiliar genre to many of the students in the unit. The Cogniti agent was instructed to provide feedback and ideas, but not re-write. The agent was also given the rubric for the assessment and other relevant examples and documentation, allowing the AI to provide specific feedback on expected standards. Our students have been working with the AI to get feedback and develop ideas for their reflections. We’ve noticed that the AI does get things wrong sometimes; being able to see students’ conversations with the AI has allowed us to identify the two times this has occurred and turn these into teaching moments.
Feedback expander
In an undergraduate biology unit, Fran van den Berg and Minh Huynh designed a Cogniti agent to help demonstrators provide better feedback to students, faster. A perennial challenge for time-pressed marking teams is to provide helpful, consistent feedback to the depth needed and in a style that is supportive and encouraging. Their Cogniti agent was instructed to expand short-form feedback comments into longer, more constructive comments. The agents were provided the assignment brief, some examples of well-written feedback, and rules to follow (such as being concise, constructive, and forward-looking – the hallmarks of good feedback). Markers would mark student reports, and note down short-form feedback that Cogniti would then help to expand. Importantly, all the assessment decisions came from humans. The markers who used the Cogniti agent found that they saved time and were able to improve the quality of feedback comments provided to students.
Case study tutor
For an international business school, we’ve been trialling a Cogniti agent that helps students work through a case study and challenges them with questions. It can be difficult for students to tackle case studies due to the complexity of real-world scenarios, a large volume of information, and the need for advanced critical thinking, analysis, and decision-making skills. This Cogniti agent was provided with a real case study text and instructions to help students understand the case study by asking them challenging questions that highlighted alternate views, encouraged evaluating evidence for claims, and guided students towards better interpretation of a case.
Agents to assist teachers
We’re also trialling a few Cogniti agents that have been specifically instructed to help educators. Here are a few examples.
- Designing for diversity coach. It can often be difficult to know where to start when it comes to incorporating inclusive practices into our teaching. This Cogniti agent has been designed to act as an educational design coach, asking you questions about current barriers to student learning and guiding you through realistic solutions.
- Rubric assistant. Rubrics are powerful tools for helping students and markers understand expected standards – but they can be difficult to write. This Cogniti agent has been trained to help you create a rubric that you could use to establish standards and criteria for assessments, by guiding you through a set of questions.
This sounds difficult — how do I control an AI?
The beauty of these modern large language models like GPT-4 is that working with them is surprisingly straightforward. No programming knowledge is needed – you just provide your instructions in plain language. The image below is a screenshot of some of the instructions that were given to Cogniti to create ‘Mrs S’.
These instructions are provided to the AI as a ‘system message’, which isn’t seen by students. This system message is more powerful than a standard prompt fed into a general-purpose generative AI chatbot, and can control the behaviour of the AI. A bit of trial and error is needed to adjust the system message, but this just involves tweaking the natural language instructions you give to the AI in order to build your AI double.
The main thing to reiterate is that the AI is not here to replace you. Our strongly-held philosophy with Cogniti is that you, the teacher, are in the AI’s driver’s seat — you build these AI doubles to support your students in specific tasks. Your AI double is there to help your students learn in ways that you see fit, so that your class time can be more informed, collaborative, and interactive — more human. Your pedagogical and disciplinary expertise remain at the foreground, and is absolutely needed to ensure that the AI doubles are working to the benefit of student learning.
Can I give it a go?
You can log in to Cogniti at https://app.cogniti.ai. If you’re from the University of Sydney or an Australian university, you can use the Australian Access Federation login. Other users can use Microsoft or Google account logins. When you log in, you can try chatting with some of the agents that have been created and shared.
Log in to app.cogniti.ai
For interested staff at the University of Sydney, we are running some introductory sessions to Cogniti soon — you are most welcome to come and find out how you might build your own AI doubles to help your students.
Tell me more!
- The Cogniti website has some help files and write-ups that may help you to consider whether this might be useful for you.
- For other resources on AI, check out our AI in Education collection on Teaching@Sydney.
- Late breaking news: OpenAI released their useful ‘GPTs’ function on 7 November 2023. We believe in the right tool for the right job, and some educators may find OpenAI’s GPT’s more suitable, whilst others will find Cogniti more suitable – here’s our brief analysis.
