Educators play a crucial role in a student’s learning journey by providing assessment feedback that is personalised, encouraging, constructive and timely. However, we often struggle with heavy workloads and with providing consistent, quality feedback that is meaningful to students. On the other hand, students, especially those new to written assessments, can find feedback to be unclear or overly critical, which can be discouraging and confusing, hindering their learning process. These challenges are magnified in large-cohort units of study where the marking load is done by large numbers of casual staff members leading to issues around standardisation and budgetary constraints.
To address this challenge, we built AI assistants using Cogniti, a generative AI tool created by educators at the University of Sydney. Cogniti allows educators to take control of large language models (LLMs) like GPT-4 with specific instructions and resources such as the assignment rubric and assignment instructions, producing tailored outputs that support student learning. The Cogniti agents that we built, designed as either a biology, data scientist or public health expert, aided in delivering constructive feedback on written assessments.
These AI agents helped our markers to improve and expand their feedback on written reports for two first year units of study in the Faculty of Science “Introduction to Statistical Methods” and “From Molecules to Ecosystems”, and a postgraduate core unit of study “Disease Prevention and Health Promotion” in the Faculty of Medicine and Health. These units have large enrolments of between 220 to 1400 students.
Our Cogniti agents were specifically created to assist our markers in enhancing and expanding feedback on written reports in all three units of study. Importantly, markers maintained control over the entire marking process, drawing upon their disciplinary knowledge and experience to ensure precise grading. Acting as a valuable tool, Cogniti helped markers by expanding brief and occasionally direct feedback, thereby relieving time pressures, frustration, and stress commonly associated with providing feedback. Additionally, it ensured that feedback was timely and consistently supportive in tone. Markers who used the feedback expander agent commented that:
Cogniti [was] most helpful in providing more detailed constructive feedback than I would normally provide…I can prompt Cogniti with something like ‘intro poor structure’ and it will give me a detailed (and kind) paragraph on how to structure an introduction and why the correct structure is important.
Providing constructive feedback is a perennial challenge for educators; Cogniti helps to support this:
Cogniti is a new fun tool to alleviate the frustrations of marking… You can take out your anger in Cogniti and it will transform it into wonderful and supportive feedback… It has especially helped me out at times when I want to pull my hair out.
The markers particularly liked that it could draw upon the assignment rubric:
I especially liked that it was linked to the rubric so the feedback was very specific.
The feedback agents also played a pivotal role in enhancing the marking process for culturally and linguistically diverse staff and first-time markers. It empowered them to offer more comprehensive feedback while boosting their confidence. This move aims to make the grading process more accessible for them.
When co-piloted by a marker, our Cogniti feedback agents helped us provide timely, constructive, and supportive feedback while working with large and diverse staff teams, tight deadlines, and sizeable cohorts. As unit coordinators, the capacity of Cogniti to standardise the tone, style, and content of feedback, while still allowing personalisation of comments was the most important aspect driving the success of Cogniti for this report feedback application. Without Cogniti, substantial coordinator time and effort is taken up in teaching and providing feedback to markers about how to constructively write feedback for students before marking starts, and then unpacking or explaining feedback comments to students after feedback is released. By using Cogniti in the feedback process, we were able to minimise these steps.
To find out more
- Register for a Cogniti workshop to learn more about the tool and start building your own AI agents.
- Read tips on how to design a good system message.
- Read other Teaching@Sydney articles about generative AI and Cogniti.
