{"id":21418,"date":"2024-10-10T23:56:47","date_gmt":"2024-10-10T12:56:47","guid":{"rendered":"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/?p=21418"},"modified":"2024-10-10T23:56:47","modified_gmt":"2024-10-10T12:56:47","slug":"using-ai-to-provide-feedback-on-an-early-feedback-writing-task-eft","status":"publish","type":"post","link":"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/using-ai-to-provide-feedback-on-an-early-feedback-writing-task-eft\/","title":{"rendered":"Using AI to provide feedback on an Early Feedback Writing Task (EFT)"},"content":{"rendered":"<p>The introduction in semester 1, 2024, of an <a href=\"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/putting-support-for-students-at-the-centre-through-early-assessment-feedback-and-personalised-communications\/\">early feedback task<\/a>\u00a0(EFT) in all 1000-level units was a welcome innovation to assist students in a successful transition to university. As detailed <a href=\"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/marking-and-feedback-of-early-feedback-tasks-eft\/\">here<\/a>, it enabled teachers to review students\u2019 work early on in the semester, and provide meaningful feedback to them about their performance. Early signs show that the introduction of EFTs has been effective in identifying struggling students early in the semester, and offering them support and encouragement in order to facilitate success.<\/p>\n<p>Most colleagues opted for multiple choice quizzes; as outlined by <a href=\"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/1475725720971205\">Enders et al.<\/a> (2020), they are an economical way of providing constructive feedback at scale.<\/p>\n<h2>The objective<\/h2>\n<p>However, in my intermediate French language unit, I wanted to assess students\u2019 <em>written<\/em> skills early in the semester. Although not a 1000-level unit, FRNC2603 French 3 has a significant percentage of first-year students entering from Year 12 HSC Beginner French, so <strong>the introduction of a written, in-class formative (0% weighting) EFT enabled us to assess students\u2019 competency and potentially adjust our teaching and learning content in accordance with their needs.<\/strong><\/p>\n<p>I was also conscious of the benefits of cohort building in facilitating a <a href=\"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/category\/core-pedagogical-principles\/foster-a-sense-of-belonging-and-community\/\">sense of belonging<\/a> to the unit, discipline and to the University. In line with my objective of creating <a href=\"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/authentic-assessment-what-is-it\/\">authenticity<\/a> in the delivery of unit content \u2013 particularly important in our discipline, as we are geographically removed from the Francophone world \u2013 the opportunity presented itself to combine cohort building, authentic teaching and learning activities, and an EFT through a European film festival that took place in Sydney at the start of semester.<\/p>\n<h2>The process<\/h2>\n<p>The EFT involved five steps:<\/p>\n<ul>\n<li>Step 1 (week 1): (cohort-building; authentic content objective) Students attended a specific French film screening at the end of Week 1. Students who were unable to attend the film screening were given the option of viewing a French film of their choice via a streaming service of their choice.<\/li>\n<li>Step 2 (before week 2 tutorial): (preparation for EFT) Students were directed to a film review and cinema website, and asked to read up on the film, in preparation for writing a personal response to the film in the form of a reflective piece. The reflection did not count for marks, although submission was mandatory. Students were not told ahead of time that their submissions would be put through an AI.<\/li>\n<li>Step 3 (week 2): (in-class written task) During one of their week 2 tutorials, without access to the Internet or any software other than an online dictionary, students wrote their reflection in French, and submitted it via Canvas. Tutors reassured students that they expected to see errors in the submissions, and explained that <strong>the objective of the activity was to ascertain the cohort&#8217;s level of competency at the beginning of semester<\/strong>, the potential need for extra support, and\/or tweaking of the curriculum in line with student performance in the EFT. Tutors monitored students as they completed the exercise to reduce covert AI use.<\/li>\n<li>Step 4 (week 2-3): Tutors copied and pasted students\u2019 work into GPT-4 (via a &#8216;vanilla GPT-4&#8217; agent in Cogniti), and asked the AI, via the same agreed-upon prompt used by all tutors, to correct submissions and provide explanations of corrections given. This process was an efficient way of providing feedback quickly on individual students\u2019 work, and of introducing students and tutors to AI. <strong>Tutors shared the AI&#8217;s comments in the Canvas Speedgrader, and provided further personalised written feedback<\/strong> e.g. where perceived hallucinations took place, or where they disagreed with the feedback given by the agent, or simply if they wanted to provide, for example, a link to where students could find help in Canvas or on the Internet.<\/li>\n<li>Step 5 (week 4): (in class) Collectively, <strong>tutors presented common errors made by students, shared the prompt given to the AI and discussed its efficacy, which prompted a broader discussion about generative AI and its place in language learning<\/strong>, which they would be using throughout the semester (see the article on\u00a0<a href=\"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/padlet-and-cogniti-a-scaffolded-approach-to-writing-practice\/\">Padlet and Cogniti: A Scaffolded Approach to Writing Practice<\/a>). Students were directed to the <a href=\"https:\/\/canvas.sydney.edu.au\/courses\/51655\">AI in Education<\/a> Canvas site created for, and in partnership with, students for further information about generative AI and its use at Sydney.<\/li>\n<\/ul>\n<h2>The upshot<\/h2>\n<p>From a teacher\u2019s perspective, the written EFT was a useful guide to understanding students\u2019 competency early in the semester and adjusting the teaching and learning content accordingly. The marking process facilitated by AI (notwithstanding hallucinations) was effective and efficient. <strong>The task provided the opportunity to talk about generative AI and introduce the vanilla GPT-4 agent.<\/strong><\/p>\n<p>Students appreciated participating in an enjoyable activity outside university (step 1); the task thus contributed to cohort building. It also encouraged students to engage authentically with their discipline via the French film festival that subsequently took place throughout March.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>The introduction in semester 1, 2024, of an early feedback task\u00a0(EFT) in all 1000-level units was a welcome innovation to assist students in a&#8230;<\/p>\n","protected":false},"author":626,"featured_media":22242,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1368,9,191,1856,2941,57],"tags":[2502,3006,3005,104,2676],"coauthors":[706],"class_list":["post-21418","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-assessment","category-faculty","category-fy-coordinators","category-foster-a-sense-of-belonging-and-community","category-learningspaces","category-teaching-tips","tag-ai","tag-early-feedback-task","tag-eft","tag-feedback","tag-writing","post-item","post-even"],"_links":{"self":[{"href":"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-json\/wp\/v2\/posts\/21418","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-json\/wp\/v2\/users\/626"}],"replies":[{"embeddable":true,"href":"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-json\/wp\/v2\/comments?post=21418"}],"version-history":[{"count":5,"href":"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-json\/wp\/v2\/posts\/21418\/revisions"}],"predecessor-version":[{"id":22243,"href":"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-json\/wp\/v2\/posts\/21418\/revisions\/22243"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-json\/wp\/v2\/media\/22242"}],"wp:attachment":[{"href":"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-json\/wp\/v2\/media?parent=21418"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-json\/wp\/v2\/categories?post=21418"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-json\/wp\/v2\/tags?post=21418"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/educational-innovation.sydney.edu.au\/teaching@sydney\/wp-json\/wp\/v2\/coauthors?post=21418"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}