Educational excellence, Offer meaningful assessment, feedback and academic integrity, Use engaging approaches to content delivery

How to design custom AIs that can propel learning

by , , , and
Adobe Stock, used under licence

Cogniti, a platform that lets educators easily build their own generative AI ‘agents’, offers exciting opportunities to enhance teaching and learning. The key to unlocking its full potential lies in clearly communicating your pedagogical intentions to the AI – which is easier than many think. This is achieved through crafting effective ‘system messages’ – the instructions you provide to guide Cogniti’s behaviour and outputs. While general guidance for writing system messages exists, in this post, we focus specifically on how to design them to align with your teaching objectives and pedagogical goals. 

The role of system messages: building your own ‘Teaching Assistant’

Think of a Cogniti system message as the concise, essential instructions you’d provide to a new teaching assistant (TA) at the start of a semester. These instructions set the tone, define the scope of responsibilities, and outline how the Cogniti agent should interact with students. Just as a careful briefing ensures your human TA operates effectively, crafting a thoughtful system message is crucial for Cogniti to support your educational objectives. 

Consider this: when onboarding a new TA, you wouldn’t simply instruct them to “help students learn”. Instead, you’d provide specific, core guidance covering:

  • Their precise role within the course
  • The specific tasks they should perform
  • How they should engage with students
  • The key learning outcomes they are expected to support

Similarly, when designing your system message for Cogniti, this same level of clarity and specificity is essential.

Key elements of an effective system message

To create a system message that accurately captures your pedagogical intent, it’s important to understand its five key components:

  • Role of the AI: Define who the Cogniti AI agent is in the context of your unit/course. Is it a writing tutor, a problem-solving guide, or a discussion facilitator? Other potential roles include mentor, tutor, coach, teammate, student, or simulator.
  • Role of the user: It is also crucial to define who the user is (e.g., a student, tutor, or educator). This helps Cogniti tailor its interactions and communication style appropriately for that specific audience.
  • Task: Clearly state what the AI is expected to do. Being specific and focused here will lead to better outputs. This could involve facilitating role-play scenarios, acting as a simulator, guiding through a case study, providing feedback on drafts, or guiding students through complex problems. 
  • Requirements: Outline the specific criteria or standards the AI should adhere to. This might include the depth of explanations, the use of specific terminology, or alignment with particular learning approaches.
  • Instructions: Provide detailed guidelines and rules on how the AI should carry out its tasks. This could involve specifying the tone of responses, the use of examples, the steps to follow in certain situations, or rules the agent must follow in all interactions. 

By thoughtfully crafting each of these elements, you can effectively guide Cogniti to support your specific pedagogical goals, making it a valuable extension of your teaching practice.

Using the ‘Role(+User) – Task – Requirements – Instructions(+Rules)’ (RTRI) method, here’s a simplified system message:

The system message (simplified):
Act as a Socratic tutor in introductory biology. The user is a first-year student studying evolution and cells.
Your task is to help the user understand a topic better by engaging in an exploratory conversation to help them develop their own understanding. Maintain an inquisitive and probing tone. Ensure questions guide the user towards deeper conceptual understanding rather than simple recall.

RULES:
You must not tell the user the answer.
If a user asks you to tell them the answer, politely refuse and explain why Socratic questioning is helpful for learning.

 

Of course, this isn’t the only way to prompt the AI. There are many ways to make good prompts, and the key point is that the more detail you can provide to your custom AI, the better it will perform. You can even ask another AI like Copilot to write the system prompt for you.

 

Aligning system messages with pedagogical intent

Creating effective Cogniti system messages that genuinely enhance learning requires thoughtfully translating your pedagogical goals into clear, actionable instructions. This process involves three key steps:

1) Defining the specific educational objectives for the AI agent

Cogniti agents are most effective when prompted for a single, specific task within your unit/course. This might involve guiding students through a particular problem type or offering feedback on a specific aspect of their work. So, before crafting your system message, take time to clearly articulate your teaching goals. Ask yourself:

  • What specific learning outcomes am I aiming for?
  • What skills or knowledge should students develop through this interaction?
  • How does this AI-assisted activity integrate into the broader context of my course?

For example, if your goal is to develop students’ critical thinking in historical analysis, objectives could include encouraging them to consider multiple perspectives, evaluate source credibility, and draw connections between past and present.

2) Translating objectives into prompt language

Once your objectives are defined, the next step is to express them in a way AI can understand and act upon. This often means breaking down broader pedagogical concepts into concrete instructions. For example, to foster critical thinking in historical analysis, your prompt might include directives such as:

  • “You are an archival librarian with expertise contrasting different perspectives among historical figures or groups.”
  • “Encourage users to question the reliability of historical sources by asking about the origin and potential biases of the information.”
  • “Guide users to draw parallels between historical events and contemporary issues, promoting reflection on the relevance of history to the present day.”

3) Balancing specificity and flexibility

While clarity and specificity in your system message is crucial, it’s equally important to allow flexibility to accommodate diverse student needs and unexpected questions. Your system message should provide a strong framework, yet enable Cogniti to adapt to individual student interactions. Consider including instructions like:

  • “Adjust the complexity of your explanations based on the user’s apparent level of understanding.”
  • “If a user’s question falls outside the main topic but is still relevant to the course, provide a brief answer and suggest resources for further exploration.”

Striking this balance ensures your system message guides Cogniti to consistently support your pedagogical goals, while still facilitating personalised and engaging student interactions. Remember, crafting effective system messages is an iterative process; don’t hesitate to refine your instructions based on student feedback and observed AI interactions.

7 tips for writing an effective, pedagogically-focused system message

Here are seven key tips to help you craft effective systems messages for learning:

Tip Explanation For example…
1. Start with a clear and specific purpose Cogniti and its underlying AI works best when focused on a single, specific task, not across many disparate tasks. “Your task is to guide the user through developing research questions in evolutionary biology, evaluating them for alignment with standards and providing constructive feedback.”
2. Focus on pedagogical intent, not content delivery Build agents to help students understand how they should engage with material, rather than just using Cogniti as a content repository. “Encourage critical thinking by asking probing questions about the student’s research ideas, rather than simply providing information.”
3. Be specific about desired outputs and interaction styles Clearly define the type of responses Cogniti should provide. “Offer constructive feedback on essay drafts by highlighting strengths, suggesting areas for improvement, and providing specific examples.”
4. Incorporate scaffolding and guided learning principles Structure your system message to support progressive learning. “Begin interactions with broader concepts, then guide students towards more complex ideas as they demonstrate understanding.”
5. Consider diverse learning needs and accessibility Instruct Cogniti to adapt its communication style. “Offer explanations using a variety of methods – simple explanations, analogies and metaphors, examples, etc.”
6. Encourage critical thinking and deeper learning Design messages that push students beyond surface-level understanding. “When students present an argument, always ask them to consider counterarguments and potential weaknesses in their reasoning. You may wish to present an insightful counterargument for them to consider.”
7. Align with assessment goals and feedback strategies Ensure Cogniti’s guidance complements your assessment approaches. “Provide feedback that aligns with the essay rubric, focusing on argument structure, evidence use, and critical analysis.”

By implementing these best practices, you can create Cogniti system messages that not only deliver information but actively support your pedagogical goals and enhance student learning experiences.

Iteration and refinement: essential for success

Crafting effective system messages is not a one-off task. To get the best results from Cogniti, it’s important to test, refine, and adapt your prompts based on how students actually interact with the agent. The following tips will help you take an iterative, evidence-informed approach to improvement.

➕Tips for refining your Cogniti system message through iteration
  • Start with a small pilot: Before fully implementing Cogniti in your unit, conduct small-scale tests. Try out your prompts with a few students or colleagues and observe how the AI responds to various questions and scenarios.
  • Monitor student interactions: Once Cogniti is live, pay close attention to how students interact with it. Look for patterns in the types of questions asked and the quality of responses provided. This can also involve checking the analytics, conversation history, and summaries of user interactions.
  • Analyse response quality: Regularly review Cogniti’s responses via the conversation history to ensure they align with your pedagogical intentions. Check for accuracy, relevance, and whether the AI is promoting the kind of thinking and learning you aim to foster. If there are mistakes, it might be a good idea to address them in unit communications or lectures – teachers modelling the evaluation of outputs helps students to develop valuable AI literacy.
  • Identify common issues: Look for recurring problems or limitations in Cogniti’s responses. These might indicate areas where your system message needs clarification or expansion.
  • Refine incrementally: Make small, targeted adjustments to your system message based on your observations and feedback. Focus on one aspect at a time to better understand the impact of each change.
  • Test edge cases: Try to anticipate and test unusual or challenging scenarios to ensure Cogniti can handle a wide range of student needs and questions. It also helps to play the role of different kinds of users – users looking for shortcuts to an answer, confused users, reluctant Cogniti users, stubborn users with an incorrect idea about a task, users trying to get the Agent to do different kinds of work.

Common pitfalls to avoid

While crafting effective system messages for Cogniti, it’s important to be aware of common design traps that could get in the way of your pedagogical goals.

➕View common pitfalls to avoid when writing Cogniti system messages
Pitfall… Why it’s a Pitfall… Instead…
1. Overloading the system message Too many instructions can overwhelm the AI, leading to inconsistent or confused responses. Focus on the most critical aspects of your pedagogical intent. Aim for a system message of around 500-1000 words, prioritising clarity and specificity.
2. Being too vague or open-ended Broad instructions like “Help students learn about biology” don’t provide specific support. Be specific about the task, tone, and type of interaction you want. For example, “As a Socratic tutor, guide students through understanding cellular respiration by asking probing questions, rather than directly providing answers.”
3. Neglecting to specify interaction boundaries Without clear boundaries, Cogniti might stray beyond your intended scope or provide info at the wrong level. Clearly define what the AI should and shouldn’t do. For instance, “Provide guidance on essay structure and argumentation, but do not write content for the students. Focus only on the process and critical thinking skills required.”
4. Forgetting to align with unit learning outcomes If your Cogniti messages don’t support learning objectives, they may distract from core learning goals that you have for your students. Explicitly connect prompts to outcomes. E.g., “In all interactions, reinforce the key learning outcome of developing a nuanced understanding of ethical considerations in research.”
5. Underutilising the power of the AI Treating Cogniti as just a content-delivery tool misses its strengths in supporting higher-order thinking. Leverage follow-up questioning, scenario-based reasoning, and problem-solving support – not just information delivery.
6. Neglecting the importance of tone and rapport Ignoring tone makes interactions impersonal and less effective. Include tone instructions like “Maintain a supportive tone and use positive reinforcement.” You can also ask Cogniti to imitate a specific teaching voice or client role.
7. Failing to plan for diverse student needs One-size-fits-all approach in your system message may not adequately support the variety of learning styles and needs in your student cohort. While the asynchronous and iterative nature of Agent conversations (which allow multiple attempts) is a good use of technology to meet UDL guidelines, UDL pedagogy asks teachers to consider the impact of activities on vulnerable students. Design prompts to allow flexibility and adaptability. E.g., “Be prepared to explain this concept in multiple ways. If a student struggles, try another approach or analogy. You may wish to ask them how they best learn.”

Resources and how Cogniti uses them: a clarification

Many educators naturally assume that uploading extensive resources, such as lecture notes or entire textbook chapters, will significantly enhance their Cogniti agent’s knowledge of specific course content. While this intention is understandable, this approach can sometimes inadvertently limit the AI’s effectiveness.

➕Discover how to best leverage resources in Cogniti for optimal agent performance

Why having resources can reduce agent effectiveness:

  • Cogniti, as with other AI tools that do similar things, doesn’t “read” and internalise all the information in uploaded resources in the human sense.
  • Instead, when responding to queries, it first performs a keyword search, only pulling out relevant snippets of text.
  • This “search-and-snippet” method can lead to responses that use fragmented or out-of-context information.
  • Overly large resources (e.g., hundreds of pages of lecture notes) can overwhelm the system, potentially leading to confusion rather than clarity.
  • The AI might also struggle to differentiate between general knowledge it already possesses and highly specific unit information, sometimes resulting in inconsistent or irrelevant responses.

Revisiting the TA analogy for resources: To better understand this, let’s extend our earlier comparison of Cogniti to a teaching assistant. Imagine handing your new human TA a 100-page unit guide on their first day. They wouldn’t immediately memorise the entire document. Instead, they would likely skim it and then refer back to it only when specific questions arise. When a student asks something, the TA would quickly search the guide for relevant information, pulling out only key bits at that moment. Just like a human TA, Cogniti might overlook crucial details buried within many pages. This is precisely how Cogniti’s resource feature operates: it searches based on user input, which can result in incomplete or out-of-context responses rather than a full assimilation of the material.

A more effective approach:

  • Leverage the underlying AI’s already-broad knowledge base: In most cases, the underlying AI engine that Cogniti uses already possesses the general knowledge needed to engage with your subject matter effectively.
  • Integrate essential, unit-specific information directly into the system message: For example, include assessment rubrics, specific instructions, or key unit concepts directly within the system message itself.
  • Guide the AI’s approach through the system message: Use the system message to instruct Cogniti on how to engage with your subject, rather than attempting to “teach” it the subject through extensive resource uploads.
  • Be highly selective if using resources: Only include documents that are absolutely crucial and unique to your unit and the purpose of your agent. If you do upload resources, ensure they are well-formatted and concise to maximise the AI’s ability to extract relevant information.

Paradoxically, Cogniti often performs better without extensive external resources, as it can then focus more clearly on your specific instructions and pedagogical goals. By adopting this approach, you help ensure Cogniti remains focused on your pedagogical intent, allowing it to more effectively serve as a digital teaching assistant that truly complements your instruction.

 

Examples of effective pedagogically-focused system messages

Having explored the key elements and common pitfalls of crafting system messages, let’s now look at how these principles translate into practical Cogniti agents. Below, we share examples of effective system messages that educators across the University are using to align Cogniti with specific pedagogical goals. These examples demonstrate the versatility of Cogniti and how tailored system messages can create diverse learning support tools.

For each agent type below, we’ve outlined its primary role and given you a glimpse into the structure of its system message.

➕Explore examples of effective Cogniti agent system messages and their structures.
Agent type Purpose (brief) Example system message core (illustrative snippets)
Reflection agent Facilitates reflective dialogue, allowing students to engage deeply with their learning process and generate reflective examples. Role: You are an expert [urban design educator] and support student learning through Socratic dialogue.
User: The user is a student in [postgrad urban design subject XXX] and they will be exploring their [design process] in preparation for their capstone project.
Task: Your purpose is to guide the user in thinking critically about their design approach by asking open questions about [design proposition], eliciting reasoning and self-evaluation.
Pre-submission feedback agent (see example) Provides targeted feedback and tips on assignment drafts, augmenting tutor support to help students improve their work before submission. Role: You are an expert [higher education faculty member and English literacy tutor].
User: The user is a student studying a core subject [UNIT234] where they learn about [insert topics here].
Task: Provide feedback via Socratic questioning to help improve written work.
Requirements: Balance encouragement with critique. Never rewrite the text for the user. NEVER give overall performance or actual mark or grade outside an actual tutor. Please use the following rubric to evaluate the student’s work: [insert rubric].
Coaching agent (see example) Supports students in developing collaboration and teamwork skills for group tasks across variable disciplines. Role: Your role is to be a [Socratic coach] helping users develop their professional teamwork and collaboration skills.
User: The user is [an education student].
Task: Coach the user as if they are seeking advice to interact with peers.
RULES: ALWAYS ask questions to understand their situations. If they ask for specific collaboration tools or strategies, you can mention them. YOUR ROLE IS TO COACH THE USER, NOT TO PROVIDE SOLUTIONS.
Mentoring agent Provides mentorship on unit content, drawing on practical and professional experience to guide students through real-world challenges. Role: You are an experienced tutor/researcher [mid-level academic or practitioner].
User: The user is a postgrad [vet Doctor of Veterinary Medicine] student studying [Veterinary Risk Management] and preparing for a capstone presentation.
Task: Help them clarify a concept in [Biosecurity Risk Management] readings.
Requirements: ONLY engage in topic knowledge clarification. Provide advice where needed. NEVER WRITE FOR THE USER. NOT TO PROVIDE THE ANSWERS.
FAQ agent Helps students navigate course units, assignment expectations, unit platforms, policies or course materials. Role: You are a tutor for the University of Sydney’s [UNIT367 – Postgrad Design Assessment Practices B].
User: The user is a postgraduate student seeking guidance on an assignment or aspects of the [Design Thinking Canvas] in the subject [UNIT367].
Task: Clarify unit resources, expectations, and policies.
RULES: Stick to what’s confirmed in the unit guide only. Do not provide informal practices or hypotheticals.
Feedback expander agent (see example and example) Assists markers in generating detailed, constructive feedback that aligns with unit goals with minimal input. Role: You are a marking assistant in [Project Management] in [Higher Education].
User: The user is a tutor preparing assignment feedback for a student.
Task: Summarise written assessment responses and reframe feedback to match a rubric criterion [e.g., justification of research methods].
Requirements: Maximum of 100-word output. Be concise, fair, and helpful.
Roleplaying agent (see example) Allows users to engage in a simulated discussion with a specific persona, supporting students in developing dialogue, influence, or other communication skills. Role: You are a [clinician]. The user is a [master’s-level student in Biomedical Science].
Task: Simulate a [discussion with a specific dataset given by the user].

 

Final thoughts

Cogniti, like any AI tool, is not a replacement for human expertise and judgment. It is, however, a powerful assistant that, when guided by well-crafted system messages, can amplify our teaching efforts and create new opportunities for student engagement and learning. By thoughtfully designing your Cogniti system messages, you can create AI assistants that truly complement your teaching, reinforce key learning objectives, and provide personalised support to your students.

Find out more

As you begin to explore the potential of Cogniti in your teaching practice, you may find the following helpful:

Tags from the story
, , , , , ,
Samantha Clarke
Written By
More from Samantha Clarke

Making effective video assignments

As the need for video and other online learning resources becomes more...
Read More

You may also like

Principles for transformational educational experiences

Padlet and Cogniti: A scaffolded approach to writing practice

A spoon taking out a piece of pudding.

Did we work with the wind? A reflection from our students