What do you do as an educator when a successful teaching innovation completely backfires with a different student cohort? In a previous Teaching@Sydney article, we described the conversion of an undergraduate Bachelor of Pharmacy (BPharm) pharmaceutical compounding unit to a competency-based*, grade-only* unit to enhance the competence of pre-registration pharmacists in pharmaceutical compounding. Given the success of that change – it led to positive student feedback and increased confidence of the teaching team in our students’ compounding abilities – we implemented a similar teaching design in the corresponding post-graduate entry dispensing unit for the Masters of Pharmacy (MPharm) cohort in 2023. What we didn’t anticipate was how differently our postgraduate students would respond. Here’s what happened – and what you can learn from our mistakes.
* For those unfamiliar with these terms, competency-based education is a framework for designing and implementing healthcare education that focuses on desired performance characteristics of healthcare professionals, establishing observable and measurable performance metrics that learners must attain to be deemed competent. Grade-only means that students receive a grade (satisfied requirements or failed requirements) for the unit, but not a mark (see this excellent Teaching@Sydney article by Ruth Weakes and colleagues that talks further on going gradeless).
The problem: When success doesn’t transfer
However, to our surprise, this has had very different results. Overall, MPharm students were much less satisfied with the unit design, with an overall USS score of 3.77, a significant decrease from previous years (4.41 in 2021 and 4.42 in 2022) and below institution, faculty and school averages; this dissatisfaction was further emphasised by student comments.
Using Brookfield’s four lenses
To explore and rethink this issue from a critically reflective standpoint, we drew on Brookfield’s famous four lenses of critical reflective practice:
- the self (our experiences as learners and teachers)
- the students (what our students are saying)
- the peer lens (what our teaching colleagues are saying)
- the literature (what the scholarly higher education literature can tell us)
This allowed us to not only think about the reasons for these different outcomes but also consider and implement potential strategies to address them.
What we discovered
The curriculum context matters
A recurring theme was that marking standards were too harsh and didn’t reflect pharmacy practice. When we examined the different implementations of the course between the two cohorts, a key difference was the absence of material on medication safety (the why, when and how medication errors occur and what can be done to prevent them). This was left out of the MPharm unit, as it is only a 3-credit point unit (compared to 4 for the BPharm) and includes an additional compulsory interprofessional learning activity. However, framing compounding activities and requirements in terms of medication safety can help students to see the relevance of these activities, which can seem technical and pedantic in isolation, to their future professional practice.
This emphasises the influence of curriculum space (or lack thereof) for successful unit delivery. Many students today are juggling responsibilities such as paid work, family obligations, travel, etc. This is particularly true of our post-graduate MPharm students. Therefore, as educators, it is essential that we ensure course workloads are manageable. At the same time, when we adjust courses to meet specific credit requirements, we must keep learning activities well aligned to meet our intended learning outcomes.
Different cohorts revealed different needs
We also observed different types of errors in the final practical skills assessment between the two cohorts. In the BPharm cohort, most errors related to not completing key parts of compounding documentation. In contrast, the MPharm cohort errors were primarily related to calculations. This suggests that BPharm students had sufficient knowledge and skills but struggled to integrate them effectively during the compounding process, while some MPharm students lacked essential pharmaceutical calculations knowledge and skills underpinning successful compounding.
BPharm students completed a calculations quiz in week 3 as an early diagnostic and feedback task. This helped to identify students who may struggle with pharmaceutical calculations. This allowed us as educators to provide remedial activities early in semester – an opportunity which was lost for the MPharm cohort. Early feedback is increasingly important given the diversity of our student cohorts. For example, a nationwide shortage of mathematics teachers means that many students—particularly those in low SES schools – do not have the opportunity to complete higher level mathematics subjects, which have previously been assumed knowledge for many healthcare courses (either because their school is unable to offer these or attract teachers qualified to teach them).
As educators, we need to challenge our assumptions about the knowledge that our students bring and take extra steps to identify students lacking foundational knowledge and provide support for them to develop this.
Thoughtful use of assessment to identify students’ strengths and areas where they need early support is key here – waiting for high-stakes final exams is no longer enough!
Our assessment design was working against us
Finally, the initial educational design included a requirement for students to be marked as satisfactory for 10 of the 15 products that they made in their weekly lab sessions, so that they were assessed on their physical compounding skills for multiple products. However, upon reflection, we felt that this requirement was not beneficial. Students argued that this is not only stressful, but redundant, as they needed to pass the final exam to pass the course anyway. They also argued it was unfair to grade them on products they were making for the first time. Teaching team observations corroborated this, with high levels of anxiety amongst students surrounding this requirement and sometimes lengthy discussions about marks awarded or deducted, rather than how students could further develop their skills.
Additionally, we felt that this did not help us to identify students in need of extra support. For example, we spoke to a student who failed their practical skills assessment due to calculation errors and saw that she had completed these calculations correctly in the lab class. When asked about this, she said her friend helped her get the right answer in the lab class so that she could reach her 10 satisfactory products. This is an example of ‘satisficing‘ behaviour – whereby students only have limited resources (e.g. time, energy, and money) and so do the minimum acceptable to achieve a desired outcome, often resulting in shortcuts. We also offered extra lab sessions to meet the requirement of 10 satisfactory products, but felt that this became a tick-box exercise. Students simply reattempted a product that they had been marked as unsatisfactory for in the past, rather than focusing on skill improvement.
The changes we made
So what happened next? We made two key changes to these courses based on our reflections and findings. Firstly, we added learning activities and assessment quizzes on pharmaceutical calculations and medication safety to the MPharm course. Medication safety material provided context for the learning and assessment activities so students could see the relevance of this to their future practice as a pharmacist. Meanwhile, the pharmaceutical calculations quiz allowed us to identify students who struggle with calculations early and provide effective remediation. We also removed the requirement for students to achieve 10 satisfactory products in their weekly lab sessions from both the undergraduate entry BPharm and MPharm, to reduce student stress and anxiety and focus on mastering relevant content and skills. To ensure that the summative assessment covered a range of products, we added a second summative practical skills assessment to both units. Now, students must be marked satisfactory for two products out of four attempts to meet the requirements for the unit.
The table below illustrates these changes in more detail:
What did the students think?
Student feedback on these changes was positive for both units, with overall Unit of Study Survey (USS) scores of 4.31/5 for BPharm and 4.57/5 for MPharm. Student comments were also positive. In particular, they appreciated the removal of the requirement to be marked as satisfactory for 10 products in their weekly lab sessions, noting that it allowed them to reflect on their mistakes and use the weekly lab sessions as practice for the practical exams.
Students also appreciated being able to focus on learning and the bigger picture rather than just making sure they ticked the boxes to pass.
Teachers noted that lab classes were less stressful and their interactions with students led to deeper engagement.
Want to try something similar in your teaching?
Here are some tips if you want to try something similar in your teaching:
- Early feedback matters! Building in opportunities for early feedback helps us as educators to identify students lacking foundational knowledge and provide support for them to develop this and succeed in the unit.
- Make sure your assessment works for you and your students. Carefully consider the purpose and timing of each assessment task. Is it about skills development or assurance that students have achieved required knowledge and skills? If the focus is skill development, then a formative task may be more appropriate. If you want assurance that they have developed appropriate knowledge and skills, then a secure summative assessment is needed (with adequate opportunities for feedback and skill development provided).
- Each cohort and teaching context is unique and one size does not fit all. Just because an innovation was successful in one teaching context does not mean it can be easily transferred. Adaptation to your individual learners’ needs and overall program structure is essential for success.
Final thoughts
Examining these courses through the lenses of our students, ourselves, our teaching colleagues, and the scholarly literature allowed us to not only identify how our units failed to support student satisfaction and achievement of learning outcomes but to devise student-centred strategies to address this. The changes we implemented resulted in increased student and teacher satisfaction, more effective use of assessments, increased student support, and greater confidence in our students’ competence to undertake pharmaceutical compounding successfully and in accordance with legal and professional standards.
