Transforming a pharmaceutical compounding unit using competency-based assessment and Universal Design for Learning (UDL)

Pharmaceutical compounding is recognised as a core pharmacy skill, both in Australia (1) and internationally (2). As part of their accreditation requirements, Australian University programs need to deliver graduates who can prepare and supply extemporaneously compounded medicines safely and accurately in accordance with current legislation and guidelines (3).

At the University of Sydney, undergraduate students develop these skills in the compulsory third-year unit PHAR3815 Pharmaceutical Skills and Dispensing A. In 2023, this unit was converted from a mark and grade unit to a competency-based, grade-only unit. The reason for this was two-fold. Firstly, a competency style of marking is already used in the unit. All tasks are marked as satisfactory or unsatisfactory and it seems disingenuous to then assign marks to these. Secondly, in the past, students were able to pass the unit without demonstrating competence at key tasks. For example, in the final written exam in 2022, less than 50% of students were marked correct for questions assessing writing an appropriate method and completing Pharmaceutical Benefits Scheme (PBS) pricing calculations [1].

In this article, we detail our experiences converting PHAR3815 from a mark and grade unit to a grade-only competency-based unit, along with some of the unit design principles used to support students in this transition.

Competency-based education in healthcare

Competency-based education (4) is a framework for designing and implementing healthcare education that focuses on desired performance characteristics of healthcare professionals. It establishes observable and measurable performance metrics that learners must attain to be deemed competent. This is in contrast to more traditional frameworks, where learning objectives focus on what the learner should know rather than what they should be able to do. Some advantages are that it is flexible and adaptable to the needs of individual learners, targets learning where it is needed most and, by ensuring that learners have developed the skills and behaviours most relevant to practice, can improve healthcare outcomes.

Features of competencies

Before we discuss how we identified and assessed the relevant compounding-related competencies, it is important to think about what a competency is. While there have been many definitions of competence and competency, here we follow Albanese et al’s (5) who outline the following 5 features of a competency:

1. Focuses on performance of an end product or “goal state” of instruction (or whether learners can use their learning to solve problems, perform procedures, communicate effectively, or make good clinical decisions rather than what and how they learn)
2. Reflects expectations external to the immediate instructional program
3. Is expressible in terms of measurable behaviour
4. Uses a standard for judging learners that is not dependent on the performance of other learners
5. Informs learners and other stakeholders what is expected of them

Defining, teaching, and assessing key competencies in PHAR3815

We used the principles outlined above to determine relevant competencies and how to assess them. We reviewed professional practice documents (such as the Pharmaceutical Society of Australia (PSA) National Competency Standards (1), Australian Pharmacy Council (APC) Accreditation Standards for University Pharmacy Programs (3) and Pharmacy Board of Australia Guidelines for Dispensing of Medicines (6) and Compounding of Medicines (7)) and drew on the practical experiences of the teaching team to determine competencies most relevant for contemporary pharmacy practice.

For PHAR3815, we defined these competencies as:

• Simple compounding of safe and effective pharmaceutical products that meets legal and professional requirements, including accurate documentation, appropriate methods, quality of the final product and appropriate labelling
• Pharmaceutical calculations
• PBS pricing calculations
• Medication safety principles

The next question was how to teach and assess these. Pharmaceutical calculations, PBS pricing calculations, and medication safety principles were assessed using timed, invigilated online quizzes. Each quiz was 30 minutes long and the pass mark was 80% (reflecting the importance of these skills and higher standard of performance needed to determine competence). Multiple choice questions were used for pharmaceutical calculations and medication safety and for PBS pricing students were given 2 compounded products and had to price each of these correctly. These topics were taught using short pre-recorded online videos, practice questions, an anonymous online discussion board where students could ask questions and consolidate their knowledge, and a summative quiz in the same format as their assessment quiz.

Compounding skills were taught and assessed in students’ weekly lab sessions and an end-of-semester practical compounding exam. Each week, students completed a 2-hour lab session where they would compound and complete associated documentation for 2 or 3 products. Before coming to the lab, they watched a short online video outlining the features of that product (including clinical uses, methods and labelling) and would complete a prework quiz to test their readiness for the lab. Assessment requirements for weekly lab sessions were:

• satisfactory completion of all prework quizzes,
• a dispensed products portfolio, requiring 10 out of 15 products completed in the lab marked as satisfactory according to the unit’s grading metrics, and
• submission of associated documentation (picture and dispensing record of each product made in the lab, skills mastery checklist where demonstrators sign off that students have completed relevant compounding skills, self and peer assessment logs and properties of pharmaceutical ingredients worksheet where students research properties of the ingredients used to make the weekly lab products).

Additionally, students completed a 1.5-hour restricted open book practical exam in their week 13 class (students who are unsuccessful have another chance in the main exam period). Here, they were given a script for a product that is the same type as those made in class but with a different formula and had 1.5 hours to write the method, complete the label and make the product. This was marked using the same grading rubric as students’ weekly lab products. It is an authentic assessment that aims to mimic what students will do in their real-life practice as a pharmacist. They completed 2 summative practice exams (of the same format as their final exam) in their timetabled classes in weeks 6 and 12 and 2 workshops where they completed practice methods for unknown products (with model answers provided) in weeks 3 and 5 to prepare them for their final exam.

Removing learning barriers using Universal Design for Learning (UDL)

In addition to the shift to competency-based assessment in PHAR3815, we also redesigned the unit’s Canvas pages following principles of Universal Design for Learning (UDL) (8). This research-based framework emphasises the importance of designing for the variability of all students from the outset when creating learning environments and can be used to proactively remove barriers to improve student learning. For our unit, this involved providing a design statement at the beginning of the semester and setting up the Canvas site with locked content and prerequisites and requirements for each module. Some of the barriers that we aimed to address were students not coming to class prepared, confusion caused by the use of multiple eLearning platforms (Canvas and LabArchives) and the purpose of each one, and students not completing their end-of-semester portfolio in a timely manner, resulting in delays to marks for the unit being finalised.

Our redesign involved restructuring the Canvas modules so that the expectations for the week are clearly set out at the beginning and all activities follow in a logical and structured way. We used the module prerequisite setting in Canvas to ensure students completed all learning tasks in one module before they could access the next one. This was crucial in having students stay up to date and come prepared for the practical classes where their competency was continually assessed. After the submission of their lab work and a short reflective task on how they could improve for the following week, students are then given access to the next module. The structured design of Canvas also allowed the coordinators to embed strategic learner check-in points, to be agile, and respond promptly to student feedback, improving the learning experience.

This UDL redesign was completed in consultation with Kria Coleman and Sarah Humphreys from the Educational Innovation team.

What happened (and lessons learnt)

243 students completed the unit in 2023 and 26 students (11% of the cohort) did not meet requirements; all failed 2 attempts of the final practical exam and 3 also did not achieve 10 satisfactory products in weekly lab sessions. This is consistent with previous years (previous fail rates: 7% in 2022 and 20% in 2021). Although all students passed their quizzes, many required multiple attempts (some students needed up to 7 attempts of each quiz). We had a particularly high fail rate (50%) on the first attempt of the PBS pricing quiz. In response to this, we held a remedial workshop in week 9 before further attempts. In this workshop, students could review their first quiz attempt and we outlined common errors and how to avoid them and provided further practice questions. This was not a planned learning activity at the start of the semester.

Importantly, students seemed to appreciate the unit even though it is very different to other units in their degree. We had positive USS results and the overall score of 4.3/5 was a slight improvement compared to previous years (4.28 in 2022 and 4.23 in 2021).

Students noted the following aspects of the unit as positives in the free text comments:

• Immediate feedback on assessment tasks (products made in class and quizzes)
• The ability to learn from mistakes and then apply this as they reattempted assessment tasks
• Compulsory quizzes were preferred to a final written exam as it helped them to keep up to date and encouraged cumulative learning throughout the semester
• Practice exams throughout semester before the final exam
• Hands-on activities that were relevant to real-life compounding
• The unit was well-organised and planned out

I loved the structured modules in the canvas page. As someone who sometimes struggle to navigate resources and structure myself to complete tasks/prework/quizzes, the organised pages with incorporated checkpoints made it a lot easier to be top of things each week.

(anonymous student feedback, 2023 USS)

What next?

Overall, the change to a competency-based unit provided a more holistic assessment of pharmaceutical compounding skills. While in this first iteration, we have not had a significant change in assessment rates or USS scores, we did notice that students present to class more prepared and confident to approach the task independently, allowing staff to provide higher level, tailored advice to students that can elevate their knowledge and skills to meet pharmacist professional standards rather than trying to introduce them to fundamental principles of compounding that cannot be adequately covered in the limited class time.

For others thinking of designing a competency-based unit of study, careful selection of measurable competencies is important, relevant to students and the broader community, and communicated so that all stakeholders know what is expected to achieve competence. Students also learn at different rates, which means that plenty of time is needed to allow multiple attempts at assessments – this is important to consider as allowing more time has flow-on effects in terms of staff, broader unit activities, and the physical space needed to run additional attempts.

References

1. Pharmaceutical Society of Australia (PSA). National Competency Standards Framework for Pharmacists in Australia. Australia: Pharmaceutical Society of Australia Ltd.; 2016. Avaialble from: https://www.psa.org.au/wp-content/uploads/2018/06/National-Competency-Standards-Framework-for-Pharmacists-in-Australia-2016-PDF-2mb.pdf Accessed 2/7/23.
2. Medina MS, Plaza CM, Stowe CD, et al. Center for the Advancement of Pharmacy Education 2013 educational outcomes. Am J Pharm Educ. 2013;77(8):Article 162.
3. Australian Pharmacy Council Accreditation Standards 2020 Performance Outcomes Framework. https://www.pharmacycouncil.org.au/resources/pharmacy-program-standards/ Accessed 27/6/23.
4. Gruppen LD, Mangrulkar RD, Kolars JC. The promise of competency-based education in the health professions for improving global health. Hum Resour Health 2012; 10:43. doi: 10.1186/1478-4491-10-4
5. Albanese MA, Mejicano G, Mullan P, Kokotailo P, Gruppen L: Defining characteristics of educational competencies. J Med Educ. 2008, 42: 248-255. 10.1111/j.1365-2923.2007.02996.x.
6. Pharmacy Board of Australia. Guidelines for dispensing of medicines. Australia: Pharmacy Board of Australia; 2023. Available from: https://www.pharmacyboard.gov.au/codes-guidelines.aspx Accessed 2/7/23.
7. Pharmacy Board of Australia. Guidelines on compounding of medicines. Australia: Pharmacy Board of Australia; 2023. Available from: https://www.pharmacyboard.gov.au/codes-guidelines.aspx Accessed 2/7/23.
8. Jawad J, O’Donovan M, Leif E, Knight E, Ford E, Buhne J. Universal Design for Learning in tertiary education: A scoping review and recommendations for implementation in Australia. 2022. Available from: https://www.adcet.edu.au/resource/10814/universal-design-for-learning-in-tertiary-education-a-scoping-review-and-recommendations Accessed June 29 2023.

[1] The PBS is Australia’s national health insurance system for medicines. When a prescription is dispensed, patients pay a set price ($30 for general patients, $7.30 for concession patients receiving government payments (such as Disability Support Payment and Jobseeker) and $0 for patients who have reached their safety net threshold). Pharmacists need to calculate the amount to claim from the government when a compounded prescription is dispensed.