Simulation Based Education Abstracts
Transfusion Medicine Preparing for the Worst: Use of Crisis Resource Management for Massive Hemorrhage Protocol Simulations
Roussakis, Maria
Introduction: Massive Hemorrhage Protocols are initiated during critical situations when patients experience massive hemorrhage. The role of medical laboratory technologists (MLTs) within the transfusion medicine laboratory is valuable, as they provide life-saving blood products to these patients in a timely manner. Although proficiency in technical skills such as transfusion testing is needed during these situations, effective interpersonal skills are critical, and include organization, teamwork, and communication. Crisis resource management (CRM) can be used as a model in workplace-based simulations to ensure a team's success in a crisis.
Methods: A literature review was conducted regarding healthcare simulation. From this review, a framework for organizing and designing a workplace-based simulation was created using CRM principles.
Results: The simulation team consists of a facilitator, technician, and actors. These roles can be assigned to senior laboratory staff and frontline bench technologists. Learning objectives were built based on CRM principles: leadership and followership, communication, teamwork, resource use, and situational awareness. A scenario script was created based on a realistic crisis event (motor vehicle accident), with script progressing based on different triggers and modifiers in which the MLT responds to. Global rating scales will serve as an assessment of technical and non-technical skills, and debriefing sessions after the simulation will serve as a reflection of the experience.
Conclusion: A recent ground-breaking study in Ontario outlined the top 10 recommendations for Massive Hemorrhage Protocols which included that healthcare professionals involved in these protocols should have formal training and participate in drills. Since MLTs are involved in issuing blood products, they should be trained using simulation scenarios. CRM is an effective framework for training these professionals and by practicing CRM behaviours before an actual crisis, clinical testing errors and patient mortality rates could decrease.
ROLE OF EQUALITY, DIVERSITY AND INCLUSIVITY IN STANDARDIZED PATIENT PROGRAMS: A NARRATIVE REVIEW
Uzelli Yilmaz, D; Azim, Arden; Sibbald, Matthew
Introduction: Integrating equality, diversity and inclusivity (EDI) in health professions curricula is a frequent institutional goal. How standardized patient programs (SPP) should support EDI in health professions training is less well described. To understand potential approaches to EDI by SPP, we developed a theoretical model and performed a narrative review of the available literature.
Methods: We identified 17 articles describing curriculum, programs or standardized patient (SP) encounters that provided frameworks for improving cultural competence or communicating with patients of diverse backgrounds. Thematic analysis resulted in three primary themes: improving cultural competence, effective communication with diverse patients and highlighting health inequalities.
Results: We identified descriptions of SP use to portray scripted scenarios involving EDI objectives, and use of SPs diversity characteristics in an unscripted way to support learning of clinical skills. We did not find descriptions of SPs providing lived experience as a source of EDI learning.
Conclusion: Developing an EDI-based SPP teaching approach that avoids culturalist divides, respects the complexity of culture, and enhances the learning of clinical skills that can be adapted to every patient’s background, is a challenge that must be addressed with subtlety. Opportunities for future research are outlined.
Code Blue: An Interprofessional Simulation-Based Workshop for Junior Learners
Azim, Arden; Tanic, Milica; Wainwright, Natalie; Nadarajah, Shamara; Uzelli Yilmaz, Derya; Sibbald, Matthew
Introduction
Simulation-based Code Blue team training is an effective intervention to improve team function and confidence among physicians and nurses. There is growing interest in applying this type of simulation-based training for undergraduate interprofessional education. In this study, we describe a novel simulation-based interprofessional Code Blue workshop for undergraduate healthcare trainees.
Methods
Medical and nursing students attended a 2-hour simulation-based workshop. Students rotated through four stations breaking down the skills involved in a Code Blue (calling for help, CPR, bag-valve mask and defibrillation). Team communication skills, such as closed-loop communication, were also integrated throughout the stations. Students then participated in a high-fidelity simulated Code Blue in interprofessional teams, followed by a facilitator-led debrief. Students rated their pre and post-workshop understanding of interprofessional roles in a Code Blue team and level of comfort responding to a Code Blue. Students were also asked to identify barriers that may have prevented them from participating in a Code Blue. Changes in role understanding and perceived confidence were also analyzed, and a thematic analysis of barriers to student participation in a Code Blue was conducted.
Results
Ten 1st and 2nd year medical students and seven senior nursing students participated in the workshop. Students’ self-rated understanding of their role within an interprofessional Code Blue team increased from 2.5 to 4.3 on a 5-point scale (p = .0014). Students’ self-reported confidence in initiating and participating in a Code Blue response also increased from 1.8 to 4.0 on a 5-point scale (p < 0.00001). Three themes emerged as barriers to participating in an in-hospital Code Blue: fear of making a mistake, lack of understanding of their roles within the interprofessional team, and lack of clinical knowledge or experience. Students identified hands-on practice and the integrative team-based simulation as the most helpful elements of the workshop.
Conclusion
Simulation-based training improves interprofessional role clarification and level of comfort participating in a Code Blue among junior learners. Fear of making a mistake as a student while participating in a Code Blue emerged as a prominent obstacle for learners and may reflect an area of intervention for pre-clinical simulation-based education.
Physicians first: simulating medical emergencies in psychiatric settings
Westcott, Sandra; Pardhan, Kaif; Snelgrove, Natasha; Brown, Michael; Harms, Sheila; Pardhan, Alim
Introduction:
Simulation is widespread in medical education yet remains underutilized in psychiatry. This technique may be particularly useful for practice managing infrequent and emergent events, such as medical emergencies in psychiatric settings. Psychiatry residents at McMaster expressed a desire to increase their knowledge and skills in acute medical management. In response, an interdisciplinary group of educators from psychiatry and emergency medicine collaborated with McMaster’s Center for Simulation-Based Learning to develop and carry out two simulations of common medical emergencies in psychiatric settings.
Methods:
Modalities included a standardized patient, simulating someone with unstable vital signs, and a high-fidelity mannequin, simulating a seizure. The focus was on medical expertise, communication, and collaboration skills for initial management. Sessions were facilitated by a senior psychiatry resident and emergency physician. Participants included first year psychiatry residents (n=7) and a registered nurse (n=1) from an inpatient psychiatry setting. Participants completed a pre- and post-survey assessing perceived comfort with managing medical emergencies. The post survey also included program evaluation and a needs assessment for future simulations.
Results:
Overall, the mean teaching effectiveness score was 5.88/7, with 7 being “extremely effective”. Both simulations were perceived by all participants as being either "mostly" or "extremely" relevant to residency training or work on inpatient psychiatry unit. There was a trend towards increased comfort with managing medically unstable patients in psychiatric settings, though statistical significance was not obtained. All respondents indicated they would be interested in participating future simulation-based training, with either standardized patients or mannequins. Adverse medication reactions, strangulation or hanging, and agitation were each identified by 75% of participants as high yield topics for future simulation activities.
Conclusion:
Psychiatrists need to be prepared to manage medical comorbidities in psychiatric settings and simulation may provide an opportunity to safely practice and enhance these skills. Evaluation of this pilot curriculum is encouraging and contributes to a small but growing body of literature on the use of simulation in psychiatry, with an emphasis on medical psychiatry and the unique opportunities this presents for interdisciplinary collaboration.
Mode of delivery: Development and implementation of an obstetrical in situ simulation program
Bloomfield, Valerie; Ellis, Susan; Pace, Julie; Morais, Michelle
Objective
Simulation is increasingly valued as a teaching and learning tool in obstetrical practice. In situ simulation assesses the hands-on and critical thinking skills demonstrated by a healthcare team within their clinical setting. We aimed to create an in situ simulation program to promote skill acquisition, enhance team work and identify underlying system limitations.
Study design
Key obstetrical emergencies were identified through a needs assessment. In situ simulations were developed to address these clinical presentations. During each simulation, latent safety threats were identified by organizers and participants. Medical management was evaluated through comprehensive emergency specific checklists. Leadership attitudes were assessed using the modified Perinatal Emergency Team Response Assessment tool. Following each simulation, team members were debriefed and qualitative and quantitative feedback was solicited and aggregated by specialty and discipline.
Results
Simulations were conducted monthly at two academic centers over 14 months. Multidisciplinary participation included medical learners, staff physicians, nursing, and allied health team members from Obstetrics, Anesthesia and Neonatology. Overall, participants reported their involvement was enjoyable. Participants reported improved communication skills, content knowledge and procedural knowledge. Participants rated the spontaneity of simulations, clinically relevant scenarios, safe learning environment and use of realistic equipment favourably. Latent safety threats were identified relating to equipment, medication, personnel, resources and technical skills (Table 1).
Conclusion
We present the successful implementation of a comprehensive in situ simulation program in two busy academic centers. In situ simulation allows for deliberate practice of obstetrical emergencies and promotes a culture of patient safety and collaborative care. The lessons learned serve as valuable data to identify limitations within our current practices and inform future policy change.
Talking the talk in junior interprofessional education: jargon or healthcare language?
Nadarajah, Shamara; Azim, A; Uzelli Yılmaz, D; Sibbald, M
Introduction: Use of jargon and complex clinical terminology has been identified as a potential barrier to interprofessional education (IPE). Healthcare terminology can be separated into two categories: inclusive terminology shared across different professions, and exclusive terminology unique to one profession. This study was designed in order to understand how complex terminology is perceived and valued by junior learners in an IPE setting.
Methods: A mixed methods study was conducted involving medical, nursing, and physician assistant students attending IPE simulation workshops on handover of care and working collaboratively. Students first reviewed scenarios used in the IPE workshops and identified terms they considered “inclusive” versus “exclusive” terminology. Then, students participated in semi-structured focus group discussions regarding attitudes/perceptions towards healthcare terminology and jargon. An iterative thematic analysis was conducted based on a verbatim transcript.
Results: 23 students analyzed 14 cases, identifying an average of 21 terms per case as healthcare terminology, representing 29% of the overall word count. Of the 290 words identified, 113 words were classified as healthcare terminology, 46 as inclusive and 17 as exclusive by more than half of the students. The most frequently identified inclusive terms were diagnoses (e.g. cellulitis) and measurements (e.g. oxygen saturation), whereas the most frequent exclusive terms were abbreviations (e.g. s/p ex lap) and procedures (e.g. splenectomy). Qualitative thematic analysis of focus group transcripts yielded 4 themes: abbreviations were commonly perceived as complex terminology, unfamiliarity with terminology was often attributed to being early in training even if exclusionary, simulation was perceived as a safe space for learning, and learning terminology was a valued objective in early IPE.
Conclusions: Students perceive a substantial amount of healthcare terminology in learning materials, which is recognized as a valuable learning objective in their early IPE experiences, but also a challenge. Categorization of healthcare language is not consistent among students and may reflect individual differences in prior clinical experiences. However, it seems that while profession-specific language can be a significant barrier to IPE, use of inclusive language, even if complex, prompts students to learn from, with and about one another.
"Tasting Menu" Abstracts
Making Sense of Negative Interpersonal Interactions in the Clinical Learning Environment
Vanstone, Meredith; Cavanagh, Alice; Bell, Amanda; Mountjoy, Margo; Whyte, Robert; Wong, Anne; Grierson, Lawrence
Background: Global meta-analyses indicate that 2/3 of medical learners are exposed to mistreatment and abuse (Fnais et al, 2014). Definitions of mistreatment in research and policy literature are wide-ranging and diverse but tend to focus on identifying and measuring discrete categories of behaviour. Previous research from our group and others indicates that these clear categorizations may not reflect the experiences of many learners, who report a spectrum of “grey area” experiences (Gan & Snell, 2014). We sought to determine the ways in which medical learners and educators understand less-than-collegial interactions with colleagues and supervisors in the clinical learning environment, seeking to describe how they made sense of their experiences, and which factors influenced their perception of the encounter.
Methods: Using the methodology of Interpretive Description, we conducted interviews with 49 Canadian medical learners (31) and educators (18) at varying points in their careers. We asked participants to tell us about encounters that they judged to be clearly mistreatment, those they were not sure about, and instances where they knew of disagreement as to whether the interaction constituted mistreatment.
Results: Our data affirms previous findings that learners and educators conceptualize negative interpersonal interactions on a spectrum. There was variability in the types of behaviour individuals labelled as mistreatment, but consistency in the factors they used to make this decision. We describe this decision-making process with the metaphor of a prism, which refracts a spectrum of white light into various component parts. The quality and features of the light which exits the prism is dependent on the material, angle, and relationship between each element of the prism. We use this metaphor to conceptualize the ways in which participants use interaction-specific factors, initiator factors, and recipient factors to make sense of the behaviors they encounter.
Discussion & Conclusion: Our findings highlight the challenge of using standardized definitions of mistreatment and abuse for educational policy. Individuals interpret their experiences in highly contextual ways, forming an assessment based on multiple factors relevant to the interaction, the initiator and the recipient. We discuss common types of interactions which may be interpreted as mistreatment and offering concrete suggestions for mitigating this potential.
Anything but cookbook medicine: Exploring the influence of supervisor practice variability on resident learning
Mithoowani, Siraj; Lieberman, Susan; Zeller, Michelle; Tseng, Eric; van Merrienboer, Jeroen
Introduction
Clinical practice variability is characterized by two or more expert clinicians who make different treatment decisions despite encountering a similar case. Thrombosis Medicine is an example of a specialty that has a large evidence base to support decision-making and where practice variability is common. Our aim was to explore how residents experience and interpret inter-supervisor clinical practice variability in Thrombosis Medicine, and how these variations influence learning.
Methods
Thirteen senior residents in Internal Medicine, Hematology or Thrombosis Medicine (PGY 3-6) participated in semi-structured interviews. Data collection and analysis occurred iteratively and concurrently in a manner consistent with constructive grounded theory methodology. Variation theory was used as a sensitizing concept (Marton, 2014). A central tenet of this theory is that learning occurs by experiencing three sequential patterns of variation: contrast, generalization, and fusion. Two investigators coded transcripts independently. Participants were recruited purposively with regard to their field of training until thematic saturation was reached. The primary investigator maintained an audit trail and reflexivity journal. Member checking was performed. Dedoose (version 8.0.35) was used for all analyses. The Hamilton Integrated Research Ethics Board reviewed the study.
Results
Clinical practice variability was common and generally viewed positively. Residents attributed clinical practice variability to supervisor personality differences, the clinical environment, patient preferences, and their own participation in the decision-making process. Clinical practice variability helped residents to discern critical aspects that influenced decision-making (contrast), group similar cases together so that the appropriate evidence could be applied (generalization), and weigh multiple variables to make a treatment decision (fusion). Residents proposed strategies to maximize learning from clinical practice variability including reflection, literature review, and case discussion.
Conclusion
Residents had a generally positive perception of clinical practice variability in the workplace. Practice variability helped residents discern critical aspects, group similar patients together and practice individualized medicine. Reflection, literature review, and case discussions were felt to maximize learning from practice variability.
Applied Theatre Within Undergraduate Medical Education: A Systematic Review
Johnston, Bronte; Jafine, Hartley
Introduction: Applied theatre has been integrated throughout various undergraduate medical education programs in both core curriculum and extracurricular activities within North America to improve students’ clinical skills and empathetic behaviours in future physicians. For the purposes of this paper, applied theatre is defined as the use of drama skills within educational practice to stimulate critical thinking about medicine and interprofessional skills among students. Currently, several medical schools across North America incorporate a variation of applied theatre within their curriculums both within and outside of the classroom. However, there is presently no compilation of current publications regarding applied theatre and undergraduate medical education; this information would be valuable to further understand the benefits of the intersections of drama and health education.
Methods: Twelve publications were obtained from online databases including: Pubmed, OVID, Web of Science, and ERIC; a specific keyword search was employed: “medical education" and “theatre (or applied drama or theater), and “North America”, and “drama”, and "medical students." Qualitative analysis was performed by coding and recoding the data through NVivo 12 software. Recoding words included: communication (collaborative relationships, non-verbal communication skills, verbal communication skills), education (curriculum, discussion, evaluation, experiential education, feedback, reflection, simulation), and personal development (compassion, empathy, enjoyment, perspective, realism, stress relief, wellbeing).
Results: Thematic analysis demonstrated the positive benefits of applied drama with improving undergraduate medical students’ communication skills, education, and personal development. Additionally, the results demonstrated how applied theatre activities overlap with the CanMEDS framework -the most widely accepted physician competency framework- highlighting their merit within medical education.
Conclusion: This systematic review provides insight into the importance of applied theatre within undergraduate medicine and how it should be further incorporated throughout the core curriculum.
The impact of the CFPC Certificates of Added Competence program on the delivery of comprehensive care in Canada
Grierson, Lawrence; Allice, Ilana; Baker, Alison; Farag, Alexandra; Jesse Guscott; Howard, Michelle; Mountjoy, Margo; Siu, Henry; Tong, X. Catherine; Vanstone, Meredith
Background: In 2015, the College of Family Physicians of Canada (CFPC) introduced Certificates of Added Competence (CAC) in four new domains of care: Care of the Elderly, Family Practice Anesthesia, Palliative Care, Sport and Exercise Medicine. To date, approximately 1600 CACs have been awarded in these four domains with two new CAC programs currently in development. While there is growing interest to further expand the CAC program, evidence regarding the impact of the current CAC program on the provision of coordinated, community-based comprehensive care Canada is lacking.
Methods: A multi-case study approach was used. 6 cases were identified to represent a variety of salient features relevant to the impact of CACs. The cases were groups of family physicians who worked in a coordinated manner. They represented a range of urban, suburban, rural and remote contexts, were located in regions with high and low concentrations of CAC-holders, served Anglophone, Francophone, and Indigenous populations, were located across Canada, and represented different remuneration arrangements and practice structures. The study concluded with a pan-Canadian survey of practicing family physicians that refined the qualitative conclusions.
Results: The study findings highlight factors relevant to the impacts of the CAC program, elucidate the differences between the operations and motivations of CAC holders from those of Enhanced Skill Practitioners with non-CAC credentials or without credentials, describe the models of care that CAC-holders adopt, outline the risks and benefits of the program, and capture the interactive elements that influence the ways that CAC holders contribute to the delivery of comprehensive, community-adaptive care in Canada.
Conclusions: Many of the factors that affect the CAC program effectiveness are outside of the CFPC scope of influence. Specifically, there are some practice arrangements which facilitate comprehensive care through CAC holders and some practice arrangements which discourage it. However, there are a number of ways in which the CAC program affects the perceptions and motivations of CFPC members, who are in the position to make change in their own communities. In this way, the program has a downstream impact on health care delivery practices, by supporting certain individuals in their negotiations for particular practice arrangements which will subsequently impact access to care within their community.
Resident burnout unchanged in the last 20 years – are we missing the point?
Naji, Leen; Singh, Brendan; Shah, Ajay; Naji, Faysal; Dennis, Brittany; Kavanagh, Owen; Banfield, Owen; Banfield, Laura; Alyass, Akram; Razak, Fahad; Samaan, Zainab; Profetto, Jason; Thabane, Lehana; Sohani, Zahra
Introduction: Burnout is increasingly recognized as a crisis in the medical profession. Our primary objective was to establish the prevalence of burnout globally among medical residents. Additionally, we sought to identify factors associated with burnout and trends over decades.
Methods: We conducted a systematic review and meta-regression. We searched 6 databases from inception. Reviewers screened 8,505 studies in duplicates for inclusion and risk of bias. We estimated pooled prevalence using a random effects model, and employed a random effects meta-regression for secondary analyses.
Results: Data from 197 studies, encompassing over 44,000 residents across 47 countries, between 1987 and 2018 were analyzed. The pooled global prevalence of burnout was 47.3% (95% CI 43.1% to 51.5%). Contrary to common belief, most literature does not support the association between age, sex, relationship status, work hours, or level of training with burnout. Rather, our meta-regression uncovered three novel findings: firstly, despite changes in duty hours and a focus on wellness, the burnout rates have remained unchanged over the past 2 decades (beta-coefficient 0.002, 95% CI: -0.009;0.013, P = 0.717). Secondly, burnout rates varied significantly by region (P=0.0002), European residents being least affected (30.8%). Lastly, specialty of training did not affect burnout rates (beta-coefficient -0.005, 95% CI: -0.110;0.099, P = 0.924).
Conclusions: Approximately half of resident physicians experience burnout worldwide, and this is largely attributed to systemic rather than individual factors. Further research to guide policy makers should aim to uncover systemic differences between European training programs and those elsewhere in the world.
Observing others suture: Do novices benefit from observing errors?
Sideris, Beth ; Kalun, Portia ; Zering, Jennifer ; Sonnadara, Ranil
Introduction
Surgical trainees spend a large portion of their time observing others, but the value of observing imperfect performances is unclear. Evidence suggests that observation of error-free performances provides learners with a correct guide for performing skills, while observing errors helps learners detect and then correct errors in their own performances. The purpose of this study was to determine if observing errors improves suturing performance, and makes individuals better at recognizing good performance.
Methods
Undergraduate students observed a video of an expert performing three simple interrupted sutures, and then attempted the task. Participants then observed and rated an expert, intermediate, or one of two novices performing the task. One novice imitated the expert by performing a shortcut (novice-shortcut), while the other novice performed the task without the shortcut (novice-authentic). Participants attempted the task a second time, observing and rating the same type of performance as previously, and then attempted the task a third time.
Results
The influence of observed performance type on participants’ suturing performance and performance rating ability were explored through two-way mixed ANOVAs. Task performance of participants who observed the expert, intermediate, and novice-authentic performances improved overall (F(2,116)=28.18, p<.001). Further, those who observed the expert and intermediate performances were significantly more accurate at rating performance compared with those who observed the novice performances (F(1,58)=18.71, p<.001).
Conclusion
Observing an expert, intermediate, or authentic novice led to improvements in performance, while observation of a novice performing a shortcut did not. Observing many errors did not help learners recognize good performances. When trainees learn novel tasks, we suggest they observe expert and intermediate performances (i.e., surgeons, fellows, senior trainees) to learn a correct guide for performance.
Assessment Abstracts
Data source prioritization among novice raters in competence committee decision making
Mann, Ruby; Acai, Anita; Sonnadara, Ranil R.
Introduction: Competence committees (CCs) make judgments about trainees’ progression towards competence using different data sources. However, the relative weighting of these data sources remains unclear, making member training and data curation a challenge. This study investigated data source prioritization in CC decision making using a sample of novice raters.
Methods: Fifty-eight undergraduate students were presented with 32 simulated resident portfolios and made decisions about whether or not to promote each resident based on their portfolio. Each portfolio was composed of five data sources indicating strong or weak performance in various combinations: numeric entrustable professional activity (EPA) data, narrative EPA data, numeric multisource feedback (MSF) data, narrative MSF data, and numeric examination data. To determine the relative weight of each source, the promotion rate of each resident was compared with the positive control portfolio, where all data sources were strong.
Results: Participants weighted EPA and MSF numeric scores more heavily than EPA and MSF narrative comments. Only 29% of participants promoted a resident whose EPA and MSF numeric scores were weak but whose other data indicated strong performance. Conversely, 90% of participants chose to promote a resident with a similar portfolio except with weak narrative EPA and MSF data.
Conclusion: Novice raters prioritized numeric data over narrative comments, potentially reflecting their perceived “objectivity” and ease of interpretation. This may have implications for understanding the decision-making processes of new CC members and member training, although further study using a sample of clinicians is required in order to build on these exploratory findings.
The lack of construct validity when assessing clinical clerks during their anesthesia rotations
Hamid, Amir; Schmuck, Mary Lou; Cordovani, Daniel
We analyze the grades anesthesiologists assign to McMaster medical students during their core anesthesia clerkship rotation. It revealed grade leniency and poor predictability of success on the Medical Council of Canada Qualifying Examination Part 1 (MCCQE1) leading us to question the validity of this assessment tool.
We examined the final evaluations from the two-week anesthesia rotation of all 205 undergraduate medical students in the class of 2018 at McMaster University. Final evaluations use an average of daily clinical evaluations assessing eight different domains of medical competence on a ten-point Likert Scale. Scores are anchored at 1 (“below expectations”) and 10 (“exceptional”) with 5 being “meets expectations.” On average, each student receives 9 daily evaluations.
The mean (standard deviation) final mark was 7.1 (0.6). As scores of 1-4 were never endorsed, no student was shown to fall below the “meets expectations” designation in any domain inherently limiting the construct validity of the assessment tool. There was no significant difference in the eight domains between pass and fail cohorts of the MCCQE1. A “Red Zone” status (defined as more than 2 standard deviations below the class cohort mean) did not predict a failing MCCQE1 score. A discriminant function analysis (DFA) was used to identify MCCQE1 pass versus fail status. It resembles a regression equation except that the goal is to predict a category rather than the value of a continuous dependent variable. The DFA, correctly identified 88.8% of the MCCQE1 failures, but only 62.4% of the MCCQE1 pass cohort. Despite the absence of grades below 5, based on a DFA it appears that it is possible to retrospectively identify students who were weakest compared to their peers based on the daily evaluations.
Although anesthesia-related content corresponds to a small fraction of what is included in the MCCQE1, several hours of direct observation in the clinical setting, done by multiple assessors, over a two-week period during clerkship, along with a one-to-one staff to student ratio, should provide reasonable insight into a student’s ability and competency as a medical graduate. Our data does not corroborate this statement. We believe that the value of avoiding a “ceiling effect”, using the full range of scores, is necessary to increase the clinical evaluation scale’s validity.
Characterizing Students’ Readiness for Interprofessional Learning Across Training Levels and Degree of Program Specialization
Brewer-Deluce Danielle; Bondy, Linda; Akhtar-Danesh, Noori; Wojkowski, Sarah.
Introduction: In interprofessional education (IPE) multiple professions learn with, from or about each other to enhance collaboration and promote quality of care. While incorporated in health professions training programs, it’s success in preparing trainees for collaborative practice is poorly understood, as baseline student opinions are unknown. In this study we explore the spectrum of IPE readiness hypothesizing that readiness varies across training level (ugrad vs grad) and degree of program specialization (general vs health professional program, HPP).
Methods: In 2019, all first year FHS students (n=1158) were invited to complete a baseline Readiness for Interprofessional Learning Scale (RIPLS) survey. The traditional RIPLS was completed by 304 students, and 71 completed a Q-methodology form of RIPLS. Only programs with >25 respondents (MSc(SLP), MSc(PT), MSc(OT), BScNursing, BHSc), were included.
Results: The RIPLS 4 subscale factor structure was confirmed in our sample via confirmatory factor analysis (RMSEA = 0.035, CFI = 0.959). A 2-way ANCOVA accounting for the effects of age and sex, suggested a significant effect of training level (p <0.05) with grad students’ average RIPLS scores exceeding undergraduates. The same pattern held for the teamwork & collaboration (p<0.05) and positive professional identity subscales (p<0.05). For the negative professional identity subscale, there were significant effects of both training level (grad > ugrad, p<0.05) and specialization (general > HPP, p<0.05). Lastly, there was a significant effect of specialization on the roles & responsibilities subscale (HPP > general, p<0.05). The Q-method, by-person factor analysis noted 3 significant factors, or groups of individuals 1) valuing teamwork in IPE (primarily ugrad students), 2) seeing IPE as beneficial for patient/client care (primarily grad students) or 3) focussing upon their self-interests (minority of all programs).
Conclusion: These results suggest that as students progress from undergraduate to graduate level, their readiness for IPE increases, while those not enrolled in a HPP (BHSc vs SLP, PT, OT and Nursing) tend to have a lesser understanding of professional roles and place greater value on learning with others. These differences across first-year students underscore the importance of recognizing individuals’ IPE readiness to ensure stage-matched educational interventions are offered and ensure long-term comparisons of IPE readiness are relevant.
Blink: Rapid ECG Diagnosis
Monteiro, Sandra; Sibbald, Matthew; Sherbino, Jonathan; LoGiudice, Andrew; Norman, Geoff
Introduction
A core component required of CBME is demonstration of competence prior to unsupervised practice. The goal of the current study was to develop an assessment tool that establishes a benchmark of performance for diagnosing ECGs.
Methods
Using a speeded protocol in which participants view diagnostic images for very brief time windows, we tested physicians’ sensitivity to detecting abnormalities in ECGs. This protocol has been shown to discriminate between novices and experts when diagnosing mammograms, however this is the first attempt to use this protocol as a competency assessment tool. We consulted experts in Cardiology to develop materials. Residents (12) and staff physicians (17) in Emergency Medicine were recruited as participants. Each participant completed two versions of the study. In both versions participants viewed diagnostic images and indicated whether the image was clinically normal or abnormal. In Version 2, participants also indicated a diagnosis. Version 1 contained 100 ECGs displayed at one of four time windows: 175ms, 250ms, 500ms and 1000ms. Version 2 contained 50 ECGs displayed at one of four time windows: 1s, 5s, 10s and 20s. For each participant in Version 1, we calculated a d’ statistic as a threshold independent, normally distributed measure of accuracy, at each exposure time. For each participant in Version 2, we calculated d’ and diagnostic accuracy. The goal was to measure expertise, from shorter to longer viewing times and also from less to more experience.
Results
In Version 1, d’ improved with viewing time (1 to 1.5), but did not differ between residents and staff. In Version 2, d’ was higher for staff (1.5) than residents (1), but there was no influence of viewing time. In Version 2, diagnostic accuracy was higher for staff than residents, and improved with viewing time.
Discussion & Conclusion
The current study demonstrates the potential for using speeded tests as indicators of experience and competence. While in-depth data gathering and additional diagnostic testing may help confirm a working diagnosis, physicians rely only on their own experience to determine if there is any underlying pathology present in a diagnostic image. The study is a novel approach to isolating the effect of direct experience on clinical expertise.
Funded Research Abstracts
Engaging effectively with Indigenous patients: Core cultural safety themes
Downey;Bernice, Lee; Alex, Simms; Abigail
Currently, through the Indigenous Health Initiative (IHI), Faculty of Health Sciences (FHS) is working collaboratively with community partners in the development of a comprehensive strategic planning process with the overall goal of responding to the Truth and Reconciliation Commission of Canada’s Final Report (TRC). Key objectives of the initiative include addressing systemic barriers for Indigenous learners, enhancing the learning environment, educating non-Indigenous faculty, identifying education research opportunities, and integrating Indigenous cultural knowledge into educational and research programs within the Faculty. This initiative is led by Dr. Bernice Downey, Assistant Professor in the Department of Psychiatry and the School of Nursing. A planning structure of the initiative includes a Steering Committee and six Working Groups that are working in parallel to support this institutional reform by addressing areas of administration, research, education, and curriculum, student support and services, faculty leadership and support, and Indigenous knowledge.
One of the IHI strategic priority is to educate FHS students, staff, and faculty members about Indigenous cultural safety, which is imperative as many of the national accreditation bodies have identified improving Indigenous health as a key priority (AFMC, 2017; CNA 2014). Through the IHI, San’yas Indigenous Cultural Safety (ICS) training is offered to ~200 staff and faculty members across FHS. This training is designed to increase knowledge and enhance self-awareness of health professionals to examine how their own culture and bias shape their health practice. This 8-week training helps learners understand the diversity of Indigenous people within Ontario; colonial policies specific to Residential Schools and Indian Hospitals; the impact of colonization on current Indigenous health care issues; significance of health disparities to health care providers; and ways in which assumptions and stereotypes affect the level of care within a healthcare setting.
The goal of this project is to assess how inter-professional learners benefit from Core Indigenous Cultural Safety training as it relates to their clinical practice and professional duties. Furthermore, using the results from this evaluation, the project team is planning to develop a sustainable and adaptable FHS specific Indigenous Cultural Safety training program for students, staff, and faculty members.
Learning to Lead: Perspectives from the Faculty of Health Sciences Academic Leadership Program
Li, Shelly-Anne; Chen, Ruth; Tong, Catherine; Wong, Anne; Chan, Teresa
Introduction: Faculty members in leadership roles require an extended range of interpersonal, managerial, and administrative abilities, and these skills may not have been developed during academic or clinical training. The Faculty of Health Sciences Academic Leadership Program (ALP) at McMaster University aims to equip faculty members with a variety of skills that will help them identify and achieve their goals as academic health leaders. These include understanding who they are as leaders, managing conflict, applying models of change management to their specific professional contexts, and negotiation strategies. To date, this program has not been evaluated. This project aims to evaluate the McMaster ALP for the purpose of improving the program.
Methods: This program evaluation takes the form of a theory-based approach known as Realist Evaluation. Individual, semi-structured interviews with 10 ALP participants in the 2017 to 2020 cohorts were conducted. Thematic analysis and interpretive description to identify themes and categories of the transcripts were applied.
Results: ALP participants enjoyed the networking opportunities with other emerging leaders in the faculty, which fostered a much-needed support network of leaders. Many participants were able to leverage this network to help navigate conflict resolution or program level issues within their workplace. The ALP also broadened their perspectives of leadership (that there is no one-size-fits-all approach), and provided participants with opportunities to reflect on their own leadership styles. Areas of program refinement included suggestions to incorporate additional non-clinically-focused scenarios that would be relevant to non-clinician faculty members. Participants also found organizational management and business theories challenging to apply in their immediate work settings.
Conclusions: Overall, the ALP helped participants reflect on their own leadership styles and approaches, and served as a ‘toolbox’ of strategies and approaches to use in their roles.
Associations between the geographical disposition of McMaster-graduated physicians before medical school, during medical school, in post-graduate training, and eventual practice
Grierson, Lawrence; Agarwal, Gina; Johnston, Neil; Bakker, Dorothy
At the foundation of medical education research is an assumed causal chain that exists between the practices and policies of health professional training and the quality of healthcare received by patients. The idea is, as we develop improved admissions and curricular approaches, we promote the graduation of better physicians; doctors that are more equipped to meet the health care needs of the communities they serve. This, in turn, fosters better health for the patients in these communities. Through this chain, medical schools realize that they are not only shaped by the health care system but that they are also active participants in shaping it. This recognition gives way to a social responsibility to engage in practices that promote health care systems that are relevant, of high quality, cost-effective,
and equitable.
One such responsibility is to address the challenge of distributing physicians to ensure equitable access to healthcare in both urban and rural areas (1, 2). The problem of physician mal-distribution has been a regular feature of the health human resource strategies of Health Canada for decades (3, 4). Patients who struggle to access physician care are less likely to receive preventative healthcare, timely diagnosis, or specialist care; measures of access that are directly linked to health outcomes (5-7). Without policy intervention, the number of Canadians reporting challenges in accessing high quality health care will continue to increase (8, 9). There are few levers to encourage physicians to arrange their independent practices in a way which ameliorates the mal-distribution of physicians (10). However, Canadian medical education does present an opportunity (11). This proposal aims to scrutinize two educational policies that are held up as promising in influencing the geographic disposition of physicians relative to underserved areas; but, for which there is a paucity of supporting evidence: the selection of medical school candidates from particular regions and the establishment of clinical education in underserved areas (10, 12-16).
International evidence on medical school selection demonstrates that candidates with particular personal traits (for e.g., racial and ethnic minority, those who grew up in rural and/or underserved communities) are more likely to practice in underserved communities (12, 14, 16-20). Although there are some practices in Canada designed to encourage admissions from applicants from certain populations, candidate selection has not yet been widely or systematically operationalized to solve the challenge of mal-distribution. However, this is possible, given that Canadian medical schools continue to experience unprecedented application pressure, with applicants vastly outnumbering available spaces (21). There are also promising opportunities for educational intervention related to the location, format, and content of medical education curricula (10). In particular, there is good reason to believe that educating physicians in underserved communities will increase the recruitment and retention of physicians in those communities (22-29). The Northern Ontario School of Medicine (NOSM), for instance, has succeeded in increasing the number of physicians serving rural, Francophone and Indigenous communities by both selecting candidates from, and educating physician trainees in, these communities (28).
Although the NOSM example suggests that such policies can be successful in Canada, the work to date has not considered the myriad ways in which medical underservice may be understood (30-32). Indeed, the vast majority of research on the mal-distribution of physicians in Canada pays singular attention to the way in which persons living in rural regions face a lack of physicians or challenges in travelling to see a physician. However, urban residents may also be underserved when access is impeded by social disadvantage (for example, ethnic minorities who speak neither official language; homeless persons; those with drug addiction; those with mental illness). It is therefore necessary to make significant strides in optimising understanding of those geographic areas that can be defined as under-served with respect to physician distribution. Indeed, the number, type and mix of local care providers in relation to population need is currently a key data gap (33, 34), and we require strong evidence to understand how personal traits and features of educational environments may influence future practice choices. Given the contextual complexity of this issue, such an endeavour must include a holistic examination of the way that physicians account for the influence of their personal and education experiences on their practice decisions.
Strategies to asses unperceived educational needs of physicians: A Scoping Review
Armson, Heather; Shaw, Elizabeth; Moncrieff, Kathleen; Roder, Stefanie; Lofft Meghan
INTRODUCTION-Assessing needs prior to developing continuing professional development (CPD) programs is a crucial step in the education process. A previous systematic literature review (Myers 1999) highlighted the need for more objective assessments. This scoping review updates the literature on uses of objective evaluations to assess physicians’ unperceived needs in CPD.
Study objectives were to (1) scope the literature since the last systematic review, (2) conduct a systematic search for studies/reports that explore innovative tools/approaches to identify physicians’ unperceived learning needs, (3) summarize/compare/classify the identified approaches, (4) map gaps in the literature to identify future areas of research.
METHODS-The scoping review approach by Arksey and O’Malley (2005) was used to systematically map the literature on approaches to unperceived needs assessment (identifying relevant studies, study selection, charting/collating data, summarizing and reporting the results). Literature was searched using electronic databases Ovid MEDLINE, EMBASE, ERIC and Cochrane. Grey literature was also searched using a Google Custom Search Engine following the Grey Matters protocol. Focus was on physicians in a CPD context. Analysis included organizing needs assessment approaches based on the conceptual framework for assessing learning outcomes in CPD (Moore et al. (2018); knowledge, competence, performance, patient and population health status).
RESULTS-The search identified 2403 articles, 76 articles were identified for inclusion in the study (54 research, 10 theoretical, 12 grey literature). Dominant strategies for unperceived needs assessment varied between research and non-research articles. Research studies predominantly reported knowledge assessment strategies (multiple-choice tests). Theoretical literature promoted performance evaluations (chart audits) and the grey literature emphasized secondary data assessment approaches (patient input and environmental scan).
CONCLUSION-The importance of assessing unperceived needs for effective CPD has been adequately described over the past two decades. Gaps between theory-based recommendations and research studies are due to resource constraints (time, cost, available data sources) rather than lack of CPD providers’ awareness. Future evaluations should incorporate multiple needs assessment strategies and make assessments actionable by describing the implementation process and resource management.
Q-Methodology: A Revolutionary Approach to Course Evaluation
Akhtar-Danesh, Noori; Wainman, Bruce; Darling, Liz; Wojkowski, Sarah; Brewer-Deluce, Danielle; Jackson, Tom
McMaster University course evaluations typically require students to rate the “effectiveness of the instructor” on a seven-point Likert scale and ask for open-ended comments. While this evaluation is often quick and easy for students to complete, the average score is difficult to interpret, and comments provided are generally inconsistent or may represent the opinions of only a few students. As a result, the information available to instructors is limited, particularly when they aim to use such information for evidence-based course improvement. By contrast, in Q-methodology, participants rank a set of student-generated statements relative to each other, creating a forced quasi-normal distribution14 which permits a more sophisticated statistical analysis of data15,16 beyond means and standard-deviations of Likert scale rankings. Q-methodology also hones qualitative feedback, requiring participants to justify their most highly positively and negatively ranked statements. Contrasting open-ended surveys, this ensures that highly contextual feedback is obtained.
Over the last 10 years, our group has pioneered Q-methodology research in education (Appendix Figure) by developing and testing Q-methodology analysis software15 and employing Q-methodology for investigating barriers to technology uptake17 and course evaluation.18,19 This work is truly cutting edge in both Q-methodology and course evaluation areas research with only seven publications18,20–25 applying Q-methodology to course evaluations. Notably, our most recent efforts were recognized internationally when Dr. Brewer-Deluce was awarded the Early-Career Anatomist Publication Award by the American Association for Anatomy18 and a prior Q-methodology paper on course evaluation won the Best Student and Postdoctoral Presentation at Experimental Biology 2017.19,25 Specifically, in the Mackinnon paper, Q-methodology uncovered three major viewpoints on an interprofessional anatomy dissection course, supporting its utility and development as a mastery-level interprofessional experience.
In the Brewer-Deluce paper, we outlined the methodological steps necessary for using Q-methodology for course evaluation and demonstrated how this novel application uncovered previously unknown areas for course improvement in a long-standing, positively reviewed course.18 Critically, these publications not only made our methods accessible to the wider academic community but enabled our course instructors to refine course components, promoting a better student learning experience.
To date, our work has focused on evaluating a relatively homogeneous cohort of individuals (e.g., students from a single program), participating in a single experience (e.g., one course) at a single time point (e.g., one cohort). Through this proposal we look to expand upon these previous successes and offer a three-tiered approach to expand each of these three aspects -- cohort, course, and chronology. Critically, we see this line of research as a continuation of our already strong research program offering a means toward systematic improvement of course evaluations.
Technology Abstracts
‘Eyeballing’ illness severity in acute care: A dissociation between intuitive and deliberate decisions via eye tracking
LoGiudice, Andrew; Sherbino, Jonathan; Watter, Scott; Norman, Geoffrey; Monteiro, Sandra, Sibbald, Matthew
Rapidly and accurately assessing the illness severity of a patient—colloquially termed ‘eyeballing’ in practice—serves a vital role in acute care disciplines. Surprisingly little is known about the cognitive underpinnings of this skill or how to teach it. Some recent work has examined eyeballing performance in light of dual process theories of decision making, proposing that a clinician’s eyeballing proficiency is driven largely by an intuitive Type 1 form of processing that hinges heavily on first-hand experience. However, no study to date has examined the key prediction that each individual clinician ought to display both types of processing. This predicted within-clinician dissociation was supported by clinician self-reports during a simulated eyeballing task. The dissociation was further substantiated by response times and several eye tracking measures. Together these data support a dual process model of eyeballing performance, with potential implications for training and assessment.
Engagement and learning in a novel spaced repetition intervention for a paediatrics academic half day curriculum
McConnery, Jason; Bassilious, Ereny; Ngo, Quang
Introduction: Post-graduate residencies utilize academic half-days (AHD) to supplement clinical learning. Spaced repetition (SR) reinforces previously learned information to improve retention. We leveraged technology to deliver a SR-based study companion for a paediatric residency program’s AHD.
Methods: We enrolled 39 residents and invited them to participate weekly for 17 weeks. We chose one lecture weekly given on AHD (day 0) for reinforcement. We emailed participants 3 key points on day 1 and a multiple-choice question (MCQ) on day 8. On day 29, we sent two challenge MCQs to test reinforced and unreinforced content from the same day 0. We tracked participation and explored satisfaction with SR. We analyzed pooled performance on challenge questions using unpaired t-test.
Results: 31 (79%) residents participated at least once, and 13 (33%) completed more than half of weekly quizzes. 37.4% of all quizzes were completed, with an average weekly engagement of 5.5 minutes. Helpfulness to learning was rated as 7.89/10 on a Likert-like scale. Reported barriers were missing related half-days or emails, and limited time. There was no significant difference in performance between reinforced (63.4%, [53.6-73.3]) and unreinforced (65.6%, [53.7-73.2]) questions.
Conclusion: Despite high satisfaction with the intervention, only 33% of residents participated over half the time. Reinforcement was not time intensive for learners. Reinforcement was not shown to significantly improve performance though results were underpowered and subject to bias. Operational barriers also likely contributed. SR is a proven strategy in medical education, therefore future implementation must consider practical and individual barriers to facilitate success.
Examining the Effects of Stereopsis in Testing Anatomical Knowledge Using a Novel Virtual Reality Educational Tool
Bak, Alex Beomju; Simms, Abigail; Sinha, Sakshi; Shin, David; Mitchell, Josh P.; Saraco, Anthony N.; Brewer-Deluce, Danielle; Wainman, Bruce C.
Introduction: Three-dimensional (3D) visualization technology has generated interest for its application in anatomy education. By adopting stereoscopic 3D digital technologies, educational institutions can mitigate issues related to the resource intensiveness of cadaver use. Thus, a Virtual Reality Bell Ringer (VRBR) was developed using Google VR SDK for Unity. VRBR displays high-resolution, 3D stereoscopic images of anatomical models on a Google Cardboard headset. The purpose of this study was to compare the effectiveness and validity of stereoscopic 3D images, 2D images and cadaveric specimens in testing anatomical knowledge with practical examinations (OSPEs) for undergraduate students with prior anatomy education. It was hypothesized that participants would perform similarly between cadaveric specimens and stereoscopic 3D images.
Methods: Undergraduate students (N = 60) were randomized to one of three testing groups (A, B or C). Each testing group was administered OSPEs in three distinct modalities: VRBR 2D images, VRBR stereoscopic 3D images or cadaveric specimens. Each participant answered 45 anatomy-based questions (15 per modality, test order randomized across groups) and completed a questionnaire assessing cybersickness and user satisfaction. Participants completed a stereo fly test and mental rotation test, which are potential covariates, to assess their stereoacuity and visuospatial ability, respectively. This study was approved by the Hamilton Integrated Research Ethics Board.
Results: In regard to cybersickness, a small fraction reported nausea (7.7%), vertigo (7.7%), and dizziness (15.4%) while responses for headache (23.1%) and fatigue (23.1%) were moderate and general discomfort (61.5%) and eyestrain (76.9%) were more prevalent. Preliminary data (N = 13) assessed via two-way ANOVA suggested that there were no statistically significant effects of test version (F(1, 22) = 0.005, p = 0.945) or stereoscopy (F(1, 22) = 0.009, p = 0.728).
Conclusion: If this trend persisted through completion of the study, it may suggest that stereoscopy does not improve the effectiveness of digital tools for testing anatomy knowledge recall. Complete data collection and analysis in March 2020 will provide a more complete picture of the validity of stereoscopy in testing anatomy knowledge recall.
Examining the effects of stereopsis in testing anatomy knowledge recall with a novel virtual reality educational tool.
Gamification in learning: Virtual laboratory simulation improves student learning outcomes & motivation
Tsirulnikov, Danielle; Suart, Celeste; Vulcu, Felicia; Mullarkey, Caitlin
Gamification or gamified learning interventions have emerged in the past decade as exciting new technologies that can be deployed in the laboratory or classroom setting to improve students’ experience and enhance learning. Previous studies on the integration of gamified interventions in education have demonstrated improved learning outcomes, test scores, and student motivation as compared to traditional teaching methods. Until recently, gamified learning interventions have most often taken the form of web-based simulations or gamified ‘apps’. McMaster University is one of the first universities to spearhead a novel initiative in partnership with Labster (a technology-enhanced learning start-up) and Google to introduce 3D virtual laboratory simulations to undergraduate students.
This study used a mixed-methods analysis approach to evaluate the impact of 3D virtual reality simulations on learning outcomes and motivation. Thirty-nine undergraduate students had their baseline knowledge evaluated on either gene expression or viral gene therapy. After this pre-test, students participated in a 3D virtual laboratory simulation on their assigned topic. Then students had a post-test to assess any gain in knowledge. Students took an exit survey following the virtual reality experience to provide feedback on the simulation and headset technology.
Student test scores increased significantly following the completion of VR simulations, suggesting that this platform can indeed enhance learning outcomes. Moreover, 92% of participants reported that they found the simulation motivating and 97% of participants reported that they gained relevant knowledge from the simulation. Strategies for mitigating common side effects of using virtual reality headsets, including eye strain, were also identified. Taken together, this data suggests that 3D virtual reality simulations represent a powerful new application of gamification that can be used in a classroom or laboratory setting.
Don’t forget about size! 3D models are ideal for anatomical learning when diameter is greater than 10cm
Yang, Jack; DeYoung, Veronica; Xue, Yuan; Nehru, Amit; Brewer-Deluce, Danielle; Wainman, Bruce
Introduction: Historically, learning anatomy was limited to studying cadaveric materials, and by extension, “life-sized” specimens. Recent technological advancements in 3D scanning and printing now allow for the production of inexpensive and durable anatomical replicas at virtually any size. This, however, creates a dilemma: what is the most effective model size to learn from? The goal of this project is to discover the appropriate size of an object to learn nominal anatomy. We hypothesize that there exists a curvilinear relationship between model size and learning, where a model too small or too big would be detrimental to learning, and an ideal intermediate size can be determined.
Methods: Undergraduate students (n=351) without prior anatomical training learned from four bones of varying sizes and features, and were then assessed on their ability to identify learned landmarks. Thoracic vertebra (VE), hemipelvis (HE), sphenoid (SP), and scapula (SC) models were 3D-printed at four different scalar sizes. The VE and HE models were printed at 50%, 100%, 200%, and 400% anatomical scale, while SP and SC models were printed at 50%, 100%, 200%, and 300% scale. Each participant was randomly assigned to a group of two models (VE/HE or SP/SC) of a specific size and randomized across the order in which they learned them. They were then tested on the respective real bone specimens, and subsequently completed a Mental Rotations Test (MRT), an Operation Span Test (OSPAN), and a qualitative survey reporting their experience with the 3D-printed models. Throughout the analysis, models were compared based on the measure of their longest dimension rather than their scalar size.
Results: Test scores ranged from 63% - 91% with the lowest scores achieved using the smallest models. Indeed, once model diameter exceeded 10cm, all scores exceeded 75%. Further, using hierarchical multiple linear regression, there was a significant effect of model size on test score, F(2,707) = 17.15, p <0.05, r2 = 0.046.
Conclusion: While significant, the negligible effect size of this regression model, suggests that people, in general, are very good at dealing with large variations in model size, with impairments existing only at smaller sizes where difficulty identifying or discriminating between points of interest may arise. This finding supports the use of 3D-printed models for learning and suggests that there may be no additional benefit of upsizing models beyond 10cm in diameter.
The Design of Technology-Mediated Continuing Medical Education for Family Physicians
Zahorka, Stephanie; Bayer, Ilana M.; Levinson, Anthony J.
Introduction: Family physicians are obligated to participate in continuing medical education (CME) to maintain their proficiency. As technology-mediated CME (eCME) solutions become increasingly accepted, it is important to study how evidence-based principles of e-learning design can be used to improve educational interventions for family physicians. This study examines the current design of eCME offerings for Canadian family physicians and seeks to inform the development of future offerings.
Methods: An environmental scan of available Canadian Mainpro+ certified eCME programs was conducted during July and August 2019. Freely-available programs were reviewed with respect to their configuration, instructional methods, and presentation. Further analysis was conducted to assess their adherence to evidence-based principles of e-learning design, as identified by Clark and Mayer (2016).
Results: Twenty-three program providers were identified (11 paid and 12 free). The freely-accessible programs included asynchronous and synchronous modules and webinars. Adherence to evidence-based principles of e-learning design varied across the programs, which may signify a lack of appreciation for their importance in supporting the transfer of knowledge.
Conclusion: Researching the application of evidence-based e-learning design best practices in the eCME of family physicians merits more attention. eCME developers should incorporate e-learning design principles into new programs to maximize the benefit to learners.
Curricula Abstracts
Social Medicine Curriculum for Family Medicine Residents: Lessons Learned
Hildebrand, Alex; Lennox, Robin; Redwood-Campbell, Lynda
Introduction: The field of social medicine seeks to address the impact of social conditions on individual and population health and the practice of medicine. This study analyzed qualitative data that was part of a larger study: a non-randomized, non-blinded controlled trial that set out to use both quantitative and qualitative data to assess the feasibility of a structured social medicine training program for family medicine (FM) residents. The goal of the intervention, the Social Medicine Stream, was to empower residents to fulfill their roles as health advocates and increase confidence in practicing social medicine. This study focuses on lessons learned about residents’ attitudes and perceived competencies through analysis of the qualitative data collected from the study.
Methods: The Stream consisted of 1) 8 horizontal electives related to social medicine domains in their PGY1 year, 2) Monthly resident-led journal clubs, and 3) A 4-week social medicine selective in their PGY2 year. The 6 social medicine elective domains were shelter, addictions, and prison medicine; and LGBTQ, Indigenous, and refugee health. The intervention cohort included 10 FM residents who opted to participate in the Stream. The control group included 7 standard curriculum FM residents who volunteered to participate in this study. All were asked to participate in focus groups to discuss their experiences in social medicine thus far in their clinical training. Focus group transcripts were analyzed using NVivo software.
Results: Themes common to both groups included: necessity of practical experience to gain competency in social medicine, and desire for more practical clinical opportunities in the program. Themes identified in the intervention group included: confidence over a widespread number of domains, worry about being competent enough, a genuine interest in increasing social medicine knowledge for themselves and others, and the belief that social medicine is important for any medical practitioner. Contrasting themes in the control group included: confidence in 1-2 domains, low interest in incorporating social medicine into future practice, and the belief that only certain practitioners in certain locations need to know about social medicine.
Conclusion: This evidence suggests that more research is needed to determine who social medicine programs, and similar programs educating physicians on health advocacy and social determinants of health, should be targeted towards.
Implementation and outcomes of an evidence based Mental health training program (Beyond Silence) for healthcare workers.
Moll, Sandra; Macedo, Luciana; Addanki, Sheila
Introduction: Healthcare workers are 1.5 times more likely to be off work due to illness or disability compared to workers in other sectors. However, many workers do not seek help due to personal, social or organization barriers. “Beyond Silence”, a two-day peer-led champion training program is designed to promote psychological health and safety in healthcare workers. Based on principles of contact-based education, the program aims to build mental health literacy, reduce stigma and promote skill development for seeking and providing help.
Methods: A mix-method, single group pre and post test design was used to evaluate the training programs. The program was implemented at 9 sites across Ontario with 123 participants. Participants included clinical, non-clinical and managers/supervisors.
Results: A large proportion of participants were women (85.4%), managers (48.8%) and had a previous history of mental health issues (53.7%). Mental health literacy, stigma, help-seeking behaviour and help-outreach behaviour was measured at baseline, follow-up, 3-months and at 6-months. A multivariate model of baseline literacy, stigma and organization resources to support mental health predicted change in literacy from baseline to 6 months.
Conclusion: The “Beyond Silence” program was found to be a beneficial and accessible tool for promoting mental health literacy and help outreach support for healthcare workers, particularly in small, rural and under-resources workplaces.
Developing a Competency Framework for Communication Skills in Psychiatry Residency Education
Snelgrove, Natasha; Zaccagnini, Marco; McCabe, Randi; McConnell, Meghan; Sherbino, Jonathan
Introduction: Communication is a core competency for Canadian physicians and psychiatrists in particular. Psychiatrists use clinical interviews as the method to establish a diagnosis, which is often integral in informing treatment (e.g. psychotherapy). Robust literature exists on teaching communication skills to medical students. However, research describing advanced communication skills for psychiatry residents is lacking. Given the transition to competency-based education in Canadian postgraduate residency education, improved understanding of progressive communication skills development in psychiatry residents is critical. This project aims to understand the progression in abilities in psychiatry residents as a first step in developing a competency framework for communication abilities.
Methods: Purposive sampling and a constructivist grounded theory approach was used to interview 14 faculty educators from the disciplines of psychiatry (9), social work (1) and psychology (4) who regularly supervise psychiatry residents at two sites affiliated with one university-based residency program. Constant comparative analysis occurred concurrently with iterative data collection until thematic sufficiency was reached to identify all themes and relationships between themes.
Results: This project identified five themes that outline the progressive development of communication skills in psychiatry residents. Three themes served as foundational and related to specific skill development, including refining pre-established relational skills, developing a skills repertoire of specific communication strategies such as de-escalation techniques, and learning to reflect upon and manage internal emotional and cognitive reactions. These competencies served as foundational pillars for the final two themes, in which residents fully develop a personalized art of flexible psychiatric interviewing. This foundation then allows residents and psychiatrists in practice to skillfully partner with all patients in co-creating treatment plans.
Conclusion: This research represents a first step in defining a communication competency framework for psychiatry residents, including the stages required for progression towards autonomous practice. Future research should further validate and test the framework and explore teaching and assessment methods.
Residency Program Changes Before Competency-Based Medical Education: A Survey of Canadian Medical Oncology Program Directors
Arora, Roochi; Kazemi, Ghazaleh; Hsu, Tina; Levine, Oren; Basi, Sanraj K; Henning, Jan-Willem; Sussman, Jonathan; Mukherjee, Som D
Background: Postgraduate medical education is undergoing a paradigm shift in many universities worldwide, transitioning from a time-based model to Competency-Based Medical Education (CBME). Residency programs may need to alter clinical rotations, educational curricula, assessment methods and faculty involvement in preparation for CBME, a process not yet characterized in the literature. The main objective of this study was to gain an understanding of the changes made within Canadian medical oncology residency programs in preparation for implementation of CBME. Methods: We conducted a cross sectional survey of Canadian medical oncology program directors. Questions centered around 5 themes: planned structural and curricular rotation changes, orientation of incoming residents and faculty to CBME, changes to learning resources for residents, changes to methods of teaching and assessment of trainees, and new educational roles for faculty members. Results: Prior to implementing CBME, all program directors had made changes to at least one clinical rotation, most commonly changing the malignant hematology rotation (74%) from a mixed inpatient and outpatient rotation to being entirely outpatient and eliminating the radiation oncology rotation (64%). Introductory rotations were altered to focus on common tumour sites, while rotations closer to the end of training were changed to increase learner autonomy. Most program directors planned to enhance resident learning with the addition of electronic teaching modules (79%), inclusion of new training experiences (71%), and changing the academic half-day curriculum (50%). Most program directors (64%) planned to change assessment methods to be entirely based on entrustable professional activities and milestones. All programs had developed a competence committee to review learner progress and most (86%) integrated academic coaches. Conclusions: Transitioning to CBME led to major structural and curricular changes within medical oncology training programs. Awareness of commonly implemented changes may help other programs who have yet to transition to CBME.
The impact of new surgical coaching tool: facilitating collaborative teaching experiences
Tran, Cindy Khai Nhi; Zering, Jennifer; Howcroft, Kathleen; Sonnadara, Ranil
Introduction:
Despite literature positioning coaching techniques as a central component of competency-based medical education (CBME), implementations of coaching are highly heterogenous in medical education to date. Previous work from our group found surgical staff and trainee experiences with coaching to be inconsistent. Notably, the focus on learner-directed goals and consistent rapport that are identified as important for successful learning in the coaching literature were largely absent. Working with a number of surgical staff and trainees, we created a surgical coaching checklist to highlight these areas. In the present study, we explored the impact of the tool on teaching and learning experiences in the operating room (OR).
Methods:
Eight staff and eight surgical trainees used the new coaching checklist for four weeks. Following this, they participated in individual interviews. Questions explored experiences with the coaching tool, barriers and facilitators to use of the tool, and perceived effectiveness of the tool. Interviews were transcribed verbatim and analyzed for themes.
Results:
Staff and trainees reported that the checklist was helpful, providing opportunities for persistent interactions, facilitating more direct feedback, and creating a consistent, structured framework for teaching in the OR. The tool was most effective when both parties believed its use would add value to the teaching encounter. Lack of preparation, individual teaching preferences, inability to communicate openly, and competing time demands impeded usage of the tool.
Conclusion:
The surgical coaching tool fosters good educational practice and more positive teaching and learning experiences, encouraging a more collaborative teaching environment. The adoption of coaching techniques continues to become increasingly important as programs transition towards CBME curriculums. Future work will further explore how the tool will help provision and interpretation of meaningful feedback and improving the culture around assessment in CBME.