In this post I will provide a brief overview of feedback and assessment as they apply to simulation based education.
Feedback / Debriefing
Providing feedback following a simulated, or even real, encounter is essential to ensure an effective educational outcome. Feedback in the setting of simulation is referred to as debriefing, it provides an opportunity to review the events of the scenario, and reflect on them. Debriefing is a lead encounters with the instructor, +/- observers, facilitating a discussion with the participant/s. Debriefing allows people time to explore and analyse their actions, with the aim of improving retention, reinforcement, and transfer of technical and non-technical skills.
Debriefing is a skill in itself requiring training and practice to ensure a high yield positive outcome. Debrief training is often included as a compontent of course instructor training, but dedicated debrief training courses are also available e.g. ECU Debriefing Skills Courses – Introduction and Advanced.
Examples of Feedback Tools / Models
- Intervene during scenario
- Useful to prevent critical error
- Disrupts participants immersion in scenario
- Positive – Negative – Positive Feedback
- Learners rapidly adapt to model
- Initial positive is ignored whilst waiting for negative feedback
- Second positive feedback perceived as a panacea following negative feedback
Plus / Delta
- Plus – What went well ?
- Delta – What would you change ?
- Simple & easy to use
- Provides framework for further discussion
- Check the learner wants and is ready for feedback
- Let the learner give comments to material being assessed
- Learner states what was done well
- Observer states what was done well
- Learner states what could be improved
- Observer states how it could be improved
- An action plan for improvement is made
Advocacy / Inquiry
- Observed Event
- Comment on observation – ‘Advocate’ on your impression
- Explore participant/s thinking – ‘Inquire’
- Develop / discover ways to improve / change performance
Getting Feedback on Feedback
We have established that feedback is an essential step in simulation based learning, and that ideally people giving feedback should be trained to do so. Given that feedback delivery is a skill in itself it is important that the debriefer seeks feedback to allow them to reflect and improve their own skills. This is an often overlooked step in simulation based education, the Center for Medical Simulation, developing the Debriefing Assessment for Simulation in Healthcare (DASH) tool, to help facilitate instructor feedback and reflection.
The DASH tool is designed to assess the instructor’s debriefing performance across six elements, these are:
- Establishes an engaging learning environment
- Maintains an engaging learning environment
- Structure debriefing in an organised way
- Provokes engaging discussions
- Identifies and explores performance gaps
- Helps trainees achieve or sustain good future performance
There are several forms of the DASH tool, one for use by trained raters, two for use by students, and two for use by the instructors themselves.
You can download the DASH forms and read more about the tool here.
Assessment in Simulation
Simulation is often employed in order to assess a candidates performance, either in a specific scenario, at a specific skill, or at a non-technical skill. When planning to assess performance using a simulated scenario, there are a number of considerations.
Is the assessment to be summative (graded performance) or formative (provding feedback) ?
Summative assessment using simulation is often employed as an OSCE examination, and for successful completion of many courses, such as APLS, ALS,and EMST. Many hospitals use formative simulated scenarios to improve teamwork, and practice management of either complex, common, or stressful encounters.
How will it comply with the objectives of an assessment tool?
- Related to objectives
- What is the desired effect on student learning ?
These particularly apply to summative assessements, but to ensure maximum effectiveness as a formative tool, all of the above should be considered.
What are the challenges to using a simulation based assessment ?
Challenges to Simulation Based Assessment include:
- Require significant planning
- Case development
- Checklist / outcome development
- Organising candidates & assesssors
- Ensure standardisation
- Summative assessment often checklist based
- Limits areas which are assessed
- Checklists cannot be all inclusive
- Limited types of encounters can be utilised for simulation
- Debrief essential in formative assessment to ensure positive outcome
- Inter-observer reliabilty
- Challenging when assessing non-technical skills e.g. communication
- Simulated performance may not match ‘real world’ performance