Evaluation of Online Simulated Scenarios for Nursing

Mike O’Driscoll,  Professor Michael Traynor

Report published: July 2021.

Introduction

In April 2020 the adult child midwifery (ACM) department at Middlesex University started using online screen-based simulation (OSBS) software sourced from Oxford Medical Simulation[1] (OMS). OMS was originally designed to be used with virtual reality hardware and delivered in the classroom setting. However, due to COVID-19 restrictions the delivery of the programme was remote and online only i.e. students participated in these simulations on their own computers, and mostly at home.

This simulation platform places students into a virtual ward or single bedded area, students are then able to control their environment and interact with the surrounding equipment as in clinical practice. There is a usually a support worker present in the room who the student can interact and communicate with. Each of the simulations features a patient presenting with a specific medical condition with certain scenarios also involving patients presenting with a variety of social or psychological issues. The student is then required to undertake a clinical assessment of that patient, using their clinical knowledge to draw conclusions and make decisions regarding patient care within a safe environment.

Evaluation

The independent evaluation of this OSBS initiative was commissioned by the ACM department in April 2020 and data collection was carried out between June and September 2020. The mixed methods evaluation was led by Mike O’Driscoll and Prof. Michael Traynor, incorporating an online survey and online focus groups with students who had participated in the OSBS initiative as well as an online focus group with staff. Relevant secondary data (such as the number of times students participated in each scenario), which was available via the OMS system, was also analysed.

Learning Objectives

Learning objectives for 3rd Year (adult and children and young people) nursing students who were to go into practice earlier than anticipated (through an opt in ‘extended placement’) because of the increased need for staff as a result of the COVID-19 pandemic: included  

  • To develop knowledge and understanding of the physical assessment of an acutely unwell patient and reflect on their practice.
  • To develop the technical and non-technical skills required when assessing and intervening with the acutely unwell patient and reflect on their practice.
  • To upskill current third year adult and children and young people nursing students to prepare them for being deployed as a result of the COVID-19 pandemic.

The OSBS initiative was also aimed at adult and children and young people third year nursing students who did not opt for the ‘extended placement’ and with adult and children and young people second year students; mental health nursing students (2nd and 3rd year); nursing associates (2nd year); third year midwifery students and PG Dip 2nd year students. For these groups there were slightly different learning objectives:

  • Enable healthcare students to develop knowledge and understanding of the physical assessment of the acutely unwell patient.
  • Enable healthcare students to develop technical and non-technical skills required when assessing and intervening with the acutely unwell patient.

Evaluation Findings

The response rate to the online survey was 25% (617 students invited, 154 wholly or partially completed questionnaires obtained). 13 students took part across two online focus groups. Seven members of staff took part in an online focus group and one took part in a one to one interview (as they had been unable to attend the focus group).

There is strong evidence of the OSBS learning objectives having been met, particularly from survey evidence regarding which skills students perceived they had gained which could be transferred into practice and learning outcomes or objectives they felt they had achieved, and this is supported by focus group evidence. A very large majority of respondents felt that they had gained skills in making clinical decisions based on their observations and prior knowledge; escalating issues to senior members of staff and using time effectively across different activities.

Respondents were very positive (mean ratings above four on a scale of 1 to 5 about the last scenario participated in, on a range of measures including realism, usefulness and overall satisfaction. Most respondents did not consider  there were serious barriers to their participation in OSBS and there were no statistically significant differences in perceived barriers to participation by scenario, age, learning style, programme or year of programme and few differences on gender. However, a considerable limitation of the evaluation is that those who took part in the evaluation had participated in the OSBS (i.e. those who experienced barriers  (or did not like OSBS for whatever  reason) were under-represented in the evaluation).

Download executive summary

Download full report

If you have any queries about this research or would like to discuss a research project please contact Mike O’Driscoll (m.odriscoll@mdx.ac.uk) or Professor Michael Traynor (m.traynor@mdx.ac.uk).


[1] See http://oxfordmedicalsimulation.com

CCRNM researchers evaluate new work-based healthcare apprenticeship models

The Learn and Earn career pathway (LECP) was established in 2017 following a pilot by Islington, Haringey and Camden community education provider networks (CEPNs) and is managed via Community Matters. It attempted to address some of the perceived barriers to employers’ engagement with apprenticeships by means of administrative assistance, financial incentives and the development of a bespoke ‘apprenticeship-plus’ model where additional training, particularly clinical skills, are included within the offer. Specifically, it was set up to promote the career pathway towards nursing and to explore the viability of using apprenticeships as a sustained funding source for training required to progress along the pathway.  

The central ‘offer’ on the pathway was a Healthcare Assistant (HCA) programme, adapted to accommodate both newcomers to the HCA role (e.g. admin or reception staff) as well as experienced HCAs who want to gain the level 3 Diploma in Clinical Healthcare Support, and to develop additional clinical skills. The programme was based on a traditional apprenticeship model but with  additional clinical skills classrooms that reflect the tasks typically undertaken by HCAs in General Practice. 

Evaluation commissioned
In 2019 a research team from the Centre for Critical Research in Nursing and Midwifery Education in Middlesex University led by Mike O’Driscoll was commissioned to carry out an independent evaluation of the Learn and Earn project. The evaluation aims included understanding the take up of each course on the Learn and Earn pathway (numbers of employers/learners, profile, progress, outcomes, reasons for take up and barriers to take up);comparing progress against targets; understanding what worked well and what did not work well and evaluating the impact of project activities at individual/business and project level.

Evaluation methods
The mixed methods process evaluation which was completed in early 2020, involved an online survey, focus group and telephone  interviews with all stakeholders  (learners, training providers, employers, CEPNs and the project managers). It also included a ‘business case’ analysis of the project, providing  a tool to inform and support employer decision-making regarding placing learners on the LECP. The business case tool, developed by Dr Wendy Knibb, provides a structure around which employers can develop their consideration of key aspects of the decision-making process such as benefits and potential non-monetary benefits of training; costs and potential non-monetary costs to training; perceived risks/challenges of training and comparing the apprenticeship model to other training options.
The evaluation also provided an evaluation toolkit, to facilitate future evaluations.

Key evaluation findings included:
Learner’s main reasons for taking part in courses were  career development (86%), followed by wanting a role which involves more healthcare expertise (83%) and job satisfaction (72%). 62% said that they had taken part in course/s because they wanted to improve their employability and just over half (51%) chose ‘get a better salary or working arrangements’ and 38% did the course/s because they ‘want a role with more contact with public’.

Level 2 HCA Apprentices

Overall satisfaction with courses, which may have been negatively impacted by some initial teething problems, was moderate – however the vast majority of current learners  (84% ) reported feeling very or fairly confident about taking part in their current course and satisfaction with current course was high for peer interaction and support (79% very or fairly satisfied), employer support (74% very or fairly satisfied) and timing / pattern of course (68% very or fairly satisfied).

Many learners felt that their course would have a positive effect on their career and their current employing organisations (or had already done so) and that their course had had a positive effect on the level of service provided to patients (especially in increasing capacity and the range of services which are offered in a GP practice).

Employers were, on the whole positive about their experiences of Learn and Earn courses and recognised benefits such as the ability to ‘grow their own’ primary healthcare workforce, i.e. to increase skills and capacity in their existing staff and to gain specific skills which the employer needed and which could be tailored to the needs of the employing organisation and to provide a better service for patients.

If you would like to commission an evaluation please contact Mike O’ Driscoll (m.odriscoll@mdx.ac.uk)

For more details of the evaluation findings please contact Community Matters (training@communitymatters.co.uk)

Report title:
Learn and Earn Project – Evaluation Report
O’Driscoll, M., Traynor, M. and Knibb, W. (2020). Middlesex University, Centre for Critical Research in Nursing and Midwifery.