Original Paper
Abstract
Background: Immersive virtual reality (IVR) is a niche technology rising in popularity in nursing education. Although there is an abundance of evidence to demonstrate the effect of virtual reality (VR) on desired learning outcomes, this evidence is limited to technical or procedural skills or managing a single patient with clinical problems. Nontechnical skills (NTS), such as communication, decision-making, teamwork, situation awareness, and managerial skills, have not been explored using IVR technology.
Objective: This study aimed to (1) investigate the potential efficacy of the IVR system virtual reality hospital (VR-Hospital, or VR-Hosp), a single-user game we developed, on nursing students’ NTS, sense of presence in the virtual clinical environment, and satisfaction and self-confidence in learning; (2) identify variables that predict NTS; and (3) explore students’ experience in using VR-Hosp.
Methods: A multimethods design with a quantitative and qualitative approach was adopted. Participants were provided with VR-Hosp with 3 scenarios in training. VR-Hosp adopted a multibed, multipatient, multitask approach and was embedded with various clinical situations. Learning outcomes were measured after the training, followed by group interviews.
Results: In total, 202 students joined the study. Results revealed high levels of satisfaction and self-confidence in learning. Significant achievement in NTS was perceived by the students. The levels of satisfaction and self-confidence in learning and the involvement and sensory fidelity domains in the sense of presence were positive predictors of NTS.
Conclusions: The promising results offer a basis for designing IVR activities for nursing education. Further investigations are imperative to determine the impact of IVR technology on learning outcomes in clinical practice.
doi:10.2196/58818
Keywords
Introduction
Background
Immersive virtual reality (IVR) is a niche technology that has been rising in popularity in nursing education. In the past decade, clinical practice opportunities have declined for nursing students due to personnel shortages and an increasing demand for clinical services [
, ]. Notably, virtual reality (VR) simulations have been recognized for their tremendous potential in nursing education and have shown benefits in performance and knowledge in emergency skills training and single-patient management [ ]. In some countries, simulations and other new technology-based training approaches have been accepted as alternatives to replace some of the required clinical hours [ ]. Their potential to replace clinical hours became more evident from the closure of clinical venues during the COVID-19 crisis [ , ]. To enable nursing students to develop the competence to solve problems in the clinical context, they not only need to apply the knowledge and skills that they have learned but also need to make decisions when facing situations that they were never taught or had never previously encountered. The question, therefore, is whether VR-based education is a plausible solution to strengthen clinical competence.Virtual Reality in Nursing Education
VR is a rapidly expending field, and its definition is complex, ranging from the use of computer-based applications to generate simulated environments depicted on a computer screen to 3D environments with interactive functions and stimuli [
]. VR can be delivered in immersive or nonimmersive modes to establish a varied perception of reality [ ]. Examples of nonimmersive modes include online or computer learning and video games. In contrast, IVR education tools or systems are usually delivered using a head-mounted device (HMD) to provide full immersion and interaction in a virtual environment. IVR 3D visualization features make it possible to interact with the virtual environment and offer a deeper sense of presence that distinguishes IVR from web-based or 2D technologies [ ]. Bystrom et al [ ] and Dang et al [ ] have defined a sense of presence as the subjective experience of participants being present within a virtual environment, which is a critical determinant of the level of engagement in immersive learning. Indeed, Dubovi et al [ ] found that students’ sense of presence within VR training is positively associated with their conceptual and procedural learning of medication administration. In clinical simulation, a higher sense of presence also allows nursing students to assume a greater degree of responsibility for patient outcomes and reflect on their clinical reasoning and problem-solving skills [ ]. We understand the benefits of repeatable training using a simulated environment for building self-confidence and self-efficacy in one’s performance without compromising safety for patients [ ]. However, few studies have evaluated the relevance of a sense of presence in the acquisition of nontechnical skills (NTS) in nursing students.Nontechnical Skills
NTS are defined as cognitive and interpersonal skills that promote worker safety and complement workers’ technical skills, which include the domains of communication, situation awareness, teamwork, leadership, and decision-making [
]. Traditional health education has primarily focused on the development of clinical knowledge and technical skills, often overlooking NTS [ ]. However, increasing evidence links failures in NTS to poor patient outcomes [ ]. In multidisciplinary settings, health professionals’ leadership, teamwork, and communication skills are crucial for clinical competence and patient safety [ ]. Additionally, in complex and dynamic environments, situation awareness (defined as an individual’s perception, comprehension, and projection of events) and critical thinking (which involves reasoning, deducing, and inducing based on this understanding) are considered essential skills for health care professionals in making effective clinical decisions [ - ]. For instance, an examination of fatal medical accident reports submitted to a third-party safety agency in Japan over a 3-year period found that approximately 50% of these incidents stemmed from failures in NTS, particularly those involving situational awareness, teamwork, and decision-making capabilities [ ]. Thus far, most IVR systems have focused on improving knowledge, mastering technical or procedural skills, developing emergency responses, or cultivating soft skills, such as empathy and communication [ - ]. Only a limited number of VR software programs have been designed for learning NTS. Examples of these 2 aspects are task prioritization according to professional guidelines [ ] and single-patient management and deterioration detection [ , ]. It is noteworthy that inconsistent results have been found in high-IVR systems for risk perception and safety training in high school students [ ]. Systematic reviews have revealed that VR is most effective in improving theoretical knowledge but not affective outcomes and NTS [ - ]. More evidence is needed to substantiate the use of VR technology to prepare nursing students to meet the clinical demands for NTS.Methods
Study Design and Objectives
This study adopted a multimethods design to investigate the efficacy of using IVR via the virtual reality hospital (VR-Hospital, or VR-Hosp system (developed by the authors and their team) on developing NTS among undergraduate nursing students. VR-Hosp (short-term patent: HK30083446) is a single-user game that was developed using Unity Pro and HTC Vive Cosmos. Its unique feature of adopting a multibed, multipatient, multitask approach aimed to create a realistic clinical environment with various situations that do not necessarily have a direct relationship to patients’ illnesses. From this, the following research objectives were derived:
- To investigate the efficacy of VR-Hosp on students’ (1) NTS, (2) sense of presence in the virtual clinical environment, and (3) satisfaction and self-confidence in learning
- To identify variables that predict NTS
Qualitative data were collected through focus groups to investigate students’ learning experiences.
Study Participants and Setting
Participants were undergraduate nursing students in a university in Hong Kong. They were recruited between 2021 and 2022 through convenient sampling from among students taking the “Fundamentals of Nursing” course, a mandatory subject for nursing students.
Virtual Reality Hospital
Conceptual Framework
A simulation model [
] was used to guide the development of VR-Hosp. This model offered a framework to structure the objectives, fidelity, and complexity of the simulation design in relation to (1) teacher and student factors and educational practices, (2) design characteristics and simulation, and (3) outcomes.Teacher Factors, Student Factors, and Educational Practices
The educational practices listed in the simulation model were active learning, feedback, interactions, expectations, diversity in learning, and time spent on tasks. In VR-Hosp, unlike traditional teaching, where learning experiences rely heavily on teachers, VR-Hosp is an immersive VR game with predetermined instructions and game flow. As such, the practice factors are relatively standardized. The activities, time, and criteria for completion were preset based on the learning objectives. Therefore, the expectations were consistent even when the tasks differed. Students were required to make distinctions between and select their actions in response to various tasks to attain the designated goals. Active participation took place since students had to play the game individually and proceed independently. They obtained prompt feedback on whether their actions were correct through answering multiple-choice questions (MCQs).
Design Characteristics and Simulation
According to the simulation model, design characteristics relate to objectives, fidelity, complexity, cues, and debriefing. The objective of using IVR in learning is to create a sense of presence that affects learning outcomes. This sense of presence is mapped on to the concept of learning space, as delineated from experiential learning theory [
]. The concept of learning space is that students learn through transactions between the person and the environment. This points to the need for fidelity and complexity in the virtual environment. Learners should be able to subjectively experience their needs, goals, unconscious influences, memories, beliefs, and events, when positioned in the dynamics, interdependence, tension, and forces of the environment.The fidelity of VR-Hosp was attained through validation of the content and coherence of the stimulus and response elements between the VR and the actual tasks, according to 4 of the 6 principles stated by Harris et al [
]: (1) face validity (whether the VR game looks and feels realistic), (2) physical fidelity (details and realism of the physical elements), (3) psychological fidelity (perceptual and cognitive features of the real task), and (4) affective fidelity (elicits emotional responses, such as stress or fear, in a similar way to the real task) [ ]. Construct validity that measures the distinction in performance between novices and experts was not examined at this stage. Ergonomic fidelity was deemed irrelevant since VR-Hosp was not designed to train students in psychomotor skills.Cues that popped up during the IVR game were essential to motivate and lead the students forward to complete the leaning tasks. In VR-Hosp, such cues were available to guide them. MCQs were also incorporated into the game. If a teacher-guided debriefing session was not available, when students played VR-Hosp individually or with peers, the MCQs would allow them to reflect on their justification for the actions that they performed.
Intervention Content
In VR-Hosp, an HMD and a controller held in the right hand were used by players to navigate the 3D virtual ward environment, where there were 3 cubicles, with 6 beds in each cubicle. VR-Hosp was evaluated by a panel of 6 experts, including 3 nursing academics with experience in developing VR games and 3 clinical mentors with rich experience in supervising clinical placements. The game was pretested by another 3 clinical mentors and 2 nursing students for acceptability in terms of content, game instructions, and game flow before it was launched.
VR-Hosp offered 3 scenarios. Each scenario lasted for approximately 10 minutes and comprised 2-3 levels of complexity tailored to students with different levels of clinical experience. The scenarios were named as follows: (1) clinical practicum orientation, (2) managing multiple tasks, and (3) prevention of errors. These 3 scenarios were developed to align with the learning objectives of VR-Hosp: (1) being aware of safety issues, (2) being alerted to contextual incidents/issues in the clinical environment, and (3) prioritizing nursing activities. The 3 scenarios featured the unpredictability of the clinical context with unexpected issues arising randomly. Each scenario within the game provided a context that allowed the students to apply and reinforce the learning objectives. By starting with simpler scenarios and progressively moving to more complex ones, the game aligned with the students’ learning journey, ensuring an appropriate level of challenge and growth.
Each time a player logged in to VR-Hosp, patient deployment, incidents, MCQs, and answer options were generated at random. The VR game used speech recognition and voice recording features (in Cantonese) for students to record dialogues in response to patients’ needs or nurse instructions before implementing care.
The voice-recording feature allowed players to respond to the requests of avatar patients or ward staff. This feature is unidirectional (ie, players must carry out the required actions before progressing to the next step in the game, with the goal of challenging players to critically reflect on the course of actions without any external assistance). Once the players completed the required actions, the MCQs appeared to provide an opportunity for them to reflect on their responses and select appropriate answers. The MCQs were developed based on the principle of reflective learning, facilitating a moment of re-evaluation and critical thinking, reinforcing the learning objectives, and promoting a deeper understanding of the scenarios [
]. In addition, the voice recordings and answers could be reviewed after the VR-Hosp session. By revisiting their interactions, students could, therefore, identify areas for improvement, reinforce their learning, and engage in meaningful discussions during the debriefing.Based on expert feedback, the response prompts were revised and optimized to provide clearer instructions and ensure a logical progression of clinical scenarios. Revisions to the MCQs were also guided by expert opinions to enhance clarity and alignment with the learning objectives. For instance, to foster critical thinking and promote a comprehensive approach to patient care, the MCQs were refined to simulate realistic dialogues, moving away from standard textbook answers. Moreover, we added answer choices that encouraged players to consider fall risk assessments instead of immediately helping the patient back to bed. The accuracy of the answers was verified, allowing for further refinements. To increase the realism of avatar patients and the clinical environment in VR-Hosp, adjustments were made to the visual and behavioral aspects of the avatars to make them more lifelike and relatable, thereby facilitating realistic patient interactions. For example, in the VR-Hosp simulation, an older male patient exhibiting an unsteady gait was depicted in the ward. His features were adjusted to more accurately reflect the movements typical of an older person. Additionally, to address the issue of VR sickness, we fine-tuned the visual and auditory elements, optimized frame rates, and implemented techniques such as smooth transitions and minimizing sudden camera movements.
displays a cubicle, depicts the VR-Hosp environment with a ringing call bell during orientation, and shows a clinical pitfall with inconsistent signage on diet and the meal delivery trolley. These scenarios require students to be attentive to virtual environments found in clinical settings.
![](https://asset.jmir.pub/assets/0c21e983e390aafe84c5395ed3c8db53.png)
Procedure
Three sets of VR-Hosp equipment, including HMDs and controllers, and 43-inch televisions mounted on movable stations, were prepared for students to practice on. There were 19 groups of students, with 12-14 students per group. In the first 30 minutes of a 2-hour session, students were given a briefing and practiced operating the controller and the HMD in the VR environment. They were presented with 1 of 3 different clinical scenarios using VR-Hosp to deliver patient care in a hospital ward setting with 3 cubicles. Afterward, they were divided into 3 teams of 4-5 students each and played VR-Hosp on their own. The members of each team took turns at being either a VR player or an observer, while the virtual game was played on the television screen at the same time to enable vicarious learning. The nurse tutor ensured each student in each group had the same amount of exposure to VR-Hosp (ie, 15 minutes), offered technical support on-site, and held a debriefing after each session.
Measures
The following outcomes were assessed immediately after the VR-Hosp session: virtual nontechnical skills, sense of presence in a virtual clinical environment, and student satisfaction and self-confidence in learning.
Virtual Nontechnical Skills
The primary outcomes were 5 personal skills measured using the virtual nontechnical skills (v-NOTECHS) system. Statements in the v-NOTECHS system were modified from the original NOTECHS rating system [
] to facilitate self-reporting on these behavioral parameters during engagement with the VR-Hosp game. The v-NOTECHS system consists of 5 domains: (1) communication and interaction (communication), 3 items; (2) situation awareness and vigilance (situation awareness), 3 items; (3) cooperation and team skills (teamwork), 5 items; (4) leadership and managerial skills (leadership), 5 items; and (5) decision-making, 5 items. One item in the communication domain of the original scale, “waited for acknowledgement from scrub nurse,” was deemed irrelevant in the VR-Hosp learning activities. Players rated the items on a 5-point Likert scale, ranging from 1for strongly disagree to 5 for strongly agree. Cronbach α coefficients in the original scale are between .77 and .87.The Presence Questionnaire Version 3.0
The Presence Questionnaire (PQ) was adopted to measure the players’ sense of presence in this virtual clinical environment [
]. The scale explores how players’ psychological state or attention shifts from the physical to the virtual environment. It consists of 4 domains: (1) involvement (involve—how natural or compelling is it to interact with the environment and control the objects?), 12 items; (2) adaptation/immersion (immersion—how much were you engaged in and focused on the assigned tasks?), 22 items; (3) sensory fidelity (sensory—the degree of coherence for stimulating multiple senses), 17 items; and (4) interface quality (interface—how much did the control or display devices interfere with concentration on the tasks?), 17 items. The highest scores for these 4 domains were 84, 50, 42, and 21, respectively. Note that the items under interface quality were negatively worded. The item scores were reversed so that the higher scores indicated less distraction and delay in the game experience. The respondents gave their ratings using a 7-point Likert scale, ranging from 1 for strongly disagree to 5 for strongly agree.Student Satisfaction and Self-Confidence in Learning
The Student Satisfaction and Self-Confidence in Learning (SSSCL) scale was selected to investigate the design of VR-Hosp [
]. The scale consists of 5 items for the satisfaction subscale (satisfaction), measuring satisfaction with the content and instructions of the game. The second subscale, self-confidence in learning (self-confidence), has 8 items, measured on a 5-point Likert scale. This subscale measures players’ self-confidence in learning associated with the development of NTS. The internal consistency of the SSSCL scale is good. Cronbach α coefficients are .92 and .82 for the satisfaction and self-confidence subscales, respectively.Qualitative Data
An independent senior research assistant who had been trained in conducting semistructured interviews led 4 online focus group sessions. Purposive sampling was used based on the participants’ sociodemographic background (ie, gender, year of study) and whether they had exposure to clinical experience (yes, no) to ensure the representativeness of the focus group sample. Each group consisted of 4-5 participants. Another assistant was present to take notes. The main question posed to the participants was, Can you share your experience with the VR-Hosp game and how it affected your learning? Further inquiries were made about the impact of the experience on their studies; the aspects they liked or disliked; and the skills, knowledge, and other benefits they acquired. Each group’s digital audio recording lasted around 45 minutes and was transcribed word for word for further analysis.
Data Analysis
Descriptive statistics were computed to show the demographic profile of the students and to capture their self-reported performance in developing their NTS using VR-Hosp. Cronbach α was used to inspect the internal consistency of the questionnaires (α≥.70: good reliability; α=.60: acceptable reliability) [
].Hierarchical multiple linear regression was used to identify the incremental predictive values of different variables on NTS. In block 1, we aimed to establish baseline relationships by considering the influence of age and gender on NTS, as these sociodemographic characteristics can affect the development of NTS [
, ]. In block 2, we accounted for experience-based factors, specifically prior clinical experience and VR game experience, which not only are associated with the development of NTS but also allowed us to establish their incremental predictive value on NTS beyond the influence of sociodemographic variables [ , ]. Considering a sense of presence has been associated with more positive outcomes in technical skills among nursing students, in block 3, the VR-Hosp game experience measured using the PQ was included to determine its predictive value for NTS, considering the influences from previous blocks [ ]. Finally, key predictors in the design of VR-Hosp and confidence in mastering the teaching content using VR, as measured using the SSSCL scale, were entered after considering the contributions in previous blocks. The Technology Acceptance Model (TAM) consists of 3 key components—computer self-efficacy, perceived usefulness, and perceived ease of use—which have been found to positively affect the behavioral intention to learn a health procedure [ ]. We used the SSSCL subscale scores because the constructs measured by these subscales closely parallel the components of TAM. For example, the self-confidence subscale of the SSSCL is highly associated with the TAM components computer self-efficacy and perceived ease of use, while the satisfaction subscale is closely linked to the perceived usefulness component [ , ].NVivo version 11 was used to manage the focus group data. Inductive content analysis was used to examine and analyze the interview data, with the aim of identifying the main categories (themes) in the data and patterns among the subcategories [
]. The unit of analysis was a statement from the transcripts of the focus groups. The exploration and interpretation of the meanings of data led to the emergence of meaningful units of subcategories, and a name was given to each subcategory corresponding to the meanings of its coding. Lastly, the subcategories were condensed to achieve the status of a theme. To ensure trustworthiness, each transcript was analyzed independently by 2 researchers (authors KC and TL), who then met to discuss the data and reach a consensus on the themes [ ]. The researchers analyzed the data until they reached the point of data saturation, when no new findings emerged.Ethical Considerations
This study was approved by the Human Subject Ethics Subcommittee of the Hong Kong Polytechnic University (approval number: HSEARS20211229002). Informed consent was obtained from all individuals included in this study. Students’ participation was entirely voluntary and would not affect their subject or curriculum in any sense.
Results
Participant Characteristics
Of the 237 undergraduate nursing students taking the preclinical VR-Hosp workshop, 202 (85.2%) students consented and participated in the study. Among the participants, who had a mean age of 20.2 (SD 1.45) years (females: n=150, 74.3%), 163 (80.7%) had no clinical experience, while the remaining students had less than 30 days of clinical experience. It is noteworthy that more than 80% (n=167) of the participants had no VR experience prior to VR-Hosp. Sociodemographic characteristics are summarized in
.Measures | Value | |
Age (years), mean (SD) | 20.2 (1.45) | |
Gender, n (%) | ||
Male | 52 (25.7) | |
Female | 150 (74.3) | |
Clinical practicum experience, n (%) | ||
No clinical experience yet | 163 (80.7) | |
Clinical placement for 15 days | 27 (13.4) | |
Clinical placement for 30 days | 12 (5.9) | |
Experience in playing IVRb games, n (%) | ||
Never | 166 (82.2) | |
1-3 years | 36 (17.8) |
aVR-Hosp: virtual reality hospital.
bIVR: immersive virtual reality.
Outcome Assessment
The overall Cronbach α coefficients of the 3 instruments were excellent (
) at .93 (v-NOTECHS system), .95 (SSSCL scale), and .92 (PQ), confirming that the construct was internally consistent (criterion α≥70).The reliability of the v-NOTECHS subscales administered to the target population were satisfactory, with Cronbach α ranging from .70 to .90 (communication=.70, situation awareness=.70, teamwork=.84, leadership=.79, and decision-making=.90). Cronbach α coefficients of the 4 PQ subscales were .90, .85, .80, and .74, respectively. The SSSCL instrument used in the study also achieved a Cronbach α of .93 for the satisfaction subscale and .91 for the self-confidence subscale.Outcome assessments | Mean (SD) | Cronbach α | ||
v-NOTECHSb | .93 | |||
Communication and interactionc | 4.3 (0.54) | .70 | ||
Situation awareness and vigilancec | 4.1 (0.55) | .70 | ||
Cooperation and team skillsc | 4.1 (0.55) | .84 | ||
Leadership and managerial skillsc | 4.1 (0.50) | .79 | ||
Decision-makingc | 4.1 (0.52) | .90 | ||
SSSCLd scale | .95 | |||
Satisfaction with the content and instructionsc | 4.3 (0.56) | .93 | ||
Self-confidence in learningc | 4.2 (0.53) | .91 | ||
PQe | .92 | |||
Involvement (maximum score=84) | 57.6 (8.71) | .90 | ||
Adaptation and immersion (maximum score=50) | 28.3 (4.95) | .85 | ||
Sensory fidelity (maximum score=42) | 39.6 (5.95) | .80 | ||
Interface quality (maximum score=21) | 11.0 (3.39) | .74 |
aVR-Hosp: virtual reality hospital.
bv-NOTECHS: virtual nontechnical skills.
cMaximum score=5.
dSSSCL: Student Satisfaction and Self-Confidence in Learning.
ePQ: Presence Questionnaire.
Efficacy on Nontechnical Skills
The survey showed that the learning outcomes for NTS were largely satisfactory, with mean scores ranging from 4.1 (SD 0.50) to 4.3 (SD 0.54) out of 5 in the v-NOTECHS scales. In the subscales satisfaction with instructions and self-confidence in learning from the SSSCL scale, mean scores of 4.3 (SD 0.56) and 4.2 (SD 0.53) were also reported, respectively, in the 5-point Likert scale. The sum of the scores for the 4 PQ domains were involvement=57.6, adaptation and immersion=28.3, sensory fidelity=39.6, and interface quality=11.0 (
).Predictions of Learning Outcomes
Self-confidence emerged as a significant predictor of 3 v-NOTECHS skills (
): situation awareness (β=.21, P=.03, adjusted R2=0.351, F2,187=2.084, P≤.001), team skills (standardized coefficient β=.49, P<.001, adjusted R2=0.392, F2,187=39.36, P≤.001), and leadership skills (β=.31, P=.002, adjusted R2=0.377, F2,187=22.32, P≤0.001). Satisfaction was documented as a significant predictor of 3 v-NOTECHS skills: communication and interaction (β=.34, P=.001, adjusted R2=0.336, F2,187=20.01, P≤.001) and decision-making (β=.39, P<.001, adjusted R2=0.392, F2,187=33.44, P≤.001).Block of variables | Communication and interaction | Situation awareness and vigilance | Cooperation and team skills | Leadership and managerial skills | Decision-making | ||||||||||||||||
β | P value | △R2 | β | P value | △R2 | β | P value | △R2 | β | P value | △R2 | β | P value | △R2 | |||||||
Block 1 | |||||||||||||||||||||
Gender | .03 | .65 | 0.006 | .04 | .54 | 0.004 | .09 | .20 | 0.010 | .14 | .06 | 0.019 | .09 | .21 | 0.008 | ||||||
Age | .07 | .31 | —a | –.05 | .52 | — | –.04 | .62 | — | .01 | .93 | — | .01 | .88 | — | ||||||
Block 2 | |||||||||||||||||||||
Gender | .04 | .61 | 0.002 | .06 | .42 | 0.017 | .10 | .15 | 0.016 | .14 | .06 | 0.001 | .09 | .20 | 0.004 | ||||||
Age | .07 | .39 | — | –.03 | .76 | — | –.04 | .16 | — | –.01 | .88 | — | –.01 | .93 | — | ||||||
VRb experience | .05 | .52 | — | .12 | .11 | — | .13 | .08 | — | .01 | .87 | — | .06 | .42 | — | ||||||
Clinical experience | 0 | .91 | — | –.08 | .31 | — | –.03 | .72 | — | .04 | .65 | — | .02 | .81 | — | ||||||
Block 3 | |||||||||||||||||||||
Gender | –.01 | .88 | 0.186c | .01 | .89 | 0.226c | .07 | .34 | 0.110c | .08 | .20 | 0.208c | .04 | .51 | 0.163c | ||||||
Age | .11 | .15 | — | .01 | .92 | — | –.02 | .81 | — | .02 | .77 | — | .01 | .89 | — | ||||||
VR experience | .03 | .67 | — | .10 | .13 | — | .12 | .09 | — | –.01 | .86 | — | .04 | .56 | — | ||||||
Clinical experience | –.02 | .78 | — | –.08 | .29 | — | –.03 | .66 | — | .03 | .69 | — | .02 | .76 | — | ||||||
Involvement | .30 | .02 | — | .25 | .05 | — | .12 | .36 | — | .18 | .15 | — | 0 | .98 | — | ||||||
Sensory fidelity | .16 | .13 | — | .27 | .01 | — | .21 | .06 | — | .11 | .26 | — | .22 | .04 | — | ||||||
Immersion | 0 | .99 | — | –.01 | .90 | — | .03 | .81 | — | .19 | .07 | — | .23 | .03 | — | ||||||
Interface quality | –.12 | .08 | — | –.03 | .62 | — | –.09 | .20 | — | –.13 | .04 | — | –.09 | .19 | — | ||||||
Block 4 | |||||||||||||||||||||
Gender | .03 | .59 | 0.142c | .03 | .57 | 0.104c | .09 | .14 | 0.256c | .11 | .07 | 0.149c | .09 | .11 | 0.217c | ||||||
Age | .09 | .19 | — | 0 | .98 | –.02 | .74 | .01 | .83 | –.01 | .87 | ||||||||||
VR experience | .04 | .56 | — | .10 | .08 | .13 | .03 | 0 | .97 | .05 | .41 | ||||||||||
Clinical experience | –.04 | .54 | — | –.09 | .16 | — | –.06 | .35 | — | .01 | .91 | — | 0 | .96 | — | ||||||
Involvement | .24 | .04 | — | .21 | .07 | — | .07 | .54 | — | .14 | .23 | — | –.07 | .56 | — | ||||||
Sensory fidelity | .14 | .13 | — | .27 | .01 | — | .21 | .02 | — | .11 | .22 | — | .21 | .02 | — | ||||||
Immersion | –.14 | .17 | — | –.13 | .19 | — | –.15 | .12 | — | .06 | .56 | — | .06 | .51 | — | ||||||
Interface quality | –.05 | .44 | — | .02 | .78 | — | –.03 | .65 | — | –.08 | .20 | — | 0 | .94 | — | ||||||
Self-confidence | .10 | .33 | — | .21 | .03 | — | .49 | <.001 | — | .31 | .002 | — | .16 | .103 | — | ||||||
Satisfaction | .34 | .001 | — | .16 | .12 | — | .08 | .41 | — | .14 | .17 | — | .39 | <.001 | — |
aNot applicable.
b VR: virtual reality.
cP<.001.
Qualitative Data
Basic patterns and coding were clustered and organized into categories (
). Content analysis yielded 3 categories corresponding to user experience and intended learning outcomes (ILOs). The first category pointed to the fidelity of VR-Hosp. The subcategories were physical fidelity, psychological fidelity, and affective fidelity. The other 2 categories deduced were found to match with the items in the SSSCL subscales and the v-NOTECHS system and, thus, were named satisfaction in learning and development of NTS. The items in the satisfaction subscale indicated the effectiveness of the VR activities in promoting enjoyment and the motivation to learn. Items in the subscale self-confidence in learning pointed to the development of expected knowledge and skills, as specified in the v-NOTECHS system measuring self-reported learning outcomes.Category I: Fidelity
Realism was noted in VR-Hosp.
Physical Fidelity
It was observed that not only “the graphics and images were constructed in detail” (student 0207) but also the narrow working space between beds was simulated (student 0304), and “a patient suddenly climbed out of bed and ran really quickly” (student 0310). The chaotic situation in clinical settings was further revealed when discrepancies were noted when the food in the meal cart differed from that indicated in the signage above the patient’s bed (student 1405).
The positive reports from students that VR-Hosp provides a realistic simulation consistent with a hospital environment were corroborated by the quantitative findings of high sensory fidelity scores in the PQ, which measures the degree of coherence in stimulating multiple senses.
Psychological Fidelity
Students continually tried to make sense of the chaotic and ad hoc incidents occurring in the virtual environment. Student 0307 mentioned:
I actually experienced the chaos of clinical practice. It feels like what I have learnt was not actually “learnt.”
Another student was stunned by having to create a voice recording in response to the assignment on patients’ needs and nurses’ tasks. The importance of communication became clear, moving the focus solely on psychomotor skills to understand patients’ needs (student 0608).
The unexpectedly immersive learning experience was closely aligned with the level of involvement—specifically, how natural or compelling it is to interact with the environment and control objects. This was particularly evident when students had to create a voice recording in response to an assignment on patients’ needs and nurses’ tasks, prompting them to reflect on the importance of communication in addressing patients’ needs.
Affective Fidelity
Tension was reported in the realistic situations embodied in the immersive interactions in VR-Hosp:
In the virtual game, I heard an alarm go off. This would not have occurred in laboratory practice. This made me feel very nervous.
[Student 0204]
I felt overwhelmed.
[Student 0811]
These qualitative findings were also corroborated by the adaptation and immersion scores, which assessed how engaged and focused participants were on the assigned tasks. Tensions observed from the realistic and immersive interactions in VR-Hosp offered them a novel learning experience beyond traditional laboratory practice.
Category II: Satisfaction in Learning
In this category, students said that this training method was helpful and effective.
Traditional teaching was somehow fragmented, and only focused on a specific area…This VR-Hosp offered us a chance to understand the workflow. In this way, we have learned better.
[Student 0706]
Not only did they come across various situations that they “did not see in textbooks” (student 1303), but they also had to “analyze information before reporting the patient’s condition” (student 1109). Students enjoyed the learning activities and asserted that the games motivated them to learn. Student 1503 said:
The game that I played was distributing meals to the patients. I did not realize that the meal signage could differ from the actual order.
Category III: Self-Confidence in Learning
Development of Communication and Interaction
Communication and interaction skills are core components of NTS. Multiple students highlighted how the VR experience helped them realize the importance of these NTS, which they had previously overlooked (students 0608, 0908, and 1306). Another participant also added:
I know communication is important, because I have to respond and find out the priorities of various situation.
[Student 0913]
Development of Situation Awareness
Situation awareness was perceived as a vital skill by many students. They had learned to be observant and alert not only to the environment but also to the patients’ actions to ensure patient safety. As a student mentioned:
Being a nurse, we have to be highly alert since so many different things could happen...What if I did not pay attention and the patient suddenly collapsed?
[Student 0410]
Another student echoed:
Many a time during laboratory practice, we perform the skills in a step-by-step manner. But in reality, it would not happen as planned. There would be sudden incidents.
[Student 1312]
Development of Decision-Making
Decision-making was 1 of the central learning outcomes. Knowledge and clinical reasoning all came into play. Moreover, prioritization was deemed “essential since tasks came up one after another” (student 0710). “[We] have to judge by ourselves” (students 0105, 0211, 0510, and 0907) was a comment that was made many times. In addition, students said, “We needed to determine the priority” (student 1301), learned to “analyze the information” (students 1008 and 1506), “think critically” (student 1401), and “know the rationale for our actions” (student 0311).
Student 1004 pointed out that “it felt so real that you would be asked to do another thing while you are busy.” Students also found that they needed to “multitask” (student 1403) and that “there is an internal timer” (student 0713). One student best summarized the experience of playing VR-Hosp:
We had to be efficient, accurate, and careful.
[Student 0406]
Discussion
Principal Findings
This study was the first of its kind to explore nursing students’ experience in using the immersive game VR-Hosp to learn NTS. Overall, findings suggested that VR-Hosp has the potential to facilitate NTS learning in order to complement current educational strategies.
Both quantitative and qualitative findings indicated the positive effects of VR-Hosp training in enhancing nursing students’ NTS through a high-stress, time-critical IVR environment that customized real-life clinical situations with multitasking and episodes of interruption, demanding heightened awareness and prompt decision-making. Sensory fidelity signified the realism and coherence of the senses, for example, sound and movement, and the examination of virtual objects from multiple viewpoints. Huang et al [
] found that a high sense of presence might generate a cognitive overload when individuals are trying to complete virtual tasks, and thereby negatively affect satisfaction. In contrast, visual, auditory, and tactile stimulations were found to be vital for novice nurses to detect cues related to patients’ conditions [ , ]. In a similar vein, sensory fidelity in VR-Hosp allowed the students to play individually with high concentration and meticulous cognitive and perceptual responses, to critically think, and make appropriate decisions. Consistent with another study, sensory modalities to imitate real-world movements were crucial in learning [ ].In our quantitative findings, critical sensory input was found to be associated with perception and comprehension of the situation [
]. More importantly, the meaningfulness and coherence of the content and activities were established as factors that promote learning and the goals of higher education [ ]. The high satisfaction and self-confidence scores, coupled with their predictions of these 2 NTS, unpacked the meaningfulness of using VR-Hosp. These findings were further complimented by students’ feedback, which indicated that the virtual environment realistically offered space for them to make decisions and react to the visual or auditory alarms, instead of providing a step-by-step guide. Students were required to multitask within a set period and to tend to episodes of sudden demands. Prioritization and communication with the patient or other members of the health care team were significant. Students had to contemplate the rationale for their actions and reconsider their justifications when answering the MCQs in response to the virtual situations. Hence, the IVR activities facilitated the building of situation awareness and vigilance, as well as decision-making.The qualitative findings provided further insights into the distinct contributions of the game’s design to the development of NTS among nursing students. Detecting and handling alarms, hazards, and clinical pitfalls were the learning activities in VR-Hosp. Students commented how VR-Hosp heightened their excitement, eliciting stress and nervousness when encountering unexpected incidents and equipment alarms during the game play. It was uncovered that the voice-recording feature in VR-Hosp made the students feel compelled to communicate with the nurse and patient during the game. They could not proceed to the next action if they did not talk to the patients and other health care workers. Such forced interactions urged them on to communicate with the avatar and engaged them in performing the designated activities. Altogether, these qualitative findings echoed the learning space concept in the experiential learning framework [
], contributing evidence of VR-Hosp’s sensory fidelity value to physical, psychological, and affective fidelity.Previous studies have reported that gender and experience in playing VR games are factors that affect satisfaction and usability scores [
]. In contrast, our quantitative findings revealed that gender and literacy in VR technology do not have an impact on situation awareness and decision-making. It was interesting to discover that the other 3 domains in the sense of presence did not predict either situation awareness or decision-making. These domains were involvement (controlling and moving in the virtual environment), immersion (proficiency and consistency with the real world), and interface quality. This means that interference in using control devices, delay in experiencing actions, and distraction in visual display did not hinder students’ confidence in their ability to develop the desired skills. It was inconclusive whether naturalistic interactions are the element that influence how the form and content of the learning modalities operate in virtual learning environments [ ]. That said, it was likely that exposure to IVR clinical practice was significant in helping novice nurses develop and master the skills of situation awareness and decision-making regardless of the control of VR devices.Overall, this study offers evidence of the sensation fidelity of the VR environment as an essential feature to achieve learning outcomes. Our findings suggest that VR-Hosp has the potential to facilitate the development of situation awareness and decision-making, complementing current educational strategies. For instance, IVR provides a cost-effective and accessible alternative to traditional pedagogy, such as high-fidelity simulation. Although high-fidelity simulation is an effective educational strategy, certain limitations, such as shortages of personnel, resources, and space and the lack of available qualified facilitators, can impact its implementation and effectiveness. However, IVR eliminates the need for physical resources and a dedicated space, allowing students to engage in realistic scenarios using VR headsets or other devices. This scalability enables a larger number of students to participate simultaneously, enhancing accessibility and reducing logistical constraints. With the use of IVR, facilitators can guide and debrief students, leveraging the recorded interactions and performance data to provide targeted feedback and facilitate reflective learning.
Limitations
This study adopted a convenience sampling method; thus, it is difficult to generalize the results. The study involved a cross-sectional survey collected after VR-Hosp practice; thus, the cause and effect of this VR teaching strategy could not be determined. NTS in the real world are often influenced by external factors, such as high uncertainty and time pressure [
]. Considering that v-NOTECHS scores were self-reported, further work is needed to objectively evaluate the learning outcomes and assess whether these skills can be sustainably translated to realistic settings. When implementing VR-Hosp, students took turns in being players and observers. This might have interfered with the immersive experience since the observers communicated with the players during the activity, for example, in locating alarms and when answering MCQs. There might also have been bias when obtaining qualitative feedback, since the process was conducted during the debriefing session moderated by the teachers. However, the information that was obtained forms a basis for future studies to compare its impact with that of other educational pedagogies. In addition, the use of HMDs in IVR can lead to VR sickness, such as nausea, dizziness, and blurred vision [ ]. Although our study did not report cases of VR sickness, its presence could negatively impact the immersive learning experience. Future research should include measures such as the Virtual Reality Sickness Questionnaire to evaluate its effect on the desired learning outcomes [ ]. Additionally, future studies using IVR may consider striking a balance between realism and incorporating elements shown to reduce VR sickness, such as narrowing the horizontal field of view, partially limiting the degrees of freedom in navigation control, and increasing tactile feedback [ ].Conclusion
VR-Hosp appears to be a promising educational pedagogy for enhancing NTS, including situation awareness and decision-making ability, in nursing students. VR-Hosp portrays a nonlinear world that challenges students to operationalize what they have learned in traditional classroom teaching and simulation practices. The findings add evidence to the determinants of learning outcomes from the aspects of a sense of presence, satisfaction, and self-confidence in learning. This should motivate the undertaking of future work on VR-based teaching and learning activities in higher education.
Acknowledgments
The research team thank Mr Joe Wong and his team, Mr Cartus Lam and Mr Jerry Wong, for their valuable advice and support in developing the virtual reality hospital software. We are grateful to the teachers and nursing students who participated in the game and provided feedback to refine and enhance the development of the game.
This work was supported by the Large Equipment Fund for Teaching 2020 and the matching fund from the School of Nursing in the Hong Kong Polytechnic University (grant number SN-1).
Authors' Contributions
All authors made substantial contributions to (1) developing the immersive virtual reality game software, (2) developing the study conception and design, (3) acquiring data and analyzing and interpreting them, and (4) drafting the paper and revising it critically for important intellectual content. All authors have agreed on the final version of the manuscript.
Conflicts of Interest
None declared.
Content analysis: overview of categories and subcategories.
DOCX File , 26 KBReferences
- Advancing healthcare transformation: a new era for academic nursing. American Association of Colleges of Nursing. Mar 1, 2016. URL: https://www.aacnnursing.org/Portals/0/PDFs/Publications/AACN-New-Era-Report.pdf [accessed 2024-06-26]
- A vision for teaching with simulation. A living document from the National League for Nursing NLN Board of Governors. National League for Nursing. Apr 20, 2015. URL: https://www.nln.org/docs/default-source/uploadedfiles/about/nln-vision-series-position-statements/vision-statement-a-vision-for-teaching-with-simulation.pdf?sfvrsn=e847da0d_0 [accessed 2024-06-27]
- Williams J, Jones D, Walker R. Consideration of using virtual reality for teaching neonatal resuscitation to midwifery students. Nurse Educ Pract. Jul 2018;31:126-129. [CrossRef] [Medline]
- Standards for education and training: recovery and emergency programme standards. Nursing & Midwifery Council. 2022. URL: https://www.nmc.org.uk/standards-for-education-and-training/emergency-education-standards/ [accessed 2024-06-27]
- Fogg N, Wilson C, Trinka M, Campbell R, Thomson A, Merritt L, et al. Transitioning from direct care to virtual clinical experiences during the COVID-19 pandemic. J Prof Nurs. Nov 2020;36(6):685-691. [FREE Full text] [CrossRef] [Medline]
- Kardong-Edgren SS, Farra SL, Alinier G, Young HM. A call to unify definitions of virtual reality. Clin Simulat Nurs. Jun 2019;31:28-34. [CrossRef]
- Hauze S, Hoyt H, Frazee J, Greiner P, Marshall J. Enhancing nursing education through affordable and realistic holographic mixed reality: the virtual standardized patient for clinical simulation. Adv Exp Med Biol. 2019;1120:1-13. [CrossRef] [Medline]
- Kononowicz AA, Woodham LA, Edelbring S, Stathakarou N, Davies D, Saxena N, et al. Virtual patient simulations in health professions education: systematic review and meta-analysis by the digital health education collaboration. J Med Internet Res. Jul 02, 2019;21(7):e14676. [FREE Full text] [CrossRef] [Medline]
- Bystrom K, Barfield W, Hendrix C. A conceptual model of the sense of presence in virtual environments. Presence: Teleop Virtual Environ. Apr 1999;8(2):241-244. [CrossRef]
- Dang BK, Palicte JS, Valdez A, O'Leary-Kelley C. Assessing simulation, virtual reality, and television modalities in clinical training. Clin Simul Nurs. Jun 2018;19:30-37. [FREE Full text] [CrossRef] [Medline]
- Dubovi I, Levy ST, Dagan E. Now I know how! The learning process of medication administration among nursing students with non-immersive desktop virtual reality simulation. Comput Educ. Oct 2017;113:16-27. [FREE Full text] [CrossRef] [Medline]
- Dunnington RM. The centricity of presence in scenario-based high fidelity human patient simulation: a model. Nurs Sci Q. Jan 2015;28(1):64-73. [CrossRef] [Medline]
- Fealy S, Jones D, Hutton A, Graham K, McNeill L, Sweet L, et al. The integration of immersive virtual reality in tertiary nursing and midwifery education: a scoping review. Nurse Educ Today. Aug 2019;79:14-19. [CrossRef] [Medline]
- Sevdalis N, Davis R, Koutantji M, Undre S, Darzi A, Vincent CA. Reliability of a revised NOTECHS scale for use in surgical teams. Am J Surg. Aug 2008;196(2):184-190. [CrossRef] [Medline]
- Lyk-Jensen HT, Jepsen RMHG, Spanager L, Dieckmann P, Østergaard D. Assessing nurse anaesthetists' non-technical skills in the operating room. Acta Anaesthesiol Scand. Aug 2014;58(7):794-801. [CrossRef] [Medline]
- Flin R, Youngson G, Yule S. How do surgeons make intraoperative decisions? Qual Saf Health Care. Jun 2007;16(3):235-239. [FREE Full text] [CrossRef] [Medline]
- Weaver SJ, Dy SM, Rosen MA. Team-training in healthcare: a narrative synthesis of the literature. BMJ Qual Saf. May 05, 2014;23(5):359-372. [FREE Full text] [CrossRef] [Medline]
- Anbro SJ, Szarko AJ, Houmanfar RA, Maraccini AM, Crosswell LH, Harris FC, et al. Using virtual simulations to assess situational awareness and communication in medical and nursing education: a technical feasibility study. J Org Behav Manag. Apr 02, 2020;40(1-2):129-139. [CrossRef]
- Saus E, Johnsen BH, Eid J. Perceived learning outcome: the relationship between experience, realism and situation awareness during simulator training. Int Marit Health. 2010;62(4):258-264. [Medline]
- Wright MC. Objective measures of situation awareness in a simulated medical environment. BMJ Qual Saf. Oct 01, 2004;13(suppl_1):65-71. [CrossRef]
- Lewis R, Strachan A, Smith MM. Is high fidelity simulation the most effective method for the development of non-technical skills in nursing? A review of the current evidence. Open Nurs J. 2012;6:82-89. [FREE Full text] [CrossRef] [Medline]
- Uramatsu M, Fujisawa Y, Mizuno S, Souma T, Komatsubara A, Miki T. Do failures in non-technical skills contribute to fatal medical accidents in Japan? A review of the 2010-2013 national accident reports. BMJ Open. Feb 16, 2017;7(2):e013678. [FREE Full text] [CrossRef] [Medline]
- Breitkreuz KR, Kardong-Edgren S, Gilbert GE, DeBlieck C, Maske M, Hallock C, et al. A multi-site study examining the usability of a virtual reality game designed to improve retention of sterile catheterization skills in nursing students. Simulat Gaming. Sep 25, 2020;52(2):169-184. [CrossRef]
- Kilmon CA, Brown L, Ghosh S, Mikitiuk A. Immersive virtual reality simulations in nursing education. Nurs Educ Perspect. 2010;31(5):314-317. [Medline]
- Plotzky C, Lindwedel U, Sorber M, Loessl B, König P, Kunze C, et al. Virtual reality simulations in nurse education: a systematic mapping review. Nurse Educ Today. Jun 2021;101:104868. [CrossRef] [Medline]
- Jorissen P, De Boi I. ImmersiMed: cross-platform simulation training. Digit Med. Oct 2018;4(4):166-172. [CrossRef]
- Forsberg E, Georg C, Ziegert K, Fors U. Virtual patients for assessment of clinical reasoning in nursing -- a pilot study. Nurse Educ Today. Nov 2011;31(8):757-762. [CrossRef] [Medline]
- Lee EJ, Ryu MJ. Development and effects of a nursing education program using virtual reality for enhancing clinical decision-making ability in respiratory disease nursing care. J Korean Acad Fundam Nurs. Nov 30, 2021;28(4):458-469. [CrossRef]
- Leder J, Horlitz T, Puschmann P, Wittstock V, Schütz A. Comparing immersive virtual reality and powerpoint as methods for delivering safety training: impacts on risk perception, learning, and decision making. Saf Sci. Jan 2019;111:271-286. [CrossRef]
- Hamilton D, McKechnie J, Edgerton E, Wilson C. Immersive virtual reality as a pedagogical tool in education: a systematic literature review of quantitative learning outcomes and experimental design. J Comput Educ. Jul 11, 2020;8(1):1-32. [CrossRef]
- Kyaw BM, Saxena N, Posadzki P, Vseteckova J, Nikolaou CK, George PP, et al. Virtual reality for health professions education: systematic review and meta-analysis by the digital health education collaboration. J Med Internet Res. Jan 22, 2019;21(1):e12959. [FREE Full text] [CrossRef] [Medline]
- Shorey S, Ng ED. The use of virtual reality simulation among nursing students and registered nurses: a systematic review. Nurse Educ Today. Mar 2021;98:104662. [CrossRef] [Medline]
- Woon APN, Mok WQ, Chieng YJS, Zhang HM, Ramos P, Mustadi HB, et al. Effectiveness of virtual reality training in improving knowledge among nursing students: a systematic review, meta-analysis and meta-regression. Nurse Educ Today. Mar 2021;98:104655. [CrossRef] [Medline]
- Jeffries PR. A framework for designing, implementing, and evaluating simulations used as teaching strategies in nursing. Nurs Educ Perspect. 2005;26(2):96-103. [Medline]
- Kolb AY, Kolb DA. Learning styles and learning spaces: Enhancing experiential learning in higher education. Acad Manag Learn Educ. Jun 2005;4(2):193-212. [CrossRef]
- Harris DJ, Bird JM, Smart PA, Wilson MR, Vine SJ. A framework for the testing and validation of simulated environments in experimentation and training. Front Psychol. 2020;11:605. [FREE Full text] [CrossRef] [Medline]
- Dewey J. How We Think: A Restatement of the Relation of Reflective Thinking to the Educative Process. Lexington, MA. D. C. Heath and Company; 1933.
- Witmer BG, Singer MJ. Measuring presence in virtual environments: a presence questionnaire. Presence. Jun 1998;7(3):225-240. [CrossRef]
- Franklin AE, Burns P, Lee CS. Psychometric testing on the NLN Student Satisfaction and Self-Confidence in Learning, Simulation Design Scale, and Educational Practices Questionnaire using a sample of pre-licensure novice nurses. Nurse Educ Today. Oct 2014;34(10):1298-1304. [CrossRef] [Medline]
- Keszei AP, Novak M, Streiner DL. Introduction to health measurement scales. J Psychosom Res. Apr 2010;68(4):319-323. [CrossRef] [Medline]
- Deng X, Chen S, Li X, Tan C, Li W, Zhong C, et al. Gender differences in empathy, emotional intelligence and problem-solving ability among nursing students: a cross-sectional study. Nurse Educ Today. Jan 2023;120:105649. [FREE Full text] [CrossRef] [Medline]
- Laschinger HKS, Finegan J, Wilk P. Context matters: the impact of unit leadership and empowerment on nurses' organizational commitment. J Nurs Adm. May 2009;39(5):228-235. [CrossRef] [Medline]
- Akhu-Zaheya LM, Gharaibeh MK, Alostaz ZM. Effectiveness of simulation on knowledge acquisition, knowledge retention, and self-efficacy of nursing students in Jordan. Clin Simulat Nurs. Sep 2013;9(9):e335-e342. [CrossRef]
- Chow M, Herold DK, Choo T, Chan K. Extending the technology acceptance model to explore the intention to use Second Life for enhancing healthcare education. Comput Educ. Dec 2012;59(4):1136-1144. [CrossRef]
- Venkatesh V, Davis FD. A model of the antecedents of perceived ease of use: development and test. Decis Sci. Sep 1996;27(3):451-481. [FREE Full text] [CrossRef]
- Wixom BH, Todd PA. A theoretical integration of user satisfaction and technology acceptance. Inf Syst Res. Mar 2005;16(1):85-102. [CrossRef]
- Graneheim UH, Lundman B. Qualitative content analysis in nursing research: concepts, procedures and measures to achieve trustworthiness. Nurse Educ Today. Feb 2004;24(2):105-112. [CrossRef] [Medline]
- Huang CL, Luo YF, Yang SC, Lu CM, Chen A. Influence of students’ learning style, sense of presence, and cognitive load on learning outcomes in an immersive virtual reality learning environment. J Educ Comput Res. Aug 05, 2019;58(3):596-615. [CrossRef]
- Burbach BE, Thompson SA. Cue recognition by undergraduate nursing students: an integrative review. J Nurs Educ. Sep 2014;53(9 Suppl):S73-S81. [CrossRef] [Medline]
- Saab MM, Hegarty J, Murphy D, Landers M. Incorporating virtual reality in nurse education: a qualitative study of nursing students' perspectives. Nurse Educ Today. Oct 2021;105:105045. [FREE Full text] [CrossRef] [Medline]
- Butt AL, Kardong-Edgren S, Ellertson A. Using game-based virtual reality with haptics for skill acquisition. Clin Simulat Nurs. Mar 2018;16:25-32. [CrossRef]
- Durso FT, Sethumadhavan A. Situation awareness: understanding dynamic environments. Hum Factors. Jun 01, 2008;50(3):442-448. [CrossRef]
- Mystakidis S, Berki E, Valtanen J. Deep and meaningful e-learning with social virtual reality environments in higher education: a systematic literature review. Appl Sci. Mar 09, 2021;11(5):2412. [CrossRef]
- Persky S, Kaphingst KA, McCall C, Lachance C, Beall AC, Blascovich J. Presence relates to distinct outcomes in two virtual environments employing different learning modalities. Cyberpsychol Behav. Jun 2009;12(3):263-268. [FREE Full text] [CrossRef] [Medline]
- Agnew Ç, Flin R, Jackson J, Raduma M. Human Factors in Patient Safety: Review of Topics and Tools. Geneva, Switzerland. World Health Organization; 2009.
- Kennedy R, Drexler J, Compton D, Stanney K, Lanham D, Harm D. Configural scoring of simulator sickness, cybersickness, and space adaptation syndrome: similarities and differences. In: Hettinger LJ, Haas MW, editors. Virtual and Adaptive Environments: Applications, Implications, and Human Performance Issues. Mahwah, NJ. Lawrence Erlbaum Associates Publishers; 2003:247-278.
- Kim HK, Park J, Choi Y, Choe M. Virtual Reality Sickness Questionnaire (VRSQ): motion sickness measurement index in a virtual reality environment. Appl Ergon. May 2018;69:66-73. [FREE Full text] [CrossRef] [Medline]
- Rebenitsch L, Owen C. Review on cybersickness in applications and visual displays. Virtual Real. Apr 26, 2016;20(2):101-125. [FREE Full text] [CrossRef]
Abbreviations
HMD: head-mounted device |
IVR: immersive virtual reality |
MCQ: multiple-choice question |
NTS: nontechnical skills |
PQ: Presence Questionnaire |
SSSCL: Student Satisfaction and Self-Confidence in Learning |
TAM: Technology Acceptance Model |
v-NOTECHS: virtual nontechnical skills |
VR: virtual reality |
VR-Hospital/VR-Hosp: virtual reality hospital |
Edited by H Ahn; submitted 26.03.24; peer-reviewed by S Kalampakorn, KYT Lim; comments to author 09.05.24; revised version received 03.06.24; accepted 10.06.24; published 10.07.24.
Copyright©Kitty Chan, Patrick Pui Kin Kor, Justina Yat Wa Liu, Kin Cheung, Timothy Lai, Rick Yiu Cho Kwan. Originally published in the Asian/Pacific Island Nursing Journal (https://apinj.jmir.org), 10.07.2024.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Asian/Pacific Island Nursing Journal, is properly cited. The complete bibliographic information, a link to the original publication on https://apinj.jmir.org, as well as this copyright and license information must be included.