Immersive virtual reality (IVR) is a niche technology that has been rising in popularity in nursing education. In the past decade, clinical practice opportunities have declined for nursing students due to personnel shortages and an increasing demand for clinical services [1,2]. Notably, virtual reality (VR) simulations have been recognized for their tremendous potential in nursing education and have shown benefits in performance and knowledge in emergency skills training and single-patient management [3]. In some countries, simulations and other new technology-based training approaches have been accepted as alternatives to replace some of the required clinical hours [4]. Their potential to replace clinical hours became more evident from the closure of clinical venues during the COVID-19 crisis [2,5]. To enable nursing students to develop the competence to solve problems in the clinical context, they not only need to apply the knowledge and skills that they have learned but also need to make decisions when facing situations that they were never taught or had never previously encountered. The question, therefore, is whether VR-based education is a plausible solution to strengthen clinical competence.

VR is a rapidly expending field, and its definition is complex, ranging from the use of computer-based applications to generate simulated environments depicted on a computer screen to 3D environments with interactive functions and stimuli [6]. VR can be delivered in immersive or nonimmersive modes to establish a varied perception of reality [7]. Examples of nonimmersive modes include online or computer learning and video games. In contrast, IVR education tools or systems are usually delivered using a head-mounted device (HMD) to provide full immersion and interaction in a virtual environment. IVR 3D visualization features make it possible to interact with the virtual environment and offer a deeper sense of presence that distinguishes IVR from web-based or 2D technologies [8]. Bystrom et al [9] and Dang et al [10] have defined a sense of presence as the subjective experience of participants being present within a virtual environment, which is a critical determinant of the level of engagement in immersive learning. Indeed, Dubovi et al [11] found that students’ sense of presence within VR training is positively associated with their conceptual and procedural learning of medication administration. In clinical simulation, a higher sense of presence also allows nursing students to assume a greater degree of responsibility for patient outcomes and reflect on their clinical reasoning and problem-solving skills [12]. We understand the benefits of repeatable training using a simulated environment for building self-confidence and self-efficacy in one’s performance without compromising safety for patients [13]. However, few studies have evaluated the relevance of a sense of presence in the acquisition of nontechnical skills (NTS) in nursing students.

NTS are defined as cognitive and interpersonal skills that promote worker safety and complement workers’ technical skills, which include the domains of communication, situation awareness, teamwork, leadership, and decision-making [14]. Traditional health education has primarily focused on the development of clinical knowledge and technical skills, often overlooking NTS [15]. However, increasing evidence links failures in NTS to poor patient outcomes [16]. In multidisciplinary settings, health professionals’ leadership, teamwork, and communication skills are crucial for clinical competence and patient safety [17]. Additionally, in complex and dynamic environments, situation awareness (defined as an individual’s perception, comprehension, and projection of events) and critical thinking (which involves reasoning, deducing, and inducing based on this understanding) are considered essential skills for health care professionals in making effective clinical decisions [18-21]. For instance, an examination of fatal medical accident reports submitted to a third-party safety agency in Japan over a 3-year period found that approximately 50% of these incidents stemmed from failures in NTS, particularly those involving situational awareness, teamwork, and decision-making capabilities [22]. Thus far, most IVR systems have focused on improving knowledge, mastering technical or procedural skills, developing emergency responses, or cultivating soft skills, such as empathy and communication [23-25]. Only a limited number of VR software programs have been designed for learning NTS. Examples of these 2 aspects are task prioritization according to professional guidelines [26] and single-patient management and deterioration detection [27,28]. It is noteworthy that inconsistent results have been found in high-IVR systems for risk perception and safety training in high school students [29]. Systematic reviews have revealed that VR is most effective in improving theoretical knowledge but not affective outcomes and NTS [30-33]. More evidence is needed to substantiate the use of VR technology to prepare nursing students to meet the clinical demands for NTS.

This study adopted a multimethods design to investigate the efficacy of using IVR via the virtual reality hospital (VR-Hospital, or VR-Hosp system (developed by the authors and their team) on developing NTS among undergraduate nursing students. VR-Hosp (short-term patent: HK30083446) is a single-user game that was developed using Unity Pro and HTC Vive Cosmos. Its unique feature of adopting a multibed, multipatient, multitask approach aimed to create a realistic clinical environment with various situations that do not necessarily have a direct relationship to patients’ illnesses. From this, the following research objectives were derived:

Qualitative data were collected through focus groups to investigate students’ learning experiences.

Participants were undergraduate nursing students in a university in Hong Kong. They were recruited between 2021 and 2022 through convenient sampling from among students taking the “Fundamentals of Nursing” course, a mandatory subject for nursing students.

Three sets of VR-Hosp equipment, including HMDs and controllers, and 43-inch televisions mounted on movable stations, were prepared for students to practice on. There were 19 groups of students, with 12-14 students per group. In the first 30 minutes of a 2-hour session, students were given a briefing and practiced operating the controller and the HMD in the VR environment. They were presented with 1 of 3 different clinical scenarios using VR-Hosp to deliver patient care in a hospital ward setting with 3 cubicles. Afterward, they were divided into 3 teams of 4-5 students each and played VR-Hosp on their own. The members of each team took turns at being either a VR player or an observer, while the virtual game was played on the television screen at the same time to enable vicarious learning. The nurse tutor ensured each student in each group had the same amount of exposure to VR-Hosp (ie, 15 minutes), offered technical support on-site, and held a debriefing after each session.

The following outcomes were assessed immediately after the VR-Hosp session: virtual nontechnical skills, sense of presence in a virtual clinical environment, and student satisfaction and self-confidence in learning.

Descriptive statistics were computed to show the demographic profile of the students and to capture their self-reported performance in developing their NTS using VR-Hosp. Cronbach α was used to inspect the internal consistency of the questionnaires (α≥.70: good reliability; α=.60: acceptable reliability) [40].

Hierarchical multiple linear regression was used to identify the incremental predictive values of different variables on NTS. In block 1, we aimed to establish baseline relationships by considering the influence of age and gender on NTS, as these sociodemographic characteristics can affect the development of NTS [41,42]. In block 2, we accounted for experience-based factors, specifically prior clinical experience and VR game experience, which not only are associated with the development of NTS but also allowed us to establish their incremental predictive value on NTS beyond the influence of sociodemographic variables [31,43]. Considering a sense of presence has been associated with more positive outcomes in technical skills among nursing students, in block 3, the VR-Hosp game experience measured using the PQ was included to determine its predictive value for NTS, considering the influences from previous blocks [11]. Finally, key predictors in the design of VR-Hosp and confidence in mastering the teaching content using VR, as measured using the SSSCL scale, were entered after considering the contributions in previous blocks. The Technology Acceptance Model (TAM) consists of 3 key components—computer self-efficacy, perceived usefulness, and perceived ease of use—which have been found to positively affect the behavioral intention to learn a health procedure [44]. We used the SSSCL subscale scores because the constructs measured by these subscales closely parallel the components of TAM. For example, the self-confidence subscale of the SSSCL is highly associated with the TAM components computer self-efficacy and perceived ease of use, while the satisfaction subscale is closely linked to the perceived usefulness component [45,46].

NVivo version 11 was used to manage the focus group data. Inductive content analysis was used to examine and analyze the interview data, with the aim of identifying the main categories (themes) in the data and patterns among the subcategories [47]. The unit of analysis was a statement from the transcripts of the focus groups. The exploration and interpretation of the meanings of data led to the emergence of meaningful units of subcategories, and a name was given to each subcategory corresponding to the meanings of its coding. Lastly, the subcategories were condensed to achieve the status of a theme. To ensure trustworthiness, each transcript was analyzed independently by 2 researchers (authors KC and TL), who then met to discuss the data and reach a consensus on the themes [47]. The researchers analyzed the data until they reached the point of data saturation, when no new findings emerged.

This study was approved by the Human Subject Ethics Subcommittee of the Hong Kong Polytechnic University (approval number: HSEARS20211229002). Informed consent was obtained from all individuals included in this study. Students’ participation was entirely voluntary and would not affect their subject or curriculum in any sense.

Of the 237 undergraduate nursing students taking the preclinical VR-Hosp workshop, 202 (85.2%) students consented and participated in the study. Among the participants, who had a mean age of 20.2 (SD 1.45) years (females: n=150, 74.3%), 163 (80.7%) had no clinical experience, while the remaining students had less than 30 days of clinical experience. It is noteworthy that more than 80% (n=167) of the participants had no VR experience prior to VR-Hosp. Sociodemographic characteristics are summarized in Table 1.

The overall Cronbach α coefficients of the 3 instruments were excellent (Table 2) at .93 (v-NOTECHS system), .95 (SSSCL scale), and .92 (PQ), confirming that the construct was internally consistent (criterion α≥70).The reliability of the v-NOTECHS subscales administered to the target population were satisfactory, with Cronbach α ranging from .70 to .90 (communication=.70, situation awareness=.70, teamwork=.84, leadership=.79, and decision-making=.90). Cronbach α coefficients of the 4 PQ subscales were .90, .85, .80, and .74, respectively. The SSSCL instrument used in the study also achieved a Cronbach α of .93 for the satisfaction subscale and .91 for the self-confidence subscale.

The survey showed that the learning outcomes for NTS were largely satisfactory, with mean scores ranging from 4.1 (SD 0.50) to 4.3 (SD 0.54) out of 5 in the v-NOTECHS scales. In the subscales satisfaction with instructions and self-confidence in learning from the SSSCL scale, mean scores of 4.3 (SD 0.56) and 4.2 (SD 0.53) were also reported, respectively, in the 5-point Likert scale. The sum of the scores for the 4 PQ domains were involvement=57.6, adaptation and immersion=28.3, sensory fidelity=39.6, and interface quality=11.0 (Table 2).

Self-confidence emerged as a significant predictor of 3 v-NOTECHS skills (Table 3): situation awareness (β=.21, P=.03, adjusted R²=0.351, F_2,187=2.084, P≤.001), team skills (standardized coefficient β=.49, P<.001, adjusted R²=0.392, F_2,187=39.36, P≤.001), and leadership skills (β=.31, P=.002, adjusted R²=0.377, F_2,187=22.32, P≤0.001). Satisfaction was documented as a significant predictor of 3 v-NOTECHS skills: communication and interaction (β=.34, P=.001, adjusted R²=0.336, F_2,187=20.01, P≤.001) and decision-making (β=.39, P<.001, adjusted R²=0.392, F_2,187=33.44, P≤.001).

Basic patterns and coding were clustered and organized into categories (Multimedia Appendix 1). Content analysis yielded 3 categories corresponding to user experience and intended learning outcomes (ILOs). The first category pointed to the fidelity of VR-Hosp. The subcategories were physical fidelity, psychological fidelity, and affective fidelity. The other 2 categories deduced were found to match with the items in the SSSCL subscales and the v-NOTECHS system and, thus, were named satisfaction in learning and development of NTS. The items in the satisfaction subscale indicated the effectiveness of the VR activities in promoting enjoyment and the motivation to learn. Items in the subscale self-confidence in learning pointed to the development of expected knowledge and skills, as specified in the v-NOTECHS system measuring self-reported learning outcomes.

This study was the first of its kind to explore nursing students’ experience in using the immersive game VR-Hosp to learn NTS. Overall, findings suggested that VR-Hosp has the potential to facilitate NTS learning in order to complement current educational strategies.

Both quantitative and qualitative findings indicated the positive effects of VR-Hosp training in enhancing nursing students’ NTS through a high-stress, time-critical IVR environment that customized real-life clinical situations with multitasking and episodes of interruption, demanding heightened awareness and prompt decision-making. Sensory fidelity signified the realism and coherence of the senses, for example, sound and movement, and the examination of virtual objects from multiple viewpoints. Huang et al [48] found that a high sense of presence might generate a cognitive overload when individuals are trying to complete virtual tasks, and thereby negatively affect satisfaction. In contrast, visual, auditory, and tactile stimulations were found to be vital for novice nurses to detect cues related to patients’ conditions [49,50]. In a similar vein, sensory fidelity in VR-Hosp allowed the students to play individually with high concentration and meticulous cognitive and perceptual responses, to critically think, and make appropriate decisions. Consistent with another study, sensory modalities to imitate real-world movements were crucial in learning [51].

In our quantitative findings, critical sensory input was found to be associated with perception and comprehension of the situation [52]. More importantly, the meaningfulness and coherence of the content and activities were established as factors that promote learning and the goals of higher education [53]. The high satisfaction and self-confidence scores, coupled with their predictions of these 2 NTS, unpacked the meaningfulness of using VR-Hosp. These findings were further complimented by students’ feedback, which indicated that the virtual environment realistically offered space for them to make decisions and react to the visual or auditory alarms, instead of providing a step-by-step guide. Students were required to multitask within a set period and to tend to episodes of sudden demands. Prioritization and communication with the patient or other members of the health care team were significant. Students had to contemplate the rationale for their actions and reconsider their justifications when answering the MCQs in response to the virtual situations. Hence, the IVR activities facilitated the building of situation awareness and vigilance, as well as decision-making.

The qualitative findings provided further insights into the distinct contributions of the game’s design to the development of NTS among nursing students. Detecting and handling alarms, hazards, and clinical pitfalls were the learning activities in VR-Hosp. Students commented how VR-Hosp heightened their excitement, eliciting stress and nervousness when encountering unexpected incidents and equipment alarms during the game play. It was uncovered that the voice-recording feature in VR-Hosp made the students feel compelled to communicate with the nurse and patient during the game. They could not proceed to the next action if they did not talk to the patients and other health care workers. Such forced interactions urged them on to communicate with the avatar and engaged them in performing the designated activities. Altogether, these qualitative findings echoed the learning space concept in the experiential learning framework [35], contributing evidence of VR-Hosp’s sensory fidelity value to physical, psychological, and affective fidelity.

Previous studies have reported that gender and experience in playing VR games are factors that affect satisfaction and usability scores [23]. In contrast, our quantitative findings revealed that gender and literacy in VR technology do not have an impact on situation awareness and decision-making. It was interesting to discover that the other 3 domains in the sense of presence did not predict either situation awareness or decision-making. These domains were involvement (controlling and moving in the virtual environment), immersion (proficiency and consistency with the real world), and interface quality. This means that interference in using control devices, delay in experiencing actions, and distraction in visual display did not hinder students’ confidence in their ability to develop the desired skills. It was inconclusive whether naturalistic interactions are the element that influence how the form and content of the learning modalities operate in virtual learning environments [54]. That said, it was likely that exposure to IVR clinical practice was significant in helping novice nurses develop and master the skills of situation awareness and decision-making regardless of the control of VR devices.

Overall, this study offers evidence of the sensation fidelity of the VR environment as an essential feature to achieve learning outcomes. Our findings suggest that VR-Hosp has the potential to facilitate the development of situation awareness and decision-making, complementing current educational strategies. For instance, IVR provides a cost-effective and accessible alternative to traditional pedagogy, such as high-fidelity simulation. Although high-fidelity simulation is an effective educational strategy, certain limitations, such as shortages of personnel, resources, and space and the lack of available qualified facilitators, can impact its implementation and effectiveness. However, IVR eliminates the need for physical resources and a dedicated space, allowing students to engage in realistic scenarios using VR headsets or other devices. This scalability enables a larger number of students to participate simultaneously, enhancing accessibility and reducing logistical constraints. With the use of IVR, facilitators can guide and debrief students, leveraging the recorded interactions and performance data to provide targeted feedback and facilitate reflective learning.

This study adopted a convenience sampling method; thus, it is difficult to generalize the results. The study involved a cross-sectional survey collected after VR-Hosp practice; thus, the cause and effect of this VR teaching strategy could not be determined. NTS in the real world are often influenced by external factors, such as high uncertainty and time pressure [55]. Considering that v-NOTECHS scores were self-reported, further work is needed to objectively evaluate the learning outcomes and assess whether these skills can be sustainably translated to realistic settings. When implementing VR-Hosp, students took turns in being players and observers. This might have interfered with the immersive experience since the observers communicated with the players during the activity, for example, in locating alarms and when answering MCQs. There might also have been bias when obtaining qualitative feedback, since the process was conducted during the debriefing session moderated by the teachers. However, the information that was obtained forms a basis for future studies to compare its impact with that of other educational pedagogies. In addition, the use of HMDs in IVR can lead to VR sickness, such as nausea, dizziness, and blurred vision [56]. Although our study did not report cases of VR sickness, its presence could negatively impact the immersive learning experience. Future research should include measures such as the Virtual Reality Sickness Questionnaire to evaluate its effect on the desired learning outcomes [57]. Additionally, future studies using IVR may consider striking a balance between realism and incorporating elements shown to reduce VR sickness, such as narrowing the horizontal field of view, partially limiting the degrees of freedom in navigation control, and increasing tactile feedback [58].

VR-Hosp appears to be a promising educational pedagogy for enhancing NTS, including situation awareness and decision-making ability, in nursing students. VR-Hosp portrays a nonlinear world that challenges students to operationalize what they have learned in traditional classroom teaching and simulation practices. The findings add evidence to the determinants of learning outcomes from the aspects of a sense of presence, satisfaction, and self-confidence in learning. This should motivate the undertaking of future work on VR-based teaching and learning activities in higher education.