1 Introduction

Virtual reality (VR), recognized for its immersive and interactive features, has considerable potential in education (Araiza-Alba et al., 2021; Cheng & Tsai, 2019; Ozyurt & Ozyurt, 2023; X. Wang et al., 2023). VR immerses users in environments with rich and detailed audiovisual effects (Thomann et al., 2024) and provides users with the opportunity to navigate virtual scenes at their own pace (Jiang et al., 2024). However, the development of VR content is a resource-intensive process with a high technological barrier to entry, making VR inaccessible to most educators (Chen & Liao, 2022a; Ranieri et al., 2022) and limiting its widespread adoption in education (Atal et al., 2023).

Conversely, spherical video-based VR (SVVR) offers a more cost-effective solution in education (Jong, 2023a). SVVR environments are constructed using panorama (360° photos and 360° videos), and enable deep immersion in real-world environments (Geng et al., 2021). This approach requires only a 360° camera to acquire media, substantially reducing the costs and technological barriers associated with constructing traditional VR environments. Teachers, leveraging this technology, can design immersive experiences tailored to their pedagogical objectives and teaching requirements.

Numerous studies have investigated the instructional effectiveness of SVVR (Chen & Hwang, 2022b; Yang et al., 2021). These studies have focused on SVVR activities created by teachers, where these activities feature a variety of educational resources such as pictures, narration, and learning tasks. These studies have highlighted the benefits of SVVR on student learning. SVVR is becoming increasingly accessible to educators as the technology improves. Nevertheless, the composition of SVVR content warrants close attention, particularly in terms of knowledge presentation and the assessment methods used in SVVR. Teachers are increasingly conscious of the necessity to carefully curate content in SVVR and evaluate its effectiveness in their teaching practice. Therefore, examining the learning content designed by teachers in SVVR is crucial. The current research on SVVR in education lacks comprehensive insight into two key areas: (i) the effect of various content types, embedded by teachers in SVVR, on the learning engagement of students (Schroeder et al., 2023), and (ii) the students’ perceptions of how these content types influence their engagement in the learning process (Tan et al., 2020).

Teachers have developed SVVR learning activities to enhance student learning engagement. The level of student learning engagement in learning is critical for both research and educational practices. Consequently, teaching effectiveness can be gauged by the quality of these teacher-developed SVVR activities and the resulting level of student engagement. In addition, there is no random assignment between students and teachers, and teacher-developed SVVRs are only implemented for students in their own classes, a relationship that characterizes nested data. This characteristic necessitates considering the nested data structure when evaluating the impact of individual teachers' SVVR implementations on student outcomes. This study sought to address the existing research gap by exploring how teacher-developed SVVR content and students' perceptions influence their engagement in learning through hierarchical linear modeling (HLM) analysis. The literature review described the teacher-student variables and reviewed the relevant literature. The literature review provides a foundation for the HLM model and research questions.

2 Literature review

2.1 Spherical Video-Based VR (SVVR)

SVVR is a type of VR that utilizes panorama videos or photos combined with interactive elements to offer learners a first-person view of real-world learning environments. SVVR has also gained popularity in educational settings (Lin et al., 2021). Its advantages include easy production methods and low production costs, facilitating the replication of real-world scenes for teaching and learning, thereby providing immersive learning experiences (Daltoè et al., 2024). For example, Chang et al. (2020) used SVVR to design a coastal geology scenario incorporating a two-stage diagnostic test strategy. This approach aided student learning in a course titled "Erosion and Deposition of Rocks." Their study demonstrated that SVVR improved student academic achievement. Similarly, W. L. Wu et al. (2021) used SVVR to provide an online learning experience of historical buildings in Rome. This method was effective in terms of enhancing students’ academic performance without adding to their cognitive load.

In addition to outdoor themes, SVVR is increasingly recognized for its potential across various curriculum areas. For instance, English learners often face challenges in practicing speaking skills. SVVR offers a convenient and immersive solution, effectively filling this gap. Integrating SVVR with peer assessment for English oral activities enables learners to identify areas for improvement in their speech and practice repeatedly, thereby reducing their anxiety (C. Y. Chen et al., 2023). Similarly, Chen and Hwang (2022b) demonstrated that SVVR can enhance students' performance in English oral presentations. The benefits of SVVR extend to writing skills. Students may lack the requisite experience or understanding of a subject, leading to underdeveloped essays. By immersing students in relevant scenarios through SVVR, teachers have found that students produce more content-rich and creative writing (H. L. Huang et al., 2020). Similar results were obtained by Shen et al. (2024) in essays on the topic of juvenile delinquency written by college students. Students who experienced prison and juvenile delinquency through SVVR gained a deeper understanding, resulting in more accurately describing juvenile delinquency penalties in their essays. These findings suggest that SVVR significantly enhances the sense of presence and immersion, enabling students to experience more diverse learning contexts than in the traditional classroom setting. Furthermore, SVVR's application extends beyond language learning to fields such as nursing education (Huang et al., 2022; C. C. Chang & Hwang, 2023) and programs for autistic learners (Schmidt et al., 2023), showcasing its versatility and effectiveness in terms of enhancing student learning outcomes.

From the above studies, it was found that SVVR can be effective in various learning areas. At the same time, the SVVR projects used in the above studies all have one characteristic in common: they were created by teachers based on the needs of their teaching rather than using commercial materials. Therefore, besides the effectiveness of SVVRs in teaching and learning, how teachers design these SVVRs is also an issue of concern.

2.2 Content design of SVVR

Well-designed information portfolios make it easier for learners to understand key content, thus enabling meaningful learning (Kalyuga & Sweller, 2014). Both cognitive load theory (Castro-Alonso et al., 2021) and reducing extraneous processing in multimedia learning (Mayer, 2014) suggest the importance of reducing students' cognitive load in multimedia environments, such as reducing the redundancy effect and following spatial contiguity. In addition, the use of elements in multimedia design is also a concern. The use of entirely irrelevant decorative images in the scene and images identical to the explanatory text does not resonate with learners (Peterson et al., 2021). Instructors should endeavor to avoid useless or repetitive information in multimedia as it occupies working memory (Oberauer et al., 2018). Effective visual signals can reduce the burden of identifying key messages and guide learners to interpret relevant information (Fiorella & Mayer, 2014).

Effective presentation and content sequencing are crucial components of teaching strategies (Reigeluth et al., 1994). Incorporating educational technology features to improve material presentation can significantly enhance student comprehension (Alavi & Leidner, 2001) and assist instructors in achieving their educational objectives (Kimmons et al., 2015). As pointed out by Makransky and Petersen (2021), the success of student learning hinges not merely on the features of Immersive Virtual Reality (IVR) but on how effectively a teacher can incorporate IVR's advantages into the design of instructional activities. In VR instructional design, blending technological advancements with pedagogical principles is pivotal, while educational VR designers must prioritize addressing students' learning needs (Oubibi & Hryshayeva, 2024). It helps students to acquire and organize knowledge and compare different concepts (Merchant et al., 2014; Stenberdt & Makransky, 2023). Similarly, SVVR content design should adhere to these educational principles. W. Li et al. (2023) emphasized the importance of clearly presenting conceptual knowledge within SVVR scenarios, which can significantly aid student learning. When students can grasp complex information within a VR setting, they are better equipped to apply this knowledge effectively in real-world contexts (Makransky et al., 2019).

Due to the high threshold of producing traditional VR, teachers often adopt commercialized VR for teaching (Castaneda et al., 2023). Although these commercial VR products are typically high-quality and well-designed, they may not fully address the specific needs of teachers. A VR development system possesses "design affordance" if it enables customization to meet user requirements (Zhou et al., 2023). SVVR offers a solution to this issue by allowing educators to design their teaching materials, suggesting that SVVR possesses design affordance. By creating their own SVVR content, teachers can tailor the virtual experiences to align more closely with their educational goals (Buchner & Hofmann, 2022). Teachers understand the intricate relationships between types of knowledge and can combine domain knowledge with appropriate pedagogical approaches. This expertise ensures that learners grasp the subject matter effectively during the learning process (Archambault & Barnett, 2010; Voogt et al., 2013). With the increasing establishment of SVVR, teachers now have greater opportunities to develop and implement their own SVVR in the classroom, allowing for more effective integration of instructional materials and learning assessments, and consequently, enhancing the prevalence of SVVR in education (Geng et al., 2021). However, research exploring how teachers can integrate SVVR features into different teaching activities remains notably scarce. Furthermore, in-depth discussions on the relationship between teacher-designed SVVR content and student learning outcomes are limited (Zou et al., 2023).

2.3 Perceived effects on active learning, repetition, and feedback of SVVR (AL, REP, and FB)

To evaluate the effects of instructional SVVR design on student learning, this study adapted the questionnaire which was proposed by Tautz et al. (2021) to evaluate the effectiveness of instruction in terms of the active learning, repetition, and feedback experienced by students in the SVVR learning process. For clarity, the terms “active learning,” “repetition,” and “feedback within the SVVR environment” are henceforth abbreviated as AL, REP, and FB, respectively.

AL, a prominently emphasized concept in education, involves students engaging actively in learning activities and reflecting on their learning process (Prince, 2004). The distinctive features and extensibility of SVVR enable a high level of presence and immersion, offering authentic learning experiences (W. C. V. Wu et al., 2023), and facilitating the shift from passive to active learning (Shadiev et al., 2020). Therefore, students in SVVR-enhanced environments actively observe contexts from an author's perspective in writing courses (H. L. Huang et al., 2020), critically engage with new knowledge (Chang et al., 2020), and organize dispersed information within the SVVR scene (C. Y. Chen et al., 2023).

REP is crucial in strengthening learning and is positively associated with improved academic performance (Zhang et al., 2022). It enhances memory, provides new interpretations of learning material, and increases comprehension (Dahlin & Watkins, 2000). Procedures that require adherence to standard operating procedures necessitate repetitive practice for familiarity, such as medical behaviors. For example, Barsom et al. (2020) implemented SVVR to train high school students in cardiopulmonary resuscitation skills. Students practiced repeatedly in the SVVR environment to become more familiar with the context of cardiopulmonary resuscitation use. It was found that students trained with SVVR performed better on a cardiopulmonary resuscitation concept test. More importantly, students performed better on the correct procedure. The repetition process is part of self-directed learning. Learners can practice unlimited times in the SVVR until they feel they have mastered the content or skill (Araiza-Alba et al., 2021). Maturation also reduces students' learning anxiety, especially their English-speaking anxiety. Chien et al. (2020) observed that SVVR created a private, repetitive space for students to practice spoken English, significantly reducing their anxiety levels. The level of anxiety about speaking English improved during the repetition process. In conclusion, SVVR offers immersive learning environments where learners can engage deeply without temporal, spatial, or financial constraints. This unrestricted access enables repeated use of SVVR for learning.

In the learning process, in addition to active learning and repetition, feedback plays a crucial role in enhancing learning effectiveness (Hattie & Timperley, 2007; Schneider & Preckel, 2017). Feedback informs students about their current learning status and the extent to which they have achieved the course’s learning objectives (C. Huang et al., 2023; Nicol & Macfarlane-Dick, 2006). Students use this feedback to work on their weaknesses, and this in turn improves their self-regulation skills. For example, students often perceive one of the difficulties in writing and speaking as their lack of ideas. Teachers should lead students to think and explore in language lessons to help them acquire rich contexts for meaningful writing and speaking. SVVR can stimulate students to reflect on and monitor what they need to improve in language lessons (Zou et al., 2023). In addition, researchers have explored the relationship between the content of feedback, the nature of tasks, and learning performance. For example, feedback that primarily focuses on explaining the task was found to have a limited effect on enhancing students' understanding. Conversely, feedback that guides students to identify critical aspects of the task and enhances their understanding of solutions significantly contributes to their comprehension of the learning content (Nelson & Schunn, 2009). Feedback offering detailed elaboration on content is more beneficial for declarative tasks. Conversely, accurate responses to student answers are crucial for procedural tasks (Merchant et al., 2014). These findings underscore the importance of meticulous design in teaching materials, encompassing activity planning to feedback provision.

According to the literature, SVVR benefits students' learning, including increasing their willingness to engage in active learning, motivating them to repeat and practice their skills, and further acknowledging the features of feedback in their learning. However, Zou et al., (2023) research highlights specific concerns, emphasizing the necessity for a balanced integration of SVVR. This approach should ensure that technology supports educational goals without dominating them. Accordingly, further research is required to refine SVVR-based material designs, focusing on sustaining student engagement, ensuring instructional effectiveness, and addressing identified issues.

2.4 Learning engagement

Learning engagement refers to the extent to which students are involved physically, consciously, or mentally in achieving desired learning outcomes (Hong et al., 2020; Mayordomo et al., 2022). Engagement is crucial because it clarifies the learning goals of students and increases their chances of achieving academic success (Lu et al., 2014). Measuring student engagement can provide important indicators of the effectiveness of programs, learning activities, and instructional resources. Educators can further develop effective student engagement strategies to enhance student learning and improve the quality of instruction (Henrie et al., 2015). P. H. Wu and Wu (2020) categorized student engagement into four types: behavioral, emotional, cognitive, and social engagement. These categories encompass various aspects of student participation, including participation in learning activities, emotional responses, physical interactions, and mental efforts.

Learning engagement may be partially attributed to instructional design. Instructional design involves systematically planning, developing, and delivering instructional materials to achieve efficient and effective knowledge acquisition processes (Dijkstra, 1997). Schmidt et al. (2018) explored the relationship between instructional activity design and student engagement. They discovered that students perceived experimental classes as offering "entertainment value" by allowing them to observe new scientific phenomena or to interact with peers. In these scenarios, students tended to view the learning activity as being less demanding, resulting in higher emotional engagement compared with cognitive engagement. Conversely, when students participated in quizzes, they exhibited limited emotional engagement. Nevertheless, they exhibited notable behavioral engagement, driven by the requirement of purposefully completing the tasks. Dubovi’s (2022) study indicated that combining declarative and procedural learning in a VR environment enhances student performance in procedural learning. A facial expression tracking system in the study captured the joyful facial expressions of students during learning, which was correlated with increased emotional engagement. Shi et al. (2024) in their research further confirmed that students were more motivated to pursue interactive and manipulative VR-supported project-based tasks, significantly leading to higher engagement in the learning process.

The level of integration of digital technology in education has increased substantially (Kang et al., 2022; Ong & Quek, 2023). Numerous studies have investigated the influence of technology on active learning, feedback, and repetition within the classroom environment (Schneider & Preckel, 2017; Tautz et al., 2021). Particularly in SVVR settings, SVVR learning environments can enhance learning interaction and engagement (Y. Wang et al., 2022; M. Li et al., 2022). Researchers have suggested that the integration of instructional design into immersive experiences affects the motivation and engagement of students, subsequently altering their attention and effort in learning (Fromm et al., 2021; Parong & Mayer, 2021). As Huang et al. (2022) found in the implementation of SVVR in blood transfusion safety courses, SVVR provides a more realistic representation of the blood transfusion process than traditional video-based methods. SVVR helps students to fully immerse themselves in the learning environment and connect with real-life situations to enhance classroom engagement.

2.5 Proposed model and research questions

While the above empirical studies have revealed the benefits of harnessing SVVR in learning and teaching, Table 1 summarizes these benefits brought and problems tackled by SVVR, from both student and teacher perspectives.

Table 1 Benefits brought and problems tackled by SVVR
Full size table

Nevertheless, the role of the teacher in these teaching and learning processes has rarely been explored. In particular, SVVR has a design affordance. Thus, this study aimed to explore student engagement patterns in teacher-developed SVVR learning environments. Understanding these patterns is essential for identifying potential obstacles to full engagement, enabling teachers to modify their instructional approaches to overcome these obstacles. This insight is invaluable for educators designing SVVR experiences, because it guides them to design and support specific learning activities.

Therefore, this study examined the effects of the type of SVVR developed by teachers and the perceived effect (AL, REP, and FB) of students on student engagement. A research model was proposed, as shown in Fig. 1. This proposed model integrates the research findings of Alavi and Leidner (2001), Lee et al. (2010), Makransky and Petersen (2021), Salzman et al. (1999), and Wan et al. (2007). Initially, Alavi and Leidner (2001) highlighted the significance of teacher-designed digital teaching activities, leading to the inclusion of teacher-developed SVVR types in this study's model. The potential for students' engagement in SVVR learning environments is hypothesized to be influenced by their perceived effectiveness of SVVR, drawing on insights from Lee et al. (2010), and Makransky and Petersen (2021). These studies collectively suggested that psychological factors could significantly predict learning outcomes. Furthermore, teacher-developed SVVR types can be used as a predictor of student engagement, as Salzman et al. (1999) suggested. Finally, building on Alavi and Leidner's (2001) assertion, the model considers teachers as moderators in the relationship between digital technology-enhanced teaching methods and student learning outcomes. In this framework, teacher-developed SVVR types are also envisioned to serve as moderators. The outcome variables include student learning engagement, which encompasses cognitive engagement (CE), behavioral engagement (BE), emotional engagement (EE), and social engagement (SE). This study sought to clarify the effects of teacher-developed SVVR type and the perceived effect of SVVR on these four aspects of learning engagement, analyzing each separately.

Fig. 1
figure 1

Proposed HLM model

Full size image

Due to the nested nature of teachers and students, this study attempted to incorporate this characteristic into the modeling. Therefore, we used HLM to analyze differences between and within teacher-developed SVVR types (Raudenbush et al., 1992). This approach offers a more precise estimation of the effects on outcome variables than conventional methods. HLM analysis can examine the direct effect of within- and between-level variables on the outcome variables. Moreover, HLM can be used to examine which between-level factors are involved in the between-group slope variation, that is, cross-level interaction effects (Aguinis et al., 2013). Consequently, this analytical approach supports our research objective, namely to explore the interplay of direct effects of variables at both the perceived effects (students-level) and teacher-developed SVVR types (teacher-level) on student engagement, and to explore how engagement changes when students with different perceived effects on SVVRs watch different types of teacher-developed SVVRs (cross-level interaction effects). The proposed hierarchical model is depicted in Fig. 1. The research questions are as follows:

  1. (1)

    How does teacher-developed SVVR type influence students’ cognitive, emotional, behavioral, and social engagement?

  2. (2)

    Do students’ AL, REP, and FB contribute to their cognitive, emotional, behavioral, and social engagement?

  3. (3)

    How does the cross-level interaction between teacher-developed SVVR type and students’ AL, REP, and FB affect their cognitive, emotional, behavioral, and social engagement?

3 Method

3.1 Participants

This study involved 33 elementary school teachers and their 841 students from various cities across Taiwan. These teachers, who expressed interest in incorporating SVVR into their teaching methods, had an average of 4.5 years of experience using information and communication technology in their teaching. They voluntarily participated in a training program which covered the use of SVVR development tools, SVVR methods for collecting SVVR content, and case studies on integrating SVVR into teaching as well as instructional design consultation. The total duration of the training activity was 8 h. The flow of the training activity included the use of SVVR development tools (3 h), the method of collecting SVVR content (2 h), the integration of SVVR case studies into the classroom (1.5 h), and the design consultation of teaching activities (1.5 h). Following their training, these teachers created their own SVVR activities for outdoor environmental inquiry courses and implemented them in their classrooms for evaluation.

The students, with an average age of 10.61 years (standard deviation [SD] = 0.90), spanning grades 3 to 6, engaged with the SVVR content created by their teachers during curriculum activities. They completed learning activities under the guidance of their teachers and subsequently were invited to anonymously complete a questionnaire after finishing the study. To adhere to ethical standards, teachers were given the option to withdraw from the training sessions or discontinue SVVR learning activities in their classrooms at any point. Similarly, student participation in the questionnaire was voluntary, and their responses were kept confidential.

3.2 Contexts

The platform used to create SVVR learning content in this study was EduVenture-VR (Jong, 2023b; Jong et al., 2020). This platform enables researchers and teachers to create and execute SVVR videos, as depicted in Fig. 2A. In EduVenture-VR, teachers can import 360° photos or videos tailored to their teaching requirements. They can then add interactive elements to these scenes based on their teaching scripts, including embedded pictures, audio, text (e.g., text supplements and subtitles), and other interactive elements (e.g., treasure hunts, voice answers, multiple choice questions, and portals). The platform provides six interactive features, illustrated in Fig. 2B, which teachers can utilize in their SVVR creations.

Fig. 2
figure 2

Design interface and selected elements window of EduVenture-VR®

Full size image

3.3 Coding the content embedded in teacher-developed SVVR

As teachers developed their SVVRs, all material was embedded according to their instructional needs. In order to explore the quality of the materials that teachers use within their SVVRs, this study coded the materials embedded by teachers. Also, Li et al. (2023) discovered a correlation between the integration of conceptual knowledge into SVVR design and improved student learning outcomes. Using Li et al.'s (2023) definition, the study regarded enhanced exposition content in SVVR as material that guides students in exploration, thought processes, and knowledge organization. For instance, as depicted in Fig. 3, a SVVR scene presents a natural ecological environment. The scene is based on a 360° photo prepared by the teacher. This environment includes pop-up SVVR elements detailing the optimal growing conditions for plants, notable species in the area, as well as their ecological relationships, allowing students to link this SVVR content with their prior knowledge during the learning process.

Fig. 3
figure 3

Example of enhanced exposition content

Full size image

In addition, the inclusion of appropriate assessment scaffolding in the SVVR learning process can positively affect student learning achievement (Anderson & Krathwohl, 2001). Well-designed assessment scaffolding guides students to generalize or interpret the connections between their knowledge and evaluate the accuracy or appropriateness of their knowledge. In this study, such assessment scaffolding is termed "thematic integration assessment." An example, displayed in Fig. 4, involves a thematic integration assessment within a SVVR scene of a natural national park. Students view the SVVR scenes about the three lakes, which helps them gain knowledge and develop a deeper understanding of these lakes. They then verbally respond to questions about their preferred lake and the reasons for their choice.

Fig. 4
figure 4

Example of thematic integration assessment scaffolding

Full size image

The study counted the quantity of enhanced exposition content and thematic integration assessments in the SVVR developed by the teachers. Two experts in SVVR content development coded the SVVR activities developed by the 33 teachers, using the definitions of enhanced exposition content and thematic integration assessment. Prior to coding, both experts confirmed their mutual understanding of these definitions and the SVVR content. They then independently coded the SVVR activities. After coding, the experts confirmed their inter-rater reliability, achieving a kappa value of 0.84, indicating consistent coding between the two experts.

3.4 Instrument

3.4.1 Student perceived effect of SVVR (student-level variable)

In this study, the questionnaire developed by Tautz et al. (2021) was adapted to measure the perceived influence of SVVR on students’ AL, REP, and FB. Each construct in the questionnaire consisted of six questions. We revised the questionnaire narratives with respect to the present research context (i.e., students using SVVR for learning in the classroom). As a measure of expert validation, the revised questionnaire was further scrutinized by a senior researcher of SVVR educational integration and a senior teacher with specific experience using SVVR for learning and teaching.

This study utilized a 5-point Likert scale, where students rated their responses from strongly disagree (1) to strongly agree (5). The Cronbach's α values for AL, REP, and FB were 0.92, 0.95, and 0.95, respectively. For example, the AL items were used to assess whether the students believed SVVR usage increased their active participation in learning, such as "By using the SVVR in this lecture, I was more active in the lecture." The REP items were used to assess whether SVVR helped the students memorize the lecture's content better, such as "Using the SVVR in this lecture helped me to better memorize the content of the lecture." The FB items were used to assess whether the students could monitor their learning progress and effectiveness, such as "By using the SVVR in this lecture, I was aware of my current level of learning.".

3.4.2 Learning engagement (outcome variable)

P. H. Wu and Wu (2020) developed a questionnaire to assess student learning engagement, which was adapted for this study. The questionnaire covers four dimensions: CE, BE, EE, and SE. The original questionnaire’s Cronbach's α values for these dimensions ranged from 0.80 to 0.85, with an overall α of 0.94, indicating good reliability. The questionnaire was modified to fit the context of SVVR usage in class. For instance, the CE items pertained to the use of learning strategies and cognitive processing, such as "I try to connect what I am learning to things I have learned before." The BE items pertained to attention, effort, and behavior in learning tasks, such as "I stay focused on watching SVVR content." The EE items pertained to student motivation and interest after using SVVR, such as "I look forward to continuing to use SVVR in the classroom." Finally, the SE items pertained to whether students could understand different perspectives, learn from peers, and help their peers during learning activities, such as "I try to understand other people's ideas in classes." Again, as a measure of expert validation, the questionnaire was further scrutinized by the aforementioned senior researcher and teacher to ensure its applicability in the present research context.

3.5 Data analysis

3.5.1 Categorizing the teacher-developed SVVR types

This study conducted a k-means cluster analysis to categorize teacher-developed SVVR types into heterogeneous clusters. It aimed to discern which teachers integrated more in-depth or comprehensive elements into their SVVR activities based on two key variables: enhanced exposition content and thematic integration assessment. The former refers to the quality and depth of the content teachers embedded into SVVRs for ensuring solid connections between the scene and other elements, while the latter evaluates whether the embedded questions in SVVRs target scaffolding students to have a holistic understanding or critical thinking about the scenario. These two elements are critical as they can significantly enhance students' engagement and understanding within the SVVR environment (Huang et al., 2024).

In fact, teachers have different ideas about integrating enhanced exposition content and thematic integration assessment into SVVRs, eventually impacting students' learning engagement in SVVR learning activities. In this study, we employed the Nbclust package on the R platform for k-means analysis to distinguish the teacher-developed SVVR types. In addition, the Ward method was used to identify the appropriate number of clusters, while Calinski and Harabasz (CH), Krzanowski and Lai (KL), SD, and the Silhouette index were included to determine the optimal number of clusters. The most effective k-means analysis result was obtained when the CH, KL, and Silhouette indices exhibited the highest values, and the SD index exhibited the lowest value (Charrad et al., 2014).

3.5.2 HLM analysis

Nested data is suitable for HLM analysis (Peugh, 2010). In this research, the SVVRs with which the students learned were developed by their teachers, and thus, the data collected from the teachers and the students were “nested.” a two-level HLM analysis to examine the effects of variables at different levels on student learning outcomes. In the proposed model, the Level 2 variable was teacher-developed SVVR type, whereas the Level 1 variables were students’ AL, REP, and FB, adjusted through group mean centering. The learning outcomes, which comprised CE, BE, EE, and SE, were evaluated after the use of SVVR in the learning activities. The HLM analysis was performed using the lme4 package of R, with maximum likelihood as the estimator. Following the recommendations of Raudenbush and Bryk (2002), we progressively modeled HLM to identify the most fitting model for this study’s variables. The HLM model used in this study included the null, mean-as-outcome, random coefficient, and the full model. Since this study has four outcome variables (CE, BE, EE, and SE), four separate two-level HLM analyses were conducted, each addressing one of the outcome variables. The model settings for null, mean-as-outcome, random coefficient, and full models are described as follows.

3.6 Null model

Initially, the null model was used to measure the variation in student engagement (Yij) across different teachers. This model comprised two variance components: teacher-level variance (U0j) and student-level variance (eij). The grand mean of student engagement across all participants was represented as the grand intercept (γ00). The null model was structured as follows:

  • \(\begin{array}{l}Level\;1:\;Y_{ij}=\beta_{0j}+e_{ij}\\Level\;2:\;\beta_{0j}=\gamma_{00}+U_{oj}\\Mixed:\;Y_{ij}=\gamma_{00}+U_{0j}+e_{ij}\end{array}\)

3.7 Mean-as-outcome model

The second stage, where the mean-as-outcome model was analyzed, centered on determining the ability of type of teacher-developed SVVR to predict student engagement (Yij). In this model, the SVVR type variable (γ01) was added to Level 2, along with the teacher-level variance (U0j) and student-level variance (eij). The grand mean of all student engagement was used as the grand intercept (γ00). The equation of the mean-as-outcome model is as follows:

  • \(\begin{array}{l}\begin{array}{cc}{Level }\,1:&{{Y}}_{{ij}}={\beta }_{0{j}}+{{e}}_{{ij}}\end{array}\\\begin{array}{ccc}{Level }\,2:&{\beta }_{0{j}}={\gamma }_{00}+{\gamma }_{01}{ SVVR }{{type}}_{{j}}+{{U}}_{0{j}}\end{array}\\\begin{array}{cc}{Mixed}:&{{Y}}_{{ij}}={\gamma }_{00}+{\gamma }_{01}{ SVVR }{{type}}_{{j}}+{{U}}_{0{j}}+{{e}}_{{ij}}\end{array}\end{array}\) 

3.8 Random coefficient model

The third stage involved a random coefficient model that was used to examine how AL, REP, FB at the student level (represented by γ10, γ20, and γ30, respectively) predicted their level of engagement (Yij). This model yielded an estimate of the grand mean of all student engagement as a grand intercept (γ00) and included variance components for student AL, REP, and FB (represented as U1j, U2j, and U3j, respectively), teacher-level variance (U0j), and student-level variance (eij). The equations for the random coefficient model are as follows:

$$\begin{array}{l}\begin{array}{cc}Level\;1:&Y_{ij}=\beta_{0j}+\beta_{1j}AL_{ij}+\beta_{2j}REP_{ij}+\beta_{3j}FB_{ij}+e_{ij}\end{array}\\\begin{aligned}Level\;2:& \ \ \ \ \beta_{0j}=\gamma_{00}+U_{0j}\\& \ \ \ \ \beta_{1j}=\gamma_{10}+U_{1j}\\& \ \ \ \ \beta_{2j}=\gamma_{20}+U_{2j}\\& \ \ \ \ \beta_{3j}=\gamma_{30}+U_{3j}\end{aligned}\\\begin{array}{cc}Mixed:&\begin{array}{l}Y_{ij}=\gamma_{00}+\gamma_{10}AL_{ij}+\gamma_{20}REP_{ij}+\gamma_{30}FB_{ij}+U_{0j}+U_{1j}AL_{ij}+U_{2j}REP_{ij}+U_{3j}FB_{ij}+e_{ij}\end{array}\end{array}\end{array}$$

3.9 Full model

The final stage involved a full model that incorporated all variables. This model yielded the prediction of student engagement (Yij) by SVVR type (γ01); the effects on engagement from AL, REP, and FB (represented by γ10, γ20, and γ30, respectively), and the effect of interactions of SVVR type with each of AL, REP and FB (represented by γ11, γ21, and γ31, respectively) on engagement. The grand mean of all student engagement is the grand intercept (γ00). The variance components include AL, REP, and FB (represented by U1j, U2j, and U3j, respectively), teacher-level variance (U0j), and student-level variance (eij). The equations for the full model are as follows:

$$\begin{array}{l}\begin{array}{cc}Level\;1:&Y_{ij}=\beta_{0j}+\beta_{1j}AL_{ij}+\beta_{2j}REP_{ij}+\beta_{3j}FB_{ij}+e_{ij}\end{array}\\\begin{aligned}Level\;2:& \ \ \ \ \beta_{0j}=\gamma_{00}+\gamma_{01}SVVR\;type_j+U_{0j}\\& \ \ \ \ \beta_{1j}=\gamma_{10}+\gamma_{11}SVVR\;type_j+U_{1j}\\& \ \ \ \ \beta_{2j}=\gamma_{20}+\gamma_{21}SVVR\;type_j+U_{2j}\\& \ \ \ \ \beta_{3j}=\gamma_{30}+\gamma_{31}SVVR\;type_j+U_{3j}\end{aligned}\\\begin{array}{cc}Mixed:&\begin{array}{l}Y_{ij}=\gamma_{00}+\gamma_{01}SVVR\;type_j+\gamma_{10}AL_{ij}+\gamma_{20}REP_{ij}+\gamma_{30}FB_{ij}+\gamma_{11}\left(AL_{ij}\ast SVVR\;type_j\right)+\\\gamma_{21}\left(REP_{ij}\ast SVVR\;type_j\right)+\gamma_{31}\left(FB_{ij}\ast SVVR\;type_j\right)+U_{0j}+U_{1j}AL_{ij}+U_{2j}REP_{ij}+U_{3j}FB_{ij}+ \\e_{ij}\end{array}\end{array}\end{array}$$

4 Results

4.1 Teacher-developed SVVR types

A k-means analysis was conducted to categorize teacher-developed SVVR types based on the quantity of enhanced exposition content and the number of thematic integration assessments. Results from this analysis, where the Ward method was used, indicated that the teachers should be grouped into two, rather than three or four, clusters for the most effective results. Specifically, this two-cluster categorization had the highest KL, highest Silhouette values, and the lowest SD index. Although the CH index was highest at 76.57 for the three-cluster categorization, the value for the two-cluster categorization was a little lower at 75.88. These results indicated that dividing teachers into two clusters is the optimal clustering outcome (Charrad et al., 2014). The results from this analysis are presented in Table 2.

Table 2 Summary k-means clustering index
Full size table

Figure 5 illustrates the categorization of teacher-developed SVVR types into two clusters. The red cluster, characterized by a higher average number of enhanced exposition contents and thematic integration assessments, is named the “enhanced type” (N = 18). Conversely, the blue cluster, notable for a lower average of these components, is named the “fundamental type” (N = 15).

Fig. 5
figure 5

k-means analyses of two-cluster classification

Full size image

4.2 Descriptive statistics of student-level and outcome variables

Descriptive statistics and Cronbach's α values for student-level variables and outcome variables are presented in Table 3. The means (SDs) for the three student-level variables, AL, REP, and FB were 4.25 (0.88), 4.27 (0.87), and 4.23 (0.88), respectively. The Cronbach's α values for these variables were 0.94, 0.95, and 0.93, respectively. For the outcome variables, the means (SDs) of CE, BE, EE, and SE were 4.24 (0.74), 4.27 (0.77), 4.38 (0.80), and 4.16 (0.88), respectively. The Cronbach’s α values for these variables ranged from 0.91 to 0.96. The Cronbach's α values indicated that the measurement of the student-level and outcome variables was highly reliable.

Table 3 Results of descriptive statistics and Cronbach's α values
Full size table

4.3 The results of HLM

This study used HLM to analyze the effects of teacher-developed SVVR type, the influence of SVVR as perceived by students, and the engagement of students in learning following their use of SVVR. Four HLM models were analyzed: The null model, the mean-as-outcomes model, the random coefficient model, and the full model. The results of these four HLM models are summarized in Appendix Table 5.

Initially, the null model was chosen to estimate the intraclass correlation coefficient (ICC = teacher-level variance / [teacher-level variance + student-level variance]). ICC was used to calculate the ratio of variance between teachers to total variance. This study subsequently calculated the ICC for 33 teachers and 841 elementary school students. The teacher-level variances (U0j) for the CE, BE, EE, and SE were 0.067, 0.078, 0.099, and 0.087, respectively. The student-level variances (eij) were 0.47, 0.50, 0.53, and 0.68, respectively. Therefore, the ICC of CE, BE, EE, and SE were 0.12, 0.13, 0.16, and 0.11, respectively. Because these ICC values exceeded the conventional threshold of 0.059 (Cohen, 1988), we considered the intergroup variation and found that HLM was suitable for analyzing CE, BE, EE, and SE. In addition, this study adopted the − 2 log likelihood (− 2LL) as an indicator of model fit, where smaller values indicated better fit. In the null model, the values of − 2LL for CE, BE, EE, and SE were 1750.2, 1808.9, 1867.3, and 2050.5, respectively. Finally, the intercepts (γ00) for CE, BE, EE, and SE in the null model were 4.26, 4.28, 4.39, and 4.16, respectively (all p < 0.001).

Second, to address research questions 1 and 2, the full model was used to determine whether students’ AL, REP, and FB predicted their engagement and to ascertain the predictive effects of teacher-developed SVVR types on students’ CE, BE, EE, and SE. Table 4 presents the results of the full model. Regarding the fixed effects, the intercepts (γ00) for CE, BE, EE, and SE were 4.11, 4.13, 4.22, and 4.02, respectively (all p < 0.001). The SVVR type (γ01) had positive effects on the four dimensions of engagement (CE, BE, EE, and SE) of 0.27 (p < 0.01), 0.28 (p < 0.01), 0.32 (p < 0.01), and 0.26 (p < 0.05), respectively. For the student-level variables, the direct effects of AL (γ10) on CE, BE, EE, and SE were 0.20 (p < 0.05), 0.36 (p < 0.01), 0.40 (p < 0.001) and 0.44 (p < 0.001), respectively. REP (γ20) indicated a positive prediction (0.21, p < 0.05) only for EE. The results for FB (γ30) indicated positive predictions on CE, BE, EE, and SE of 0.35 (p < 0.001), 0.27 (p < 0.05), 0.17 (p < 0.05), and 0.22 (p < 0.05), respectively. Finally, only one cross-level interaction, AL*SVVR type, γ11, had a negative prediction on EE (− 0.24, p < 0.05). In the random effects, the teacher-level variances (U0j) and the student-level variances (eij) for the four dimensions of learning engagement ranged from 0.066 to 0.097 and from 0.17 to 0.24, respectively.

Table 4 Results from the full model
Full size table

Finally, the − 2LL values for CE, BE, EE, and SE were 1231.2, 1121.9, 1029.5, and 1293.1, respectively. A comparison of the − 2LL value across all HLM models revealed that the full model had the smallest − 2LL value. Thus, incorporating the teacher-developed SVVR types and the perceived influence of SVVR by students provided a robust model fit for explaining the engagement of students.

4.4 Cross-level interaction

This study investigated the effect of cross-level interaction between SVVR type and the influence of SVVR as perceived by students on their learning engagement. Specifically, the interaction term (AL*SVVR type) was significantly and negatively associated with EE (γ11 = − 0.24, p < 0.05). As illustrated in Fig. 6, both groups had high EE when using enhanced-type SVVR. Notably, in the group with lower levels of AL, the effect of SVVR type on EE was more pronounced than in the group with higher levels of AL. When enhanced-type SVVR activities were introduced in teaching, a strong positive effect on EE was observed, particularly in the group with lower AL. This finding suggests that enhanced-type SVVR activities are particularly effective for improving EE in students who exhibit lower levels of AL.

Fig. 6
figure 6

Cross-level interaction of AL and EE

Full size image

5 Discussion

This study investigated the influence on student learning engagement of type of SVVR developed by elementary school teachers and of AL, REP, and FB in SVVR-based learning environments. The results indicated that enhanced-type SVVR had a positive effect on the engagement of students. REP positively influenced students’ EE. Additionally, the cross-level interaction of AL and SVVR type (AL*SVVR type) negatively affected their EE.

5.1 Effect of types of SVVR on engagement

The study discovered that enhanced-type SVVR positively affects the learning engagement of students. Enhanced-type SVVR is characterized by a higher quantity of enhanced exposition content and thematic integration assessments. Conversely, fundamental-type SVVR incorporates fewer of these elements. Ponce et al. (2018) indicated that the use of multimedia instructional materials, which help students visualize concepts and organize their knowledge, improves student comprehension and retention. In addition, engaging different cognitive systems (e.g., visual and auditory) in processing content can increase the efficiency of message processing (Alazmi & Alemtairy, 2024; Sweller, 2022) and can increase student engagement (Kalyuga, 2023). Dirkx et al. (2014) noted that students tend to retain high-order learning content more effectively. Learning materials that address key terms or points constitute deep content and are representative of the field (Tauber et al., 2018). The provision of deep content supports meaningful learning, even for learners previously unfamiliar with the content (Kostiainen et al., 2018) and promotes better learning engagement (Zepke, 2013). Therefore, providing key terms or a precise focus in SVVR environments can enhance student attention, guide them toward learning the content critically, and improve their engagement.

Beyond the provision of quality content, the use of assessment as a learning scaffold is also effective (Anderson & Krathwohl, 2001). In addition to enhancing knowledge retention (C. H. Lai et al., 2021), assessment immerses students more deeply in the learning environment by enabling quick recall and reorganization of recently viewed content (Fletcher, 2016). However, in a SVVR environment, if an assessment solely focuses on knowledge recitation, students might perceive it as merely another form of quiz, which could potentially distract them from the immersive and engaging aspects of SVVR. Martinez-Garza et al. (2013) emphasized that the efficacy of learning is contingent upon the meticulous design of multimedia content. In fact, Yang et al.'s (2024) study reached a similar conclusion that the inclusion of learning scaffolds in the form of visual cues can help learners organize content and improve their presentation skills. Consequently, learning content in SVVR should encompass deeper content, such as key terms, well-designed assessments, and precise focus. Merely duplicating course materials into a SVVR environment without proper consideration may lead to reduced learning efficiency.

5.2 Effect of REP on EE

The study's analysis of student-level variables revealed that AL and FB were associated with all four dimensions of engagement. This finding underscores the importance of SVVR in helping students engage with feedback and actively reflect on their learning. For example, using multiple scenarios and embedding learning content into SVVR supports student exploration and helps them form connections between knowledge and the learning context. Additionally, adding diverse assessment modules with feedback or labeling prompts in SVVR enhances the student capability to refine knowledge, thus promoting active engagement in the exploration process.

Furthermore, this study corroborates previous findings that REP is associated only with student EE. Specifically, REP, involving the repetitive learning of a particular content set, has been recognized as a factor contributing to boredom, especially in language education (Kruk & Zawodniak, 2020; Pawlak et al., 2020). Students often experience boredom and lack of focus when they perceive learning activities as being linear and repetitive. Interestingly, when learning tasks deviate from traditional repetition patterns, student engagement, including their motivation and interest, tends to be activated (Kormos & Préfontaine, 2017). Moreover, providing students with more opportunities for active exploration can stimulate and sustain positive emotions (Mastria et al., 2017). The present study demonstrates that repetition in a SVVR environment can yield effects similar to those in conventional teaching settings.

5.3 Effect of cross-level interaction (AL*SVVR type) on EE

The cross-level interaction analysis revealed that enhanced-type SVVR more effectively promotes student EE compared with fundamental-type SVVR. Regardless of their level of AL, students exhibited high EE when using enhanced-type SVVR. This type of SVVR plays a critical role in supporting student EE. Notably, the difference in EE was more significant among students with low levels of AL than those with high levels when interacting with different SVVR types. This indicates that students with low AL experienced a greater increase in EE when exposed to enhanced-type SVVR, as opposed to their counterparts with high AL levels. These results highlight the importance of content design (Radianti et al., 2020). The enhanced-type SVVR offers more than basic exploratory contexts and descriptions; it incorporates extensive supportive information and assessments into the SVVR environment. This approach is designed to aid students in developing their analytical and reasoning skills. For example, by incorporating complementary and specific elements in the SVVR scene, students can gather information from different perspectives and autonomously assemble a coherent understanding. Importantly, with the freedom to browse in SVVR, students can access various types of information depending on what they are interested in. This design method helps prevent cognitive overload, which might result from an excess of information presented in a single element (Albus et al., 2021) and aligns with principles for designing presence pedagogy in VR environments as proposed by Doumanis et al. (2019). These principles, such as distributed cognition, allow for the interaction and connection of information within SVVR, which is a crucial factor for achieving EE. Furthermore, the well-designed assessments in this study guided students to organize, connect, and explain their prior knowledge and the knowledge they acquired in the virtual environment. Open-ended questions provided a meaningful link between subject knowledge and student experiences.

This result also indicates that students with low AL, when engaging with SVVR content that complies with pedagogical design principles, can display EE levels similar to those of high AL students. This demonstrates the importance of pedagogical design principles in SVVR development. Zhao and Yang (2023) and Y. C. Chang et al. (2021) demonstrated that providing adequate learning tasks and guidance in VR influences student interest and self-efficacy. The current study has confirmed that these factors have a considerable effect on students with lower achievement levels. In addition, educational technology studies have found benefits of robust pedagogical design for low-achieving students. For example, Liu et al. (2010) found that computerized concept maps positively affected the reading comprehension of students who struggled with English as a foreign language. In addition to aiding students with high levels of AL in maintaining their performance, a well-designed knowledge construction process significantly improves the learning of students with low-level AL (Venn et al., 2023). The cross-level interaction results of this study not only highlight the importance of considering pedagogical design principles in SVVR development, but also verify the importance of providing students with knowledge connections and distributed cognition in SVVR environments.

6 Conclusion

In the growing literature on the implementation of SVVR in educational settings (Ranieri et al., 2022; Smutny, 2022), many studies have focused on comparing the effectiveness of SVVR with traditional instructional media (C. Chen & Yuan, 2023; W. Huang et al., 2021; W. C. V. Wu et al., 2023). In this study, we explored the impact of varying SVVR content on student engagement, specifically examining the effectiveness of teacher-developed SVVR types. We discovered that the enhanced-type SVVR significantly increased student engagement, underscoring the importance of meticulous organization of SVVR content. This finding builds upon and expands the theoretical frameworks established by Alavi and Leidner (2001), Lee et al. (2010), Makransky and Petersen (2021), Salzman et al. (1999), and Wan et al. (2007), among others. While these earlier studies highlighted the influence of virtual reality features on student learning, our research further demonstrates that how SVVRs deliver information and facilitate students' knowledge organization also plays a crucial role in enhancing learning outcomes.

Although these studies have demonstrated the potential of SVVR for education, a gap remains in relation to how different types of approaches to SVVR content designed by teachers influence learning outcomes. Teachers offering students appropriate learning tasks and opportunities for active exploration opportunities is crucial when designing SVVR activities. Such an approach encourages students to make knowledge connections and enhances their engagement in SVVR learning environments. When developing SVVR activities, teachers or content designers should focus on educational goals, student needs, and expectations to improve student perceptions and engagement when using SVVR. This study highlighted the importance of teacher-developed SVVR type in education, which is an understudied topic. The diversity in ideas among teachers is noteworthy. Future research should include a wider variety of SVVR activities developed by teachers. For example, the incorporation of more elements of teacher-developed SVVR activities to categorize SVVR types will yield findings that are more useful for the formulation of SVVR-based pedagogy. Additionally, future research could recruit teachers to participate in SVVR teaching activities for specific learning topics to understand more accurately how SVVR-developed types affect student learning.

In conclusion, the results of HLM in this study indicated that the types of SVVR developed by teachers and the interaction between these SVVR types and student AL significantly influence student learning engagement. This study provides critical insights for future content developers regarding factors to consider when designing SVVR activities and learning activities. It contributes to the body of knowledge on teacher-developed SVVR types and their capacity to enhance student learning outcomes in formal classroom environments.