Retroalimentacion digital

Retroalimentacion digital

• 1.
  Contents lists availableat ScienceDirect Studies in Educational Evaluation journal homepage: www.elsevier.com/locate/stueduc How digital reflection and feedback environments contribute to pre-service teachers’ beliefs during a teaching practicum Christopher Neil Prilop ⁎ , Kira Elena Weber, Marc Kleinknecht Leuphana University of Lüneburg, Institute of Educational Science, Department of Teacher Education and School Development, Universitätsallee 1, 21335 Lüneburg, Germany A R T I C L E I N F O Keywords: Teaching practicum Digital environment Reflection Feedback Beliefs Self-efficacy A B S T R A C T This study investigated the impact of digital reflection and feedback environments on pre-service teachers’ beliefs about teaching and learning (constructivist/traditional) and self-efficacy before and after a teaching practicum. While pre-service teachers in one condition only self-reflected and received feedback in face-to-face sessions (n = 65), in the other conditions they also participated in text- (n = 19) or video-based (n = 22) digital reflection and feedback environments. Test results showed traditional beliefs increased in the text-based en- vironment. Constructivist beliefs decreased in non-video conditions in comparison to the video-based digital environment. Self-efficacy was fostered in all conditions. Content analysis showed that pre-service teachers’ self- reflections and feedbacks were more positive in the video-based condition and self-reflections displayed a higher level of knowledge-based reasoning. Implications for future research are discussed. 1. Introduction Educational beliefs, such as teacher self-efficacy or constructivist and traditional beliefs about teaching and learning, have significant impact on the instruction teachers deliver (Pajares, 1992). Traditional beliefs result in transmission- and control-based teaching, whereas constructivist beliefs lead to classroom practices that place emphasis on students constructing their own knowledge (Stipek, Givvin, Salmon, & MacGyvers, 2001). Teacher self-efficacy can be defined as the perceived capability to successfully teach students even if they are challenging or unmotivated (Moulding, Stewart, & Dunmeyr, 2014). High teacher self- efficacy (Zee & Koomen, 2016) and constructivist teacher beliefs, con- trary to traditional beliefs (Kunter et al., 2013), have been positively associated with high student achievement. Educational research in- dicates that reflection can be considered a key component for changing teachers’ constructivist and traditional beliefs about teaching and learning (Kagan, 1992) or fostering teachers’ self-efficacy (Bandura, 1997). Feedback is a necessary stimulant in the reflection process. It leads to deeper reflection and provides a change of perspective (Hammerness et al., 2005). Reflection and feedback sessions can be implemented in teaching practicums during pre-service teacher education. However, reflection and feedback sessions are not a standard component of teaching prac- ticums due to lack of resources, especially if experts are supposed to be present (Lee & Wu, 2006). Digital reflection and feedback environments can provide a remedy as they are time- and location-independent (Wu & Kao, 2008) and have been shown to improve (pre-service) teacher’ professional knowledge concerning cognitive and meta-cognitive com- ponents (e.g., Kleinknecht & Gröschner, 2016; Rosaen, Lundeberg, Cooper, Fritzen, & Terpstra, 2008; Santagata, Zannoni, & Stigler, 2007). In such digital environments pre-service teachers reflect and are pro- vided with feedback on lessons they taught by uploading a textual description or a video sequence of their classroom practice: They can then interact online with peers, mentors or experts concerning their teaching skills (Bonk, Malikowski, Angeli, & East, 1998; So, Pow, & Hung, 2009). Prior studies that did not apply digital reflection and feedback environments during teaching practicums were able to show that text- or video-based interventions can foster (pre-service) teachers’ self-efficacy and constructivist beliefs (e.g., Gold, Hellermann, & Holodynski, 2017; Gröschner, Schindler, Holzberger, Alles, & Seidel, 2018; Heemsoth & Kleickmann, 2018; Philipp et al., 2007). So far, affective components of (pre-service) teachers’ professional knowledge, such as self-efficacy and constructivist beliefs, have not been investigated in studies applying digital reflection and feedback environments. Therefore, the present study extends previous research on the im- pact of reflection and feedback on pre-service teachers’ self-efficacy and beliefs about teaching and learning. We investigated how participating https://doi.org/10.1016/j.stueduc.2019.06.005 Received 25 February 2019; Received in revised form 20 June 2019; Accepted 22 June 2019 ⁎ Corresponding author at: Leuphana University of Lüneburg, Institute of Educational Science, Department of Teacher Education and School Development, Universitätsallee 1, C1.207, 21335 Lüneburg, Germany. E-mail addresses: [email protected] (C.N. Prilop), [email protected] (K.E. Weber), [email protected] (M. Kleinknecht). Studies in Educational Evaluation 62 (2019) 158–170 0191-491X/ © 2019 Elsevier Ltd. All rights reserved. T
• 2.
  in one ofthree different reflection and feedback environments (face-to- face, text-based, video-based) contributed to pre-service teachers’ self- efficacy and beliefs about teaching and learning in our teaching prac- ticum. With this approach the present study expands the insights on changes of beliefs about teaching and learning and self-efficacy within an ecologically valid setting. 1.1. Teachers’ educational beliefs Teachers’ professional knowledge forms the basis for mastery of teaching situations (Kunter et al., 2013). It is based on a variety of sources, such as pedagogical knowledge, content knowledge, or peda- gogical content knowledge, but also on affective-motivational compo- nents like beliefs, values, and motivation, or meta-cognitive compo- nents such as self-regulation (Blömeke, 2014; Shulman, 1987). Of these components beliefs seem to play a crucial role (Fenstermacher, 1986) and have the potential to be “the single most important construct” (Pajares, 1992, p. 329) in teacher education. In general, beliefs can be summarized as an individual’s “convic- tions, philosophy, tenants, or opinions” (Haney, Lumpe, & Czerniak, 2003, p. 367). Therefore, the influence they have in an educational setting is extremely strong because they are responsible for teachers’ decision-making and actions (Bandura, 1986). Educational beliefs can be divided into various sub-constructs, for example, regarding the self- efficacy of teachers or teachers’ traditional and constructivist beliefs about teaching and learning (Calderhead, 1996; Woolfolk Hoy & Burke- Spero, 2005). 1.1.1. Changing teachers’ beliefs about teaching and learning Much research has concentrated on teachers’ beliefs about teaching and learning (e.g., Fennema et al., 1996; Lui & Bonner, 2016). In general, teachers’ beliefs about teaching and learning can be classified into traditional and constructivist beliefs, resulting in traditional and constructivist classroom practices (Barkatsas & Malone, 2005). Tradi- tional beliefs follow transmissive/behaviorist theories of learning (Voss, Kleickmann, Kunter, & Hachfeld, 2013), while constructivist beliefs emanated from Piaget’s theory of cognitive constructivism (Staub & Stern, 2002). Teachers with traditional beliefs believe that children learn by “receiving clear, comprehensible and correct information about … procedures and by having the opportunity to consolidate, automatize, and generalize the information they have received by practicing the demonstrated procedures” (Goldsmith & Schifter, 1997, pp. 22-23). Basically, students reach the right answer by following step- by-step rules and procedures (Stipek et al., 2001). Consequently, tra- ditional instruction is characterized by classroom practices such as teacher-centered direct instruction, following textbooks or summative assessment (Bonner & Chen, 2009; Stipek et al., 2001). Teachers who hold constructivist beliefs believe in instruction that emphasizes self- regulated construction of knowledge by the students (Stipek et al., 2001). Focus is not put on control but rather on mentoring and guiding the students during their learning process (Stipek et al., 2001). As a result, constructivist instruction features work in groups or formative assessment (Bonner & Chen, 2009). Constructivist beliefs of teachers have been shown to positively af- fect student achievements, contrary to teachers who hold traditional beliefs. Peterson, Carpenter, and Fennema (1989) analyzed the impact of elementary school teachers’ beliefs on student outcomes. They were able to show that students that attended classes of teachers who held constructivist beliefs outperformed students of teachers with traditional beliefs concerning problem-solving tasks. Staub and Stern (2002) ex- tended on this study by investigating primary student learning gains regarding mathematical word problems in a longitudinal study. Stu- dents of teachers with constructivist beliefs displayed higher achieve- ments in mathematical word problems than students of teachers who held traditional beliefs. Staub and Stern attributed this finding to teachers with constructivist beliefs selecting more structure-oriented tasks. Fennema et al. (1996) analyzed the effects of a professional de- velopment course focusing on constructivist teaching practices on pri- mary school teachers’ beliefs and instruction. Teachers were taught to engage their students in problem-solving activities and encourage stu- dents to discuss their mathematical thinking. Fennema et al. (1996, p. 430) found that “gains in students’ concepts and problem-solving per- formance appeared to be directly related to changes in teachers’ in- struction”. Kunter et al. (2013) investigated the impact of teachers’ professional knowledge on instructional quality and student develop- ment in a large scale study with secondary mathematics classes. They found that constructivist beliefs positively predicted students’ achievements. Research indicates that teachers have recently been shifting from traditional to constructivist beliefs. Handal (2003) reviewed a large body of research on teachers’ beliefs and determined that pre-service teachers mostly shared traditional beliefs about teaching and learning whereas in-service teachers displayed a slight inclination to con- structivist ideas. However, recently, Lui and Bonner (2016) were able to establish that pre- and in-service teachers lean towards constructivist views. This is supported by the OECD’s Teaching and Learning Inter- national Survey (OECD, 2009) concerning in-service teachers. Kagan (1992) identified three prerequisites effecting change in pre- service teachers’ beliefs by reviewing 40 studies in this field. First, pre- service teachers must have the opportunity to interact and study stu- dents in classroom settings. Second, university courses need to focus on what pre-service teachers actually require in the classroom. This in- cludes theory but more importantly procedural knowledge and strate- gies. Third, during teaching practicums pre-service teachers must be able to work with in-service teachers and require opportunities for self- reflection. These findings were elaborated on by assessing pre-service primary school teachers’ beliefs about mathematics during a field experience (Ambrose, 2004). Ambrose came to the conclusion that a change in beliefs can be stimulated by four mechanisms: (a) they can have emotion-packed, vivid experiences that leave an impression; (b) they can become immersed in a community such that they become enculturated into new beliefs through cultural transmissions; (c) they can reflect on their beliefs so that hidden beliefs become overt; (d) they can have experiences or reflections that help them to connect beliefs to one another and, thus, to de- velop more elaborate attitudes. (p. 95) However, he also points out that reflection alone might not be en- ough to form new beliefs, but that “[v]ivid experiences [may need] to be coupled with reflection” (p. 96). 1.2. Fostering pre-service teachers’ self-efficacy Like traditional and constructivist beliefs about teaching and learning, teacher self-efficacy is a well-researched field (Zee & Koomen, 2016). Teacher self-efficacy can be defined as the belief in one’s ability to successfully teach students and reach desired outcomes (Tschannen- Moran & McMaster, 2009). A substantial body of research suggests that self-efficacy has comprehensive implications for teaching quality and, thus, student learning (Bandura, 1997; Zee & Koomen, 2016). Self-ef- ficacious pre-service teachers are more open-minded concerning teaching approaches and focus on meeting their students’ needs (Woolfolk Hoy & Burke-Spero, 2005). These findings are supported by a large scale study by Vieluf, Kunter, and Vijver (2013). They observed positive correlations between teacher self-efficacy and job satisfaction and effective classroom teaching practices such as structuring, student orientation and enhanced activities. In general, self-efficacious teachers display higher commitment to the teaching profession (Chesnut & Burley, 2015). Consequently, if teachers lack self-efficacy, this can have a negative effect on their students’ self-efficacy, engagement, C.N. Prilop, et al. Studies in Educational Evaluation 62 (2019) 158–170 159
• 3.
  motivation, and achievement(Zee & Koomen, 2016). Apart from ne- gative effects on student achievement, the worst case is that a lack of self-efficacy can result in teacher burnout (Schwarzer & Hallum, 2008). According to Bandura (1994) self-efficacy can be promoted by four sources. First, pre-service teachers need to experience success, however they must also learn to overcome obstacles and persevere (mastery experience). Second, observing peers reaching their goals is beneficial because student teachers perceive efforts as more manageable and at- tainable (vicarious experiences). Third, pre-service teachers need to be assured that they possess the ability to achieve certain aims to succeed in given activities (social persuasion). Fourth, handling emotional and physical reactions to demanding tasks needs to be learned (physiolo- gical cues). As a result, sources of self-efficacy form “a self-reinforcing cycle of either success or failure” (Tschannen-Moran & McMaster, 2009, p. 229). Teaching practicums influence pre-service teachers’ self-efficacy substantially (Klassen & Durksen, 2014). Though there is a general consensus that student teaching increases pre-service teachers’ self-ef- ficacy (e.g., Colson, Sparks, Berridge, Frimming, & Willis, 2017; Hoy & Woolfolk, 1990; Wenner, 2001), some studies also indicate detrimental effects (e.g., Lin & Gorrell, 2001; Pendergast, Garvis, & Keogh, 2011). This seems to depend on the sources of self-efficacy. Pre-service tea- chers can experience mastery experience while teaching for the first time or make vicarious experiences by observing peers teaching. Fur- thermore, reflection and feedback sessions can provide social persua- sion by mentors or peers and can also help achieve mastery experience (Bandura, 1994). Woolfolk Hoy and Burke-Spero (2005) analyzed stu- dent teachers’ and beginning teachers’ self-efficacy during practical experiences. They were able to establish that student teachers became more self-efficacious, whereas beginning teachers’ self-efficacy declined in their first year of teaching. Student teachers’ and beginning teachers’ self-efficacy was significantly correlated with the support (verbal per- suasion) they received. Consequently, teaching practicums should not follow sink-or-swim approaches but be carefully scaffolded for pre- service teachers. Tschannen-Moran, Woolfolk Hoy, and Hoy (1998) emphasize that support is required during first teaching experiences to reduce complexity (verbal persuasion). They suggest an “apprenticeship approach” (p. 236) which is characterized by the student teacher being able to concentrate on and develop one set of teaching skills at a time (mastery experience). Furthermore, verbal persuasion in the form of feedback is needed to highlight positive achievements. The importance of coaching is also focused on by Woolfolk Hoy (2005). Woolfolk Hoy’s study showed that pre-service teachers’ self-efficacy correlated posi- tively with the amount of support mentor teachers offered (verbal persuasion). Furthermore, Tschannen-Moran and McMaster (2009) in- vestigated the effect of coaching after a professional development course on primary teachers’ self-efficacy. In combination with a coaching session the self-efficacy effects were the strongest (mastery experience/ verbal persuasion). Self-efficacy actually decreased for a higher proportion of participants of courses without follow-up coaching. 1.3. Reflection and feedback As educational research shows, reflection and feedback during teaching practicums can be considered a potential source to increase pre-service teachers’ self-efficacy (Woolfolk Hoy, 2005) and change constructivist and traditional beliefs about teaching and learning (Ambrose, 2004). Reflection and feedback sessions entail pre-service teachers inviting peers, mentors or university teachers to observe a lesson and receive feedback immediately afterwards (Lu, 2010). Re- flection and feedback sessions “stimulate reflection” (Hammerness et al., 2005, p. 380) and lead to “a self-critical, investigative process wherein teachers consider the effect of their pedagogical decisions on their situated practice with the aim of improving those practices” (Tripp & Rich, 2012b, p. 678). Specifically, this reflection process can be considered a combination of internal and external feedback loops (Narciss, 2013). Teachers assess their own classroom practice by com- paring it to standards or reference levels (internal feedback). Assess- ments by peers or experts (external feedback) add to the internal feed- back. External feedback consists of conveying a criteria-based assessment of the teaching performance to the teacher and identifying possible strengths and weaknesses (Sluijsmans, Brand-Gruwel, van Merriënboer, & Martens, 2004). A comparison of the external and in- ternal feedback leads to decision-making and action (Narciss, 2013). Consequently, reflection and feedback sessions in teacher education can be considered a combination of self- and peer/ expert-assessment (Sluijsmans, Dochy, & Moerkerke, 1999). External feedback can be provided by learners of equal-status (peer feedback) or experts with clear ‘knowledge authority’ (expert feedback) (Gielen, Peeters, Dochy, Onghena, & Stuyven, 2010, p. 305). The lack of knowledge authority can lead to active negotiations of meaning and construction of knowledge concerning peer feedback, while expert feedback may lead to passiveness and misinterpretation of the in- formation (Yang, Badger, & Yu, 2006). Moreover, the equal-status re- lationship of peers generates trust and thus promotes deeper self-re- flection, as shortcomings are more openly discussed (Topping, 2005). Yet, feedback provided by experts has been shown to be of higher quality than peer feedback (Prins, Sluijsmans, & Kirschner, 2006). As a result, research indicates stronger improvements in performance from expert feedback (Yang et al., 2006). However, as both peer and expert feedback can help pre-service teachers to discover and reflect on aspects that would otherwise go unnoticed (Wu & Kao, 2008), pre-service teachers should be provided with as much feedback as possible during their teaching practicum (Lee & Wu, 2006). 1.4. Digital video-based reflection and feedback Reflection and feedback sessions are increasingly being integrated into teacher education (Joyce & Showers, 2002; Kraft, Blazar, & Hogan, 2018). However, reflection and feedback sessions are not a standard component of teaching practicums or limited in extent (Valencia, Martin, Place, & Grossman, 2009). This is largely due to constraints concerning time and location (Lee & Wu, 2006). Therefore, recent studies applied digital platforms for reflection and feedback sessions, and analyzed their effects (e.g., Gregory et al., 2017; Kleinknecht & Gröschner, 2016; So et al., 2009; Wu & Kao, 2008). In general, digital reflection and feedback environments enable students to “participate […] anytime and anywhere” (Wu & Kao, 2008, p. 45). As pre-service teachers usually do their teaching practicums at different schools this increases the opportunities for them to interact (So et al., 2009). Hence, by participating in digital environments pre-service teachers encounter a higher amount of effective and ineffective teaching practices which creates “a knowledge base for teaching” (So et al., 2009, p. 783). Furthermore, the fact that participants can take part remotely and at different times, enables experts to contribute to the reflection and feedback process (Lee & Wu, 2006). Studies were able to show that digital video-based reflection and feedback environments during teaching practicums significantly con- tribute to (pre-service) teachers’ professional knowledge and, conse- quently, student performance. For example, research established im- provements of (pre-service) teachers ability to analyze instruction (Santagata et al., 2007), depth of reflection (Kleinknecht & Gröschner, 2016), noticing (Rosaen et al., 2008), professional vision (Weber, Prilop, Glimm, & Kleinknecht, 2018a) and revealed that teachers par- ticipating in digital video-based reflection and feedback produced higher student achievements (Gregory et al., 2017). However, to date, studies have not analyzed the impact of digital video-based reflection and feedback during practicums on affective components of (pre-ser- vice) teachers’ professional knowledge such as beliefs. C.N. Prilop, et al. Studies in Educational Evaluation 62 (2019) 158–170 160
• 4.
  1.5. Video andtext as tools in teacher education Digital reflection and feedback environments can be enhanced by text (e.g., Bonk et al., 1998) or video cases (e.g., So et al., 2009) which can be beneficial on several levels. Text and video representations of classroom practice make it possible to read or watch situations re- peatedly and, thus, to concentrate on specific aspects. Although both are less complex than live teaching observations, they reduce com- plexity to a different degree (Yadav et al., 2011). Text cases are char- acterized by putting only necessary information into focus, excluding seemingly unimportant details (Syring et al., 2015). Events are narrated in sequential order (Clark & Paivio, 1991). On the contrary, video clips capture the multitude of simultaneous processes taking place in the classroom setting (Borko, Whitcomb, & Liston, 2009). Teaching se- quences can be revisited with different foci, making an in-depth ana- lysis possible (Sherin, 2007). Video as a means of stimulated recall (Powell, 2005) has been shown to entail high emotional involvement and deeper engagement (Seidel, Stürmer, Blomberg, Kobarg, & Schwindt, 2011). When teachers analyze their own teaching videos they activate contextualized or tacit knowledge (Borko, Jacobs, Eiteljorg, & Pittman, 2008; Powell, 2005; Seidel et al., 2011). However, applying video in teacher education re- quires carefully designed learning environments (for a discussion of existing frameworks, see Kang & van Es, 2018). Classroom videos of other teachers provide less contextual information than live observa- tions (Sherin, 2007) and therefore need to be contextualized for lear- ners to grasp the classroom’s culture, atmosphere and environment (Körkkö, Morales Rios, & Kyrö-Ämmäla, 2019). Furthermore, specific observation targets need to be determined to avoid attentional biases (Derry, Sherin, & Sherin, 2014). At the same time this can counteract cognitive overload (Derry et al., 2014; Syring et al., 2015). Moreover, various studies have shown that the video type (own classroom video vs. other teacher’s classroom video) entails different effects. Though own video results in higher activation (Seidel et al., 2011), teachers analysing other video are more critical and develop more consequences and alternatives than teachers working on their own material (Kleinknecht & Schneider, 2013). Hellermann, Gold, and Holodynski (2015) revealed that combining both types (own and other video) re- sulted in higher learning gains concerning professional vision than just own video. Regarding pre-service teachers, Santagata and Guarino (2011) established that analysing videos by peers increases motivation as pre-service teachers find their classroom practice more achievable. Only a few studies compared the use of video or text systematically. Choi and Yang (2011) revealed that video fostered students’ empathy more than text-based situations. Students developed their emotional awareness more by being able to sympathize with individuals depicted in videos. Furthermore, Syring et al. (2015) investigated the effects of video and text cases on pre-service teachers’ cognitive load and moti- vational-emotional processes. Working with video cases caused a higher cognitive load for pre-service teachers in comparison to text-based cases. However, analyzing video cases lead to more joy and higher immersion than their counterpart. 1.6. Video and text as means of reflection and feedback Textual reflection and feedback have been incorporated into pre- service teacher education by applying (online) learning journals (Apel, 2003; Gläser-Zikuda, 2007). Learning journals are used by student teachers to record relevant teaching situations to reflect on situations according to criteria, and make their reflections the subject matter of feedback. Kleinknecht and Gröschner (2016) conducted an intervention concerning pre-service teachers’ reflection and noticing of teaching events over the period of a teaching practicum. They found that journal writing and video-based environments lead to differential effects. In the journal writing condition pre-service teachers were able to explain their evaluations of classroom practice. On the other hand, participants of the video-based intervention group focused on more positive classroom events in reflections of their own teaching than the text-based group, and feedback provided in the video-based environment presented more teaching alternatives to recipients. Concerning teachers’ beliefs about teaching and learning, Heemsoth and Kleickmann (2018) did not make use of learning journals but written accounts of teaching situations. They investigated whether being presented with good, problematic, or balanced (good and problematic) examples of teaching changed pre- service teachers’ beliefs. Results showed that constructivist beliefs were only fostered in the balanced condition, and that comparing good and problematic teaching events also led to a stronger decrease of tradi- tional beliefs in comparison to only one type. Regarding pre-service teachers’ self-efficacy Gold et al. (2017) were able to show that ana- lyzing written descriptions of other teachers’ classroom practice sig- nificantly fostered pre-service teachers’ self-efficacy during a seminar, however, not more productively than video-based analyses. Tripp and Rich (2012a) specifically investigated the effects of video reflection and feedback on in-service teacher change. Teachers in their study reported that after video reflection they “were more likely to believe another teacher’s recommendations once they witnessed the said event on video” (p. 737). Furthermore, they trusted feedback more, as they were able to see the evidence “with their own eyes” so that “they could no longer deny or pretend that their teaching did not need improvement” (p. 738). At the same time they could also monitor their own progress on video. Moreover, the video reflections created “vivid images” (p. 738) which they remembered in later teaching and that led to change of practices. Tripp and Rich also assessed that a dissonance between events on video and descriptions in the self-reflection elicited questions from participants and, thus, a deeper discussion. Video re- flection was also applied to foster in-service teachers’ self-efficacy by Gröschner et al. (2018). Participants reflected on their own lessons concerning classroom discourse. Contrary to Gröschner and colleagues’ hypothesis, teachers’ self-efficacy did not increase significantly in the video group in comparison to a non-video group. However, the sample size was very small (video group: n = 6; non-video group: n = 4) and the self-efficacy pre-posttest displayed a positive tendency for the video condition. 1.7. Aims of the present study We analyzed what effect a conventional face-to-face reflection and feedback setting (face-to-face condition), a text-based digital environ- ment (T-Reflect condition) or a video-based digital format (V-Reflect condition) had on pre-service teachers’ beliefs about teaching and learning and self-efficacy during a teaching practicum. In a nutshell, our research questions are as follows: (1) To what extent do face-to-face, text-based and video-based reflec- tion and feedback environments (face-to-face, T-Reflect, V-Reflect) impact pre-service teachers’ constructivist and traditional beliefs about teaching and learning during a teaching practicum and do they differ in effect? (2) To what extent do face-to-face, text-based and video-based reflec- tion and feedback environments (face-to-face, T-Reflect, V-Reflect) have an effect on pre-service teachers’ self-efficacy during a teaching practicum and how do they differ in impact? (3) To what extent do pre-service teachers’ self-reflections and feed- backs differ in the text-based and video-based reflection and feed- back environments (T-Reflect, V-Reflect)? We assume that the video-based environment (V-Reflect) has a more positive impact on pre-service teachers’ beliefs about teaching and learning than the text-based setting (T-Reflect) and the face-to-face set- ting because video offers more vivid experiences (Ambrose, 2004) and likely leads to more balanced reflections and feedback (Heemsoth & Kleickmann, 2018; Kleinknecht & Gröschner, 2016). Furthermore, we C.N. Prilop, et al. Studies in Educational Evaluation 62 (2019) 158–170 161
• 5.
  assume the face-to-facecondition to have more impact on pre-service teachers’ beliefs about teaching and learning than the T-Reflect condi- tion because reflections and feedback are not only based on individually chosen instances of classroom reality (Kleinknecht & Schneider, 2013). Moreover, we presume that pre-service teachers in the video-based condition (V-Reflect) show more improvement than the face-to-face and text-based group (T-Reflect) concerning self-efficacy because video provides sources of self-efficacy at a higher degree (Gröschner et al., 2018; Lee & Wu, 2006). However, we assume the text-based environ- ment (T-Reflect) will profit from the personally chosen focus of self- reflection in comparison to the face-to-face condition (Gold et al., 2017; Kleinknecht & Gröschner, 2016). Regarding self-reflections and feedbacks in the text- and video- based reflection and feedback environments we expect pre-service teachers’ self-reflections to be more in-depth and positive in the V- Reflect condition than in the T-Reflect group (Choi & Yang, 2011; Seidel et al., 2011; Sherin, 2007). 2. Method 2.1. Design We applied a quasi-experimental pre-post-design (see Fig. 1). The intervention was conducted in a teaching practicum lasting four weeks. Pre-service teachers’ beliefs about teaching and learning (traditional, constructivist) as well as their self-efficacy were ascertained by ques- tionnaires. Questionnaires were administered before and after the teaching practicum. 2.2. Participants In total, 109 Bachelor student teachers in their fourth semester at a German university completed the teaching practicum.1 A limited number of participants had to be excluded from the sample as they did not complete all pre-posttests. The final sample consisted of 106 par- ticipants (93.7% female, Mage = 23.11, SDage = 4.43). From the final sample 65 student teachers (92.3% female, Mage = 23.35, SDage = 4.61) were assigned to the face-to-face-condition, 19 student teachers to the T-Reflect-condition (84% female, Mage = 22.58, SDage = 1.71) and 22 student teachers volunteered for the V-Reflect- condition (96% female, Mage = 22.64, SDage = 3.97). Student teachers could not be assigned to the video-based condition (V-Reflect) randomly because all video-recordings are subject to strict private data policies in Germany. Therefore, student teachers had to agree to being videotaped. As only a limited number of schools and parents agreed to video-recordings, we planned with a larger conven- tional face-to-face group. Primarily we wanted to keep the digital en- vironment groups (T-Reflect, V-Reflect) equal in size. However, as we had to place students in teams at schools, the V-Reflect-condition re- sulted in a slightly higher number of participants than the T-Reflect- condition. Participants in the three conditions did not differ from each other in terms of age, F(2,103) = 0.13, p = .88, previous teaching experience, F (2,78) = 1.33, p = .27 and self-assessed knowledge of classroom management, F(2,103) = 1.07, p = .34. Before the teaching practicum we also assessed student teachers’ feedback quality as it could have an impact on self-efficacy and beliefs. We evaluated the feedback quality by applying a video-based tool (Prilop, Weber, & Kleinknecht, 2018) and coding written feedback with an adapted version of Prins et al.’s (2006) Feedback Quality Index. No significant differences between conditions were found for participants’ feedback quality, F (2,103) = 1.72, p = .19. As self-reflection and feedback could also be influenced by what pre-service teachers are able to perceive, we mea- sured their professional vision of classroom management with a stan- dardized test (Weber et al., 2018a) developed by Gold and Holodynski (2017). Pre-service teachers’ professional vision of classroom manage- ment did not differ between the condition (face-to-face, T-Reflect, V- Reflect), F(2,103) = 0.32, p = .72. After the teaching practicum we also assessed how many informal feedback sessions the participants took part in with mentors or peers. Participants of the face-to-face-, T-Reflect- and V-Reflect-environments did not show any differences, F (2,103) = 0.14, p = .87. 2.3. Teacher education in Germany In Germany2 students of teacher education have to complete a Ba- chelor and Master degree before entering a 1.5 year practical teaching period. Only after completing this practical phase they are fully quali- fied teachers. During their Bachelor degree students have to complete two school practicums. The first is a purely observational practicum of three weeks in the second semester. In the fourth semester they have to participate in a four-week practicum that requires them to teach four lessons on their own. This is the practicum our study focused on. In preparation for the teaching practicum all students had to be enrolled in a lecture on didactics and methods and an accompanying seminar (Weber, Gold, Prilop, & Kleinknecht, 2018b). Both courses dealt with different theoretical concepts and teaching methods. Tradi- tional and constructivist teaching settings were focused on in two ses- sions of the seminar. 2.4. Intervention During the teaching practicum each student had a team partner at the same school and a tandem partner at a different school. Students were supposed to observe their team partners teaching and visit their tandem partners once. Students in all conditions were visited by uni- versity teachers for feedback and reflection sessions. University tea- chers observed a lesson taught by a student and subsequently provided feedback after self-reflection by the student. Team and tandem partners also joined the feedback and reflection sessions and offered feedback. Participants of the face-to-face-condition were visited twice while stu- dents of the T-Reflect- or V-Reflect-condition were visited once. Apart from these feedback and reflection occasions students of the T-Reflect- and V-Reflect-condition also had to take part in two online feedback and reflection sessions (see Fig. 1). 2.4.1. Procedure: digital reflection and feedback sessions Reflection and feedback sessions on the digital platforms followed a specific scaffolded sequence. Pre-service teachers reflected on instances of teaching which they perceived as critical to successful classroom management. In the T-Reflect-condition pre-service teachers described these situations, while the V-Reflect-group uploaded a five to ten minute video-sequence of their choice. We chose classroom management be- cause it is considered a prerequisite of successful teaching, yet hard to accomplish for pre-service teachers (Wolff, van den Bogert, Jarodzka, & Boshuizen, 2015). Consequently, classroom management was also dealt with in the preparation for the practicum, both in the lecture and seminar. This course of action was chosen because sufficient domain knowledge can be considered essential for meaningful feedback (Van Zundert, Sluijsmans, Könings, & Van Merriënboer, 2012). More im- portantly, concentrating on and developing one set of teaching skills at 1 The sample used in this study was also subject to analyses in other fields (Weber et al., 2018a; Prilop et al., 2018). 2 Though the general structure of teacher education is the same throughout Germany, there are variations as to when and for how long students have to take part in practicums. For a more extensive discussion of teacher education in Germany please see Arnold, Gröschner, and Hascher (2014) and Cortina and Thames (2013). C.N. Prilop, et al. Studies in Educational Evaluation 62 (2019) 158–170 162
• 6.
  a time, inthe sense of an “apprenticeship approach”, has proven ben- eficial (Tschannen-Moran et al., 1998, p. 236). We assume that pre-service teachers require highly structured re- flection and feedback settings (Kleinknecht & Gröschner, 2016). Hence, pre-service teachers completed structured reflection and feedback cy- cles (see Fig. 2). First, students composed a self-reflection (internal feedback). They were asked to follow a three-step approach consisting of a description of their classroom management, an evaluation of it and possible alter- native teaching approaches (Kleinknecht & Gröschner, 2016). Second, students received two instances of peer feedback concerning their self- reflection (external feedback). In order to increase feedback quality, students were reminded of general rules (i.e. Base your feedback on specific classroom situations. Suggest alternatives. Ask thought-provoking questions.). Third, peer comments were followed by a feedback by a university teacher (external feedback). Fourth, the reflection and feedback cycle was completed by a second self-reflection of the pre- service teacher which took feedbacks into account, the so-called feed- back balance (consequences from internal feedback + external feed- back). In the face-to-face sessions a lesson observed directly before formed the basis of the reflection and feedback cycle. 2.4.2. Reflection and feedback environments The video-based reflection and feedback environment required pre- service teachers to upload short videos of teaching to our platform. We used moodle enhanced by the plug-in vShare (Huppertz, Massler, & Plötzner, 2005). Team partners had to film each other twice with cameras provided by the university. Then pre-service teachers chose a 5–10 min sequence for the V-Reflect-platform and uploaded it. This was the starting point for the reflection and feedback cycle. The interface (see Fig. 3) of the T-Reflect-environment looked the same as the V-Re- flect-platform but did not offer the possibility of embedding a video. 2.5. Instruments and data collection To assess pre-service teachers’ development of beliefs about Fig. 1. Timetable of the Quasi-Experimental Study. Fig. 2. Structured reflection and feedback cycle (modified according to Kleinknecht & Gröschner, 2016). C.N. Prilop, et al. Studies in Educational Evaluation 62 (2019) 158–170 163
• 7.
  teaching and learning(traditional, constructivist) as well as self-effi- cacy, student teachers completed three scales – one on self-efficacy and two on beliefs (see Fig. 1). All pre-service teachers received a perso- nalized link to the questionnaires on the online survey platform unipark (Questback, 2017). In order to prevent methodological artefacts, pre- service teachers were asked to rate the statements presented to them without knowing what they measured (Please rate the following state- ments.) Furthermore, presentation of statements was randomized. To analyze the content of the self-reflections and feedbacks on the video- and text-based reflection and feedback environments, pre-service teachers’ self-reflections and feedbacks of one structured reflection and feedback cycle were collected from the online platform. 2.5.1. Pre-service teachers’ beliefs about teaching and learning Participants’ beliefs about teaching and learning were evaluated with two scales which were developed by Fennema, Carpenter, and Loef (1990) and translated for German teacher education research by Staub and Stern (2002). They validated the translation by having it re-trans- lated into English by a native speaker of English with good command of German. We used the abbreviated version of Staub and Stern’s (2002) scale that was applied in a large scale assessment of teachers’ profes- sional knowledge at German and Swiss schools (Rakoczy, Buff, & Lipowsky, 2005). As both scales were created for mathematics teachers, we altered them for a more general context. The first scale evaluated teachers’ constructivist beliefs, i.e., that students should try and solve problems alone before solutions are presented by teachers. Six items (e.g., “Students learn best when they can discover ways to the solution themselves.”) were ranked on a four-point Likert scale (1 – strongly disagree/4 – strongly agree). Internal consistency in our study was sa- tisfactory (α = .66/.82). The second scale assessed participants’ tradi- tional beliefs, i.e., that students need to be taught specific procedures before they can start solving problems themselves. It comprised ten items that were also ranked from strongly disagree (1) to strongly agree (4). Example items are, “Students learn best from presentations and ex- planations of their teachers” (α = .74/.72). Consequently, high scores on the first scale indicate the belief that learners should construct their own knowledge whereas high scores on the second scale suggest the participant believes in students as receivers of knowledge. 2.5.2. Pre-service teachers’ self-efficacy Pre-service teachers’ self-efficacy was measured by an established scale developed for the German context by Schwarzer and Schmitz (1999). It consists of ten items assessing teachers’ self-efficacy in terms of professional performance and development, interaction with stu- dents, parents and colleagues or dealing with stress. Items, such as “I am confident that I will be able to take care of individual problems of students even better in the future”, or “Even if there are disturbances in my class, I am confident to be able to stay calm” had to be judged on a four-point Likert scale ranging from 1 = strongly disagree to 4 = strongly agree. A re- liability analysis of the pre- and posttest results indicated a satisfactory internal consistency (α = .94/.70). 2.5.3. Content analysis of self-reflections and feedbacks Self-reflections and feedbacks were analyzed according to guide- lines for quantifying qualitative data from Chi (1997). Self-reflections and feedbacks were segmented into paragraphs. Paragraphs (thematic units) were used as units of analysis as they offer sufficient granularity to answer our research question (Chi, 1997; Seidel et al., 2011). Three coding systems were applied to analyze the units of analysis (see Table 1). To analyze the quality of reflection and feedback we applied a coding system originally developed by Schwindt (2008) that has been used to establish the depth of knowledge-based reasoning processes by various studies (e.g., Kleinknecht & Gröschner, 2016; Seidel et al., 2011). We coded whether pre-service teachers did not notice Fig. 3. Interface of the video-based reflection and feedback environment. Table 1 Content analysis of pre-service teachers’ self-reflections and feedbacks: Coding system, units of analysis, inter-coder reliability (percentage and Cohen’s Kappa). Units of analysis Total (V-Reflect/T- Reflect) % κ Self-reflection Knowledge-based reasoning 228 (149/79) 0.87 .83 Direction of evaluation 228 (149/79) 0.94 .92 Feedback Knowledge-based reasoning 143 (99/44) 0.91 .88 Direction of evaluation 143 (99/44) 0.99 1 C.N. Prilop, et al. Studies in Educational Evaluation 62 (2019) 158–170 164
• 8.
  (code = 0),only noticed (1), noticed and evaluated (2), noticed, eval- uated and reflected on consequences (3), or noticed, evaluated, re- flected on consequences and developed alternatives when perceiving positive or critical teaching events. Code 4 is regarded as the highest “reflective form of articulation” (Seidel et al., 2011) according to this coding system. Furthermore, the direction of evaluation was coded based on a coding system by Kleinknecht and Gröschner (2016). We coded whether teaching events were not evaluated (0), negatively evaluated, or positively evaluated (2). All units of analysis were independently coded by two coders (from a team of three coders) to establish reliability. Inter-coder reliability reached international standards for all categories (see Table 1). 2.6. Methods of analysis We analyzed pretest differences between conditions (face-to-face, T- Reflect, V-Reflect) with ANOVAs. Furthermore, we ran paired t-tests to evaluate pre-posttest differences of the individual groups in terms of beliefs about teaching and learning (traditional, constructivist). Furthermore, a factorial 3 × 2 ANOVA with one between-participant factor (condition: face-to-face, T-Reflect, V-Reflect) and one within-par- ticipants factor (time: pre-test, posttest) was performed to analyze which condition changed beliefs about teaching and learning more. We specifically looked at the interaction of condition × time to assess the effect of conditions. A significant interaction would indicate an increase of traditional or constructivist beliefs within the three conditions. For closer inspection 2 × 2 analyses of variance were conducted between individual groups. Moreover, we conducted paired t-tests to assess differences between the pre- and posttest within the individual groups (face-to-face, T-Reflect, V-Reflect) with respect to self-efficacy. A 3 × 2 analysis of variance was performed to analyze differences between all groups. We set condition as between-participant factor with three levels (face-to-face, T-Reflect, V-Reflect) and time as within-participant factor with two levels (pre-test, posttest). Again, we were looking for sig- nificant interactions (condition × time) that would suggest different gains of self-efficacy within the three conditions. Furthermore, 2 × 2 ANOVAs were conducted for a more direct comparison of individual groups. Moreover, independent t-tests were performed for self-reflec- tions and feedbacks to analyze differences between pre-service teachers’ knowledge-based reasoning on the highest level in T-Reflect and V- Reflect. To analyze whether pre-service teachers’ produced more posi- tive or negative self-reflections or feedbacks, we compared their nega- tive and positive evaluations with dependent t-tests for each group se- parately. Finally, we investigated whether there were significant differences between the groups concerning the direction of evaluation of pre-service teachers’ self-reflections and feedbacks a 2 × 2 MANOVA was conducted. Condition was set as between-participant factor (T- Reflect, V-Reflect) and direction of feedback (positive evaluation, ne- gative evaluation) as within-participant factor. We performed a statistical power analysis for sample size estimation of ANOVAs using GPower 3.1.9.2 (Faul, Erdfelder, Buchner, & Lang, 2009). Effect size was set at medium, 0.5, using Cohen’s (1988) criteria. Estimated sample size needed to be N = 15 with an alpha of .05 and power of .80. Thus, our sample size can be considered adequate. For all statistical analyses the alpha value was set at p < .05. Unequal sample sizes were automatically adjusted by SPSS (Rusticus & Lovato, 2014). Although normal distribution of data was slightly violated in the face- to-face group, we applied parametric analyses as ANOVA is robust against this kind of violation (Schmider, Ziegler, Danay, Beyer, & Bühner, 2010). Sphericity was given in the ANOVAs. 3. Results 3.1. Check of baseline equivalence As differences between the conditions (face-to-face, T-Reflect, V- Reflect) before the intervention could diminish the estimated sig- nificance of the intervention baseline equivalence was checked. We found that participants in all conditions held a high degree of constructivist beliefs ranging between 3.37 and 3.55 at pre-test (see Table 2). The three groups did not differ in constructivist beliefs about teaching and learning at pre-test, F(2,103) = 2.07, p = .13. Hence, participants’ constructivist beliefs can be seen comparable. Overall, pre- test results of pre-service teachers’ traditional beliefs indicated low le- vels for all conditions ranging from 2.12 and 2.23 at pre-test. Again, no significant differences could be found for all of the groups, F (2,103) = 0.69, p = .50. Furthermore, participants in all conditions were comparable concerning pre-service teachers’ self-efficacy, F (2,103) = 1.107, p = .33. 3.2. Effects on beliefs about teaching and learning Whereas pre-service teachers in the face-to-face condition and the video-based group stagnated concerning their constructivist beliefs, pre-service teachers’ in the text-enhanced learning environment dis- played a negative tendency, however, non-significant (p = .051). Regarding traditional beliefs pre-service teachers remained at a low level in the face-to-face- and V-Reflect-condition. The T-Reflect-inter- vention group yielded significant increases of traditional beliefs with a medium effect size, d = 0.60, p < .01. The ANOVA of constructivist beliefs of all conditions (face-to-face vs. T-Reflect vs. V-Reflect) revealed significant main effects for time and interaction time × condition (see Table 3). The significant interaction effect indicates diverging pre-posttest differences between the face-to- face-, T-Reflect- and V-Reflect-group. 19% of the overall variation in performance can be attributed to the impact of the condition, ηp 2 = 0.19. This effect was more closely examined in pairwise ANOVAs which showed that 29% of variance can be explained by differences between the V-Reflect- and T-Reflect-group, ηp 2 = 0.29, and 21% of variation can be attributed to differences between the V-Reflect- and face-to-face-condition, ηp 2 = 0.21. No significant main effects of time or interaction were established for the ANOVAs of traditional beliefs be- tween all groups as well as pairwise comparisons. 3.3. Effects on self-efficacy We found that all groups increased their self-efficacy significantly Table 2 Means (M), standard deviations (SD) and effect size for repeated measures (d) for beliefs about teaching and learning (constructivist/ traditional) and self- efficacy. Pre-test Post-test Δ t df p d M SD M SD Constructivist beliefs (max. 4 points) Face-to-face 3.55 0.33 3.46 0.42 −0.09 −1.84 64 .07 0.24 T-Reflect 3.37 0.43 3.18 0.58 −0.19 −2.09 18 .051 0.38 V-Reflect 3.55 0.36 3.61 0.40 0.07 1.03 21 .32 0.16 Traditional beliefs (max. 4 points) Face-to-face 2.23 0.34 2.20 0.39 −0.03 −0.62 64 .54 0.08 T-Reflect 2.12 0.39 2.34 0.36 0.22 2.88 18 < .01 0.60 V-Reflect 2.17 0.39 2.17 0.35 0.005 0.07 21 .95 0.00 Self-efficacy (max. 4 points) Face-to-face 3.11 0.25 3.27 0.27 0.16 5.18 64 < .001 0.62 T-Reflect 3.04 0.25 3.16 0.23 0.12 2.24 18 < .05 0.51 V-Reflect 3.13 0.35 3.29 0.30 0.15 2.24 21 < .05 0.50 Note: nFace-to-face = 65, nT-Reflect = 19, nV-Reflect = 22, significant effects printed bold. C.N. Prilop, et al. Studies in Educational Evaluation 62 (2019) 158–170 165
• 9.
  over the courseof the practicum (see Table 2). The face-to-face-group yielded a medium effect size, d = 0.62, p < .001, as did the T-Reflect-, d = 0.51, p < .05, and V-Reflect-condition, d = 0.50, p < .05. These results indicate that all practicum formats fostered pre-service teachers self-efficacy. The ANOVAs we conducted confirmed these results by revealing significant main effects for time (see Table 3). As there were no inter- action effects our findings suggest that self-efficacy does not increase due to a specific learning environment but the practical placement it- self. 3.4. Effects on self-reflection and feedback We found that pre-service teachers in the V-Reflect condition pro- duced significantly more self-reflections with knowledge-based rea- soning at the highest level (description, evaluation, consequences, and alternatives) than participants of the T-Reflect condition, t(55) = 2.80, p < .007, d = 1.06 (see Table 4). Concerning pre-service teachers’ feedbacks no significant differences were established, t(55) = 1.45, p = .15, d = 0.39. This indicates that the video-based environment enhances self-reflections. Furthermore, t-tests of within-group comparisons of positive and negative evaluations revealed that there were no significant differences in pre-service teachers’ self-reflections of the V-Reflect condition, t (30) = 1.79, p = .08, d = 0.5, but significant differences with large effect for the feedbacks that were produced in the video-based en- vironment, t(30) = 3.06, p < .005, d = 0.93. In the T-Reflect condition no significant effects were found for the comparison of positive and negative evaluations in the self-reflections, t(25) = 0.93, p = .36, d = 0.33, or feedbacks, t(25) = 0.94, p = .36, d = 0.32. This indicates that pre-service teachers’ self-reflections in the V-Reflect condition are more negative but at the same time receive significantly more positive feedback as a response, unlike the T-Reflect condition where positive and negative evaluations do not differ significantly for self-reflections and feedback. However, descriptively pre-service teachers’ self-reflec- tions and feedbacks in the T-Reflect condition were predominantly ne- gative. Concerning differences between the groups in the direction of eva- luations a 2 × 2 MANOVA showed that self-reflections of participants of the V-Reflect condition contained significantly more positive, F(1, 55) = 9.93, p < .003, ηp 2 = .15, and negative evaluations, F(1, 55) = 4.14, p < .047, ηp 2 = .07, than pre-service teachers in the T- Reflect condition. Regarding differences between the direction of eva- luations of feedback a 2 × 2 MANOVA revealed no differences for ne- gative evaluations, F(1,55) = 0.04, p < .85, ηp 2 = .001, but significant differences for positive evaluations, F(1,55) = 17.94, p < .001, ηp 2 = .25. This indicates pre-service teachers in the V-Reflect condition self-reflected more positive and negative events and received more positive feedback than participants of the T-Reflect condition. 4. Discussion Educational beliefs, such as teacher self-efficacy and traditional and constructivist beliefs about teaching and learning, are influential ele- ments of teachers’ professional knowledge, and can therefore have significant effects on student achievement (Pajares, 1992; Shulman, 1987). For this reason, the present study investigated the impact of digital reflection and feedback environments on pre-service teachers’ beliefs about teaching and learning and self-efficacy during a four-week practicum. In terms of beliefs about teaching and learning no significant effects could be established for the comparison of pre-service teachers’ constructivist beliefs before and after the practicum. However, the study revealed that pre-service teachers’ traditional beliefs are fostered in the text-based digital environment. Furthermore, analyses of Table 3 Results of repeated measures analysis of variance for the intervention effect on beliefs about teaching and learning (constructivist/ traditional) and self-efficacy. Main effect Time Main effect Condition Interaction effect Time × Condition F p ηp 2 F p ηp 2 F p ηp 2 Constructivist beliefs F2F vs. VR vs. TR 4.62 .045 .20 2.87 .08 .14 4.08 < .05 .19 VR vs. TR 0.65 .43 .04 3.99 .06 .19 7.34 < .05 .29 F2F vs. VR 2.49 .13 .11 1.34 .26 .06 5.59 < .05 .21 F2F vs. TR 1.37 .26 .07 11.44 .003 .39 0.06 .80 .004 Traditional beliefs F2F vs. VR vs. TR 3.76 .07 .17 0.47 .63 .03 1.98 .16 .10 VR vs. TR 4.34 .052 .19 0.78 .39 .04 4.34 .052 .19 F2F vs. VR 0.06 .79 .004 0.19 .67 .009 0.36 .56 .02 F2F vs. TR 0.31 .58 .02 5.89 .03 .25 1.76 .20 .09 Self-efficacy F2F vs. VR vs. TR 18.54 < .001 .49 0.96 .39 .05 0.18 .83 .009 VR vs. TR 11.85 .003 .38 1.17 .29 .06 0.22 .64 .01 F2F vs. VR 14.05 .001 .46 2.18 .16 .09 0.18 .89 .001 F2F vs. TR 9.87 .005 .34 0.002 .97 .00 0.004 .95 .00 Note: F2F – face-to-face, TR – T-Reflect, VR – V-Reflect, nF2F = 65, nTR = 19, nVR = 22, significant effects printed bold. Table 4 Content analysis of pre-service teachers’ self-reflections and feedbacks in the text- (T-Reflect) and video-based (V-Reflect) digital environments. Self-reflection Feedback T-Reflect V-Reflect T-Reflect V-Reflect M SD M SD M SD M SD Knowledge-based reasoning No description 1.23 1.37 1.65 0.91 0.00 0.00 0.23 0.50 Description 0.23 0.59 0.77 0.80 0.19 0.40 0.97 0.87 Description and evaluation 0.19 0.40 0.71 0.86 0.27 0.45 0.32 0.60 Description, evaluation and consequences 0.77 0.91 0.61 0.92 0.46 0.86 0.65 0.84 Description, evaluation, consequences and alternatives 0.62 0.57 1.06 0.63 0.77 0.59 1.03 0.75 Direction of evaluation No evaluation 1.23 1.37 1.74 0.96 0.00 0.00 0.16 0.37 Negative evaluation 1.04 0.77 1.77 0.96 0.96 0.66 1.00 0.86 Positive evaluation 0.77 0.91 1.29 1.01 0.69 1.01 1.97 1.22 Note: Means and standard deviations represent the number of codes. C.N. Prilop, et al. Studies in Educational Evaluation 62 (2019) 158–170 166
• 10.
  variance showed significantdifferences of the development of con- structivist beliefs in the face-to-face-, text- and video-condition. The face-to-face- and T-Reflect-settings seem to decrease constructivist be- liefs, while they remain at a high level in the video-based learning environment. Regarding pre-service teachers’ self-efficacy the study revealed that this belief increased for teacher students of all conditions from pre- to posttest. Furthermore, none of the reflection and feedback settings (face-to-face, T-Reflect, V-Reflect) yielded significantly better results than the others. Quantitative qualitative content analysis of pre- service teachers’ self-reflections and feedbacks in the text- (T-Reflect) and video-based environments (V-Reflect) showed that pre-service tea- chers displayed higher level knowledge-based reasoning in their self- reflections in the V-Reflect condition than in the T-Reflect condition, however, not concerning their feedbacks. Furthermore, in the V-Reflect condition pre-service teachers’ self-reflections contained more positive than negative evaluations. Evaluative parts of the feedbacks in the video-based digital environment were more positive than in the text- based environment. Regarding pre-service teachers’ beliefs about teaching and learning, results support Lui and Bonner’s (2016) findings. Pre-service teachers possess strong constructivist beliefs, while traditional beliefs are on a low level. However, our findings extend previous research in this area in the following ways. On the one hand, the T-Reflect-condition fostered traditional beliefs. On the other hand, the face-to-face- and T-Reflect- environments seem to decrease constructivist beliefs contrary to the video-enhanced platform. According to Kagan (1992) this could have been caused by interactions with students, university courses that are coherent with school issues and occasions for self-reflection during teaching practicums. As all students attended the same preparatory course and had the same opportunities for classroom interactions, it can be assumed variances are based on the difference between environ- ments of self-reflection and feedback. In the T-Reflect-condition self- reflections were based on pre-service teachers’ written memory of the classroom situations. Various studies (e.g., Van den Bogert, Van Bruggen, Kostons, & Jochems, 2014; Wolff et al., 2015) determined that novices generally concentrate on negative events such as managing discipline and disruptive student behavior when analyzing teaching. Our content analysis did not find any statistically significant differences of negative and positive evaluation in the self-reflections, however, descriptively a tendency towards more negative evaluations can be observed. At the same time feedbacks were based on these self-reflec- tions. Pre-service teachers in the V-Reflect condition received more positive evaluations of their teaching in the feedbacks than in the T- Reflect condition. As feedbacks in the T-Reflect condition were not predominantly positive, it is likely this did not counterbalance control- focused, traditional beliefs that were supported by negative teaching events. Although pre-service teachers in the V-Reflect conditions dis- played considerable negative evaluations in their self-reflections we assume the availability of a classroom video lead to feedback con- taining more positive evaluations and, thus, acted as a counterbalance. This is supported by Tripp and Rich’s (2012a) finding that video clips enabled teachers to “gain new perspectives” (p. 735) on their teaching. They found colleagues provided new insights or asked thought-pro- voking questions that changed their impressions of classroom events. As feedbacks had to be solely based on the self-reflections, it can be as- sumed that new insights were only limited. Furthermore, video se- quences are highly immersive and can lead to knowledge-based rea- soning at the highest levels (Seidel et al., 2011). Our content analysis confirmed this finding. Pre-service teachers in the V-Reflect condition displayed more knowledge-based reasoning at the highest level than participants of the T-Reflect condition. This leads to deeper engagement (Sherin, 2007) and creates “vivid images” (Tripp & Rich, 2012a, p. 738) that influence future situations. Hence, videos can function as the “emotion-packed, vivid experiences” (Ambrose, 2004, p. 95) that can cause a more balanced, i.e. positive and negative reflection. Further- more, it is likely that initial self-reflections in the face-to-face group predominantly dealt with negative teaching events. However, as peers and experts were present and observed classroom interactions, we as- sume their feedback shifted pre-service teachers’ self-reflection and placed positive events at the fore. It can be assumed that the combi- nation with more balanced reflections of their own videos and feedback by peers changed pre-service teachers’ focus. This concurs with findings in a different domain. Heemsoth and Kleickmann (2018) were able to establish that reflecting on good and problematic examples of teaching at the same time increased constructivist beliefs, unlike only good or problematic examples. They concluded that being able to “compare teaching examples [enables] more conscious experiences […] which lead to higher changes in teacher students’ beliefs” (p. 19). Thus, self- reflection in combination with a video clip of classroom events might have had the same effect. Our findings concerning pre-service teachers’ self-efficacy align with previous studies (e.g., Colson et al., 2017; Gold et al., 2017; Hoy & Woolfolk, 1990; Wenner, 2001), however, also expand the insights concerning digital reflection and feedback environments. In general, the practicum seems to offer student teachers various sources that en- hance self-efficacy (Bandura, 1994). First, pre-service teachers were required to predominantly focus on their classroom management skills during the practical placement. By placing their attention on this par- ticular set of skills it is likely that mastery experience was attainable. They were able to work on classroom management skills such as monitoring or managing momentum (Gold & Holodynski, 2017), re- peatedly. The increase of self-efficacy indicates that pre-service tea- chers experienced success in their teaching practice. However, they probably also encountered obstacles in their teaching experience. Overcoming these obstacles can also be considered vital for mastery experience (Bandura, 1994). During face-to-face- and/or digital re- flection and feedback sessions they received support by university or mentor teachers and peers (Tschannen-Moran et al., 1998). Conse- quently, they were able to receive verbal confirmation that their skills were improving. Furthermore, pre-service teachers observed their peers in the classroom or on the video-based platform or read about specific accounts of classroom management. Observing or reading about peers in similar circumstances can be seen as vicarious experiences (Bandura, 1994). Accounts of other student teachers struggling or succeeding makes handling the complexity of the classroom more attainable. It also provided pre-service teachers with the opportunity to expand their knowledge of classroom management as they were able to read or watch how their peers acted in a multitude of situations (Li, Liu, & Steckelberg, 2010; So et al., 2009). While teaching practice of in-service teachers might seem hard to live up to, actions of equal-status peers are more achievable and foster self-efficacy (Santagata & Guarino, 2011). Although no differences in the development of student teachers’ self- efficacy could be found between conditions our study adds to prior research. It shows that digital reflection and feedback environments pose a viable substitute for face-to-face reflection and feedback ses- sions. 4.1. Implications for teacher education Our findings provide a more comprehensive understanding of di- gital reflection and feedback environments and pre-service teachers’ beliefs about teaching and learning and self-efficacy. Two aspects are perhaps the most intriguing. On the one hand, increases of self-efficacy are not determined by the type of reflection and feedback setting and are promoted in all conditions. On the other hand, constructivist beliefs do not decrease in the video-enhanced environment, contrary to face- to-face or text-based reflection and feedback. Consequently our findings shed some light on the practicality of digital reflection and feedback environments. Teaching practicums need to allow for balanced reflection so that constructivist beliefs do not decrease. Text-based and face-to-face en- vironments do not seem to be able to accomplish this. Our results C.N. Prilop, et al. Studies in Educational Evaluation 62 (2019) 158–170 167
• 11.
  indicate that pre-serviceteachers require a second source of informa- tion, both for their own reflections and for their feedback. Video as a second source of information helps pre-service teachers to put their own teaching and that of others into perspective. Otherwise one-sided re- flections can promote traditional beliefs. Furthermore, by providing immersive experiences, that increase trust in feedback (Tripp & Rich, 2012b), and can be relived repeatedly (Sherin, 2007) can cautiously be assumed to counteract self-reflections that harm constructivist beliefs (Heemsoth & Kleickmann, 2018). In his review of research on mathe- matics teachers’ beliefs Philipp (2007) concluded that the “essential ingredient [to create a constructivist mindset in teachers] is reflection upon practice” (p. 309). However, this conclusion falls short. Our study shows that all reflection is not equal. Face-to-face reflection and feedback requires ample resources (Lee & Wu, 2006). In conventional practicum settings students and experts need to travel between schools for face-to-face feedback and reflection sessions (Wu & Kao, 2008). Although students as well as experts need to view uploaded classroom videos and read reflections, the additional travel time in conventional settings can be saved. Because conventional feedback and reflection sessions require an extensive amount of travel time, not all programs involve proper reflection and feedback sessions. Digital video environments make time- and location-independent re- flection and feedback possible. Overall, the advantages of providing and receiving feedback in different locations and at different times, enabling experts to participate more readily, but especially the added benefit of video analysis (e.g., Borko et al., 2009; Tripp & Rich, 2012b) make digital video-enhanced environments a viable substitute to face-to-face feedback and reflection sessions. 4.2. Limitations and future directions Our study faced some limitations that encourage future research in this field. First, recording classroom videos requires participants to volunteer to be filmed due to data privacy. Consequently students could not be assigned randomly. We checked for baseline equivalence of in- itial levels of self-efficacy and traditional and constructivist beliefs about teaching and learning to see if groups differed before the prac- ticum, which they did not. However, self-selection could have had an impact on motivation of participants. Therefore this aspect constrains broader implications of our findings. Second, our sample size is relatively small for the video- and text- based conditions. However, research comprising video-recordings rarely consist of larger samples (e.g., Gröschner et al., 2018; Tripp & Rich, 2012a). This can also be attributed to data policy. Before being able to conduct video studies in German schools, consent from schools, parents and teachers needs to be acquired. If the application of video turns into a more common feature of teacher education, we expect more schools or parents to be willing to participate in the future. At the moment especially in-service teachers who did not work with classroom videos during their university degrees, know too little about the ad- vantages of video application. Consequently, they are reluctant to participate or to advise school students’ parents to let their children participate. We consider this a bottom-up process. The more pro- spective teachers work with classroom videos in teacher education, the easier it will be to recruit teachers to take part in video-based studies and courses in the future. Moreover, to allow for broader implications the sample should consist of students from different universities, and possibly different countries in the future. Third, ecological validity challenges controllability. Our study was conducted in an authentic practicum setting. Hence, specific classes with especially demanding school students could have had an impact on pre-service teachers’ beliefs about teaching and learning or self-ef- ficacy. Furthermore, feedback by mentors or peers could have had an impact, too. In future research, the quality of feedback should be in- vestigated. Fourth, following Heemsoth and Kleickmann (2018) we assumed that differences in constructivist beliefs are based on classroom videos providing a more balanced option for reflection and feedback. As the use of video in teacher education is becoming more prevalent, we feel future research should investigate differential effects between videos of “good”, “problematic” or “balanced” teaching. This could pose far reaching implications for the application of specific video types in teacher education. Fifth, we did not measure other aspects of pre-service teachers’ professional knowledge. As other studies (e.g., Kunter et al., 2013) were able to show an increase in one area of professional knowledge can lead to a decrease in another, future research should focus on interactions between different knowledge areas and analyze the effects of digital environments. 5. Conclusions The present study confirms that reflection and feedback environ- ments are beneficial to pre-service teachers’ self-efficacy. We were able to extend existing findings by focusing on digital reflection and feed- back settings. More importantly, we were able to establish that not all types of reflection are beneficial. Reflection and feedback environments with video support seem to maintain constructivist beliefs more pro- ductively than face-to-face and text-based settings. Text-based en- vironments even fostered traditional beliefs. Our content analysis in- dicated that this might be an effect of a higher degree of reflection in the V-Reflect condition and more positive feedback in comparison to the T-Reflect condition. Future research should therefore focus on the ap- plication of video in practicums, but also in university courses, re- garding pre-service teachers’ self-efficacy and traditional and con- structivist beliefs about teaching and learning. Our study was able to show that video-based reflection and feedback not only fosters cogni- tive and meta-cognitive components of (pre-service) teachers (e.g., Kleinknecht & Gröschner, 2016; Rosaen et al., 2008; Santagata et al., 2007) but also affective components. Acknowledgements We would like to thank the pre-service teachers who participated in this study and the school directors and teachers who provided them with the opportunity of a practical placement. Furthermore, we are grateful to our student assistants Karoline Glimm, Johanna Meyn and Jule Fischer for helping to plan the practicum. References Ambrose, R. (2004). Initiating change in prospective elementary school teachers’ or- ientations to mathematics teaching by building on beliefs. Journal of Mathematics Teacher Education, 7(2), 91–119. Apel, H. (2003). Onlinejournal – Lernreflexionen online [Online journal – Reflection for learning online). MedienPädagogik: Zeitschrift für Theorie und Praxis der Medienbildung, 7, 1–11. Arnold, K.-H., Gröschner, A., & Hascher, T. (2014). Schulpraktika in der Lehrerbildung: Theoretische Grundlagen, Konzeptionen, Prozesse und Effekte [School practicums during teacher education. Theory, concepts, processes, and effects]. Münster: Waxmann. Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Englewood Cliffs, NJ: Prentice-Hall. Bandura, A. (1994). Self-efficacy. In V. S. Ramachaudran (Vol. Ed.), Encyclopedia of human behavior: Vol. 4, (pp. 71–81). New York: Academic Press. Bandura, A. (1997). Self-efficacy: The exercise of control. New York: Freeman. Barkatsas, A., & Malone, J. (2005). A typology of mathematics teachers’ beliefs about teaching and learning mathematics and instructional practices. Mathematics Education Research Journal, 17(2), 69–90. Blömeke, S. (2014). Framing the enterprise: Benefits and challenges of international studies on teacher knowledge and teacher beliefs – Modeling missing links. In S. Blömeke, F.-J. Hsieh, G. Kaiser, & W. H. Schmidt (Eds.). International perspectives on teacher knowledge, beliefs and opportunities to learn – TEDS-M results (pp. 3–19). Dordrecht: Springer. Bonk, C. J., Malikowski, S., Angeli, C., & East, J. (1998). Web-based conferencing for preservice teacher education: Electronic discourse from the field. Journal of Educational Computing Research, 19(3), 269–306. Bonner, S. M., & Chen, P. P. (2009). Teacher candidates’ perceptions about grading and constructivist teaching. Educational Assessment, 14(2), 57–77. C.N. Prilop, et al. Studies in Educational Evaluation 62 (2019) 158–170 168
• 12.
  Borko, H., Jacobs,J. K., Eiteljorg, E., & Pittman, M. E. (2008). Video as a tool for fostering productive discussions in mathematics professional development. Teaching and Teacher Education, 24(2), 417–436. Borko, H., Whitcomb, J., & Liston, D. (2009). Wicked problems and other thoughts on issues of technology and teacher learning. Journal of Teacher Education, 60(1), 3–7. Chesnut, S. R., & Burley, H. (2015). Self-efficacy as a predictor of commitment to the teaching profession: A meta-analysis. Educational Research Review, 15, 1–16. Chi, M. T. H. (1997). Quantifying qualitative analyses of verbal data: A practical guide. Journal of the Learning Sciences, 6(3), 271–315. Choi, H., & Yang, M. (2011). The effect of problem-based video instruction on student satisfaction, empathy, and learning achievement in the Korean teacher education context. Higher Education, 62(5), 531–561. Clark, J. M., & Paivio, A. (1991). Dual coding theory and education. Educational Psychology Review, 3(3), 149–210. Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Hillsdale, NY: Erlbaum. Colson, T., Sparks, K., Berridge, G., Frimming, R., & Willis, C. (2017). Pre-service teachers and self-efficacy: A study in contrast. Discourse and Communication for Sustainable Education, 8(2), 66–76. Cortina, K. S., & Thames, M. H. (2013). Teacher education in Germany. In M. Kunter, J. Baumert, W. Blum, U. Klusmann, S. Krauss, & M. Neubrand (Eds.). Cognitive activation in the mathematics classroom and professional competence of teachers. Mathematics tea- cher Education (pp. 49–62). Boston, MA: Springer. Derry, S., Sherin, M., & Sherin, B. (2014). Multimedia learning with video. In R. Mayer (Ed.). The Cambridge handbook of multimedia learning (pp. 785–812). Cambridge, UK: Cambridge University Press. Faul, F., Erdfelder, E., Buchner, A., & Lang, A.-G. (2009). Statistical power analyses using G*Power 3.1: Tests for correlation and regression analyses. Behavior Research Methods, 41(4), 1149–1160. Fennema, E., Carpenter, T. P., & Loef, M. (1990). Teacher belief scale: Cognitively guided instruction project. Madison: University of Wisconsin. Fennema, E., Carpenter, T. P., Franke, M. L., Levi, L., Jacobs, V. R., & Empson, S. B. (1996). A longitudinal study of learning to use children’s thinking in mathematics instruction. Journal for Research in Mathematics Education, 27(4), 403–434. Fenstermacher, G. (1986). Philosophy of research on teaching. Three aspects. In M. C. Wittrock (Ed.). Handbook of research on teaching (pp. 37–49). (3rd ed.). New York: Macmillan. Gielen, S., Peeters, E., Dochy, F., Onghena, P., & Stuyven, K. (2010). Improving the ef- fectiveness of peer feedback for learning. Learning and Instruction, 20(4), 304–315. Gläser-Zikuda, M. (2007). Lehramtsstudierende reflektieren und evaluieren ihr Unterrichtshandeln - zum Potenzial des Tagebuch-Ansatzes [Student teachers reflect and evaluate their teaching – on the potential of a journal approach]. Gruppendynamik und Organisationsberatung, 38(1), 43–57. Gold, B., Hellermann, C., & Holodynski, M. (2017). Effekte videobasierter Trainings zur Förderung der Selbstwirksamkeitsüberzeugungen über Klassenführung im Grundschulunterricht [Effects of video-based trainings for promoting self-efficacy in elementary classroom management]. Zeitschrift für Erziehungswissenschaft, 20(1), 115–136. Gold, B., & Holodynski, M. (2017). Using digital video to measure the professional vision of elementary classroom management: Test validation and methodological chal- lenges. Computers & Education, 107, 13–30. Goldsmith, L. T., & Schifter, D. (1997). Understanding teachers in transition: Characteristics of a model for the development of mathematics teaching. In E. Fennema, & B. S. Nelson (Eds.). Mathematics teachers in transition (pp. 19–54). Mahwah, NJ: Lawrence Erlbaum Associates, Inc. Gregory, A., Ruzek, E., Hafen, C. A., Mikami, A. Y., Allen, J. P., & Pianta, R. C. (2017). My teaching partner-secondary: A video-based coaching model. Theory Into Practice, 56(1), 38–45. Gröschner, A., Schindler, A.-K., Holzberger, D., Alles, M., & Seidel, T. (2018). How sys- tematic video reflection in teacher professional development regarding classroom discourse contributes to teacher and student self-efficacy. International Journal of Educational Research, 90, 223–233. Hammerness, K. M., Darling-Hammond, L., Bransford, J., Berliner, D. C., Cochran-Smith, M., McDonald, M., et al. (2005). How teachers learn and develop. In L. Darling- Hammond, J. Bransford, P. LePage, K. Hammerness, & H. Duffy (Eds.). Preparing teachers for a changing world: What teachers should learn and be able to do (pp. 358– 389). San Francisco: Jossey-Bass. Handal, B. (2003). Teachers’ mathematical beliefs: A review. The Mathematics Educator, 13(2), 47–58. Haney, J. J., Lumpe, A. T., & Czerniak, C. M. (2003). Constructivist beliefs about the science classroom learning environment: Perspectives from teachers, administrators, parents, community members, and students. School Science and Mathematics, 103(8), 366–377. Heemsoth, T., & Kleickmann, T. (2018). Learning to plan self-controlled physical edu- cation: Good vs. problematic teaching examples. Teaching and Teacher Education, 71, 168–178. Hellermann, C., Gold, B., & Holodynski, M. (2015). Förderung von Klassenführungsfähigkeiten im Lehramtsstudium. Die Wirkung der Analyse eigener und fremder Unterrichtsvideos auf das strategische Wissen und die professionelle Wahrnehmung [Fostering classroom management skills in teacher education: Effects of analysis of one’s own or other teachers ‘classroom videos on strategic knowledge and professional vision]. Zeitschrift für Entwicklungspsychologie und Pädagogische Psychologie, 47(2), 97–109. Hoy, W. K., & Woolfolk, A. E. (1990). Socialization of student teachers. American Educational Research Journal, 27(2), 279–300. Huppertz, P., Massler, U., & Plötzner, R. (2005). V-share: Video-based analysis and reflection of teaching experiences in virtual groups. Proceedings of the International Conference on Computer Support for Collaborative Learning, 245–253. Joyce, B. R., & Showers, B. (2002). Student achievement through staff development (3rd ed.). Alexandria, VA: ASCD. Kagan, D. M. (1992). Implications of research on teacher belief. Educational Psychologist, 27, 65–90. Kang, H., & van Es, E. A. (2018). Articulating design principles for productive use of video in preservice education. Journal of Teacher Education, 70(3), 1–14. Klassen, R. M., & Durksen, T. L. (2014). Weekly self-efficacy and work stress during the teaching practicum: A mixed methods study. Learning and Instruction, 33, 158. Kleinknecht, M., & Gröschner, A. (2016). Fostering preservice teachers’ noticing with structured video feedback: Results of an online- and video-based intervention study. Teaching and Teacher Education, 59, 45–56. Kleinknecht, M., & Schneider, J. (2013). What do teachers think and feel when analyzing videos of themselves and other teachers teaching? Teaching and Teacher Education, 33, 13–23. Körkkö, M., Morales Rios, S., & Kyrö-Ämmäla, O. (2019). Using a video app as a tool for reflective practice. Educational Research, 61(1), 22–37. Kraft, M. A., Blazar, D., & Hogan, D. (2018). The effect of teacher coaching on instruction and achievement: A meta-analysis of the causal evidence. Review of Educational Research, 88(4), 547–588. Kunter, M., Klusmann, U., Baumert, J., Richter, D., Voss, T., & Hachfeld, A. (2013). Professional competence of teachers: Effects on instructional quality and student development. Journal of Educational Psychology, 105(3), 805–820. Lee, G. C., & Wu, C.-C. (2006). Enhancing the teaching experience of pre-service teachers through the use of videos in web-based computer-mediated communication (CMC). Innovations in Education and Teaching International, 43(4), 369–380. Li, L., Liu, X., & Steckelberg, A. L. (2010). Assessor or assessee: How student learning improves by giving and receiving peer feedback. British Journal of Educational Technology, 41(3), 525–536. Lin, H.-L., & Gorrell, J. (2001). Exploratory analysis of pre-service teacher efficacy in Taiwan. Teaching and Teacher Education, 17(5), 623–635. Lu, H.-L. (2010). Research on peer coaching in preservice teacher education – A review of literature. Teaching and Teacher Education, 26(4), 748–753. Lui, A. M., & Bonner, S. M. (2016). Preservice and inservice teachers ‘knowledge, beliefs, and instructional planning in primary school mathematics. Teaching and Teacher Education, 56, 1–13. Moulding, L., Stewart, W. P., & Dunmeyr, M. (2014). Pre-service teachers’ sense of effi- cacy: Relationship to academic ability, student teaching placement characteristics, and mentor support. Teaching and Teacher Education, 41, 60–66. Narciss, S. (2013). Designing and evaluating tutoring feedback strategies for digital learning environments on the basis of the Interactive Tutoring Feedback Model. Digital Education Review, 23, 7–26. OECD (2009). Creating effective teaching and learning environments: First results from TALIS. Paris, France: OECD Publications. Pajares, M. F. (1992). Teachers’ beliefs and educational research: Cleaning up a messy construct. Review of Educational Research, 62(3), 307–332. Pendergast, D., Garvis, S., & Keogh, J. (2011). Pre-service student-teacher self-efficacy beliefs: An insight into the making of teachers. Australian Journal of Teacher Education, 36(12), 46–57. Peterson, P. L., Carpenter, T., & Fennema, E. (1989). Teachers’ knowledge of students’ knowledge in mathematics problem solving: Correlational and case analysis. Journal of Educational Psychology, 81, 558–569. Philipp, R. A. (2007). Mathematics teachers’ beliefs and affect. In F. Lester (Ed.). Second handbook of research on mathematics teaching and learning (pp. 257–315). Reston, VA: National Council of Teachers of Mathematics. Philipp, R. A., Ambrose, R., Lamb, L. L. C., Sowder, J. T., Schappelle, B. P., Sowder, L., et al. (2007). Effects of early field experiences on the mathematical content knowl- edge and beliefs of prospective elementary school teachers: An experimental study. Journal for Research in Mathematics Education, 38(5), 438–476. Powell, E. (2005). Conceptualising and facilitating active learning: Teachers’ video-sti- mulated reflective dialogues. Reflective Practice, 6(3), 407–418. Prilop, C.N., Weber, K.E., & Kleinknecht, M. (submitted). Digital video-based feedback environments in teacher education: Effects on pre-service teachers’ feedback com- petence on teaching practice during practicum. Computers in Human Behavior. Prins, F., Sluijsmans, D., & Kirschner, P. A. (2006). Feedback for general practitioners in training: Quality, styles, and preferences. Advances in Health Sciences Education, 11, 289–303. Questback (2017). Unipark - EFS survey, version summer 2017. Köln: Questback GmbH. Rakoczy, K., Buff, A., & Lipowsky, F. (2005). Befragungsinstrumente [Survey instru- ments]. In E. Klieme, C. Pauli, & K. Reusser (Vol. Eds.), Dokumentation der Erhebungs- und Auswertungsinstrumente zur schweizerischdeutschen Videostudie “Unterrichtsqualität, Lernverhalten und mathematisches Verständnis" [Documentation of survey and evaluation instruments of the Swiss-German video study “Teaching quality, learning behavior, and mathematical comprehension“]. Materialien zur Bildungsforschung [Materials for educa- tional research]: Vol. 13. Frankfurt am Main: GFPF. Rosaen, C. L., Lundeberg, M., Cooper, M., Fritzen, A., & Terpstra, M. (2008). Noticing noticing: How does investigation of video records change how teachers reflect on their experiences? Journal of Teacher Education, 59(4), 347–360. Rusticus, S., & Lovato, C. (2014). Impact of sample size and variability on the power and type I error rates of equivalence tests: A simulation study. Practical Assessment, Research & Evaluation. 19(11), 1–10. Santagata, R., & Guarino, J. (2011). Using video to teach future teachers to learn from teaching. ZDM, 43(1), 133–145. Santagata, R., Zannoni, C., & Stigler, J. W. (2007). The role of lesson analysis in pre- service teacher education: An empirical investigation of teacher learning from a C.N. Prilop, et al. Studies in Educational Evaluation 62 (2019) 158–170 169
• 13.
  virtual video-based fieldexperience. Journal of Mathematics Teacher Education, 10(2), 123–140. Schmider, E., Ziegler, M., Danay, E., Beyer, L., & Bühner, M. (2010). Is it really robust? Reinvestigating the robustness of ANOVA against violations of the normal distribu- tion assumption. Methodology: European Journal of Research Methods for the Behavioral and Social Sciences, 6(4), 147e151. Schwarzer, R., & Hallum, S. (2008). Perceived teacher self-efficacy as a predictor of job stress and burnout: Mediation analyses. Applied Psychology: An international review, 57(1), 152–171. Schwarzer, R., & Schmitz, G. S. (1999). Skala zur Lehrerselbstwirksamkeitserwartung (WIRKLEHR) [Scale for teacher self-efficacy]. In R. Schwarzer, & M. Jerusalem (Eds.). Skalen zur Erfassung von Lehrer- und Schülermerkmalen [Scales for assessing teacher and student attributes] (pp. 60–61). Berlin: Freie Universität Berlin. Schwindt, K. (2008). Lehrpersonen betrachten Unterricht: Kriterien für die kompetente Unterrichtswahrnehmung [Teachers observe instruction: criteria for competent perception of instruction]. Münster, Germany: Waxmann. Seidel, T., Stürmer, K., Blomberg, G., Kobarg, M., & Schwindt, K. (2011). Teacher learning from analysis of videotaped classroom situations: Does it make a difference whether teachers observe their own teaching or that of others? Teaching and Teacher Education, 27(2), 259–267. Sherin, M. (2007). New perspectives on the role of video in teacher education. In J. Brophy (Vol. Ed.), Advances in research on teaching: Vol. 10, (pp. 1–27). Bingley, UK: Emerald. Shulman, L. (1987). Knowledge and teaching: Foundations of the new reform. Harvard Educational Review, 57(1), 1–22. Sluijsmans, D. M. A., Brand-Gruwel, S., van Merriënboer, J. J. G., & Martens, R. L. (2004). Training teachers in peer-assessment skills: Effects on performance and perceptions. Innovations in Education and Teaching International, 41(1), 59–78. Sluijsmans, D., Dochy, F., & Moerkerke, G. (1999). Creating a learning environment by using self-, peer- and co-assessment. Learning Environment Research, 1(3), 293–319. So, W. W., Pow, J. W., & Hung, V. H. (2009). The interactive use of a video database in teacher education: Creating a knowledge base for teaching through a learning com- munity. Computers & Education, 53(3), 775–786. Staub, F. C., & Stern, E. (2002). The nature of teachers’ pedagogical content beliefs matters for students’ achievement gains: Quasi-experimental evidence from elemen- tary mathematics. Journal of Educational Psychology, 94(2), 344–355. Stipek, D. J., Givvin, K. B., Salmon, J. M., & MacGyvers, V. L. (2001). Teachers’ beliefs and practices related to mathematics instruction. Teaching and Teacher Education, 17(2), 213–226. Syring, M., Bohl, T., Kleinknecht, M., Kuntze, S., Rehm, S., & Schneider, J. (2015). Videos oder Texte in der fallbasierten Lehrerbildung? Effekte unterschiedlicher Medien auf die kognitive Belastung und die motivational-emotionalen Prozesse beim Lernen mit Fällen [Videos or texts in case-based teacher education? An examination of the effects of different media on cognitive load and motivational-emotional processes in case- based learning]. Zeitschrift für Erziehungswissenschaft, 18(4), 667–685. Topping, K. (2005). Trends in peer learning. Educational Psychology, 25(6), 631–645. Tripp, T. R., & Rich, P. J. (2012a). The influence of video analysis on the process of teacher change. Teaching and Teacher Education, 28(5), 728–739. Tripp, T. R., & Rich, P. J. (2012b). Using video to analyze one’s own teaching. British Journal of Educational Technology, 43(4), 678–704. Tschannen-Moran, M., & McMaster, P. (2009). Sources of self-efficacy: Four professional development formats and their relationship to self-efficacy and implementation of a new teaching strategy. The Elementary School Journal, 110(2), 228–245. Tschannen-Moran, M., Woolfolk Hoy, A., & Hoy, W. K. (1998). Teacher efficacy: Its meaning and measure. Review of Educational Research, 68(2), 202–248. Valencia, S. W., Martin, S. D., Place, N. A., & Grossman, P. (2009). Complex interactions in student teaching: Lost opportunities for learning. Journal of Teacher Education, 60(3), 304–322. Van den Bogert, N., Van Bruggen, J., Kostons, D., & Jochems, W. (2014). First steps into understanding teachers ‘visual perception of classroom events. Teaching and Teacher Education, 37, 208–216. Van Zundert, M. J., Sluijsmans, D. M. A., Könings, K. D., & Van Merriënboer, J. J. G. (2012). The differential effects of task complexity on domain-specific and peer as- sessment skills. Educational Psychology: An International Journal of Experimental Educational Psychology, 32(1), 127–145. Vieluf, S., Kunter, M., & van de Vijver, F. J. R. (2013). Teacher self-efficacy in cross- national perspective. Teaching and Teacher Education, 35, 92–103. Voss, T., Kleickmann, T., Kunter, M., & Hachfeld, A. (2013). Mathematics teachers’ be- liefs. In M. Kunter, J. Baumert, W. Blum, U. Klusmann, S. Krauss, & M. Neubrand (Eds.). Cognitive activation in the mathematics classroom and professional competence of teachers: Results from the COACTIV project (pp. 249–271). (Mathematics Teacher Education; Vol. 8). New York: Springer. Weber, K. E., Prilop, C. N., Glimm, K., & Kleinknecht, M. (2018a). Video-, Text- oder Live- Coaching? Konzeption und Erprobung neuer Formate der Praktikumsbegleitung [Video-, text- or live-coaching? Conception and implementation of new formats of internship-supervision]. Herausforderung Lehrer_innenbildung, 1(1), 90–119. Weber, K. E., Gold, B., Prilop, C. N., & Kleinknecht, M. (2018b). Promoting pre-service teachers’ professional vision of classroom management during practical school training: Effects of a structured online- and video-based self-reflection and feedback intervention. Teaching and Teacher Education, 76, 39–49. Wenner, G. (2001). Science and mathematics efficacy beliefs held by practicing and prospective teachers: A 5-year perspective. Journal of Science Education and Technology, 10(2), 181–187. Wolff, C. E., van den Bogert, N., Jarodzka, H., & Boshuizen, H. (2015). Keeping an eye on learning: Differences between expert and novice teachers’ representations of class- room management events. Journal of Teacher Education, 66(1), 68–85. Woolfolk Hoy, A. (2005). What predicts student teacher self-efficacy? Academic Exchange Quarterly, 9(4), 123–127. Woolfolk Hoy, A., & Burke-Spero, R. (2005). Changes in teacher efficacy during the early years of teaching: A comparison of four measures. Teaching and Teacher Education, 21(4), 343–356. Wu, C.-C., & Kao, H.-C. (2008). Streaming videos in peer assessment to support training pre-service teachers. Educational Technology & Society, 11(1), 45–55. Yadav, A., Phillips, M., Lundeberg, M., Koehler, M., Hilden, K., & Dirkin, K. H. (2011). If a picture is worth a thousand words is video worth a million? Differences in affective and cognitive processing of video and text cases. Journal of Computing in Higher Education, 23(1), 15–37. Yang, M., Badger, R., & Yu, Z. (2006). A comparative study of peer and teacher feedback in a Chinese EFL writing class. Journal of Second Language Writing, 15(3), 179–200. Zee, M., & Koomen, H. M. Y. (2016). Teacher self-efficacy and its effects on classroom processes, student academic adjustment, and teacher well-being. Review of Educational Research, 86(4), 981–1015. C.N. Prilop, et al. Studies in Educational Evaluation 62 (2019) 158–170 170