The impact of an online technology course on pre-service teachers’ technological knowledge: Strategies and design

This study investigated how pre-service teachers ’ technology integration knowledge changed as a result of participating in an online educational technology course for a semester in the Fall of 2020 and the Spring of 2021. The pre-service teachers ’ technology integration knowledge development was assessed through their self-reported technological pedagogical content knowledge (TPACK). A total of 194 participants completed a validated TPACK pre-and post-test survey. The results of multilevel modeling analysis suggested that the participants had significant gains in all TPACK domains. Strategies used in designing the online educational technology course were examined for their contribution to pre-service teachers ’ TPACK development. Educational technology-based course included activities that facilitated the integration of the components of TPACK framework through peer interactions, lesson planning, and peer feedback. The findings have implications for future research and provide guidance for the design of effective online learning environments to improve pre-service teachers ’ technology integration knowledge.


INTRODUCTION
Preparing pre-service teachers to develop technology integration knowledge and use technology in ways that impact student learning is an important objective of teacher preparation programs.Research indicates that the COVID-19 pandemic initiated an unprecedented shift in K12 education that demonstrated the importance of K12 teachers' technology integration knowledge and the potential of technology in facilitating student learning (Bower et al., 2014;Wekerle & Kollar, 2021).Acquiring technological knowledge (TK) necessary for effective teaching and learning entails developing an understanding of the complex web of relationships between technology, pedagogy, and content knowledge (CK) (Koehler & Mishra, 2009).It requires going beyond mere competence with the latest technological tools (Zhao, 2003).A model that considers how pre-service teachers' knowledge domains intersect in order to effectively teach and engage students with technology is referred to as technological pedagogical content knowledge (TPACK) (Mishra & Koehler, 2006) and builds upon Shulman's (1986) conception of pedagogical content knowledge (PCK).
Given a shift in education to incorporate computer-based electronic technologies in instruction, learning a subject matter with technology is a significantly different experience from acquiring the skills to teach that subject matter with technology (Niess, 2005).
Historically, teacher preparation programs had a great emphasis on CK (Shulman, 1986), which was viewed as separate from pedagogy and technology (Mishra & Koehler, 2006).Mishra and Koehler (2006) proposed TPACK as a distinct form of knowledge that considers the specific knowledge of content and technological tools that translates to effective teaching and learning.The consideration of the role technology plays in instruction and student learning experiences has advanced the theoretical framework proposed by Shulman (1986) and defines technology integration and its application as a knowledge-based system.
Despite the fact that TPACK is probably the most prominent model that explains the specific knowledge base required for the educational use of digital technologies (Chai et al., 2011a(Chai et al., , 2013;;Schmid et al., 2021), there is limited research on how specific experiences in an online technology-based course result in the development of pre-services teachers' TPACK main domains as well as their TPACK integrated domains.Voithofer and Nelson (2020) explain there is a great emphasis on learning through experience and course-field integration.Preservice teachers' knowledge development in main knowledge domains (TK, pedagogical knowledge [PK], and CK) was examined in several studies.However, the results of the integrated TPACK knowledge domains (technological content knowledge [TCK], PCK, technological pedagogical knowledge [TPK], and technological pedagogical content OPEN ACCESS knowledge [TPCK]) have not been consistently insignificant (Wang et al., 2018).Moreover, there is limited research on how online course design strategies such as scaffolding, teacher-student collaboration, and alignment of theory and practice could help explain the effectiveness of course development and its contribution to pre-service TPACK growth.The purpose of the current study is to advance research in this area by assessing the impact of an online educational technology course on pre-service teachers' TPACK development using multilevel analysis.

LITERATURE REVIEW
TPACK Framework Mishra and Koehler (2006) proposed the technological Pedagogical content framework to describe the complex interplay between technology, content, and pedagogy.This framework highlights the importance of technology, pedagogy, and content for understanding effective teaching with technology.It explicitly acknowledges that effective use of technology is rooted in three primary forms of knowledge as well as the intersections of these knowledge domains: TK, PK, CK, PCK, TCK, TPK, and TPCK.Mishra and Koehler (2006) define TPACK as the knowledge of how technological tools can be used to represent subject-specific activities or topic-specific activities to facilitate student learning.Tseng (2016) explains that it can be interpreted as teachers' understanding of when, where, and how to enhance student learning of content using appropriate pedagogy and supporting technologies.
TPACK consists of multiple subject-specific domains of knowledge.TK refers to the knowledge of the application of technologies.PK describes the knowledge of practices, procedures, or methods necessary for teaching and learning.CK is the knowledge of the subject matter.PCK refers to the knowledge of teaching methods that make the subject matter more understandable to the learners.TCK is described as the knowledge of how subject matter representation is influenced by technology.TPK refers to the knowledge of the application of teaching approaches applied to the use of technology.TPCK is understood as the knowledge of using various technologies to implement teaching methods for different subject content (Mishra, & Koehler, 2006;Schmid et al., 2021).According to this framework, preservice teachers are required to have a well-developed knowledge base in their subject integrated with the development of their knowledge of technology and pedagogy.Niess (2005) argues that as students begin the teacher preparation program, some of the development of their knowledge of the subject matter may be integrated with the development of their knowledge of teaching, learning, and technology.They often learn about teaching and learning with technology outside their development of their knowledge of the subject matter and pedagogy.TPACK facilitates the development and consideration of these knowledge domains concurrently.Although TPACK framework has gained recognition among researchers, educators, and teachers (Herring et al., 2016;Voogt et al., 2013;Wekerle & Kollar, 2021), it does not have an agreed-upon theoretical definition.Willermark (2018) stated that TPACK is defined as either knowledge of or competence in technology integration.TPACK as knowledge focuses on teachers incorporating TK into the structure of PCK and the surrounding context.Conversely, TPACK based on performance highlights planning, implementation, and evaluation of teaching activities and competencies.

Strategies for Technology Integration
Studies reported different strategies to prepare pre-service teachers to integrate technology in their future classrooms.Existing literature indicates three pathways to TPACK development including standalone technology courses, embedded instructional strategies in technology or methods courses, and field experiences (Voithofer & Nelson, 2020).Voithofer and Nelson (2020) explain there is a great emphasis on learning through experience and course/field integration.Combined strategies and conditions at the macro and micro levels are believed to advance pre-service teachers' TPACK development.Agyei and Voogt (2015) stated that efforts to develop pre-service teachers' technology integration knowledge are either directly related to the preparation of pre-service teachers or to the creation of conditions conducive to technology integration at the institutional level.They argued that the strategies implemented by numerous teacher education programs are diverse and often conflicting.
Common enacted strategies include delivery of technology integration content, hands-on technology skill-building activities, practice with technology integration in the field, and technology integration reflections (Agyei & Voogt, 2015).The key themes that have been reported regarding pre-service teachers' TPACK development include the conditions at the institutional level, such as technology planning and leadership, training, and access to resources (Tondeur et al., 2020).The key strategies at the micro level include role modeling, reflection, instructional design, lesson plans, lesson presentation, collaboration, authentic or real class experiences, and feedback (Aktas & Ozmen, 2020;Tondeur et al., 2012Tondeur et al., , 2020)).Studies have shown linking conceptual or theoretical information to practice seems to significantly enhance pre-service teachers' preparation to use technology (Agyei & Voogt, 2015).Aktas and Ozmen (2020) has reported positive relationships between these strategies and pre-service teachers' TPACK development.Some of the course-based strategies that have been applied in courses designed to develop pre-service teachers' technology integration knowledge include alignment of theory and practice, collaboration with peers, scaffolding, and practicing in authentic settings (Agyei & Voogt, 2015).The focus on technology integration development strategies indicates a shift in research from theory to the application of frameworks such as TPACK to design and determine specific technology interventions that could lead to successful adoption of technology (Niess, 2005;Redmond & Lock, 2019;Wang et al., 2018).
Research on TPACK has also shifted to examining the relationship between TPACK knowledge domains (Chai et al., 2011a).Research suggests that while the identification of pre-service teachers' knowledge development in main knowledge domains (TK, CK, and PK) was reported in many studies to be a significant predictor of TPACK, examining the integrated TPACK knowledge domains (TCK, PCK, TPK, and TPCK) remains a challenge due to insignificant results (Wang et al., 2018).This suggests that further investigation is needed to understand pre-service teachers' TPACK development.

The Present Study
The ability to use technology effectively in instruction is becoming increasingly important.Despite the fact that TPACK has provided some insights into the use and application of emerging technologies, teacher education programs have struggled to effectively and adequately prepare pre-service teachers to integrate technology in their future classrooms (Goktas et al., 2008).Research indicates that few teachers feel comfortable using technology in their teaching and graduating teachers are still entering schools unprepared to use technology in meaningful ways (Farjon et al., 2019;Mitchell, 2019;Voithofer & Nelson, 2020).While extensive literature exists on pre-service teachers' TPACK and their own evaluations of TPACK constructs (Voogt et al., 2013), little is known about the relationship between TPACK main domains, pre-service teachers' TPACK integrated domains, and the types of of course strategies that could facilitate pre-service teachers' technology integration knowledge development.
There is a need to examine whether individual components of TPACK account for differences in their technology-integration knowledge development.Given that TPACK is contextually bound (Mishra & Koehler, 2006) course teaching strategies are regarded as relevant for pre-service teachers' TPACK growth.The purpose of the current study is to assess the impact of an online technology course on pre-service teachers' TPACK development using multilevel analysis.It measures pre-service teachers' TPACK growth and provides evidencebased research that can facilitate the implementation of strategies in the development of online technology courses in designed for pre-service teachers to successfully integrate technology in their classrooms.The following research questions guided this study: 1. To what extent does an online instructional technology course impact pre-service teachers' development of TPACK domains and their intersections over time?
2. To what extent do instructional strategies of an online technology course support pre-service teachers' TPACK development?

METHODOLOGY Context
The asynchronous 15-week online course was delivered using the WordPress blogging platform.A course site was set up on the platform, and each student created their own site, which was then connected to the main course site.Students were asked to generate blog posts for each assignment.These posts were then aggregated to main course site using tags and categories.This open platform allowed each student to both create a portfolio of their work as well as be able to access the work of their peers.This facilitated peer review and sharing of resources.
TPACK framework was first introduced in week two through readings and videos.In week three students were asked to develop a lesson plan based on the primary constructs of TPACK (technology, pedagogy, and content).They articulated each of these constructs in their lesson plan, and then reviewed one other peer's assignment, providing feedback about how the constructs were articulated.During the following week, students revised their initial lesson plan based on the feedback they received.They also included additional intersecting constructs of TPK, TCK, and PCK.During subsequent weeks in the course, students were guided to develop TPACK-focused lesson plans using a variety of instructional technologies such as learning management systems, online learning resources, social media, and content-specific technologies.In addition, students were gradually introduced to technology-rich assessment practices and professional learning networks for future learning opportunities.
The course adopted a constructionist approach to learning, where students created text-based lesson plans that were shared through blog posts, which could be used as starting points for peer review and feedback from the instructor (Papert, 1991).This method mirrors what would be required of them as they moved into their student teaching and future teaching practice.Further, it allowed them to construct artifacts of learning that could be shared as a digital portfolio and as a resource for them to pull from in their future teaching practice.The open access of the platform, along with the navigational tools of tags and categories, provided them future access to all students' work and lesson plans curated by content areas and grade levels.

Participants
The data were collected from 194 pre-service teachers enrolled in a semester-long two-credit instructional technology course during either Fall 2020 or Spring 2021 in a university located in the southeastern United States.Of these teachers, 34 (17.53%) were males and 160 (82.47%) were females.
The majority of teachers (n=158) were 18-24 years of age (81.44%).About 3.61% (n=7) of teachers were Black or African American, 50% (n=97) were Asian, 12.37% (n=24) were American Indian or Alaska Native, 28.87% (n=56) were White, and 5.15% (n=10) were others.About 32.47% (n=63) of teachers had a college degree, 10.82% had some graduate school, 2.06% (n=4) had a master's degree, 41.75% (n=81) had a high school diploma, and 12.89% (n=25) had other educational training.Almost all the participants majored in education.Over the course of the semester, participants were asked to complete Pamuk et al.'s (2015) TPACK survey as part of the course.The course is a requirement for several teacher preparation programs and is an elective in others.Due to restrictions brought on by the COVID-19 pandemic, this course was modified to be conducted asynchronously online.

Measures & Data Collection
Participants' overall TPACK and knowledge in the seven subdomains were assessed using Pamuk et al.'s (2015) instrument (Table 1).Overall TPACK is the composite score of participants' selfreport knowledge in the seven subdomains.The survey includes 37 items in total, and they were rated on a 4-point Likert scale (1=strongly disagree, 4=strongly agree).Table 1 presents sample items, the number of each TPACK domain of the survey, and the reliability of each construct at each time point.Participants completed the surveys, which also included demographic data, during the 2 nd and 15 th weeks of the course.

Data Analysis & Model Development
The data set consisted of repeated measures of TPACK scores (i.e., pre-and post-test).Thus, repeated observations were nested within individual pre-service teachers.A clustered data structure justified the selection of multilevel modeling (MLM) analysis (Hoffman, 2015).MLM was employed to examine changes in each pre-service teacher's overall TPACK and TPACK sub-domains over time frame of course.A function of MLM is to separate within-group individual effects from between-group aggregate effects.MLM is frequently applied in aggregating nested data as it provides a more exact likelihood specification that avoids the assumptions of within-cluster normality and within-study variances (Garson, 2019).Two-time or repeated measures data used in this study may be seen as a special case of hierarchical data.Pre-service teachers' overall TPACK and its subdomains (TK, PK, CK, TPK, TCK, PCK, and TPCK) are outcome variables at level 1.At this level, TPACK measures are recorded at each time point.Student ID or pre-service teacher ID is a unit of analysis at level 2. This unit of analysis has two rows of data-one row for each survey administration.Pre-and post-data are nested within students at level 2. The model in this study adjusted estimates of intercept (mean) of dependent variable at level 1 on grouping variable at level 2. It considered TPACK and its subdomains to be nested within pre-service teachers.As TPACK scores at two-time points were included as a fixed effect and pre-service teachers were included as random effects, testing if time has a linear effect on TPACK and its subdomains and whether the latter at level 1 are affected by variation among pre-service teachers.
The strategy for model building was developed in line with Hoffman's (2015) recommendations.In the first step, an empty means, random intercept model was built to inspect the intraclass correlation coefficient (ICC).The null model may be used as a baseline model to test whether the values of the dependent variable (TPACK and its subdomains) at level 1 cluster within groups formed by the grouping variable (pre-service teacher) at level 2, thereby violating the data independence assumption of OLS regression and indicating the need for MLM (Garson, 2019).In the second step, a fixed linear time, random intercept model was built to examine if there was a linear change in overall TPACK and each TPACK sub-domain over time on average.Random linear time models were not permitted given that the data only contained two-time points.Owing to relatively small sample of the data, restricted maximum likelihood (REML) was used.Fixed effects were evaluated using Wald tests.Stata 16 was used for analyses.

Descriptive Statistics & Bivariate Correlations
Table 2 presents descriptive statistics for overall TPACK and each TPACK subdomain at each time point.No variables were substantially non-normal.Descriptively, there was an increase in overall TPACK and all TPACK subdomains.Table 3 presents bivariate correlations.The same knowledge domains measured at different time points were significantly and positively correlated with one another.All TPACK domains were more strongly correlated with one another at the second time point.

Pre-Service Teachers' TPACK Development
A null model was performed to examine whether the scores of TPACK and its subdomains at level 1 cluster by pre-service teacher ID variable) at level 2. Garson (2019) explained that this is mathematically equivalent to finding that there is a significant ICC based on the grouping variable.The closer ICC is to zero, the more likely it is to be nonsignificant, meaning that the level 1 outcome variable is independent of the level 2 grouping variable, and MLM is not needed.Cohen et al. (2013) stated that if the value of ICC is above 0.058, differences resulting from grouping justify MLM examination.
As Table 4 shows, for main knowledge domains, around 37-60% of the original outcome variation is cross-sectional and due to betweenperson mean differences over time.On the contrary, for the integrated knowledge domains and overall TPACK, about 16-31% of the original outcome variation is cross-sectional and due to between-person mean differences over time.In this case, using ordinary least squares (OLS) regression would generate coefficients that are inappropriate due to clustering effects.In the second step, a series of fixed linear time, random intercept models were built to evaluate the average change in each TPACK subdomain and overall TPACK from the beginning of the semester to the end of the semester.
As Table 5 shows, participants' pre-TK was 3.03 and post-TK was 3.20.TK significantly increased by .17units from time 1 to time 2. Adding the fixed linear time predictor significantly explained about 12.50% of residual variance.
As Table 6 shows, participants' pre-CK was 3.11 and post-CK was 3.23.CK significantly increased by .12units from time 1 to time 2. Adding the fixed linear time predictor significantly explained about 14.29% of residual variance.
As Table 7 shows, participants' pre-PK was 3.16 and post-PK was 3.32.PK significantly increased by .16units from time 1 to time 2. Adding the fixed linear time predictor significantly explained about 16.67% of residual variance.
As Table 8 shows, participants' pre-PCK was 2.99 and post-PCK was 3.19.PCK significantly increased by .20 units from time 1 to time 2.  Adding the fixed linear time predictor significantly explained about 26.67% of residual variance.
As Table 9 shows, participants' pre-TCK was 3.19 and post-TCK was 3.34.TCK significantly increased by .15units from time 1 to time 2. Adding the fixed linear time predictor significantly explained about 6.25% of residual variance.
As Table 10 shows, participants' pre-TPK was 2.98 and post-TPK was 3.29.TPK significantly increased by .31units from time 1 to time 2. Adding the fixed linear time predictor significantly explained about 31.82% of residual variance.
As Table 11 shows, participants' pre-TPCK ws 3.00 and post-TPCK was 3.23.TPCK significantly increased by .23 units from time 1 to time 2. Adding the fixed linear time predictor significantly explained about 20.00% of residual variance.
As Table 12 shows, pre-TPACK was 3.06 and post-TPACK was 3.26.TPACK significantly increased by .20 units from time 1 to time 2. Adding the fixed linear time predictor significantly explained about 37.50% of residual variance.In sum, there was a significant increase in each TPACK subdomain and overall TPACK from time 1 to time 2. Participants reported having the highest increase in TPK (b=.31; residual variance explained=31.82%)and least in CK (b=.12; residual variance explained=14.29%).

DISCUSSION & IMPLICATIONS
This study investigated the development of pre-service teachers' perceived TPACK in a 15-week online educational technology course and revealed several important findings.Regarding research question one, there was a significant increase in all perceived knowledge domains, which supports the effectiveness of the online educational technology course to develop pre-service teachers' critical knowledge for technology integration.Educational technology courses are not always effective in developing all aspects of pre-service teachers' TPACK.In the study by Habowski and Mouza (2014), pre-service teachers' CK and PCK did not significantly increase over the course of one semester.Cengiz (2015) did not find significant increases in TK and TPCK at the end of the course.Kaplon-Schilis and Lyublinskaya (2017) even reported that there was a non-significant decrease in pre-service teachers' PK.Therefore, simply engaging teachers in the intervention of an educational technology course will not necessarily develop preservice teachers' TPACK.The design of the course plays a major role (Aktas & Ozmen, 2020).
Of all the knowledge domains, teachers reported the largest increase in TPK and the least increase in CK in this study.This finding is not unexpected given that the curriculum of the course does not specifically focus on deepening pre-service teachers' knowledge in their teaching subjects.Rather, involving teachers in the activities that foster the interaction between different primary knowledge domains is emphasized.It should be noted that there is a discrepancy in the growth pattern of TPACK in the context of educational technology courses in the existing literature.Jin (2017) and Kaplon-Schilis and Lyublinskaya (2017) both reported the largest increase in TPCK, while Durdu and Dag (2017) and Habowski and Mouza (2014) reported the largest increase in TCK.Tokmak et al. (2013) reported the largest increase in TK for math teachers, TPK for science teachers, and TPCK for literacy teachers.As with this study, Cengiz (2015) found the largest increase in TPK.On the other hand, Chai et al. (2011b) reported the largest increase in CK, a primary knowledge domain.Despite the seeming differences, one consistent pattern across all these studies and the current finding is that educational technology courses seem to be effective in developing pre-service teachers' knowledge in technologyrelated domains, particularly in the integrated areas.
Regarding research question two, the current findings further support that embedding instructional strategies in technology or methods courses is a viable way to develop pre-service teachers' perceived TPACK (Voithofer & Nelson, 2020).The introduction of the theoretical framework through readings and videos at the very beginning of the semester followed by lesson planning activities helped these pre-service teachers see the link between conceptual information and their practice, which has been demonstrated as a critical strategy to develop TPACK (Agyei & Voogt, 2015).The constructionist approach employed in the course supported by the open access of the WordPress blogging platform along with the navigational tools of tags and categories afforded pre-service teachers opportunities to engage in hands-on technology skills-building activities, collaboration, feedback, and technology integration reflection and provided them with plenty of vicarious experiences (i.e., role modeling).
All of these elements align with the recommendations in the literature and, hence, effectively support the development of preservice teachers' TPACK as reported in the current study (Agyei & Voogt, 2015;Aktas & Ozmen, 2020;Tondeur et al., 2012Tondeur et al., , 2020)).Although it is impossible to single out which element contributes the most to teachers' growth in TPACK in this study, we believe that providing pre-service teachers plenty of opportunities to interact with one another is critical in online educational technology courses.Several studies have shown that although online learning features convenience and flexibility, students reported feeling alone and disconnected with peers in remote learning environments (Song, 2004;Vonderwell, 2003).Students' perceptions of peer interaction have also been shown to be associated with their motivation to learn in online settings (Lin et al., 2017).Therefore, although all the aforementioned elements are important, quality interaction between peers should be an indispensable part of any curriculum during the process of designing an online educational technology course.

CONCLUSIONS, LIMITATIONS, & FUTURE RESEARCH
The purpose of this study was to examine pre-service teachers' technology integration knowledge development through MLM in a technology-based course.The study investigated whether individual components of TPACK account for differences in pre-service teachers' technology-integration knowledge development.Course instructional strategies were considered in the examination of pre-service teachers' TPACK growth.The data was collected from 194 pre-service teachers in Fall of 2020 and Spring of 2021 at two-time points through a validated TPACK survey.The results of MLM analysis suggest that the participants had significant gains in all TPACK domains, which suggests that instructional strategies are viable in developing preservice teachers' TPACK.In other words, a technology-based course that includes activities that facilitate the integration of components of TPACK framework through peer interactions, lesson planning, and peer feedback can play a critical role in enhancing pre-service teachers' TPACK.
One of the main contributions to the existing literature is the learning and research context of this study.Although previous studies offer several important insights into developing pre-service teachers' TPACK, the recommended strategies are mainly based on the findings in the face-to-face context (Agyei & Voogt, 2015;Aktas & Ozmen, 2020).Given the unique nature of online learning environments, it is not certain if those recommendations can be extended to remote settings.This study not only lends further support for the "best practices" in the existing literature but also demonstrates how to implement those micro-level strategies in a fully online learning environment.
The data was gathered through a self-reported measure, which can be considered a limitation of the present study.Self-reported measures tend to be subjective in nature as they could be influenced by the characteristics of the context and the participants.Also, the results of this study were limited to the data collected from one course.Future research should examine longitudinal data that includes several online educational technology courses over more than one semester could provide insights into pre-service teachers' technology integration knowledge development over an extended period of time, as well as the long-term impact of online technology-based courses.Future studies could also consider an analysis of pre-service teachers' learning strategies, challenges, and practices that may yield better outcomes.Qualitative measures such as observations and interviews can provide in-depth insights into the connections between pre-service teachers' technology integration knowledge development and their various contexts or learning environments.

Table 5 .
Change in TK over time

Table 6 .
Change in CK over time

Table 7 .
Change in PK over time

Table 8 .
Change in PCK over time

Table 9 .
Change in TCK over time

Table 10 .
Change in TPK over time

Table 11 .
Change in TPCK over time

Table 12 .
Change in TPACK over time