Research shows that teachers matter more for student achievement than any other in-school factor. Thus, how we prepare teachers is an essential component of improved student achievement. The 2015-16 State of Education in Tennessee priority of empowering Tennessee’s teachers includes a call to continue improvements to educator preparation. Responding to this call, SCORE convened a half-day SCORE Institute focused on promising, research-based practices, as well as next steps to advance educator preparation in Tennessee.
A recently published study by University of Michigan researchers Matthew Ronfeldt and Shanyce L. Campbell details how the authors investigated the potential for using teacher observation ratings as a measurement for evaluating educator preparation programs (EPPs). This topic is timely – as we continue to advance effective educator preparation, we must consider the best ways to hold programs accountable for preparing strong teachers.
The research team looked at program completer information, and teacher observation scores collected by the Tennessee Higher Education Commission and the Tennessee Department of Education on 40 of the 43 educator preparation programs in Tennessee. With regard to the remaining three institutions, the authors eliminated all EPPs with fewer than 10 graduates in a 3-year period and combined all of those small programs into one additional “small schools” category. The sample of teachers included in the study consisted of program completers who graduated between 2009-10 and 2012-13 and were employed during the 2011-12 and 2013-14 school years.
The authors had two major questions about the relationship between EPPs and their graduates’ observation rankings:
1. Are there differences between EPPs in terms of graduates’ observation ratings?
2. How do EPP rankings based on graduates’ Tennessee Value-Added Assessment System (TVAAS) scores compare with EPP rankings based on graduates’ observation ratings
The authors drew several important conclusions, including:
The difference between observation ratings from top-performing EPPs and low-performing EPPs is comparable to the difference in observation ratings between first and second-year teachers. The authors’ study identified three institutions that produced candidates who performed significantly better than the state average and five institutions that produced candidates who scored significantly worse than the state average according to observation scores.
It may be valuable to consider rating individual programs within EPPs, as overall institutional ratings can mask profound differences across programs. The authors assessed differences in observational ratings of candidates across programs—such as elementary education—that are housed within institutions. They found that some institutions house programs that rank significantly better or significantly worse in observation ratings than the institution as a whole. For instance, one institution rated as average according to observation ratings houses a secondary undergraduate program that scores among the lowest in the state.
The authors found a positive and statistically significant relationship between graduates’ observation scores and their TVAAS scores. This relationship remained consistent for elementary and secondary teachers and for reading and math teachers. However, the magnitude of this relationship is modest, suggesting different elements of EPP performance are measured by TVAAS and observation ratings. Further, there could be value in considering multiple measures to obtain a full picture of EPP performance.
Stakeholders in educator preparation can draw many conclusions from this study related to how the state holds programs accountable and how those methods provide critical feedback for programs to improve. The findings of this study should inform conversations about these two important topics because accountability and program feedback may have profound effects on how EPPs instruct their candidates. Accountability might also ensure that teacher preparation candidates are held to the same rigorous standards as practicing teachers.
Though important, how Tennessee holds educator preparation programs accountable represents just one facet of EPP improvement. SCORE’s research brief on educator preparation, From Preparation to Practice: Research on Improving Effective of Early-Career Teaching, details additional elements of the complex work surrounding educator preparation program improvement and the many critical conversations that will need to occur as we continue to engage in this very important work.