Testimony by Kevin W. Meuwissen, Ph.D., Assistant Professor, Department of Teaching and Curriculum, Warner School of Education, University of Rochester
Rochester, New York
New York State Assembly Standing Committees on Higher Education and Education; Joint Hearing on New Statewide Teacher and School Building Leader Certification Requirements
Wednesday, April 30, 2014
Hamilton Hearing Room, Legislative Office Building
Albany, New York
We at the Warner School of Education acknowledge the importance of teacher education and certification to high-quality K-12 teaching. And we agree with Darling-Hammond’s claim in her 2010 Center for American Progress report that performance assessments like edTPA surpass paper-pencil certification exams as measures of contextualized, enacted teaching practice, and as tools for delineating the complexities of teaching. However, we also find that New York’s implementation of edTPA, up to the present, does not realize the potential benefits of a movement toward more authentic assessment and evaluation. While we support the Regents’ decision to provide teaching candidates who fail edTPA this year with an alternative pathway to certification until 2015, more extensive piloting and support is necessary to reduce problems with edTPA implementation and provide schools of education and their teaching candidates with valuable benchmarks to better prepare for the assessment. Even under this proposed timeframe, New York’s edTPA implementation schedule remains uniquely short in comparison to schedules in some other states.
Four questions frame this testimony. First, what does scholarship tell us about the use of performance assessments for teacher certification? Second, how does New York State’s implementation of edTPA align with what we know from that scholarship? Third, what are the consequences of New York’s edTPA implementation for candidates and teacher educators? And fourth, what policy recommendations are warranted in light of the responses to these questions?
What does scholarship tell us about the use of performance assessments of teaching for certification?
On the first question, evidence pertaining to outcomes of performance-based certification assessments is scant because such assessments are relatively new. Much of that evidence comes from research on a program that served as a model for
: the Performance Assessment for California Teachers (PACT). Darling-Hammond, Newton, and Wei explain that performance assessments like the PACT hold promise as predictors of teaching effectiveness and opportunities for candidates to develop their planning, instruction, assessment, and reflective practices. For example, Bunch and colleagues studied eight elementary teaching candidates’ experiences with the PACT and found that completing the assessment helped cultivate a focus on the challenges of teaching linguistically diverse learners in multicultural contexts. Yet research bears out dilemmas of PACT implementation as well. Okhremtchouk and others determined that most teachers and administrators in their candidates’ internship schools were unfamiliar with the PACT, its aims, and its requirements; and further, constraints in some schools significantly impeded candidates’ capacities and efforts to complete the assessment.
Research by Lit and Lotan, Peck and McDonald, and Peck, Gallucci, and Sloan explores the effects of PACT implementation on teacher education programs. Their studies suggest that performance assessment of candidates can encourage collaborative, longitudinal program review via analyses of benchmark evidence – specifically, complete candidate work samples, not just summative scores – but that such review also requires the negotiation of many complex factors. That negotiation includes: (1) discussing divergent perspectives on the assessment, its purposes, and its consequences; (2) maintaining important local values and priorities, like teacher research and community involvement, while meeting external certification demands; (3) interpreting the assessment’s language; (4) deciding how, if at all, to integrate the assessment across the program or into coursework; (5) developing tools and strategies for focusing candidates on continuous professional growth despite the summative nature and technological complexity of the assessment; and (6) including numerous faculty members and other institutional stakeholders in those activities.
How does New York State’s implementation of edTPA align with what we know from that scholarship?
There are clear implications of these studies for enacting edTPA. A performance assessment for certification can benefit teaching candidates and teacher educators under two conditions: (1) there must be time for teacher education programs, internship schools, and candidates to work systematically and collaboratively to plan for implementation and address procedural and technical challenges thereof; and (2) comprehensive benchmarks and transparent evaluation criteria must be available to teacher educators as tools for analysis and decision making within their programs. Unfortunately, New York’s implementation of edTPA has met neither condition. As Cochran-Smith, Piazza, and Power indicate, New York’s teacher educators have sparse access to stratified edTPA benchmarks for their own research and development, even currently. And the subcontracting of evaluation to a corporate entity likely will limit teacher educators’ access to edTPA portfolio materials moving forward.
Also questionable is the State Education Department’s decision to set the edTPA’s cut score at 41 for secondary certification candidates after two years of piloting and field testing. By contrast, the State of Washington set its passing standard at 35 after five years of piloting and field testing. Commissioner King has not explained how New York’s cut score was determined, though he has indicated that he expects a higher rate of failure on certification exams than in the past because of it. This fusion of opaqueness and rapid implementation leaves many teacher educators deeply skeptical that the State Education Department and Board of Regents have an interest in providing opportunities and resources to strengthen their programs via the assessment. That alternative credentialing programs like Teach For America are held to looser standards corroborates this skepticism. It is worth noting that 15 states are in some degree of edTPA development and implementation. Of those states, New York is the only one with a timeline of fewer than five years from introduction to consequential use.
What are the consequences of New York’s edTPA implementation for teaching candidates and teacher educators?
The following examples demonstrate how edTPA’s hasty implementation negatively impacted candidates and teacher educators at the Warner School of Education this year. Before April 2014, edTPA guidelines prohibited teacher educators and candidates’ peers from “suggesting changes to be made in an edTPA draft” and “offering formal feedback” on video clips that might be selected for submission. In our program, candidates generally complete edTPA via innovative instructional units they design for and enact within their clinical field experiences. This places teacher educators and candidates in the tenuous position of formatively analyzing and discussing instructional artifacts, including video, as part of their course experiences, yet also trying to adhere to the edTPA’s support restrictions. Earlier this month, the Stanford Center for Assessment, Learning, and Equity (SCALE) released substantially revised guidelines that better enable authentic feedback on teaching products that candidates also might select for edTPA. While this is a commendable modification, it is one that should happen during piloting and benchmarking, not in the first year of consequential use.
Another dilemma stems from some K-12 schools’ requirement that candidates use EngageNY modules, which are intended to align with state Common Core tests but are still in development, during their student teaching placements. One elementary candidate remarked:
These modules are very teacher-centered, with a lot of whole-class instruction. And to think that the political details of the War of 1812 are meaningful and developmentally appropriate to second graders is ridiculous. So edTPA is mandated by the state. But if I want to do well on it, and teach in ways that I know are beneficial for my kids, I have to find ways to teach outside of the EngageNY modules, which also come from the state.
In a similar vein, a secondary English language arts candidate indicated:
I’m teaching a creative writing unit, which limits what I can submit for edTPA. The rubrics prioritize Common Core standards, like comprehending and constructing meaning from complex text features, which means that if I try to submit lessons and video about teaching kids to peer edit, for example, they’re not going to fit the rubric criteria very well.
These candidates’ comments succinctly and effectively illustrate edTPA’s entanglement with other state policy levers that also have been rushed to implementation. We must take into consideration that edTPA was executed concurrently with two other new certification exams, alongside K-12 curriculum mandates and high-stakes tests that have altered the landscapes of placement schools dramatically. Keeping in mind that a stated goal of edTPA is to strengthen teaching across the state, it also is important to imagine the unintended consequences of its implementation on young people considering a teaching career. Their numbers have declined in recent years, and we can only expect that situation to worsen if edTPA is perceived to be an unfair assessment for which it is difficult to prepare.
What policy recommendations are warranted in light of the responses to these questions?
Donald Campbell coined an aphorism that the more far-reaching an evaluation tool is for decision-making, the more subject it will be to corruption pressures, and the more apt it will be to distort the processes it is intended to monitor. New York State’s enactment of edTPA thus far has corrupted the purposes and conceivable benefits of performance assessment for state certification. The Regents’ safety net policy, adopted on April 29, is a positive step, but likely will not eliminate the challenges identified in this testimony. It may provide teacher education institutions, candidates, and K-12 schools with an opportunity to allay them, but only if those institutions have access to comprehensive, high-quality benchmarking data. We recommend that the New York State Department of Education provide those data, including completed portfolios and scoring rubrics from the current year, to institutions with certification programs and grant those institutions time to learn from them and productively manage the practical dilemmas of edTPA in their candidates’ interests.
We also recommend that the Department be held accountable for increasing the effectiveness and transparency of its policy rationales and decisions. Again, California’s PACT offers a lesson for consideration. Wei and Pecheone describe in great depth a scoring protocol that involves extensive training and calibration, double-score verification by local and external raters, and a complex auditing system, with interventions for institutions that produce unreliable results. In New York, not only is edTPA scoring conducted by the Pearson Corporation and removed from the local context completely, it is described vaguely with language like “evaluators are monitored throughout the scoring process.” Given edTPA’s implementation challenges thus far, including a frantic search for proficient raters in the midst of consequential use, skepticism among candidates and teacher educators about the integrity of the evaluation process seems justifiable. More transparent and accountable policymaking by the State Education Department could alleviate some of that skepticism. Research by Cohen and Hill suggests that when policy reforms are not accompanied by intellectual, political, and fiscal collaboration among state agencies and the professional stakeholders implementing them, those reforms tend to fail. The State Education Department and Board of Regents should take that research into consideration.
Finally, we suggest that policymakers consider a more permanent solution for candidates who have successfully completed their teacher education programs and other state certification exams, but not edTPA. A transitional certification option already exists for those who enter the field through alternative credentialing programs like Teach For America. It seems fair and reasonable that candidates prepared via accredited programs have the same opportunity to demonstrate successful practice and maintain their certification as those who are not.
Dr. Kevin W. Meuwissen
Assistant Professor, Department of Teaching and Curriculum
Warner School of Education, University of Rochester
Link to written testimony with complete references and list of other signatories