Contributed by:
School districts often struggle to recruit and retain effective math teachers. Alternative-route certification programs aim to expand the pool of teachers available; however, many alternate routes have not been able to attract large numbers of teacher candidates with undergraduate degrees in math. In response, some districts, including Baltimore, Philadelphia, Washington D.C., and New York City, have developed alternative programs with a math immersion component to recruit candidates who do not have undergraduate majors in math. Such programs provide potential math teachers with intensive math preparation to meet state certification requirements while at the same time maintaining an early-entry approach in which individuals who have not completed a teacher preparation program can become qualified to teach with only five to seven weeks of coursework and practice teaching. Four years since its inception, the New York City Teacher Fellows Math Immersion program supplies 50 percent of all newly certified math teachers to New York City public schools. In this study, we find that Math Immersion teachers have stronger academic qualifications than their College Recommending (traditionally certified) peers, although they have weaker qualifications than Teach for America teachers. However, despite stronger general academic qualifications Math Immersion teachers produce somewhat smaller gains in math achievement for middle school math students than do College Recommending teachers and substantially smaller gains than do Teach for America teachers.
1.
NBER WORKING PAPER SERIES
RECRUITING EFFECTIVE MATH TEACHERS: HOW DO MATH IMMERSION TEACHERS COMPARE?:
EVIDENCE FROM NEW YORK CITY
Donald Boyd
Pam Grossman
Karen Hammerness
Hamilton Lankford
Susanna Loeb
Mathew Ronfeldt
James Wyckoff
Working Paper 16017
http://www.nber.org/papers/w16017
NATIONAL BUREAU OF ECONOMIC RESEARCH
1050 Massachusetts Avenue
Cambridge, MA 02138
May 2010
We are grateful to the New York City Department of Education and the New York State Education
Department for the data employed in this paper. We benefited from insights by Vicki Bernstein and
Mark Thames. We also thank the program directors and other administrators who provided us with
details of their preparation programs. Thanks to participants at the AEFA meetings and seminar participants
at the University of Pennsylvania for comments on an earlier draft. We appreciate financial support
from the U.S. Department of Education, IES Grant R305E06025 and the National Center for the Analysis
of Longitudinal Data in Education Research (CALDER). CALDER is supported by IES Grant R305A060018
to the Urban Institute. The views expressed in the paper are solely those of the authors and may not
reflect those of the funders. Any errors are attributable to the authors. The views expressed herein
are those of the authors and do not necessarily reflect the views of the National Bureau of Economic
Research.
© 2010 by Donald Boyd, Pam Grossman, Karen Hammerness, Hamilton Lankford, Susanna Loeb,
Mathew Ronfeldt, and James Wyckoff. All rights reserved. Short sections of text, not to exceed two
paragraphs, may be quoted without explicit permission provided that full credit, including © notice,
2.
Recruiting Effective Math Teachers: How Do Math Immersion Teachers Compare?: Evidence
from New York City
Donald Boyd, Pam Grossman, Karen Hammerness, Hamilton Lankford, Susanna Loeb,
MathewRonfeldt, and James Wyckoff
NBER Working Paper No. 16017
May 2010
JEL No. I21,I28
ABSTRACT
School districts often struggle to recruit and retain effective math teachers. Alternative-route
certification programs aim to expand the pool of teachers available; however, many alternate routes
have not been able to attract large numbers of teacher candidates with undergraduate degrees in math.
In response, some districts, including Baltimore, Philadelphia, Washington D.C., and New York City,
have developed alternative programs with a math immersion component to recruit candidates who do
not have undergraduate majors in math. Such programs provide potential math teachers with intensive
math preparation to meet state certification requirements while, at the same time maintaining an
early-entry approach in which individuals who have not completed a teacher preparation program can
become qualified to teach with only five to seven weeks of coursework and practice teaching. Four
years since its inception, the New York City Teacher Fellows Math Immersion program supplies 50
percent of all new certified math teachers to New York City public schools. In this study, we find that
Math Immersion teachers have stronger academic qualifications than their College Recommending
(traditionally certified) peers, although they have weaker qualifications than Teach for America
teachers. However, despite stronger general academic qualifications Math Immersion teachers
produce somewhat smaller gains in math achievement for middle school math students than do
College Recommending teachers and substantially smaller gains than do Teach for America teachers.
Donald Boyd Karen Hammerness
The Center for Policy Research School of Education
University of Albany Stanford University
135 Western Ave. Stanford, CA 94305
Albany, NY 12222 [email protected]
Hamilton Lankford
Pam Grossman School of Education, ED 317
School of Education University at Albany
Stanford University State University of New York
Stanford, CA 94305 Albany, NY 12222
[email protected] [email protected]
(more on next page)
3.
Susanna Loeb
524 CERAS, 520 Galvez Mall
Stanford University
Stanford, CA 94305
and NBER
Mathew Ronfeldt
School of Education
Stanford University
Stanford, CA 94305
James Wyckoff
Curry School of Education
University of Virginia
P.O. Box 400277
Charlottesville, VA 22904-4277
4.
I. Introduction
For well over a decade school districts across the U.S. have struggled to recruit and retain
effective math teachers. This problem appears to be more acute in schools serving high poverty student
populations (Boyd et al., 2005; Boyd et al., 2009; Hanushek et al., 2004). Historically, this has meant
that many middle and high school math teachers are teaching out of field (Ingersoll, 2003). NCLB
attempted to address this issue by requiring that all children in core academic subjects be taught by a
highly qualified teachers (HQT) beginning in 2005-06. To be highly qualified a teacher must, among
other things, have state certification and demonstrated knowledge in the subject area. States were afforded
substantial discretion in how they met the HQT requirements. Nonetheless, there is evidence that not all
teachers meet the HQT standard and that children in high poverty schools are much more likely to be
taught math by a teacher who does not meet this requirement (Peske and Haycock, 2006).
In response to the shortage of qualified math teachers, school districts have employed a variety of
strategies. Some of these strategies, including paying a one-time signing bonus or a subject-area bonus,
largely target the distribution of teachers between districts while leaving the overall pool of candidates
relatively unchanged. Other strategies, such as alternative-route certification programs, expand the pool
of teachers. For example, the New York City Teaching Fellows Program provided nearly 12,000 new
teachers to New York City schools from 2003 to 2008. However, many alternate routes, including the
Teaching Fellows, have not been able to attract large numbers of teacher candidates with undergraduate
degrees in math or science. For example, fewer than 10 percent of the math certified teachers who
entered teaching in New York City in 2007-08 through the New York City Teaching Fellows program
had an undergraduate major in mathematics. More recently, several teacher residency programs that
focus on math, such as Math for America, have been directing substantial effort to the recruitment and
preparation of highly qualified math candidates. While these programs have attracted individuals with
undergraduate degrees in Mathematics from very strong undergraduate institutions, to date we know little
about the effectiveness of the teachers from these programs compared to those from alternative
certification or tradition teacher preparation programs.
In response to the need for qualified math teachers and the difficulty of directly recruiting
individuals who have already completed the math content required for qualification, some districts,
including Baltimore, Philadelphia, Washington D.C., and New York City, have developed alternative
certification programs with a math immersion component to recruit otherwise well-qualified candidates,
who do not have undergraduate majors in math. Such programs provide candidates with intensive math
preparation to meet state certification requirements while, at the same time maintaining the early-entry
approach common in alternative pathways in which individuals who have not completed a teacher
preparation program can become a qualified teacher with only five to seven weeks of coursework and
1
5.
practice teaching. This approach is becoming increasingly widespread but to date there is little evidence
of the effectiveness of teachers that enter through this immersion route.
The New York City Teaching Fellows program was among the first to employ a math immersion
component in the recruitment of math teachers. Prior to 2003, in the absence of sufficient numbers of
teachers who met the math major requirement, New York City employed many uncertified (temporary
license) teachers to teach math. These uncertified teachers disproportionately taught low-performing
students who frequently were from non-white and low-income families.1 As of September 2003, the New
York State Board of Regents required all districts to hire certified teachers. To address this shortage in
math and in other subjects, the New York City Department of Education created the alternative
certification pathway, the New York City Teaching Fellows (NYCTF). NYCTF was successful in
recruiting new teachers to NYC schools. For example, for the 2007-08 school year, there were 11
applicants to the Fellows program for every vacancy filled by a Fellow. However, recruiting math
teachers is often difficult. New York State requires that math teachers receive 30 semester hours of
undergraduate mathematics coursework, typically equivalent to a math major, which is not so different
from the requirements in many other states. Few college graduates meet this requirement and even fewer
of these graduates desire to enter teaching. Thus, even with the creation of the alternative certification
route, New York City finds it difficult to recruit sufficient numbers of teachers with substantial math
coursework or a math undergraduate major.
In response to the continued shortage of qualified math teachers, the district developed the Math
Immersion component of the New York City Teaching Fellows. Math Immersion began as a small pilot
in 2002-03, just as NYCTF was beginning, and, depending on the year, supplies nearly 50 percent of all
new middle and high school math teachers in New York City. Math Immersion seeks to increase the
supply of math teachers by reducing entrance requirements and providing opportunities for teaching
candidates interested in mathematics to complete the math required to be qualified, without returning to
college for an additional degree. By design, the Math Immersion program recruits individuals who did
not major in math but who demonstrate evidence of math proficiency by having a math related
undergraduate major (e.g., economics or science) or who have math related work experiences.
In this study, we examine the following research questions:
How does the background and preparation of Math Immersion teachers compare to math teachers
entering through other pathways?
For a detailed discussion of the sorting of teachers in New York see Lankford, Loeb and Wyckoff (2002).
Research in other states has demonstrated very similar patterns ((Betts, Reuben & Danenberg , 2000; Clotfelter,
Ladd, Vigdor & Wheeler, 2007; and Peske & Haycock 2006).
2
6.
How do the achievement gains of the students taught by Math Immersion teachers compare to
those of students taught by math teachers entering through other pathways?
How does the retention of Math Immersion candidates compare to math teachers entering through
other pathways?
II. Background
Linking teacher preparation and pathways into teaching to student learning is a complex process.
Student outcomes are influenced directly by the teacher workforce but also by other school inputs and
external factors such as student background and environment. Because of these complexities linking
teacher preparation to student achievement is difficult to model empirically. On top of this, the teacher
workforce and each teacher’s decisions of where to teach and how to teach is influenced by many
institutional factors such as state and district policies, by teacher preparation pathways, and even by
student performance. Teacher preparation, alone, is difficult to describe and measure, as it comprises
many elements from subject-matter, to pedagogy, to child and youth development and classroom
management. In addition, quality of implementation likely is at least as important as content coverage in
With the increasing availability of rich data on students, teachers and schools in recent years,
researchers have begun to develop a range of empirical models to examine the relationship between how
teachers are prepared and the outcomes of their students. Most of these models either compare the
learning gains of students taught by teachers in the same school or compare the learning gains of the same
students taught by different teachers in different years. Recent rigorous research using these approaches
to assess the effectiveness of alternative routes to teaching shows that individuals entering teaching
through highly selective early-entry routes are either as effective in teaching math as teachers entering
through traditional preparation programs or become so within the first few years of their careers, (Decker
et al. 2004; Boyd et al. 2006; Kane et al. 2007; Harris and Sass, 2008; and Constantine et al. 2009).
However, there is wide variation in the selection and preparation requirements of both traditional
and alternative preparation programs, and comparing broad categories of pathways into teaching does
little to uncover the effects of program or pathway characteristics. In some instances the difference
between an alternative route and a traditional route can be more a matter of timing of requirements than a
difference in requirements (Boyd et al, 2008). In other cases there are dramatic differences in the
requirements that teachers must fulfill to become certified through alternative and traditional preparation
programs, (Feistritzer, 2008; Grossman and Loeb, 2008). Nearly all of the research examining the relative
effectiveness of various forms of teacher preparation has been limited to exploring relative differences in
the gains of student achievement for teachers from different programs (e.g. Boyd et al, 2006; Harris and
3
7.
Sass, 2008; Decker, Mayer, & Glazerman, 2004; Raymond, Fletcher, & Lucque, 2001; Xu, Hannaway
and Taylor, 2007) without attempting to understand the many components of teacher preparation. There
are a few exceptions to this focus on program effects. Constantine et al., 2009 provide a detailed
description of differences in programs in their analysis. Boyd et al. 2009 assess the effects of preparation
program characteristics for elementary school teachers on student learning and Harris and Sass, 2007,
examine the extent to which a teacher's specific preparation coursework is associated with achievement
gains in her students.
Thus, several studies have examined the effectiveness of teachers from alternative pathways and
some have included middle school math outcomes. In addition, a few studies have examined the
relationship between preparation features and classroom achievement gains. On the other hand, to our
knowledge, no prior research has systematically examined the specific preparation and effectiveness of
math teachers, in particular, nor has it examined the effectiveness of routes into math teaching based on a
math-immersion model.
Recruiting Math Teachers. New York City hires between 6,000 and 9,000 new teachers every
year. In many years prior to the 2003-04 school year, uncertified teachers (temporary license teachers)
constituted as much as fifty percent of all new hires. The New York State Board of Regents required that
effective as of 2003-04 virtually all teachers must be certified. In anticipation of this, in 2000 the Regents
had created the opportunity for districts to hire alternatively certified teachers. In response, the New York
City Department of Education working with the New Teacher Project created the New York City
Teaching Fellows program (NYCTF) and soon thereafter the Math Immersion component of NYCTF
(NYCTF-MI). These changes dramatically altered the composition of entering teachers to New York
City Public Schools. Figure 1 shows that uncertified teachers were largely replaced by NYCTF and
NCTF-MI teachers, although there has also been meaningful increases in the number of College
Recommending teachers in recent years.
Figure 1 reflects the hiring of all teachers in New York City while, for this analysis, we are
particularly interested in math teachers. The change in pathways for math teachers was even greater than
the changes overall. Prior to 2003, NYCDOE relied heavily on uncertified teachers, because sufficient
numbers of College Recommending math teachers were unavailable. In addition, from 2003-04 through
2007-08 New York City expanded the total number of math teachers by 18 percent due to increasing
enrollments and reductions in class size.2 As a result, New York City needed to recruit between 600 and
800 new math teachers per year during this period. When other sources of supply were unavailable, New
York City turned to the Math Immersion program to meet demand. For each year starting in 2005-06
until 2009-10 that meant that approximately 20 percent of Math Immersion Fellows did not meet internal
Based on correspondence from Vicki Bernstein, New York City Department of Education, 9/14/09.
4
8.
selection standards for the NYCTF. The problem is more acute in the recruitment of math teachers than
other teachers as only about 12 percent of non-Math Immersion Fellows in that period failed to meet
internal standards.3 Below we explore whether the need to go beyond selection standards affected student
performance. Figure 2 shows the number of new teachers who are certified in math.4 In recent years
Math Immersion has supplied nearly half of all new math teachers, far more than any other pathway into
math teaching. College Recommending programs have shown strong growth in recent years, but as of
2008 still only supplied about 30 percent of new math teachers.
New York City has come to rely heavily on Math Immersion for its new math teachers,
accentuating the importance of a better understanding the effectiveness of these teachers and this
approach to pre-service preparation. Dramatic changes in other pathways would be needed to fill the
demand for middle and high school math teachers if the Math Immersion program were eliminated. In
this analysis we compare Math Immersion to other current pathways as a means to understand their effect
on student achievement and teacher retention.
III. Data and Methods
The data for this analysis come from three distinct sources: extensive administrative data,
information about teacher preparation programs obtained from document reviews and interviews with
administrators in teacher preparation programs, and from a survey of teachers. We describe each of these
datasets in turn below.
Administrative data. We employ administrative data on students, teachers and schools drawn from
a variety of databases from the New York City Department of Education, the New York State Education
Department and the College Board. Student achievement exams are given in grades 3 through 8. All the
exams are aligned to the New York State learning standards and each set of tests is scaled to reflect item
difficulty and are equated across grades and over time.5 Tests are given to all registered students with
limited accommodations and exclusions. Thus, for nearly all students the tests provide a consistent
assessment of achievement for a student from grade three through grade eight. Since the Math Immersion
program was initiated in the 2003-04, we include data for all teachers who teach students with math
achievement outcomes from 2003-04 through 2007-08. The dependent variables in our models come from
annual student achievement exams given in grades four through eight to almost all New York City
students. The student data, provided by the New York City Department of Education (NYCDOE),
Based on correspondence from Vicki Bernstein, New York City Department of Education, 9/14/09.
For purposes of this graph a teacher is defined as having math certification if at the time she entered teaching she
held either an elementary/middle school or a secondary school math certification.
The mathematics exams in all grades are developed by CTB-McGraw Hill. New York State employs CTB-
McGraw Hill for its 4th and 8th grade ELA exams. In 2003 New York City switched from CTB to Harcourt Brace
for its 3rd, 5th-7th grade exams. At that time there was an equating study done to accommodate the switch in exams.
5
9.
consists of measures of gender, ethnicity, language spoken at home, free-lunch status, special-education
status, number of absences, and number of suspensions for each student who was active in any of grades
three through eight that year.
For most years, the data include scores for approximately 65,000 to 80,000 students in each
grade. Using these data, we construct a set of records with a student’s current exam score and his or her
lagged exam score. For this purpose, a student is considered to have value added information in cases
where we had a math score for the current year and a score for the same subject in the immediately
preceding year for the immediately preceding grade. All student achievement scores have been
normalized by grade and year to have a zero mean and a unit standard deviation.
To enrich our data on teachers, we match New York City teachers to data from New York State
Education Department (NYSED) databases, using a crosswalk file provided by NYCDOE that links their
teacher file reference numbers to unique identifiers employed by NYSED. We draw variables for NYC
teachers from New York State data files as follows:
Teacher Experience: For teacher experience, we use transaction-level data from the NYCDOE
Division of Human Resources to identify when individuals joined the NYCDOE payroll
system in a teaching position. When this information is missing or when the value is less than
the value in the NYSED personnel master files, we use the NYSED data.
Teacher Demographics: We draw gender, ethnicity, and age from a combined analysis of all
available data files, to choose most-common values for individuals.
Test performance: We draw information regarding the teacher certification exam scores of
individual teachers and whether they passed on their first attempts from the NYS Teacher
Certification Exam History File (EHF).
Pathway: Initial pathway into teaching comes from an analysis of teacher certification data plus
separate data files for individuals who participated in Teach for America or the New York
City Teaching Fellows Program.
College Recommending: We obtain indicators for whether an individual had completed a
college-Recommending teacher preparation program and, if so, the level of degree obtained
(bachelor’s or master’s) from NYSED’s program-completers data files.
Program Data. The information on preparation programs comes from a data collection effort in
the spring and summer of 2004 designed to characterize the preparation received by individuals entering
teaching in 2004-05 but also applicable to surrounding cohorts. We focus specifically on the 18
institutions that prepare about two-thirds of the College Recommending teachers hired in NYC schools in
recent years. Within these institutions, we concentrated on the pre-service preparation at 25 college-
6
10.
recommending math certification programs, as well as the preparation provided by two large alternative
route programs: the New York City Teaching Fellows and Teach for America.
We rely on a number of data sources to document information about programs: state documents,
institutional bulletins and program descriptions, NCATE documents when available, and institutional
websites to find information about requirements and course descriptions. In documenting information
about courses, whenever possible we use the information that is closest to what is actually taught. For
example, we ask programs for the names of instructors who taught math methods for the cohorts
completing programs in 2004, and use this list rather than the list of faculty included in the state
documents. In addition, we interview program directors and directors of field experiences about the
curriculum, structure, and field experiences in their programs. We also documented the curricular
requirements in each program, focusing specifically on the number of required courses in math methods
and in math content, as well as required courses related to learning, assessment, diverse learners, and
classroom management. To further document the preparation received in mathematics, we collected
syllabi from both math content and math methods courses whenever possible. In our analyses of
preparation to teach mathematics, we looked at the overall emphasis on the teaching of mathematics, as
represented by the percentage of the curriculum that focused on math, as opposed to an emphasis on less
subject-specific preparation. Because participants in these various pathways complete their coursework at
different times, it is important to remember that students in the College Recommending programs will
have completed all of these requirements prior to teaching full-time as a teacher of record; in both TFA
and the NYC Teaching Fellows, participants complete 6-8 weeks of initial coursework prior to becoming
full-time teachers, completing the rest of the requirements during their first 2-3 years of teaching.
Surveys. In the spring of 2005 we conducted a survey of all first-year New York City teachers in
which we ask detailed questions about their preparation experiences, the mentoring they received in their
first year, and their teaching practices and goals. Our overall response rate is 71.5 percent and the
response rate fo Respondents were asked to consider the preparation they received prior to entering the
classroom—what is typically referred to as pre-service teacher education. For teachers who entered
through TFA or NYC Teaching Fellows, this referred to the 6-8 weeks of preparation, generally offered in
the summer. r each pathway is nearly or slightly above 70 percent.
The survey asked all respondents a variety of questions regarding their general teacher
preparation, mentoring and current working environment. 6 In addition, we surveyed middle and high
school math teachers specifically about several aspects of their current teaching and their preparation to
teach math. We received completed surveys from 603 respondents including 210 Teaching Fellow Math
Immersion teachers (NYCTF-MI), 130 Teaching Fellows (NYCTF), 22 Teach For America teachers
The survey can be found at www.teacherresearchpolicy.org.
7
11.
(TFA), 129 College Recommending teachers (CR), and 112 teachers from “other” preparation routes
(“other path”).
We employ factor analysis of survey items to measure the extent to which programs emphasize
various aspects of preparation. These factors and the survey questions on which they are based are
summarized in Appendix B. For this purpose, we identify factors for opportunities to learn about
teaching math; their subject matter preparation in math, their preparation in specific teaching strategies,
their preparation for special education students, the quality of their field experience and the overall
opinion of the quality of their teacher preparation program.
Methods. In describing teacher preparation programs we employ data from our analysis of
program documents and interviews with program administrators that is summarized in tabular form. We
employ the factors constructed from the survey questions in regression analysis to examine whether
teachers prepared in certain pathways and programs identify similarities in their preparation that
differentiates it from that of other pathways. These regressions also include controls for the school
context in which teachers work and their personal characteristics.
As described above, a number of factors potentially complicate the identification of aspects of
teacher preparation that may influence the achievement of students taught by these teachers. First,
teaching candidates select their teaching pathway, preparation institution and program. This selection is
important because of the need to account for it in our assessment of program effects. Also by identifying
the features of pathways that attract individuals with the greatest potential, programs can recruit more
effective teachers. Second, different pathways into teaching can lead teachers into schools and
classrooms with different characteristics. For example, even at the pathway level there exist systematic
differences in the observable characteristics of the students they teach (see Table 1). On average the
students of Math Immersion teachers appear to be meaningfully more challenging to teach than the
students of College Recommending teachers. The students of Math Immersion teachers have math
achievement scores that average nearly 30 percent of a standard deviation lower than those of students of
College Recommending teachers. They are also more likely to be eligible for free lunch and are more
likely to be absent. By the same measures, the Math Immersion teachers have students who appear less
challenging than other New York City Teaching Fellows teachers or Teach for America teachers.
Because these differences likely influence student outcomes, our empirical models must be able to control
for them if we are to identify the effects of preparation as distinct from placement.
There are two parts to our multivariate analysis of the effects of math preparation. In the first, we
explore the effect of pathways by estimating the mean differences in value-added to student achievement
in math of teachers from different preparation pathways. We net out the effects of student, classroom and
8
12.
school influences from the effects of preparation pathway. The model for estimating pathway effects is
based on the following equation:
Aijst = β0 + β 1Aijs(t-1) + Xitβ 2 + Cijstβ 3 + Tjstβ 4 + Πj + νs + ε ijst (1)
Here, the achievement (A) of student i in year t with teacher j in school s is a function of his or her prior
achievement, time-varying and fixed student characteristics (X), characteristics of the classroom (C),
characteristics of the teacher (T), indicator variables (fixed effect) for the preparation pathway, e.g.,
College Recommending, the teacher completed (Π), a fixed-effect for the school (ν), and a random error
term (ε). Student characteristics include race and ethnicity, gender, eligibility for free or reduced-price
lunch, whether or not the student switched schools, whether English is spoken at home, status as an
English language learner, the number of school absences in the previous year, and the number of
suspensions in the previous year. Classroom variables include the averages of all the student
characteristics, class size, grade, and the mean and standard deviation of student test scores in the prior
year. All pathway effects are estimated relative to Math Immersion.
Because the field is not settled on the appropriate specification for estimating student
achievement gains, we estimate a variety of alternative specifications. Instead of estimating current
achievement as a function of prior achievement, we employ achievement gains. For each of these models
we substitute student fixed effects for school fixed effects. All models cluster errors at the teacher level.
Whether or not to include teacher characteristics depends upon the question at hand. If we want
to know whether teachers from Math Immersion are more effective than teachers from another pathway
then there is no reason to include fixed teacher characteristics, such as SAT or certification exam scores.
In fact, the benefit of one pathway may come from its ability to recruit and select high quality candidates.
However, if we want to separate the selection from the preparation aspects of programs, then it is
important to control for teachers’ initial characteristics. These controls are particularly important for the
parts of our analysis that look at the effects of program characteristics on preparation, as opposed to
programs overall. The teacher characteristics that we include are age, gender, race and ethnicity, whether
they passed their general knowledge certification exam on the first attempt, SAT scores and a series of
indicator variables summarizing the ranking of their under graduate college. We estimate a variety of
alternative specifications for Equation 1, including: using gains scores as the dependent variable while
omitting lag scores as independent variables, employing student fixed effects rather than school fixed
effects and by limiting the sample to only individuals who begin teaching in New York City in 2004 or
In addition to exploring the average effects of pathways, we are interested in a series of related
questions. How does the effect of pathways differ based on teaching experience—that is do the students
of novice teachers in Math Immersion experience different achievement gains from the students of novice
9
13.
teachers in other pathways and how do these patterns change as teachers become more experienced? To
examine this question we interact pathways with teaching experience for each of the first four years of
IV. RESULTS
In this section we address each of the three research questions in turn.
Question 1: How does the background and preparation of Math Immersion teachers compare to
math teachers entering through other pathways?
Attributes of Math Teachers: There are meaningful differences between the attributes of math
immersion teachers and teachers who enter through pathways other than NYCTF, particularly the College
Recommending pathway. As shown in Table 2, Math Immersion teachers, both those teaching in high
school and middle school, are a more diverse group of teachers than their College Recommending
peers—they are substantially more likely to be male, Black and Hispanic. They also tend to perform
better on most measures of academic ability, including the math and verbal SAT exams, the Liberal Arts
and Sciences Test (LAST), New York’s general knowledge certification exam, and the math/science sub-
score of the LAST, although they perform slightly worse on the Content Specialty Test in Math (CST
Math) and the secondary pedagogy exam (ATS Secondary). Not surprisingly Math Immersion teachers
are fairly similar to other NYCTF teachers but perform less well on all measures of academic ability than
TFA math certified teachers.
Many of the Math Immersion teachers who become math certified either have a math related
undergraduate major (49 percent) or math related work experience (19 percent).7 Although it appears
that a substantial percentage of Math Immersion teachers do not have math related majors or work
experiences, we do not have information on college course work which is another way candidates may
have met the Math Immersion eligibility criteria. As shown in Table 3, among Math Immersion teachers
there are some differences between those with math related backgrounds and those without such
backgrounds. On many measures, however, Math Immersion teachers who do not have math related
backgrounds have qualifications that are at least as strong, and sometimes even stronger, when compared
to those with math related backgrounds.
NYCTF math teachers and the subcomponent of Math Immersion teachers are prepared at several
different institutions. Table 4 shows that four campuses are responsible for the vast majority of these
teachers. There are many similarities, but some interesting differences across the attributes of math
We obtain information about undergraduate major and work experiences based on a program information obtained
from the New York City Teaching Fellows.
10
14.
certified teachers prepared at these campuses. 8 Table 5 shows that many of the demographic
characteristics are very similar across campuses, though Campus C’s teachers tend to be somewhat older
and are more likely to be male, while Campus A’s teachers are more likely to be Black. There is
remarkable consistency across many of the measures of ability, with the exception that Campus C’s
relatively small Math Immersion program has teachers who outperform several other campuses on the
pedagogy exam. On the SAT math and verbal tests, Math Immersion teachers at Campus Z perform
better, while those at Campus A appear to perform worse than the other campuses.
Among the College Recommending programs, a similarly small number of campuses account for
most of the math certified teachers. Three institutions R, S, and T account for 40 percent of all the math
certified teachers produced by College Recommending programs hired by New York City schools over
the five years 2004-08. Each year, most programs produce only a handful of math certified teachers who
are hired in New York City.
Differences in Preparation Between Math Immersion and College Recommending Pathways: Our
reviews of program requirements across 25 College Recommending and 5 Math Immersion programs
suggest that there is relatively little variation between pathways but substantial variation within each
pathway with regard to required coursework. Table 6 shows the average number of courses and course
credits required across several key components of pathways, where we have separated the graduate and
undergraduate College Recommending programs. As these results show, the average Math Immersion
program requires roughly as many or more courses and credit hour in most components of the programs,
including math content and math methods, as either the average graduate or undergraduate College
Recommending program. There are two exceptions. The undergraduate College Recommending
programs require more classroom management and learning about learners than do Math Immersion
programs (1.75 credit hours v. 0.6 credit hours for classroom management and 4.5 v. 2.4 credit hours for
learning about learners). College Recommending graduate programs are between the other groups on
These findings are often, but not always, supported by our survey of teachers regarding their
perceptions of the preparation they received in their programs. Table 7 presents the results of regression
analyses where factors created from teachers responses to survey questions regarding their perceptions of
the opportunities they had to engage in various preparation activities during pre-service education are
regressed on preparation pathways where all pathways are relative to the Math Immersion pathway as
well as school context factors. As shown, teachers from College Recommending programs cite
significantly greater general opportunities to learn about the teaching of math, preparation in specific
Pseudonyms are provided for the campuses in order to protect the confidentiality of the institutions and
participating faculty.
11
15.
teaching strategies, greater quality of field experiences and more opportunities to learn preparation for
working with special education students. There is no difference in perceptions of opportunities to learn
math content between College Recommending teachers and Math Immersion teachers. It is also the case
that Teach for America teachers report more opportunities to learn in specific strategies and better field
experiences but less opportunity for math subject matter preparation, as was the case with regular
Teaching Fellows. Regular Teaching Fellows also report fewer opportunities to learn teaching of math
but more opportunities in the preparation of specific strategies. Again, it is important to remember that the
survey asked specifically about opportunities to learn prior to entering the classroom as a full-time
teacher; teachers in both TFA and NYC Teaching Fellows, including Math Immersion, were still taking
courses to fulfill program requirements.
Although we find only modest differences in the average program requirements between Math
Immersion and College Recommending programs, we do observe much greater differences among
programs within each pathway.
Variation Within Preparation Pathways: While Math Immersion is in some senses a single
program, the preparation experiences of NYCTF-MI teachers can be quite different depending on which
institution they attend. College Recommending programs also establish differing program requirements
within the broader requirements established by New York State. To understand the preparation in each
program, we accessed program documents and accreditation materials as well interviewed program
directors and field coordinators.
A Math Immersion Teaching Fellow could be prepared in mathematics and general pedagogy in
very different ways, depending upon the campus at which he or she was prepared. As Table 8 suggests,
the programs vary in terms of their course requirements.9 There are three telling aspects of this analysis.
First, there are remarkable differences across campuses in their math content and math methods
requirements, ranging from one 3-credit course in math content required by at Campus Z to 5 or more
courses required by Campuses A, B and C. The range in requirements for math methods was smaller. In
sum, Math Immersion Fellows could receive different emphasis on math content or math methods
depending on the campus they attend. Second, there is a range of requirements in general pedagogy10
across these programs. As seen in Table 8, only two of the five campuses required courses on assessment,
and, despite the continued emphasis upon and discussions about the role technology should play in
Our categorization of the courses (whether they are considered subject matter content courses or methods; whether
they are general pedagogy courses, or courses about learners) is based upon and consistent with an earlier analysis
we conducted on childhood education programs at many of these same institutions.
“General pedagogy” in our analysis refers to any courses that were not specific to the teaching of a content area,
but rather had to do with general issues of teaching—such as coursework in technology, assessment;
interdisciplinary or general methods courses that did not focus upon a particular discipline; courses in literacy across
the content areas.
12
16.
teacher education programs, only one campus required coursework in technology. Finally, of the five
Fellows campuses, four programs required at least one course in learning or child development.11
However, again, as with the preparation in other areas reported thus far, the requirements in learning
range substantially. Variation across the other components of preparation programs was not meaningful.
In sum, the most striking variation across programs lies with whether programs put greater
emphasis on math content and methods, or more emphasis on more general preparation for teaching that
was not specific to teaching mathematics topics, courses or issues. For instance, two of the Math
Immersion fellows programs are structured around heavier requirements in general courses on pedagogy
and learners and learning (Campus Z and Campus D), and require fewer courses in math and math
methods. Campus Z has particularly weak requirements in Math content. Campus Z program requires 3
credits in mathematics content, and 6 credits in methods; these requirements represent 9 of the total of 39
credits, or 23 percent of the total required. On the other hand, at Campus C, math methods and math
content credits represent 30 of the required 47 credits, for 63 percent of the total requirements. Two
campuses stand out for their curricular emphasis on math content and math methods in their course
requirements: Campus C and Campus A.
We also examined program documents and interview program administrators of College
Recommending programs in mathematics who supply the majority of math teachers from College
Recommending programs for New York City public schools (See Table 9). The programs we reviewed
included a total of 25 programs at 16 campuses, 14 of these programs were graduate programs, 11 were
undergraduate programs. Of the 16 institutions, 10 are private and 6 are public. All of the institutions
that offered NYCTF Math Immersion programs also offered College Recommending programs in
We find a substantial range in requirements in mathematics content. For graduate programs in
the teaching of mathematics, requirements ranged from no courses required in math content, to five
courses in math content (See Table 9). In part, these lower requirements in math content may be due to
the fact that a number of the graduate programs required math preparation prior to entry—in many of
these programs, incoming applicants were required to have been math majors, although there is
substantial variation among undergraduate programs in math content, too. In terms of math methods
courses, we find a similar range with regard to requirements; almost half of the programs required just
one mathematics methods course and four programs required either three or four courses. In sum, the
range of requirements in math methods appears to be somewhat similar to the range seen in the Math
In this category, consistent with prior analysis, we included courses that focus upon learners and learning; courses
on child development; courses on classroom management; courses on diverse learners or diverse language learners;
and courses on children with special needs.
13
17.
Immersion programs. The variation in requirements for preparation in learning and learners and that in
classroom management in College Recommending programs also is similar to that in Math Immersion.
As summarized in by the standard deviations of required courses and credit hours for Math Immersion
and College Recommending programs (Table 6), the variation of within pathway course requirements
substantially exceeds the variation between pathways. This is perhaps not surprising in that New York’s
alternative preparation pathways are best characterized as allowing for differences in the timing of
meeting requirements rather than allowing for different requirements.
In light of our program analysis which reveals that one program, Campus Z, stands out as having
the fewest requirements in math-related preparation to teach, we examine the results of the survey
comparing the responses of students from campus Z to students from the other Math Immersion
campuses. To explore differences among Math Immersion programs across our measures of teacher
preparation, we estimate models including indicator variables for each campus within the Math
Immersion pathway where the comparison group is teachers prepared at Program Z. Because a teachers’
perspective on her preparation may be influenced by the context in which she is teaching at the time she
completes the questionnaire, we also estimate models that include school context factors as controls.
As compared to teachers from Campus Z, Table 10 shows that teachers from other campuses
score higher across survey factors measuring preparation program attributes. Though the coefficients are
only sometimes statistically significant, they are consistently positive. When we group together all other
campuses and compare them to Campus Z (bottom row), teachers from all other campuses report having
significantly more opportunities to learn teaching math and more preparation to use specific teaching
practices, however there are no differences in their perceptions of opportunities to learn math. These
results are consistent with many, but not all, of the findings from our program review. Additionally,
teachers from other campuses report higher quality field experiences.
Based on our review of the structure and content in Math Immersion and College Recommending
preparation programs in mathematics and based on teacher reports of their preparation, there appears to be
substantial variation within and across pathways. We now explore whether different pathways influence
gains in student achievement outcomes.
Question 2: How do the achievement gains of the students taught by Math Immersion teachers
compare to those of students taught by math teachers entering through other pathways?
Are teachers entering teaching in New York City through the Math Immersion program more or
less effective than math teachers from other pathways? Based on their preparation and their background
there are reasons to believe that NYCTF-MI teachers may have different effects on students than do other
teachers. By definition, Math Immersion teachers do not have an undergraduate major in their subject
14
18.
area, which is commonly required for teachers entering through the College Recommending pathway.
However, Math Immersion teachers also tend to have stronger academic credentials than teachers from
other pathways with the exception of those entering through Teach for America. To explore the relative
effectiveness of Math Immersion teachers in improving student achievement outcomes, we estimate
several value-added models for students taking standardized math achievement exams in grades 6-8.
We should note that to more fully examine math achievement we would like to have value added
measures for high school mathematics but such data do not currently in exist in New York City, or most
other districts. This does raise a potentially important methodological issue of the placement of math
teachers between middle school and high school. There is anecdotal evidence that many math teachers
prefer to teach in high school and that many preparation programs steer their strongest students toward
teaching positions in high schools, where content knowledge may be even more important. To assess
whether there is any evidence of this and more importantly if such placements differentially affect some
pathways or programs (a sample selection issue), we examine the qualifications of high school and middle
school math certified teachers by pathway in Table 2 and by program in Table 5.
As shown in Table 2, the qualifications of math certified teachers over the 2004-08 period is
generally stronger for teachers in high school than those in middle schools across each pathway. For
example, the College Recommending teachers in high school have SAT math scores that are 7.9 percent
higher than College Recommending teachers in middle school, while comparable differences for Math
Immersion and TFA are 4.6 percent and 9.6 percent respectively. The differences for the Content
Specialty test are 4.4 percent for College Recommending, 2.4 percent greater for Math Immersion and no
difference for TFA. To the extent that these measures of qualifications have some predictive ability of a
teacher's value added, then we would expect high school teachers from each pathway to more effective.
However, these differences do not suggest that one pathway is being systematically affected by teacher
sorting to high school. Similar comparisons can be made among the Math Immersion programs. As
shown in Table 5, each of the Math Immersion programs places teachers with somewhat stronger
qualifications in high school relative to the teachers from their pathway who teach in middle school.12
These differences vary but across every measure Program Z has the smallest difference between middle
and high school teacher, suggesting the Program Z's middle school teachers may be relatively more
effective compared other pathways than its high school teachers.
In general we find that most of the independent variables characterizing individual students, the
class of the student, and the experience of teachers produce math achievement gains in grades 6 through 8
It is also the case that Math Immersion teachers who do not meet internal acceptance standards but who were
admitted due to excess demand are somewhat more likely to teach in middle school than high school compared to
their colleagues who met internal recruitment standards (58 v. 52 percent). (Correspondence with Vicki Bernstein,
New York City Department of Education, 9/14/09.)
15
19.
as suggested by theory and found in most other research employing administrative data (see Table 11).
All of the student attributes affect achievement. For example, prior achievement is an important predictor
of current achievement, Asian students outperform whites, while Black and Hispanic students have lower
achievement than whites. Students who have changed schools perform substantially more poorly than
those who are not mobile, as do students with more absences and suspensions, other things equal. The
attributes of class peers also influences student achievement in the expected ways. As has been found in
several previous studies, increasing experience as a teacher improves student math achievement for the
first four or five years, with additional experience having no meaningful effect on achievement. This
effect includes both changes in an individual teacher’s ability to improve achievement and the changing
composition of the workforce. If teachers who are less effective are disproportionately more likely to
leave middle school math classrooms then at least some of the gains to experience may reflect this
The focus of this research is the effect of the pathway through which a teacher enters teaching,
and in particular the relative effect of math immersion, the omitted pathway in the estimates found in
Table 11. These estimates suggest that on average, students of Math Immersion teachers in grades 6-8
have smaller gains in math achievement than students of teachers from the College Recommending,
Teaching Fellows, and TFA pathways. Coefficients reflect effect sizes. In gauging effect size
magnitudes it is useful to compare coefficient estimates to the effect of student gains produced by the first
year of teaching experience, which most observers regard as important to student achievement. In this
context, the effect of having a Teach for America teacher relative to a Math Immersion teacher is roughly
the same as the first year of teaching experience (about 0.05). The additional achievement of students of
College Recommending (0.016) and regular Teaching Fellows (0.021) relative to Math Immersion
teachers is estimated to be about 40 percent as large as the first year of teaching experience, and in models
with school fixed effects these estimates are statistically significant at the 10 percent level.
Although there are significant differences between the mean effects of some of the pathways,
there is also substantial overlap of the distribution of teacher value added. Figure 3 shows the distribution
of the teacher fixed effects by pathway.13 The distribution of TFA teachers is generally shifted to the
right, but they also have a meaningful number of relatively more effective teachers as indicated by the
The figure plots the persistent component of a teacher’s effectiveness by employing an empirical Bayes estimator
similar to that suggested in Kane, Rockoff and Staiger (2008). The estimate of teacher effectiveness results from a
regression of student math achievement identical to equation 1 with teacher experience as the only measure of
teacher attributes. The residuals from this regression are shrunken to adjust for the measurement error associated
with the estimates. We should note that while the estimates of effectiveness for each individual teacher are
unbiased, the estimates by pathway taken together to form the distribution of teacher effectiveness over adjusts the
overall distribution of teacher effects. Even so, there is substantial overlap among the pathways.
16
20.
bump in the distribution between effect sizes of 0.4 and 0.6. Although the distributions diverge in some
interesting ways, it is clear that most of the teachers from one pathway are indistinguishable from teachers
who entered through other pathways.
To explore the robustness of these findings, Table 12 compares these estimates across a variety of
model specifications. We examine the consequences: of employing student fixed effects rather than
school fixed effects, of including teacher controls (age, gender, race and ethnicity, whether they passed
their general knowledge certification exam on the first attempt, SAT scores and a series of indicator
variables summarizing the ranking of their under graduate college), and of employing achievement gains
rather than levels as the dependent variable. In general, the effect of gains rather than levels result in only
minor changes in the estimated effects of pathways (columns 1, 3 versus 2, and 4). Similarly employing
student fixed effects rather than school fixed effects as controls changes the estimated coefficients in
small ways, though the regular Teaching Fellows and College Recommending pathways are now
statistically significantly different from Math Immersion at the 5 percent level or better (e.g., column 1 v.
However, due to excess demand from 2004-08, the NYCTF program accepted some applicants
who fell below their internal selection standards. During this period 9 percent of the math immersion
teachers who taught students in our value-added analysis did not met these standards (NYCTF-MI
Below), 51 percent met these criteria (above) and 40 percent did not receive a rating (NYCTF-MI NA).
As shown in column 9 of Table 12, these ratings identify meaningful differences in Math Immersion
teachers. The comparison group is now the Math Immersion teachers who exceeded the selection
threshold. These teachers are on average relatively more effective that their colleagues who were rated
below the threshold (0.044), although the difference is not statistically significant. The difference
between Math Immersion and College Recommending is eliminated when compared to the Math
Immersion teachers who exceeded the threshold and the difference with TFA is reduced. Our best
estimates of the effect of Math Immersion are those presented in column 1, but the results of column 9
indicate that excess demand for math teachers during those years plays a role in the differences between
Math Immersion and other pathways.
Including teacher controls substantially reduces the magnitude of the pathway coefficient
estimates (Table 12, columns 3, 4, 7 and 8) . In general we believe that teacher preparation programs
perform two functions—selection and preparation, and should be judged on the combined effect.
However, we also find it interesting to attempt to disentangle these components by including teacher
controls that can be viewed as proxies for variables programs use in determining admissions. Admittedly
these are not great controls for the characteristics that likely differentiate teachers at point of application.
However, the effect of including the teacher controls that we can observe has the effect of reducing the
17
21.
TFA pathway effect by more than half (0.055 to 0.018) in the model estimated in levels with school fixed
effects. This is consistent with the notion that TFA is very good at recruiting and identifying teachers
who are ultimately effective in producing achievement gains. This also suggests that our proxies for
teacher qualifications are important in improving student achievement.14 In addition, we estimate the
same models presented in Table 12 but limiting the sample to only teachers who began their careers in
2003-04 or later. These results are very similar to those presented in Table 12, however they indicate that
College Recommending teachers outperform Math Immersion teachers (effect size =0.035).15 To
understand this result better, we explore the relationship between experience and pathway in more detail.
The timing of teacher preparation is much different for teachers entering through alternative
certification pathways such as Math Immersion than for teachers entering through College
Recommending programs in New York. State certification requires both pathways to meet essentially the
same requirements but at different points relative to becoming the teacher of record. While College
Recommending teachers meet all of the requirements for an initial teaching license prior to becoming a
classroom teacher, alternatively certified teachers in New York complete an intensive pre-service
component during the summer prior to becoming a classroom teacher, then enroll in a masters program in
education that is typically completed during the first two to three years as a teacher.16
Due to these timing differences, it is useful to explore how the effects of pathways may differ
systematically with the early years of teaching experience. We might expect that teachers entering
through alternative certification pathways might be less effective in their first year or two of teaching but
that the gap would close as they both gained more experience and completed their preparation
requirements.17 Table 13, shows the interaction effects of pathway and experience for a variety of model
specifications. The comparison group is first year Math Immersion teachers. As is expected, the
effectiveness of teachers in all pathways increases with experience.18 Table 14 provides an easier means
of comparing the relative effectiveness of each pathway at each level of experience. Table 14 shows the
difference at each level of experience between each pathway and Math Immersion and whether that
difference is statistically significant. Students of Math Immersion teachers typically have smaller math
achievement gains at every level of experience than those of College Recommending and Teaching
Fellows teachers. However, these differences are typically not statistically significant at the 10 percent
Boyd et al. 2008 explore the effect of teacher qualifications in detail.
Full results available from the authors.
For more details on certification requirements in New York State, see
http://www.highered.nysed.gov/tcert/certificate/typesofcerts.htm
In earlier work, we found precisely this result (Boyd et al, 2006).
Based on these estimates we can distinguish whether these gains to experience reflect teachers becoming more
adept at improving student achievement over time or a composition effect of less effective teachers leaving the
workforce. Based on other work we believe that both explanations contribute to the results presented.
18
22.
level. Math Immersion teachers are estimated to be less effective than TFA teachers at each level of
experience, although these effects are statistically significant only in the first and second years, which
likely reflects the small sample sizes in both groups, as the point estimates remain relatively large.
However, these differences largely disappear when we include variables intended to measure teacher
qualifications. Math Immersion teachers appear to be more effective than teachers in the Other category,
although these differences are statistically significant only in the first two years without the teacher
Our earlier analysis of the structure and content of the preparation that Math Immersion Teachers
received revealed substantive variation across the five programs that prepared the vast majority of Math
Immersion Teachers. Further we found some differences in the students who participated in each of these
programs. To explore whether these differences resulted in differential student achievement gains, we
estimated models that included all pathways but also identified the specific institutions through which
Math Immersion teachers were prepared, see Table 15. Here teachers enrolling at Campus Z are the
comparison group. These results suggest that Campuses B, C and E appear to outperform Campus Z in
most model specifications and Campus D does so less consistently. When Campus Z is eliminated from
the estimation of pathway effects (Table 11) there are no differences between College Recommending,
Teaching Fellows and Math Immersion teachers. Students of TFA teachers have substantially better math
achievement than those of teachers from the other pathways.19 Taken together, these results suggest that
the specific implementation of Math Immersion in programs can importantly affect teacher preparation
and resulting student achievement.
In trying to understand the relatively less effective performance of teachers from Campus Z, we
refer back to our analysis of program requirements and of the survey results. As described above,
Campus Z had the fewest requirements in math and math methods of all the Math Immersion campuses,
while Campus C had the greatest followed closely by A and B. Given the few programs training Math
Immersion teachers, we can not hope to make causal statements of the effects of program design on
outcomes, but these results do suggest that the relative focus on math content and math pedagogy offered
by a program may influence a teacher’s ability to improve math achievement.
Question 3: How does the retention of Math Immersion candidates compare to math teachers
entering through other pathways?
The students of individuals who enter teaching through the Math Immersion program appear to
have math achievement gains that are somewhat lower than those of College Recommending and
substantially lower than TFA teachers, other things equal. Most policy makers appropriately place great
Results available from authors on request.
19
23.
weight on student outcomes as means of evaluating alternative policies and programs. Increasingly,
teacher attrition has become an important issue and there is concern that individuals who enter teaching
through alternative certification routes, such as Math Immersion, are less likely to remain in teaching.
Teacher attrition is potentially troubling for several reasons—there is very strong evidence that the
effectiveness of teachers improves during their first four or five years (see Rockoff, 2004; Rivkin et al,
2005, Boyd et al. 2008b) and as a result losing teachers who have gained experience directly influences
student achievement, other things being equal. There are indirect effects as well. High turnover rates
make it difficult for school leaders and teachers to work together effectively thus compromising the
learning environment. Finally, the costs associated with recruiting and mentoring new teachers represents
a substantial investment that could easily be employed in other ways (see, for example, Barnes et al.
We employ personnel files from the New York City Department of Education to explore teacher
attrition. These files identify each time a teacher changes status, e.g., retire, transfers schools, take a leave
of absence, etc. Using these data we define a teacher in any given year as someone employed as a teacher
as of October 15th of that academic year.20 Teachers are defined as remaining in the same school if their
personnel records indicate they began the next academic year teaching in the same school; they are
defined as having transferred to another school in NYC at the beginning of the next academic year they
are a teacher in a different school; and they are defined as leaving teaching in New York City public
schools if personnel records show they have retired, quit or on leave and not returning for more than one
Descriptive statistics characterizing the attrition rates for math-certified teachers by pathway in
grades 6 through 12 are shown in Table 16. Math Immersion teachers had relatively low first year attrition
but in years 2 through 4, Math Immersion teachers, like teachers from other alternative certification
pathways experienced a higher likelihood of transferring and leaving the New York City public school
system. By the end of what would have been their fourth year, more than 40 percent of Math Immersion
teachers have left teaching in New York City and fewer than a third remain in their original school. This
is meaningfully higher attrition than College Recommending teachers, 31 percent of whom have left New
York City teaching while about half remain in their original school. Math Immersion teachers persist in
teaching at somewhat greater levels than other New York City Teaching Fellows, and at much greater
This definition would exclude individuals in a year who may be teaching under some other title, such as a
substitute teacher; those who are not teachers, and an individual who began teaching in a given year after October
15th. Individuals who began after October 15th and who continued as a teacher in the subsequent year are included
for that year.
There are cases where individuals are not teachers in NYC public schools for more than a year and subsequently
return to teach, but these cases are relatively rare. It is also true that teachers who have left teaching in NYC may be
teaching in other school districts or in an administrative position in NYC.
20
24.
levels than Teach for America teachers. By the conclusion of the fourth year, nearly 80 percent of TFA
teachers have left teaching in New York City public schools, while fewer than 10 percent remain in their
original schools.
How would the academic gains of students differ as a result of school officials systematically
filling job openings by hiring teachers entering through one pathway versus another? The answer, in part,
depends upon the relative effectiveness of teachers at each level of experience across pathway as
discussed above. However, it is also necessary to account for differences in retention rates across
pathways. This follows from the meaningful gains in teacher value-added associated with increased
experience over the first few year of teaching. If one pathway consistently has higher turnover even if its
teachers do well relative to those in other pathways with the same experience, the pathway may not be
providing the most effective teachers, on average.
How does the average value-added of teachers vary across pathways once differences in teacher
retention rates are taken into account? We address this question using the following simulation. Suppose
that school officials hired an arbitrary number of new teachers (e.g., 1000) from each of the pathways. For
subsequent years, the teachers hired from each pathway are allowed to age through the experience
distribution, applying the pathway dependent retention rates implied in Table 16. Teachers who leave are
replaced by teachers with no prior experience from the same pathway. These new hires in turn age
through the system. In this way, it is possible to simulate how the experience distribution of teachers from
each pathway would evolve over time and differ across pathways thus allowing us to estimate how such
differences affect the average value-added of the teachers from each pathway. These results are shown in
Table 17. The most striking result is that the clear advantage that TFA teachers had at every level of
experience (see the value added estimates from Table 13 replicated in the bottom panel of Table 18)
dissipates as the very high attrition of TFA teachers following their second and third years of experience
causes many more TFA teachers to be replaced by novices. Because of its lower attrition the College
Recommending pathway develops a small advantage relative to the Math Immersion and is roughly
equivalent to regular the Teaching Fellows and TFA pathways.
V. Conclusion
Math Immersion was born of necessity to assist in filling the vacancies when uncertified teachers
were barred from teaching and insufficient numbers of College Recommending or alternatively certified
teachers who met the existing math certification requirements were available to teach in New York City.
Remarkably four years since its inception, the Math Immersion preparation pathway supplies 50 percent
of all new certified math teachers to New York City public schools. Given the prominence of the Math
Immersion pathway in supplying math teachers to NYC schools, it is important to examine the design of
21
25.
the program and its effects on student achievement.
In general, we find that Math Immersion teachers have stronger academic qualifications, e.g.,
SAT scores and licensure exam scores, than their College Recommending peers, although they have
weaker qualifications than Teach for America teachers. In addition, Math Immersion teachers are found
in some of the most challenging classrooms in New York City. In this respect, the program has
succeeded in attracting teachers with stronger academic backgrounds to teach in high needs schools.
However, despite stronger general academic qualifications Math Immersion teachers are
responsible for somewhat smaller gains in math achievement for middle school math students than are
College Recommending teachers, although in many cases these differences are not statistically
significant. Math Immersion teachers have substantially smaller gains than Teach for America teachers.
These results are robust to a variety of alternative specifications. However, Math Immersion teachers are
more likely to leave teaching in New York City than are their College Recommending peers, but
substantially less likely to do so than Teach for America teachers. In simulating the impact of attrition on
the effectiveness of different pathways, the College Recommending pathway develops a small advantage
relative to Math Immersion but is roughly equivalent to Teach for America and regular Teaching Fellows.
Based on the value-added and attrition results, one might be tempted to conclude that New York
City should be hiring more TFA and College Recommending teachers and looking to dismantle the Math
Immersion program. However, such a conclusion ignores the fact that for many years prior to the
creation Math Immersion New York City hired a very large number of uncertified teachers; many of these
teachers taught middle and high school math classes precisely because there were insufficient numbers of
College Recommending teachers certified in math who were willing to staff these low-performing
schools. While the number of math teachers prepared through College Recommending programs has
increased in recent years, these programs are still not preparing sufficient math teachers to fill the
demand. Additionally, due to reduced demand for teachers beginning in 2008-09, the Math Immersion
program has been able to raise the standards by which it accepts applicants. It will be interesting to assess
whether this change affects the average effectiveness of new cohorts.
Recruiting and preparing high quality teachers to meet the demand of K-12 schools is a massive
undertaking and many high needs schools have found it very difficult to recruit and retain effective
teachers. While there is a great deal to learn regarding the effective recruitment and preparation of
teachers, there is already ample evidence that each pathway produces teachers who range in effectiveness,
with some very effective teachers and some teachers who are less so. Similarly, within pathways
programs vary in their effectiveness. This suggests that the policy discussion about teacher preparation
should be focused on the features of programs and pathways that contribute most importantly to
successful teachers and not whether one pathway outperforms another. Rather we believe that
22
26.
policymakers are well advised to invest in the development of programs that draw on the most promising
features of the more successful existing programs.
As we have argued earlier, programs can influence their outcomes through both the recruitment
and selection of promising candidates and strong preparation. The analysis in this paper suggests that on
average TFA teachers produce student achievement gains in middle school math that exceed those of
teachers from other pathways with comparable experience. TFA has invested heavily in the recruitment
and selection of its Corps members and this effort appears to account for a substantial portion of the
difference between TFA and Math Immersion or College Recommending teachers. However, this
advantage is largely eliminated once the much higher attrition of TFA teachers is taken into account.
Additionally, TFA recruits far fewer teachers into New York City schools than do either the Teaching
Fellows or College Recommending pathways. However, other programs could learn from TFA regarding
the selection of candidates who are effective teachers in low-performing classrooms.
Selection, however, is only one part of the equation. We also suspect, although we have only
limited evidence to support the hypothesis, that a teacher’s preparation in math content and pedagogy may
influence the math achievement of his/her middle school students. We found evidence regarding the
positive influence of math content and the nature of field experiences when we examined the attributes of
teacher preparation programs in childhood education (Boyd et al. 2009). The somewhat weaker
performance of Math Immersion teachers relative to College Recommending teachers in light of the
stronger academic skills of Math Immersion teachers also may suggest that preparation can improve
teacher effectiveness; and the TFA advantage in middle school mathematics may in part signal the
importance of strong math content knowledge as well. In addition, the more circumstantial evidence on
the impact of a program with limited content preparation suggested by the weak effects of program Z also
suggests that programs invest in math-specific preparation, in both content and pedagogy.
One of the implications of this line of reasoning is to design and evaluate programs that combine the
recruitment of academically strong candidates with high quality preparation in math content, math
pedagogy, and field experiences that provide them with opportunities to observe effective teachers and
practice their teaching skills in closely supervised classrooms of high needs students. Another
implication to explore is the notion that the availability of teachers from a variety of pathways benefits
schools that have been traditionally difficult to staff because each pathway is able to recruit some good
teachers for these schools. The variability of teachers within each pathway points both to the importance
of better understanding effective recruitment and preparation and to the importance of monitoring and
supporting teachers once in the classroom.
Improving the quality of math teaching in our schools will require more systematic and rigorous
evaluation of the selection and preparation components of teacher education. State departments of
23
27.
education must take the lead in these efforts, given their role in determining teacher licensure
requirements. The federal Race to the Top initiative provides states with the policy and financial leverage
to embrace this challenge.
24
28.
Figure 1: Number of Teachers Entering New York City Public Schools by Pathway, 2002-
4,500
4,000
3,500
CR
3,000
2,500
2,000
NYCTF
1,500
TL
1,000
TFA
500
0
2002 2003 2004 2005 2006 2007 2008
Figure 2: Number of Entering Math Certified Teachers New York City,
by Pathway, 2002-2008
450
400
350
NYCTF-MI
300
250
CR
200
150
100
NYCTF
50
TFA
0
2003 2004 2005 2006 2007 2008
25
29.
Figure 3: Distribution of Teacher Value Added by Pathway, with Empirical Bayes
Shrinking, 2004-2008
0
-0.6 -0.4 -0.2 0 0.2 0.4 0.6
CR NYCTF-MI TFA
26
30.
Table 1: Attributes of Students Taught by First-year Teachers by Pathway, Grade 8, 2006
Student Attributes CR NYCTF NYCTF- TFA other
MI
Lagged Math Achievement 0.238 -0.125 -0.051 -0.139 -0.061
Proportion Black 0.292 0.277 0.322 0.442 0.403
Proportion Hispanic 0.358 0.496 0.493 0.527 0.372
Proportion Free Lunch 0.547 0.664 0.635 0.619 0.66
Classsize 27.6 27.8 26.9 26.3 26.1
Lagged Student Absences 12.3 13.4 13.1 14.8 13.5
Lagged Suspensions 0.037 0.064 0.062 0.023 0.042
Table 2: Attributes of Entering Math Certified New York City Teachers by Pathway, 2004-2008
CR NYCTF NYCTF-MI TFA
Teacher Attributes High Middle High Middle High Middle High Middle
School School School School School School School School
Female 0.648 0.732 0.446 0.563 0.479 0.546 0.492 0.551
Black 0.073 0.105 0.130 0.197 0.142 0.200 0.082 0.141
Hispanic 0.065 0.046 0.068 0.066 0.085 0.074 0.066 0.043
Age 29.7 28.9 30.4 29.1 31.1 30 23.6 23.5
Last Score 255 251 273 268 274 271 279 279
CST Math Score 262 251 268 263 257 251 268 269
SAT Math 600 556 626 611 616 589 710 648
SAT Verbal 506 483 580 545 577 564 627 623
Number of Teachers 478 157 195 64 1098 542 61 98
27
31.
Table 3: Attributes of Entering Math Certified Math Immersion Teachers by Whether They
Had Math Related Major or Math Related Work Experience, 2004-2008*
Math Not Math
Teacher Attributes Related Related
Female 0.444 0.556
Black 0.192 0.132
Hispanic 0.090 0.073
Age 32.0 29.5
Last Score 270 277
Last Science/math sub-score 275 283
CST Math Score 255 254
ATS Secondary Score 251 253
SAT Math 594 622
SAT Verbal 554 595
* Coded as math related if individual had either math related undergraduate major or math related work
experience, not math related otherwise if not missing. Math related majors included: math, accounting
astronomy, biochemistry, biology, business, chemistry, computer science, economics, engineering,
finance, information systems, physics, and statistics. Math related work experiences included:
engineering, financial, and public accounting.
Table 4: Number of New York City Teaching Fellows Prepared by Various Campuses by
Math Immersion and Math Certification Status, 2004-2007
All Teachers by Institution
Math Immersion
Status A B C D Z
NYCTF-MI 290 536 75 270 441
NYCTF-Not MI 1082 1077 751 185 1431
Total 1372 1613 826 455 1872
Math Certified Teachers by Institution
A B C D Z
NYCTF-MI 290 536 75 270 441
NYCTF-Not MI 46 78 19 35 75
Total 336 614 94 305 516
28
32.
Table 5: Attributes of Entering Math Certified NYCTF Teachers by Math Immersion Status
and Preparing Campus, 2004-2008
A B C D Z
Teacher Attributes High Middle High Middle High Middle High Middle High Middle
Scho School School School School School School School School School
ol
Female 0.484 0.566 0.477 0.558 0.360 0.520 0.468 0.534 0.509 0.538
Black 0.222 0.320 0.139 0.209 0.082 0.240 0.108 0.165 0.125 0.127
Hispanic 0.032 0.062 0.123 0.086 0.102 0.080 0.088 0.046 0.069 0.093
Age 30.4 29.2 31.1 29.2 37.2 30.2 31.7 31.9 29.9 29.7
Last Score 274 270 275 272 271 268 272 268 277 276
Last Science/math sub-score 278 272 281 276 275 280 281 279 284 282
CST Math Score 259 251 255 250 258 243 255 249 257 257
ATS Secondary Score 241 241 262 250 247 276 253 256 245 254
SAT Math 611 563 609 589 633 582 271 262 625 616
SAT Verbal 567 542 578 576 566 594 618 586 589 586
573 542
Number of Teachers 191 99 371 165 50 25 154 116 322 119
29
33.
Table 6: Required Courses and Credit Hours for Key Courses, College Recommending
and Math Immersion Programs, Means and Standard Deviations
College Math Math Classroom Learning Assess- Special Ed Diversity
Recommending Courses Methods Management ment
Programs
Graduate programs
Courses
Mean 1.64 2.00 0.29 1.29 0.50 0.57 0.50
Standard deviation 1.78 1.11 0.61 0.73 0.52 0.65 0.65
Credits
Mean 4.93 5.79 0.86 3.75 1.29 1.71 1.36
Standard deviation 5.34 3.29 1.83 2.16 1.44 1.94 1.91
Undergraduate programs
Courses
Mean 3.82 1.36 0.64 1.73 0.00 0.36 0.36
Standard deviation 3.76 0.50 0.67 0.90 0.00 0.50 0.67
Credits
Mean 11.00 4.71 1.75 4.50 0.25 1.33 1.58
Standard deviation 11.29 1.38 2.26 2.70 0.00 1.66 2.46
Math Immersion
Programs
Courses
Mean 4.20 2.80 0.33 1.00 0.40 0.40 0.25
Standard deviation 1.92 0.84 0.58 0.00 0.55 0.55 0.50
Credits
Mean 12.60 8.40 0.60 2.40 1.20 1.20 0.60
Standard deviation 5.77 2.51 1.34 1.34 1.64 1.64 1.34
30
34.
Table 7: Teachers' Perceptions of Their Preparation by Preparation Pathways, 2005 Survey of First Year Teachers
Preparation in Field General Opps Subject Matter
Specific Experience to Learn Preparation in Preparation for
Pathway Strategies Quality Teaching Math Math SPED students
College
Recommending 0.331 0.441 0.386 0.038 0.358
[2.99]*** [3.91]*** [3.54]*** [0.33] [3.13]***
Teaching Fellows 0.274 -0.052 -0.350 -0.462 0.215
[2.50]** [-0.46] [-3.32]*** [-4.12]*** [1.91]*
Teach For America 0.604 0.810 -0.007 -0.561 0.272
[2.74]*** [3.65]*** [-0.03] [-2.48]** [1.22]
Other Path 0.004 0.230 0.371 0.320 0.436
[0.04] [1.87]* [3.31]*** [2.74]*** [3.73]***
N 558 528 543 541 551
* In addition to the pathway indicator variables each regression contains school context factors, which include a factor representing: teacher
influence on planning and teaching, administrative quality, staff collegiality and support, student attitudes and behavior, school facilities, and
school safety.
31
35.
Table 8: Math Immersion Programs: Key Course Requirements (Courses and Credits)
Campus Math Math Classroom Learning Assessment Special Diversity Technology Total Req’d
Course Methods Mgt Education Credits
Campus A 5 (15) 3 (9) 1 (3) 1 (3) 0 0 0 0 46-49
Campus B 5 (15) 3(9) 0 1(3) 0 0 0 0 48
Campus C 6 (18) 4 (12) 0 1(3) 0 0 0 0 47
Campus D 2 (6) 1 (6) 0 0 1(3) 1 (3) 0 1 (3) 39
+ 2 courses
(6 credits)
prior to
entering
program*
Campus Z 1 (3) 2 (6) 0 1 (3) 1 (3) 1 (3) 1 (3) 0 39
*Program does not pay for or provide for these two prior mathematics courses.
32
+1-315-636-4485
+91 98151-74050 
