Urban Charter School Study Report on 41 Regions

Transcript

1 Urban Charter School Study Report on 41 Regions 2015

2 © 201 5 CREDO Center for Research on Education Outcomes Stanford University Stanford, CA http://credo.stanford.edu CREDO, the Center for Research on Education Outcomes at Stanford University, was established to improve empirical evidence about education reform and student performance at the primary and rganizations and policymakers in secondary levels. CREDO at Stanford University supports education o using reliable research and program evaluation to assess the performance of education initiatives. CREDO’s valuable insight helps educators and policymakers strengthen their focus on the results from innovative programs, cu rricula, policies and accountability practices. Acknowledgements CREDO gratefully acknowledges the support of the State Education Agencies and School Districts who contributed their data to this partnership. Our data access partnerships form the foundati on of CREDO's work, without which studies like this would be impossible. We strive daily to justify the confidence you have placed in us. The views expressed herein do not necessarily represent the positions or policies of the organizations official endorsement of any product, commodity, service or enterprise mentioned in noted above. No this publication is intended or should be inferred. The analysis and conclusions contained herein are exclusively those of the authors, are not endorsed by any of CREDO’s s upporting organizations, their governing boards, or the state governments, state education departments or school districts that participated in this study. The conclusions of this research do not necessarily reflect the opinions or official position of the Texas Education Agency, the Texas Higher Education Coordinating Board, or the State of Texas. i credo.stanford.edu

3 Table of Contents ... Highlights of the Report v Introduction ... 1 y ... 3 Defining Urbanit Data and Methods ... 4 ... 5 Demographics Performance ... 8 Learning Gains by Student Subgroups ... 16 Impact of Urban Charter Attendance on Annual Learning Gains by School Level, Growth Period, and 24 Years of Enrollment ... Sc -level Quality Comparisons ... 26 hool Correlates of Performance ... 34 Implications 37 ... credo.stanford.edu ii

4 Table of Tables 1. Transformation of Learning Gains ... Table 5 6 ... Table 2: Selected Student Demographics by Urban Region and School Sector (Tested Students) Table 3: Impact of Charter Enrollment on Learning Gains Relative to Average Achievement of All Schools in Region – Math ... 12 ... 17 Table 5: Impact of Charter Enrollment on Annual Average Learning Gains for All Urban Regions Table 6: Impact of Charter Enrollment on Annual Learning Gains in Math by Region and Sub -population 20 ... Table 7: Impact of Charter Enrollment on Annual Learning Gains in Reading by Region and Sub - populatio n ... 22 Table 8: Impact of Urban Charter Attendance on Annual Learning Gains by School Level ... 24 Table 9: Impact of Urban Charter Attendance on Annual Learning Gains by Growth Period ... 25 Table 10: Impact of Urban Charter Attendance on A nnual Learning Gains by Years of Enrollment 26 ... level Quality Comparisons – Table 11: School- Region Urban Charter School Study Results and 2013 41- National Charter School Study Results ... 27 Table 12: School- Level Quality Comparisons by Region - Math ... 29 Table 13: School- Reading ... 31 Level Quality Comparisons by Region – Table 14: Correlations between Math or Reading Effect Sizes and Other Factors ... 34 credo.stanford.edu iii

5 Table of Figures Math 9 Figure 1: Impact of Charter Enrollment on Average Annual Learning Gains by Region – Figure 2: Impact of Charter Enrollment on Average Annual Learning Gains by Region – Reading 10 credo.stanford.edu iv

6 Highlights of the Report This report presents an investigation conducted by CREDO over the past two years. We examined charter school performance in urban areas, driven by our ongoing effort to identify successful models for educating America’s students, particularly students of color and students living in poverty. We sought to determine whether urban charter schools have different performance than other schools results that differ from the in their communities. In addition, we asked if urban charter schools present charter school landscape as a whole, as estimated in the 2013 National Charter School Study. Finally, if differences were identified in urban charter schools, could we provide any insight into which elements nces in results? of the urban charter sectors might correlate with differe Using student level data obtained via data sharing agreements with our state education agency -07 through 2011- 12. partners, we studied 41 urban areas in 22 states covering the school years 2006 The outcome of interest was the academic a dvancement in one year’s time of a typical student in a charter school compared to the same measure for a virtual peer from local traditional public schools in the same location as the charter school. se see the full report for greater detail on each of Highlights of the findings are presented below. Plea these findings. Our findings show urban charter schools in the aggregate provide significantly higher 1. levels of annual growth in both math and reading compared to their TPS peers. Specifically, students enrolled in urban charter schools experience 0.055 standard deviations (s.d.’s) greater growth in math and 0.039 s.d.’s greater growth in reading per year than their matched peers in TPS. These results translate to urban charter students receiving the equivalent of roughly 40 days of additional learning per year in math and 28 additional days of learning per year in reading. 2. When learning gains for urban charter students are presented for individual urban regions, regions with larger learning gains in charter schools outnumber those with smaller learning gains two -to -one. In math, 26 urban regions post learning gains for charter school students that outpace their TPS counterparts . C harter schools in 11 urban areas have gains, and four regions have equivalent learning gains . In reading, smaller math in math charter school students in 23 of the 41 regions demonstrate larger learning gains than their while 10 regions have smaller gains TPS peers, harter schools in eight regions have similar . C student learning gains in reading compared to TPS peers . 3. Learning gains for charter school students are larger by significant amounts for Black , Hispanic, -income , and special education students in both math and reading. Students low who are bot h low -income and Black or Hispanic, or who are both Hispanic and English credo.stanford.edu v

7 Language Learners , especially benefit from charter schools ains for these subpopulations , G amount to months of additional learning per year. Positive results for charter school 4. students increased on average over the period of the In the 2008- study. 09 school year, charter attendance on average produced 29 additional days and 24 additional days of learning in reading. By the 2011- 12 of learning for students in math harter students received 58 additional learning days in math and 41 additional school year, c relative to their TPS peers. days in reading , (see CREDO’s National Charter 5. Compared to the charter school landscape as a whole ), the 41 urban charter regions School Study 2013 have improved results at both ends of the quality spectrum: they have larger shares of schools that are better than TPS alternatives and smaller shares of under -performing schools. Specifically, 43 percent of urban charter schools deliver larger learnin g gains in math than the local TPS alternative, with 33 percent showing equivalent results and 24 percent posting smaller learning gains. In reading, 38 percent of urban charter outpace their TPS peers, 46 percent fare the same , schools and only 16 percen t of urban charter schools have smaller gains each year. 6. Despite the overall positive learning impacts, there are urban communities in which the majority of the charter schools lag the learning gains of their TPS counterparts, some to distressingly large d egrees. In some urban areas, cities have no schools that post better gains than their TPS alternatives and more than half the schools are significantly worse. The results reported in this study continue to build a record of many charter schools operating in challenging environments that repeatedly demonstrate the ability to educate all students to high levels. While some urban charter sectors continue to struggle, successful charter schools are growing in number and expand the ev idence base that schools and communities can organize and operate public schools that deliver the academic progress their students need to be successful in school, work, and life . credo.stanford.edu vi

8 Urban Charter School Study Report on 41 Regions 2015 Introduction Charter schools are a prominent and growing component of the public school system in the United 1 . The charter , with roughly States 00 charters across the country enrolling over 2.5 million students 6,4 is regularly treated as a monolithic set of scho sector ols, but recent research has made clear that across the U.S. there are in fact charter markets with dramatically different student profiles, distinct 2 . Previous CREDO state level studies, in ructures, a governance and oversight st nd academic quality n to other recent analyses of charter school performance, have identified individual charter additio TPS ) peers, particularly those markets substantially outperforming their traditional public school ( serving students in urban areas . CREDO decided to investigate w hether urban charter schools do in fact have differential performance than that found in our 2013 N ational Charter School Study for the charter sector as a whole and, if so, what the drivers of these differences in quality might be. In this report, CREDO used its unprecedented data holdings to investigate the student profiles and CRE academic performance of a large portion of the major urban regions in the U.S. DO included in this forty -one major urban regions for which we have analysis istrative and school level student level admin data. A complete list of urban regions included in this analysis can be found in the section “Defining 1 National Alliance for Public Charter Schools (2014). ”Details from the Dashboard: Estimated Number of Public Charter Schools and Students,” Washington D.C. Retrieved on 8 December, 2014 from: -an -Report -February - http://www.publiccharters.org/wp- content/uploads/2014/02/New d-Closed 20141.pdf 2 Center for Research on Education Outcomes (2013). “National Charter School Study,” retrieved on 8 December, 2014 from: http://credo.stanford.edu/documents/NCSS%202013%20Final%20Draft.pdf 1 credo.stanford.edu

9 Urbanity” below. cument, as well as in the content found online at In this do , we address the following major questions: urbancharters.stanford.edu Across the major urban school systems in the U.S., what is the range of performance of charters • TPS traditional public schools ( )? and Do urban charter schools tend to cause higher or lower growth with different • student e results vary by region ? subgroups, and how do thes Are there trends with respect to the quality of urban charter and TPS? • • s and TPS in urban school systems across the U.S., Which students are being served by charter -enrollment) performance of their both with respect to their demographics and the initial (pre students? urban charter schools in the aggregate provide significantly higher levels of annual Our findings show grow Specifically, students enrolled in urban th in both math and reading compared to their TPS peers. charter schools experience 0.055 standard deviations ( s.d.’s ) greater growth in math and 0.039 s.d.’s greater growth in reading per year than their matched pe ers in TPS. These results translate to urban charter students receiving the equivalent of roughly 40 days of additional learning per year in math and 3 . See Table 1 below for an expanded look at how gains 28 additional days of learning per year in reading in learning are translated from standard deviations to days of learning. The remainder of the Multi Summary is organized as follows. The section “Defining Urbanity” -Region details the process CREDO used to identify urban regions and schools for inclusion in this analysis. The techniques used to following section, “Data and Methods,” briefly discusses the data and analysis compare academic att ainment across urban regions and school sectors. Greater detail can be found in the technical appendix for interested readers. The next two sections, “Demographics” and “Performance,” present major findings aggregated across all urban regions with respect to the characteristics of students served and their academic performance. The succeeding section, "Correlates of Charter School Performance" takes a broad view of the results and considers whether factors in the evolution of the charter schools or attribut es of the communities themselves are associated with the performance results we estimate; while not causal in nature, the exercise is still suggestive of conditions that may elevate the performance of charter schools over time. The final section, ,” combines specific findings across each urban region to derive broader conclusions “Implications about the sta te of charter and TPS in urban school systems across the United States . 3 Eric A. Hanushek, Paul E. Peterson and Ludger Woessmann. Is the US Catching Up? International and State Trends in Student Achievement. Education Next , Vol. 12, No. 4. Fall 2012. credo.stanford.edu 2

10 Defining Urbanity to conducting an investigation of urban sc was to determine The first challenge hool systems in the U.S. which school systems to include in the analysis. CREDO considered multiple factors when identifying 4 , the size of each region’s regions for inclusion, including total population size of the metropolitan area primary school district(s), the total number of charter schools in the region, and the growth of the charter sector over tim e. Included urban regions are listed below, grouped by state: Arizona (Mesa, Phoenix, Tucson ), • • Colorado (Colorado Springs, Denver) , California (Bay Area, Central CA, Southern CA, • ), South Bay • District of Columbia, • Florida (Fort Myers, Jacksonville, Miami, Orlando, St. Petersburg, Tampa, West Palm Beach), • Georgia (Atlanta), Illinois (Chicago), • • Indiana (Indianapolis), • Louisiana ( New Orleans), • Massachusetts (Boston), • Michigan (Detroit), • Minnesota (Minneapolis), • Missouri (St. Louis), • Nevada (Las Vegas), • New Jersey (Newark), • New Mexico (Albuquerque), • New York (New York City), • Ohio (Cleveland, Columbus), • Pennsylvania (Philad elphia), • Tennessee (Memphis, Nashville), Texas (Austin, Dallas, El Paso, Fort Worth, Houston, San Antonio), • Wisconsin (Milwaukee). • The next step was to identify the specific schools for inclusion, which includes defining exactly what constitutes an “urban school,” as well as defining the boundaries of an urban region. These may seem to be straightforward tasks, but doing so in a cons istent manner across communities that differ in geography (disperse vs. compressed), population stability (high vs. low mobility), and permeability 4 United States Census Bureau (2013). Population Estimates: Metropolitan and Micropolitan Statistical Areas, retrieved on 12 December 2014 from: http://www.census.gov/popest/data/metro/totals/2013/ 3 credo.stanford.edu

11 (drawing only from other urban schools vs. drawing from suburban schools) required a consistent set of selec - tion rules. The resulting rigorous and comprehensive criteria required the development of a multi state process to address the often messy realities of urban regional and school classification. The . specific approach CREDO developed to deal with these iss Technical Appendix ues is covered in the Data and Methods As evidenced by the list of included urban regions above, a large number of states are covered in this each of these urban regions required negotiated agreements and partnerships with analysis. Including -two states, ensuring compliance with the twenty the state education agencies (SEA) in each of the ure the protection of Family Education Records Privacy Act (FERPA) provisions, among others, to ens student data . sed to create a matched student Information provided by the states was u database containing charter records and a matched group of comparison TPS students over the six years from the 1,018,510 6/0 7 to the 2011/12 school year . CREDO’s matching process uses the Virtual Control Record (VCR) 200 protocol, matching each charter student with up to seven traditional public school students based on 5 The matched data set contains over 80% of all prior test scores and demographic characteristics. charter students -one urban regions in this analysis. in the forty The impact analysis follows the approach used in prior CREDO studies of national charter performance, such as the National Charter School Study released i n 2013. Similar statistical methods are used to control for differences in student demographics and eligibility for program supports, such as free and reduced price lunch programs and special education status. Use of the VCR method assures that the maining relevant difference between charter students and their comparison group is the only re decision to attend either a charter or TPS . in the same urban region Results in the national analysis are presented in two formats. First, and most common to researche rs, results are presented in standard deviation units, which allows for comparison of students across grades, states, and time. The se results are also translated into “days of learning ,” to provide a reference by which non- real world” impact of charter enrollment on different technical readers can judge the “ student subgroups. A crosswalk of standard deviation units to “days of learning” is provided in Table 1 below. 5 For additional information on the Virtual Control Record method, please refer an explanatory infographic located here . credo.stanford.edu 4

12 6 Table 1. Transformation of Learning Gains Gain Growth (in days of learning) (in standard deviations) 0.00 0.0 0.01 7.2 36.0 0.05 0.10 72.0 0.15 108.0 144.0 0.20 0.25 180.0 0.30 216.0 Demographics Because charter schools are schools of choice they may not have a student population that exactly mirrors the districts from which they draw students. These differences are important for understanding er schools. Any substantial differences are also which families elect to enroll their students in chart important to note as they signal the need for careful control of student differences when examining the performance of charter schools compared to TPS. udent demographics were compared between the charter and TPS sectors St f the forty -one in each o urban regions. In general, u rban school systems serve a disproportionately low income and minority student body compared to the student distribution within their states . Given the variation in student demographi average s in the charter and TPS sectors cs across urban sectors, comparing demographic across all urban regions included in this analysis is less instructive than identifying trends found among multiple regions individually. In other words, statistical tests comparing pooled average student 6 Eric A. Hanushek, Paul E. Peterson and Ludger Woessmann. Is the US Catching Up? International and State Trends in Student Achievement. Education Next , Vol. 12, No. 4. Fall 2012. 5 credo.stanford.edu

13 demographics across all regions may obscure point of comparis on for results derived from the stronger the surrounding TPS in the same urban sector. each urban charter sector, which is s of English Language Learner ( ELL ) students, students in poverty , and students The percentage in the most recent year of available data are receiving special education services provided in Table 2 below are based on the number of tested students in our presented below. Note that all of the figures data and may differ from aggregate enrollment statistics in each urban region due to differences in testing practices and classification procedures across regions and sectors. Table 2: Selected Student Demographics b y Urban Region and School Sector (Tested Students) % English Language Learners % Students in Poverty % Special Education Charter TPS Charter TPS Charter TPS Region querque 16 11 15 40 69 Albu 12 8 4 9 5 58 76 Atlanta 10 17 18 68 56 Austin 10 3 4 22 24 72 Bay Area 60 Boston 17 21 8 30 79 75 3 3 15 18 72 75 Central CA 13 7 10 93 89 Chicago 11 15 21 2 0 83 99 Cleveland Colorado Springs 8 9 7 47 46 5 5 15 5 Columbus 76 72 16 23 Dallas 9 20 10 81 70 DC 16 19 6 6 76 68 Denver 12 34 29 77 71 10 7 9 8 14 87 85 Detroit 8 12 16 72 74 El Paso 6 7 8 3 14 44 74 Fort Worth Fort Myers 14 1 3 35 65 10 19 8 13 Houston 78 74 6 11 Indianapolis 13 5 13 76 72 Jacksonville 9 13 3 2 52 56 Las Vegas 10 4 14 11 65 10 4 5 1 Memphis 45 45 6 3 Mesa 6 2 7 41 56 Miami 7 12 7 9 79 78 10 Milwaukee 21 11 15 81 83 Minneapolis 10 14 33 22 79 65 Nashville 2 1 6 8 91 72 credo.stanford.edu 6

14 % English Language % Special Education Learners % Students in Poverty Charter TPS Charter TPS Charter TPS Region 82 6 1 1 6 97 New Orleans 14 14 5 12 New York City 81 82 15 0 10 85 86 Newark 4 11 14 6 Orlando 51 73 11 Philadelphia 11 13 3 7 77 87 5 4 4 56 64 Phoenix 6 11 10 13 San Antonio 82 65 9 South Bay 3 5 28 20 58 46 Southern CA 5 6 17 21 68 76 St. Louis 15 4 10 87 90 10 12 0 3 42 61 St. Petersburg 6 27 14 3 Tampa 44 66 7 Tucson 5 8 3 3 47 58 West Palm Beach 15 15 3 5 72 55 The urban regions with the largest share of students in poverty are Chicago, Cleveland, Detroit, Milwaukee, Newark, New York City, New Orleans, and St. Louis, where over 80% of students served by both the charter and TPS sectors qualify for free or reduced price lunches (according to tested student data). Comparing the charter and TPS sectors in each region, we see that charter schools enroll a disproportionately large number of students in poverty (greater than a 10% differential) in Austin, the Bay Area, Dallas, Minneapolis, Nashville, San Antonio, the South Bay and West Palm Beach. In contrast, the TPS sectors enroll substantially more students in poverty than do charters in Albuquerque, Atlanta, Fort Worth, Las Vegas, Mesa, New Orleans, Orlando, Philadelphia, Cleveland, Fort Myers, St. Petersburg, Tampa, and Tucson. The urban regions with the largest share of ELL students are Austin, the Bay Area, Central California, Dallas, Denver, Minneapolis, the South Bay, and Southern Califor nia, where both the charter and TPS sectors serve at least 15% ELL students. Charter schools in Denver, Minneapolis , and the South Bay enroll at least 5 percentage points more ELL students than do the TPS in their regions. Conversely, the TPS sectors in Bo ston, Detroit, Fort Worth, Houston, Las Vegas, New York City, Indianapolis, Orlando, and St . Louis enroll at least 5 percentage points more ELL students than do the charter sectors in their regions. The urban regions with the largest share of tested studen ts receiving special education services are Albuquerque, Austin, Boston, Chicago, Cleveland, Columbus, Denver, Washington D.C., Fort Myers, Tampa, Indianapolis, Minneapolis, Newark, New York City, Orlando, Philadelphia, Milwaukee, San credo.stanford.edu 7

15 Antonio, St. Louis, a West Palm B each , where both the charter and TPS sectors serve at least 10% nd . Tampa is the only urban region where the charter sector serves at least 5 special education students percentage points more special education students than their local TPS (albe it by a lot, 27% for charter Miami, and Newark, St. Louis, Cleveland, St. Milwaukee, vs. 14% for TPS). However, the TPS sectors in sburg all serve at least 5 percentage points more special education students than the charter Peter sectors in their regions. rban charter schools enroll a greater proportion of female student s It is also important to note that u region than urban TPS in nearly every pically 1 or 2 percentage points, the . While the difference is ty difference is most significant gender among tested students in Newark, where the charter schools in our TPS. data enroll nearly 7% more girls than local Detailed demographic information for each urban region can be found in the individual state . workbooks located here Performance Since charter schools may have students who are not perfectly representative of the TPS populations in their communities, judgments about school performance require techniques that assure equivalent ts are examined. Comparisons of academic growth made between charter and TPS students are studen conducted using CREDO’s virtual control record ( VCR ) technique. Based on stringent external reviews both the internal an d external validity of the se findings is and our own internal testing, confidence in to this report for further explanation). merited (see the Technical Appendix The analysis estimates the average one- academic progress of charter school students compared to year a similar period for matched TPS students. The impact of charter enrollment relative to local TPS for math and readin g can be found in Figu res 1 and 2 below. credo.stanford.edu 8

16 Figure 1: Impact of Charter Enrollment on Average by Region – Math Annual Learning Gains All Regions Albuquerque Atlanta Austin Bay Area Boston Central CA Chicago Cleveland Colorado Springs Columbus Dallas DC Denver Detroit El Paso Fort Myers Fort Worth Houston Indianapolis Jacksonville Las Vegas Memphis Mesa Miami Milwaukee Minneapolis Nashville New Orleans New York City Newark Orlando Philadelphia Phoenix San Antonio South Bay Southern CA St. Louis St. Petersburg Tampa Tucson West Palm Beach 0.35 0.30 0.25 0.20 0.15 0.10 -0.30 -0.25 -0.05 -0.10 -0.15 -0.20 0.00 0.05 -0.35 Standard Deviations credo.stanford.edu 9

17 Fig ure 2 on Average Annual Learning Gains by Region – Reading : Impact of Charter Enrollment All Regions Albuquerque Atlanta Austin Bay Area Boston Central CA Chicago Cleveland Colorado Springs Columbus Dallas DC Denver Detroit El Paso Fort Myers Fort Worth Houston Indianapolis Jacksonville Las Vegas Memphis Mesa Miami Milwaukee Minneapolis Nashville New Orleans New York City Newark Orlando Philadelphia Phoenix San Antonio South Bay Southern CA St. Louis St. Petersburg Tampa Tucson West Palm Beach 0.10 0.05 0.15 0.25 0.20 0.00 -0.05 -0.10 -0.15 -0.20 -0.25 Standard Deviations credo.stanford.edu 10

18 When all of the urban regions are pooled together, urban charter schools on average have significantly greater growth in math and reading than urban TPS. to see an infographic on Math Click here results for all regions combined. Click here to see an infographic on results for all regions combined. Reading Specifically, students enrolled in urban charter schools receive the equivalent of 40 additional days of learning growth (0.055 s.d.’s) in math and 28 days of additional growth (0.039 s.d.’s) in reading compared to their matched peers in TPS. These figures compare favorably to those found for the national charter sector as a whole, where the national CREDO’s National Chart er School Study found average impact of charter enrollment was 7 additional days of learning per year in reading (0.01 s.d.’s) and no significant difference in math. s mask a more nuanced with earlier studies of charter school performance, the aggregated result As pattern. Figures 1 and 2 above show there is great variation in student results across regions. For math , the effect of attending charter schools ranges from a negative effect of - .14 s.d.'s in Las Vegas to a in Boston compared to the learning of TPS peers. positive effect of .32 The pattern of charter school performance across the urban regions is positive on balance. There are outpace their TPS charter school students more regions where counterparts than regions where urban urban charter students lag behind them . Twenty -six regions have noticeably better learning gains in a year’s time compared to 11 re gions whose results lag behind their local yea rly TPS gains in m ath. For reading, students in 23 regions outpace the learning gains of their TP S peers while in 10 regions their learning gains are smaller. I n both subjects there are regions where the marginal improvement of charter school learning over TPS is dramatic: gains for charter studen ts in the Bay Area, Boston, D.C., Memphis, New Orleans, New York City and Newark are much stronger than their TPS peers in Math. The gains for annual stand out with respect to Bay Area, Boston, Memphis, Nashville and Newark also reading. charter school students in in perspective, it gain or loss associated with enrollment in a charter school of the To put the magnitude is valuable to consider the absolute level of academic achievement of each urban region relative to the credo.stanford.edu 11

19 rest of their state. if a region’s charter sector For example, achieves modest positive gains relative to their local TPS, to what extent should we expect students enrolled in this charter sector to “catch up” egion in the marginal chart over time with other students in their state? By considering er effect in each r region as a w of their urban relation to the average achievement hole, we can get a sense of the extent to which charter students will catch up (or fall behind) relative to the rest of their stat e. (Note that the measures of growth cannot be added directly to the achievement measures, as they are created from different distributions.) Estimated charter impacts are presented in the first column, color coded to aid tterns of perfo rmance across urban regions. Lighter colored cells represent a larger identification of pa This co mparison can be seen in T advantage for the below. charter sector. ables 3 and 4 : Impact of Charter Enrollment on Learning Gains Relative to Average Achievement of All Scho Table 3 ols in Region – Math Average Achievement in Region at Marginal harter Effect C Start of Study Key greater than 0.08 Albuquerque -0.019* 0.038 .02 to .08 0.018** Atlanta 0. 182 - - .02 to .02 -0.011 Austin 0.016 .08 to - .02 - 0.190** Bay Area 9 - 0.03 less than .08 - Boston 0.324** 98 0.4 - -0.003 Central CA 163 0. - 0.023** Chicago 0. 404 - Cleveland 0.043** 0. - 716 Colorado Springs 0.022** 111 0. -0.004 Columbus 472 0. - 0.041** Dallas - 0.030 0.134** DC 2 0.00 0.077** Denver 536 0. - Detroit 0.090** 688 - 0. El Paso -0.089** - 0.020 Fort Worth -0.140** - 0.232 -0.063** Fort Myers 013 0. 0.023** Houston 0.048 - Indianapolis 0.066** 0. 265 - credo.stanford.edu 12

20 Average Achievement Marginal in Region at harter Effect Start of Study Key C greater than 0.08 Jacksonville 0.018 0. 157 - .02 to .08 Las Vegas -0.114** - 0. 051 .02 to .02 - 0.135** Memphis 472 0. - .02 - .08 to - Mesa -0.063** 198 0. .08 - less than 0.029** Miami 271 - 0. 0.091** Milwaukee 0. 841 - 0.077** Minneapolis 493 0. - 0.071** Nashville 380 - 0. New Orleans 0.119** - 0. 412 0.145** New York City - 190 0. 0.233** Newark 0.675 - Orlando -0.014 0.220 - Philadelphia 0.059** 595 - 0. -0.080** Phoenix 036 0. - -0.030** San Antonio - 0.061 0.055** South Bay 0. 135 0.080** Southern CA 0. - 170 St. Louis -0.001 - 0. 034 0.002 St. Petersburg 81 0.0 - Tampa 0.047** 0.108 - Tucson 0.045** 0.230 - West Palm Beach -0.033** 0. 065 credo.stanford.edu 13

21 Table 4 : Impact of Charter Enrollment on Learning Gains Relative to Average Achievement of All Schools Reading in Region - Average Marginal Ach ievement in Region at Charter Effect Key Start of Study -0.006 0.066 greater than 0.08 Albuquerque -0.145 .02 to .08 Atlanta 0.031** -0.013 -0.027 -.02 to .02 Austin 0.130** -0.067 -.08 to - .02 Bay Area Boston 0.236** -0.587 less than - .08 0.018* -0.204 Central CA -0.373 Chicago 0.002 0.056** -0.624 Cleveland Colorado 0.094 0.024** Springs Columbus 0.016* -0.48 Dallas 0.036** -0.069 DC 0.002 0.097** 0.036** -0.575 Denver -0.638 Detroit 0.070** -0.034** El Paso -0.069 Fort Worth -0.073** -0.164 -0.066** 0.038 Fort Myers Houston 0.018** -0.093 Indianapolis -0.271 0.077** -0.085 Jacksonville -0.026* Las Vegas -0.076** -0.079 Memphis 0.164** -0.424 Mesa -0.049** 0.133 Miami 0.016** -0.318 credo.stanford.edu 14

22 Average Achievement in Region at Marginal Charter Start of Effect Study Key 0.041** -0.743 greater than 0.08 Milwaukee 0.006 .02 to .08 -0.525 Minneapolis -0.275 -.02 to .02 Nashville 0.112** 0.087** -0.414 -.08 to - .02 New Orleans New York City 0.033** -0.29 less than - .08 -0.722 Newark 0.216** -0.184 Orlando -0.006 Philadelphia 0.056** -0.628 Phoenix -0.043** -0.064 San Antonio -0.009 -0.032** 0.136 South Bay 0.066** 0.060** -0.152 Southern CA St. Louis -0.037 0.009 St. Petersburg -0.041** -0.054 Tampa 0.004 -0.147 Tucson -0.194 -0.001 West Palm -0.083** 0.018 Beach regional Click here to see an infographic association of achievement and charter effects for Math. Click here to see an infographic regional association of achievement and charter effects for Reading. credo.stanford.edu 15

23 As can be seen in the infographics and T ables 3 4 above, by comparing the annual learning gains and n, multiple associated with charter enrollment to the average achievement of each urban regio (TPS and charter schools combined) , such as Boston, scenarios become apparent. Many urban regions , find themselves faced with large region and Nashville Detroit, Indianapolis, Memphis, -wide but within the region have high quality charter achievement deficits relative to their state’s average compared to their region’s local . These charter sectors appear to provide their students sectors TPS with strong enough annual growth in both math and reading that continuous enrollment in an average charter school can erase the typical deficit seen among students in their region (Annual Charter Impact able 9 below, suggest yearly growth incr eases as students persist by Years of Enrollment, presented in T in charter schools, increasing the likelihood of students “catching up” in these regions). set urban charter sectors find themselves in regions with large region -wide achievement Another of relative to their state’s average and relatively moderate positive impacts on student growth deficits relative to local TPS . For example, students enrolled in charter schools in Cleveland, Miami, and emic growth than expected in their region’ Milwaukee can expect to see higher levels of acad s local TPS, but this charter lift is not enough for the average charter student to offset the achievement deficit of the region relative to the rest of the state in both math and reading . Two urban charter sectors, New York City and South Bay , stand out for providing positive gains for their rving a student body with achievement equal to or higher students in both math and reading and se than the average achievement within their state. Continuous enrollment in these charter sectors can be expected to result in steady movement up the state’s distribution of academic achievement. , already achieving Alternatively, the charter sectors in Las Vegas and Fort Worth provide their students below the state average, with lower levels of academic growth in math and r eading each year relative Continuous enrollment in these charter schools will cause an already low achieving student local TPS. base to fall further behind the average student in their state each year. A final subset of charter sectors , such as those in Fort Myers, Mesa, and West Palm Beach , provide their students with lower levels of annual growth in math and reading and serve a student body that performs similarly to or better than their state’s average achievement level. If these charter sectors do not find a way to increase the average level of academic growth among their students, they risk allowing their students to fall behind the rest of their state in academic achievement . Learning Gains by Student Subgroups When th e impact of u rban charter schools is studied for students in different subgro up s, we see that would have nearly experiences greater growth in charter schools than they every group of students credo.stanford.edu 16

24 otherwise realized r the charter sector at large, in their local TPS. Mirroring the findings fo disadvantaged students tend to receive the strongest positive benefits from enrollment in urban charter schools. Black and Hispanic students, students in poverty, English language learners, and students receiving special educ ation services all see stronger growth in urban charters than their matched peers in urban TPS. These results are partially offset, however, by the negative impact on and for math and reading growth experienced by White students enrolled in urban charter schools . The math results for white urban charter students compare Native American students in math .07 s.d.'s; the reading results were the same. Asian favorably to the impact nationally, which was - and retained students see mixed impacts on math students and reading growth as a result of for each enrollment in charter schools. The impact of urban charter enrollment relative to local TPS subgroup can be seen in Table 5 below. : Impact of Charter Enrollment on Annual Average Learning Gains for All Urban Regions Table 5 MATH READING Group DAYS OF DAYS OF LEARNING EFFECT SIZE LEARNING EFFECT SIZE 0.039** Overall 40 0.055** 28 Black 0.051** 36 0.036** 26 0.029** 22 Hispanic 0.008** 6 White - 0.047** - 36 - 0.021** - 14 Asian 0.012** 9 0.001 0 Native American - 0.097** - 70 - 0.033 0 0.033** 24 Poverty 0.024** 17 ELL 0.041 0 0.071 0 0.012* 9 0.007 0 Retained Special Ed 0.013** 9 0.018** 13 credo.stanford.edu 17

25 Group READING MATH DAYS OF DAYS OF EFFECT SIZE EFFECT SIZE LEARNING LEARNING Black Students Pov erty 0.082** 59 0.061** in 44 anic Hisp Students in 0.067** 48 Pov 0.035** erty 25 Hisp anic Students with ELL Status 0.10** 72 0.11** 79 Compared to the results found for all charter schools in CREDO’s 2013 national report, urban charter deficits found in schools achieve higher levels of average growth by reducing or eliminating educational the charter sector more generally. For example, Asian students enrolled in urban charter schools receive small positive benefits in math (~ 8 days of additional growth) and no significant impact in reading relative to their peers in TPS. Across all charter schools in the 2013 National report, Asian students were found to receive the equivalent of 29 fewer days of learning relative to their peers in math, while also showing no significant difference in reading performance compared to their peers in TPS. National Charter School Study, urban charter schools tend to Continuing a trend found in CREDO’s 2013 do best in serving students with multiple disadvantages. This can be seen by comparing the average academic growth of Black and Hispanic stu dents in poverty in charter s and TPS. Across all urban regions, Black students in poverty receive the equivalent of 59 days of additional learning in math and 44 days of additional learning in reading compared to their peers in TPS. Hispanic students in po verty experience the equivalent of 48 days of additional learning in math and 25 days of additional learning in reading in charter schools relative to their peers in TPS. Of particular note is the fact that, across all urban charter sectors, Hispanic Engl ish Language Learner (ELL) students ELL students in TPS ; in other words, advance each year in math on par with White, non- Hispanic ELL charter students realize no learning gap each year. Reading gains for this group , like many other subgroups, lags White, non- ELL students in TPS, but the ir performance relative to their TPS Hispanic ELL peers is positive. Hispanic ELL students enrolled in charter schools receiving the credo.stanford.edu 18

26 equivalent of only 22 days less growth in reading compared to White, non -ELL students enrolled in TPS. By comparison, Hispanic ELL students enrolled in urban TPS receive 29 fewer days of learning growth in math and 65 fewer days of learning in reading per year compared to that of White, non -ELL TPS students. Compared to the national charter sector, urban charter schools also perform significantly better with three additional subgroups whose performance depressed the aggregate performance of Black and Hisp anic students in the 2013 report : Black students not in poverty, Hispanic students not in poverty, and Hispanic students who are not ELL. Nationally, charter schools perform no differently than TPS in either math or reading with Black students who are not in poverty. Urban charter schools, however, provide significantly higher gains in both math (43 days additional learning) and reading (29 days additional learning) compared to local urban TPS with Black students not in poverty . Hispanic students not in pov erty perform no differently in urban charters and TPS. This compares favorably to the national charter sector, where Hispanic non -poverty charter students saw significantly lower performance in both math (29 fewer days of learning) and reading (9 fewer day s of learning) relative to ELL students in urban charter schools perform significantly their peers in TPS. Finally, Hispanic non- better than their peers in urban TPS, receiving the equivalent of 40 additional days of learning in math and 22 additional days of learning in reading per year of enrollment. In the national charter sector, Hispanic non- ELL students receive no benefit in math and only 7 additional days of learning in reading per year. Table 6 chievement, broken down by urban below shows the impact of charter enrollment on math a region. Estimated impacts are presented in each cell, which are color coded as we ll to aid identification of patterns of performance within and across urban regions. Lighter colored cells represent a larger advantage for th e charter sector for that subgroup. Charter sectors with positive impacts greater than 0.08 standard deviations ( ) per year receive the lightest coloring, followed by those with positive s.d.’s etween 0.02 and 0.08 . Charter sectors with yearly impacts between - 0.02 s.d.’s and 0.02 impacts b s.d.’s 0.02 and - 0.08 s.d.’s receive a s.d.’s receive a neutral color, charter sectors with impacts between - darker shade, and charter sectors with annual negative growth impacts greater than - 0.08 s.d.’s receive the darke st shade. For example, the column presenting marginal charter effects for White students is generally “darker” than the column for students in poverty , suggesting that urban charter sectors tend to perform better among students in poverty than for White st udents generally. Results for reading are sim below. ilar and can be found in Table 7 In light of the substantial variation in sample sizes between included urban regions, and t o aid the reader’s ability to identify patterns in charter impact across region s, estimates of charter impact are shaded without regard to statistical significance. For readers interested in p values associated with . each of the estimates presented below, they can be found in the state level workbooks presented here credo.stanford.edu 19

27 Table 6: Impact of Charter Enrollment in Math by Region and Sub -population on Annual Learning Gains Poverty Overall Urban Regions ELL SPED Black Hi sp anic Asian White St udents 0.029 0.041 0.013 0.033 0.012 -0.047 0.055 All Regions 0.051 -0.019 0.016 0.088 0.023 -0.058 -0.031 -0.040 -0.021 Albuquerque 0.018 Atlanta 0.041 -0.048 0.105 -0.005 -0.043 -0.041 -0.025 0.124 -0.036 -0.006 -0.082 -0.078 -0.077 -0.161 Austin -0.011 0.190 0.060 0.006 -0.100 0.160 0.160 0.160 -0.010 Bay Area Boston 0.043 0.114 0.051 0.272 0.290 0.175 0.208 0.324 -0.003 0.039 0.085 -0.040 0.072 -0.059 -0.076 -0.184 Central CA 0.039 -0.007 0.004 -0.042 0.029 -0.074 0.013 Chicago 0.023 0.043 0.022 -0.059 -0.043 Cleveland -0.100 * -0.057 0.050 Colorado 0.022 -0.007 0.021 0.088 0.068 0.007 0.048 0.019 Springs Columbus -0.004 0.043 -0.067 -0.013 0.009 0.020 -0.031 -0.095 Dallas 0.041 0.034 0.005 0.039 -0.003 0.006 -0.086 -0.050 DC 0.071 0.059 0.107 0.072 0.020 -0.089 -0.100 0.134 0.077 0.037 0.026 -0.051 -0.044 0.061 -0.067 -0.045 Denver 0.031 -0.059 -0.058 0.070 0.051 0.072 0.187 Detroit 0.090 -0.089 -0.007 -0.069 0.080 El Paso -0.102 0.023 -0.208 -0.231 Fort Myers -0.063 -0.029 -0.753 0.013 -0.086 -0.039 -0.023 -0.048 Fort -0.140 -0.068 0.027 0.196 -0.170 -0.132 -0.080 -0.131 Worth Houston 0.023 -0.018 0.019 0.017 -0.027 0.069 0.004 -0.017 Indianapolis 0.026 0.096 0.011 0.084 -0.009 * -0.047 0.066 0.014 0.017 -0.051 -0.026 Jacksonville 0.005 -0.041 0.021 0.018 -0.067 Las Vegas 0.080 0.034 0.055 -0.114 -0.178 -0.105 -0.119 Memphis 0.135 -0.037 -0.012 0.016 0.149 0.147 * -0.020 -0.039 Mesa -0.002 0.096 0.039 -0.063 -0.034 0.012 -0.081 Miami 0.029 0.036 0.156 -0.033 0.006 -0.007 * -0.039 credo.stanford.edu 20

28 Poverty Urban Regions Overall White ELL SPED Black Hi sp anic Asian udents St 0.052 0.091 -0.020 -0.022 0.094 0.016 0.148 0.050 Milwaukee 0.077 0.091 0.011 0.045 0.071 Minneapolis 0.138 0.051 -0.164 0.006 0.049 -0.065 0.071 0.104 0.179 0.096 Nashville 0.059 0.119 0.002 -0.044 0.032 0.109 0.076 0.096 0.126 New Orleans 0.040 0.028 -0.013 New York City 0.134 0.102 -0.019 -0.005 0.145 0.013 1.933 -0.002 0.217 0.171 0.046 0.127 Newark 0.233 -0.014 -0.069 -0.031 -0.019 0.048 0.121 * -0.042 Orlando Philadelphia 0.024 0.100 -0.005 0.039 0.037 -0.022 0.050 0.059 -0.058 Phoenix -0.010 0.051 0.011 -0.080 -0.017 -0.146 -0.117 San Antonio -0.030 0.078 0.013 0.057 -0.110 -0.103 -0.054 -0.123 South Bay 0.114 0.073 -0.053 -0.102 0.010 -0.043 -0.053 0.055 0.037 0.025 -0.014 0.034 0.067 0.015 -0.035 Southern CA 0.080 -0.001 -0.023 0.123 0.074 St. Louis 0.010 0.001 0.006 0.031 St. Petersburg 0.002 0.008 0.028 0.001 0.038 0.010 -0.051 Tampa 0.047 0.026 -0.146 0.076 0.107 -0.018 0.258 -0.048 Tucson -0.078 -0.006 -0.020 0.093 0.058 0.198 0.090 0.045 West Palm 0.049 -0.017 0.042 -0.033 -0.057 -0.088 -0.159 -0.040 Beach Color indicates size of charter impact on growth in standard deviations. * Value not reported due to small N. Key less than - .08 - .08 to - .02 - .02 to .02 .02 to .08 greater than .08 credo.stanford.edu 21

29 Table 7 : Impact of Charter Enrollment on Annual Learning Gains in Reading by Region and Sub - population Poverty Urban Regions Overall ELL SPED Black Hi sp anic Asian White St udents 0.008 0.071 0.018 0.024 0.001 -0.021 0.039 All Regions 0.036 -0.006 0.017 0.075 -0.029 -0.102 -0.016 0.040 -0.005 Albuquerque 0.031 Atlanta 0.068 -0.064 0.079 0.005 -0.066 -0.200 -0.046 0.072 0.042 0.061 -0.079 -0.040 -0.038 -0.123 Austin -0.013 0.130 0.031 0.076 -0.005 0.119 0.076 0.113 0.037 Bay Area Boston 0.082 0.161 0.057 0.140 0.196 0.074 0.131 0.236 0.018 -0.004 0.106 0.022 0.080 -0.023 -0.052 -0.015 Central CA 0.049 -0.016 0.005 -0.046 -0.041 -0.104 -0.148 Chicago 0.002 0.056 -0.096 0.032 -0.002 Cleveland 0.062 0.307 0.052 0.170 Colorado 0.024 -0.011 0.012 0.143 0.035 0.010 0.022 0.031 Springs Columbus 0.016 0.065 0.000 -0.043 -0.015 0.020 -0.115 -0.067 Dallas 0.036 0.039 0.038 0.099 -0.013 -0.009 -0.042 -0.064 DC 0.048 0.029 0.104 0.051 0.033 -0.056 -0.063 0.097 0.036 0.030 0.040 0.072 -0.019 0.000 -0.009 -0.046 Denver 0.035 -0.054 -0.049 0.047 -0.041 -0.356 0.133 Detroit 0.070 -0.034 0.021 0.010 0.108 El Paso -0.076 0.113 0.041 -0.160 Fort Myers -0.066 -0.005 -0.813 0.045 -0.141 -0.075 -0.217 -0.046 -0.113 -0.045 0.260 0.075 -0.073 -0.094 -0.021 -0.071 Fort Worth Houston 0.018 0.001 0.087 0.004 -0.022 0.030 0.017 -0.006 Indianapolis 0.022 0.087 0.040 0.063 -0.021 0.132 0.039 0.077 -0.011 Jacksonville -0.008 -0.251 -0.010 -0.026 -0.097 0.025 -0.010 Las Vegas -0.076 0.006 0.022 -0.041 -0.065 -0.086 -0.047 -0.058 Memphis -0.004 0.010 0.014 0.164 0.152 -0.015 * -0.019 Mesa -0.049 -0.007 0.174 0.084 -0.045 -0.032 -0.036 -0.057 credo.stanford.edu 22

30 Poverty Urban Regions Overall White ELL SPED Black Hi sp anic Asian udents St 0.016 0.040 -0.021 -0.036 -0.016 * -0.040 Miami 0.046 0.027 0.041 0.023 0.061 0.057 -0.015 0.054 0.022 Milwaukee Minneapolis 0.053 -0.015 0.036 0.019 0.044 -0.090 -0.166 0.006 0.063 0.210 0.023 0.112 0.088 0.434 0.022 Nashville 0.041 0.087 -0.001 0.041 New Orleans 0.075 0.066 0.061 0.141 0.071 New York City 0.033 0.039 0.001 0.029 0.003 0.000 -0.130 -0.099 0.186 0.020 -0.005 0.009 0.170 * 0.063 Newark 0.216 -0.006 -0.005 -0.018 -0.127 Orlando 0.016 -0.140 -0.029 0.060 Philadelphia 0.056 0.027 0.042 -0.006 0.040 0.004 0.047 0.028 Phoenix -0.043 0.002 0.053 0.028 -0.039 -0.020 -0.024 -0.066 San Antonio 0.061 0.062 0.091 -0.135 -0.097 0.022 -0.060 -0.032 0.037 0.054 -0.034 0.047 0.048 -0.009 0.004 South Bay 0.066 0.060 0.024 0.070 0.001 Southern CA 0.033 0.007 -0.001 0.016 St. Louis 0.009 -0.010 0.066 -0.031 0.020 -0.035 -0.130 0.052 St. Petersburg -0.041 -0.006 0.818 -0.037 -0.061 -0.012 0.107 -0.028 Tampa 0.024 -0.122 0.018 0.042 -0.035 * -0.067 0.004 0.055 0.004 -0.072 0.010 Tucson -0.019 -0.022 0.010 -0.001 West Palm -0.122 -0.083 0.041 -0.074 -0.025 -0.078 -0.112 -0.097 Beach Color indicates size of charter impact on growth in standard deviations. reported due to small N. * Value not Key less than - .08 - .08 to - .02 - .02 to .02 .02 to .08 greater than .08 credo.stanford.edu 23

31 Impact of Urban Charter Attendance on Annual Learning Gains by School Level, Growth Period Years of Enrollment , and the aggregate yearly impact of charter enrollment across all urban regions, we In addition to analyzing were interested to see if charter school impacts were consistent across grade spans, the results o f which are presented in Table 8 Table 9 presents the impact of charter atte ndance by growth below. period . Growth periods cover two successive school years and use test scores from each to observe the reveal change from one year to the next. s in quality Progressing across several periods can trend time . Table 10 provides the impact of charter attendance separated among urban charter schools over by year of enrollment. Disaggregating the average charter effect by year of enrollment allows us to changes in the impact of urban charter schools between a student’ s first year identify enrollment and of subsequent years in the charter sector . : Impact of Urban Charter Attendance on Annual Learning Gains Table 8 by School Level READING MATH DAYS OF DAYS OF EFFECT SIZE LEARNING EFFECT SIZE LEARNING Charter Elementary 0.056** 40 0.046** 33 Charter Middle 73 0.063** 45 0.101** Charter High School 0.044** 32 0.012** 9 Charter Multilevel 0.01** 7 0.016** 12 above separates out the impact of urban charter attendance by school level. While urban Table 8 charter schools provide higher levels of annual learning growth at all school levels, the strongest positive impacts come from charter middle schools (73 additional days of learning per year in math and 45 additional days of learning per year in reading). Urban charter elementary school s are also found to provide strong positive impacts in both math and reading, while urban charter high schools are strongest in math. credo.stanford.edu 24

32 Another view of the impact of charter schools on student learning addresses their performance over time. As the charter schools gain experience and the community gains understanding of schools of choice, performance could change. For example, charter schools could adapt over time to the needs of their students, or families could more readily identify schools that meet the needs of their children; both of these possibilities might translate into better results over time. Alternatively, as more charter schools open and attract later adopters, there is a chance that the quality of the schools could move to t the overall quality of the broader range of schools. A study of the performance of more closely reflec er time appears below in Table 9 . charter schools in the urban regions ov Growth Period Table 9 : Impact of Urban Charter Attendance on Annual Learning Gains by MATH READING iod Growth Per DAYS OF DAYS OF Ending in: EFFECT SIZE EFFECT SIZE LEARNING LEARNING 2008 2009 0.040** 29 0.033** 24 - 42 2010 0.058** - 0.042** 30 2009 2010 - 2011 0.057** 4 1 0.037** 27 - 2012 0.081** 58 0.057** 41 1 201 Similar to the national charter sector, urban charter schools show a general upward trend in quality over time , achieving positive annual impacts of 58 additional days of learning in math and 41 additional days of learning in reading by the final growth pe riod in this analysis. This is consistent with both the findings for the national charter sector in CREDO’s 2013 National Charter School Study and the recent 7 ve . It is important to note that results presented abo emphasis on quality improvement in the sector control for changes in student demographics and achievement each year and therefore isolate the real charter impact in separate growth periods. A single school can also be represented in each growth period if it was open and had tested students each yea r of analysis. That said, the charter sector is dynamic and thus the cohort of charter schools is not the same in each year, due to a combinat ion of the establishment of new urban charter schools and the closure of existing ones . 7 - For example, National Association of Charter School Authorizers: http://www.qualitycharters.org/one -million- lives.html million- lives/one 25 credo.stanford.edu

33 Table 10 below provides th e annual impact of charter attendance separated by year of enrollment. annual impact of charter enrollment presented earlier is broken down in to a Specifically, the average nd rd st year in charter” effect, a “2 year in charter effect,” a “3 year in chart er effect,” and a “4+ years in “1 charter effect.” Table 10 : Impact of Urban Charter Attendance on Annual Learning Gains by Years of Enrollment MATH READING DAYS OF DAYS OF EFFECT SIZE EFFECT SIZE LEARNING LEARNING 1st Year in 0.01** 7 -0.01** -7 Charter nd Year in 2 Charter 58 0.06** 43 0.08** rd 3 Year in Charter 0.12** 86 0.06** 43 4+ Years in ** 108 0.10 72 Charter 0.15** The impact of urban charter attendance s a s trong positive trajectory by year of enr ollment (Table show 10) . The longer students stay enrolled in charter schools, the larger the annual benefit of charter attendance becomes . These trends are strong enough that by the time a student spends four o r more years enrolled in an urba n charter school, we can ex pect their a nnual academic growth to be 108 days greater in math and 7 their peers in TPS. Given these trends , it is in reading per year than 2 days greater un reasonable to expect many urban charter sectors to continue to improve in quality. Trends in not ty are also presented for each urban region, which can be found in individual state charter quali . here workbooks School- level Quality Comparisons Much of the discussion about CREDO’s earlier work has centered on school- level comparisons of the performance of charter schools versus the alternative schooling options their students face. These computations group charter school students by their school of enrollment each year and compare the credo.stanford.edu 26

34 average academic progress to the average of their similarly -grouped virtual peers. These school- level measures are then statistically tested in pairs to see if the charter school is performing better, worse or no d ifferent than its corresponding school. Consistent with the general tenor of findings earlier in this report, the school quality comparisons for urban charter schools are more positive than was found for the sector as a whole in the 2013 National Charter School Study. The relative comparisons appear in Table 11 below. level Quality Comparisons – 41- Table 11: School- Region Urban Charter School Study Results and 2013 National Charter School Study Results Worse Better Same 41 Urban Regions -- Math Overall 33 43 24 Reading Overall -- 41 Urban Regions 16 38 46 Worse Same Better 2103 National Study Math Overall -- 40 31 29 2013 National Study Reading Overall -- 25 19 56 credo.stanford.edu 27

35 At both ends of the quality scale, urban charter schools post more positive results than was found across the national scene in 2013. The proportion of the urban schools that have significantly poorer results than the TPS alternative is decresed in both math and reading. The more notable improvement occurs at the high end of the quality spectrum. In both tested subjects, the proportion of urban charter schools that out -perform their local TPS is more than 10 percentage points larger than was found in the 2013 national study. The school- level quality comparisons for individual region s take the aggregate results into even sharper relief. These comparisons appear in Tables 12 and 13. credo.stanford.edu 28

36 Table 12: School- Math Level Quality Comparisons by Region - Overall 43 33 24 Albuquerque 39 17 43 Atlanta 35 45 20 Austin 46 17 38 Bay Area CA 17 59 24 Boston 92 8 Central CA 28 22 50 Chicago 24 38 38 Cleveland 44 42 15 Colorado Springs 33 33 33 Columbus 21 56 23 Dallas 53 31 16 DC 32 9 59 Denver 25 42 33 Detroit 60 8 33 El Paso 44 56 Fort Myers 33 67 Fort Worth 50 50 Houston 31 34 35 Indianapolis 25 38 38 Jacksonville 44 33 22 Worse Better No Different credo.stanford.edu 29

37 Table 12 (Continued) Overall 24 43 33 Las Vegas 69 31 Memphis 38 19 43 Mesa 48 43 10 Miami 42 21 38 Milwaukee 60 37 3 Minneapolis 7 37 56 Nashville 11 56 33 New Orleans 29 56 15 New York City 22 14 64 Newark 23 77 Orlando 33 17 50 Philadelphia 20 19 61 Phoenix 24 30 46 San Antonio 41 28 31 South Bay CA 22 22 57 Southern CA 34 43 23 St. Louis 42 32 26 St. Petersburg 33 33 33 Tampa 50 13 38 Tucson 17 24 59 West Palm Beach 33 17 50 Worse No Different Better credo.stanford.edu 30

38 Table -Level Quality Comparisons by Region – Read ing 13: School Overall 38 46 16 Albuquerque 57 22 22 Atlanta 65 25 10 Austin 38 42 21 Bay Area CA 33 57 10 Boston 81 19 Central CA 38 28 34 Chicago 22 24 55 Cleveland 51 15 34 Colorado Springs 61 33 6 Columbus 29 9 63 Dallas 45 45 10 DC 63 33 3 Denver 42 38 21 Detroit 4 51 45 El Paso 67 33 Fort Myers 33 67 Fort Worth 30 70 Houston 34 21 45 Indianapolis 52 48 Jacksonville 10 70 20 Better No Different Worse credo.stanford.edu 31

39 Table 13 (Continued) Overall 38 46 16 Las Vegas 38 62 Memphis 29 71 Mesa 52 39 10 Miami 50 33 17 Milwaukee 44 53 3 Minneapolis 32 14 54 Nashville 22 11 67 New Orleans 42 39 19 New York City 16 43 41 Newark 69 31 Orlando 67 17 17 Philadelphia 61 25 14 Phoenix 65 19 16 San Antonio 9 19 72 South Bay CA 52 26 22 Southern CA 38 48 14 St. Louis 50 17 33 St. Petersburg 57 29 14 Tampa 20 10 70 Tucson 69 17 14 West Palm Beach 57 43 Worse Better No Different of the 41 regions are The individual region results show cause for concern and for celebration. six . In math, more than dramatically lower performing than their TPS counterparts in one or both subjects Central California, El Paso, Fort Worth, Las Vegas and West Palm 50 percent of the charter schools in have Beach , Mesa significantly lower learning gains. The same is true for Las Vegas and West Palm credo.stanford.edu 32

40 Beach in reading. The fact that only six reg . There is an urgent ions have these results is cold comfort need to address the primacy of academic rigor in the charter schools in these communities A more positive way to summarize the regional differences is to consider the number that have the share of schools performing badly and/or have a majority of their schools performing at minimized levels superior to the local TPS alternatives. These regions demonstrate the quality can focus at either end of the spectrum to achieve overall strength in the reg ion. Looking at math results, seven regions less than 10 percent have PS alternatives. of their schools significantly underperforming their T Fourteen regions have more than 50 percent of their schools outperforming their local TPS options. In reading, twelve regions have less than 10 percent performing worse than the local TPS and ten regions have 50 percent or more of their schools showing results that are superior to TPS. Importantly, a substantial number regions manage to accomplish both targets : small shares of low performing schools and a majority of charters outperforming their local TPS. For reading, the Bay Area in California, Boston, DC, Detroit, Indianapolis, Memphis and Newark accomplish this result. For math, Boston, DC, Detroit, Milwaukee, Minneapolis the Bay Area in California, and Newark do the same. Charter schools in Boston, Detroit , the District of Columbia and Newark stand out for meeting the dual standard in both math and reading. These four communities of charter schools prov ide essential examples of school- level and system -level commitments to quality that can serve as models to other communities. credo.stanford.edu 33

41 Correlates of Performance the question, "Can we explain why Knowing the charter effect sizes of so many regions naturally raises the differences across regions exist?" Proving a causal relationship between the performance of districts and any potential explanatory factors is impossible -- there is no way to systematically alter some regions to see if their performance changes as a result. Regardless, it is still interesting to , maturity of the movement in the state, or other observable consider if size of the charter community factors track with performance. We computed Spearman Rank Order correlations of a number of descriptors of the charter schools in each region. Spearman R ank correlations are a variant of the better know Pearson correlations; the test ons on the two variables under consideration. In of association is based on the rank order of the regi other words, we ranked the regions by their charter academic growth effects and then tested how closely the rank order of other factors, such as the overall number of K -12 students in a region or the percen t of students enrolled in charter schools, matched the performance ranking. The resulting correlation coefficients appear in Table 1 4. 4: Correlation Math or Reading Effect Sizes and Other Factors Table 1 s between A BLES VARI MATH READING Reading 0.89* Structure of the Charter Sector Year State Charter Law Enacted 0.10 - 0.07 - State Charter Law Ranking in 2012 -0.07 0.09 Number of Schools 0.24 0.23 Number of TPS 0.20 0.20 Number of Charter Schools 0.34* 0.27 Student Population Total Students in 2006 -0.08 0.01 credo.stanford.edu 34

42 VARI A MATH READING BLES 0.26 0.30 Total Charter Students in 2006 - - 0.01 0.07 Total Students in 2010 0.36* 0.40* Total Charter Students in 2010 0.05 - 0.08 Percent Special Education Students in 2010 0.14 Percent English Language Learners in 2010 0.16 Percent Students in Poverty in 2010 0.32* 0.38* 0.52* - 0.54* Percent White in 2010 - 0.50* Percent Black in 2010 0.49* Percent Hispanic in 2010 - 0.31 - 0.31* Percent Asian/Pacific Islander in 2010 0.15 0.06 Percent Native American in 2012 0.25 - 0.40* - -racial in 2010 -0.22 -0.13 Percent Multi 0.02 -0.14 Student Count of Primary School Districts Charter Student Count of Primary Schools 0.21 0.17 Market Share Percent Charter Schools 0.12 0.06 Charter Share of in Region 0.16 0.31 School District Largest Percent Charter Students in 2006 0.27 0.30 Percent Charter Students in 2010 0.46* 0.48* Difference in Percent Charter Students (d=2010 2006) 0.45* 0.51* - credo.stanford.edu 35

43 The factors we considered group into four clusters: Structure of the Charter Sector, Student and Populations, Market Share. As far as variables pertaining to the structure of the charter sector, such as the maturity of the sector or the perceived quality of the charter law (using the National Alliance for Public Charter Schools State Charter Law rankings), neither factor had a significant correlation with the comparative student learning gains over TPS peers. However, the Student Population variables suggest that increased maturity of the sector in a given region may have an influence, because the absolute number of charter students was not significant in 2006, but became significant in 2010. Similarly, the share of a region's students who were enrolled in a charter school followed a similar trend, not significant in 2006 but becoming significant in 2010. The pattern suggests that there may be some role of critical mass in fostering better performance across the charters in a region. This idea is supported by the finding that the larger the jump in charter share of public students, the higher the region's performance. Several school- level student profile variables were found to be significant. The percent of students in each region who are in poverty or who were Black or Hispanic was positively associated with learning gains in both math and reading across the regions. While the results might be counter -intuitive -- these groups are typically considered less academically prepared -- the correlations are consistent with the expressed mission of m any urban charter school operators to provide high -quality education choices specifically for these students. Finally, the larger the share of White students in a region, the less advantage charter schools bestow on them compared to their TPS peers. Tracing back through region - specific findings, the result makes sense: regions with large shares of White s tudents tended to have above average starting achievement in TPS and weaker annual academic progress in charter schools. credo.stanford.edu 36

44 Implications 1. Urban charter schools vary in quality, but that variation clusters around a higher average level of performance than the national charter sector as a whole. Compared to the results found for the national charter sector in CREDO’s 2013 National Charter School Study, urban charter schools on average achieve substantially greater levels of growth in math and reading relative to local TPS. Despite this advantage in aggregate performance, urban charter sectors exhibit similar levels of variation in academic quality aroun d this average, both across sectors and often within each sector as well. While a handful of the highest performing charter sectors have figured out a way to provide superior, or at least equivalent, levels of academic growth relative to local TPS for every student subgroup (e.g. Boston and Newark), many strong charter sectors nonetheless fail to provide strong growth for every sector of their student population. 2. Urban charter schools tend to reflect the strengths and weaknesses of the national charter se ctor. In many respects, urban charter schools achieve their high average levels of performance by essentially “doubling down” on the strengths of the broader charter movement. In most urban regions with strong charter sectors, the major drivers of these effects are their high performance with students in poverty, Black and Hispanic students, and English Language Learners. Also similar to the national charter sector, urban charter schools tend to see their aggregate performance dragged down by relatively low level s of growth provided to their White and Asian students, although these deficits are typically smaller than those found for the national sector. Attempts to identify correlates of performance point to two themes . 3. The first was accumulated success ov er time, both in attracting larger numbers of students into the region's charter schools and maintaining a strong pace of growth in the region. The second was the focus on students of color and poverty; where regions had schools that enrolled larger share s of these students, the regional results were stronger. This suggests a focused model with continuing success in providing students who are often disenfranchised in local schools better opportunities to grow academically. credo.stanford.edu 37

45 4. Many urban regions could be nefit by finding a “sister city.” Many urban regions stand to benefit from identifying and learning from an urban charter sector that has figured out how to achieve substantially higher levels of growth with similar students. For example, cities like Orla ndo and Fort Myers can look to and learn from the success of Miami’s charter sector with ELL students, who see the equivalent of 112 additional days of learning per nver year in math relative to their peers in TPS. Similarly, members of the charter sector in De could benefit from taking a drive to Colorado Springs to see how they achieve such strong results with their special education population. Many schools, in both the charter and TPS sector, pride nd develop best practices in education. themselves on their willingness to experiment, refine, a We hope the findings in this report can serve as a road map to guide that process. 5. The best urban charter sectors provide extraordinary opportunities to learn how best to serve the most disadvantaged students. ) e results presented throughout this document (and online at urbancharters .stanford.edu Th provide ample evidence that some urban charter sectors have figured out how to create cademic growth to their most disadvantaged students. This is dramatically higher levels of a important for at least two reasons. First, these urban regions can serve as models from which all public schools serving disadvantaged student populations may learn . Second , and perhaps more impo rtant, these charter sectors clearly refute the idea that some groups of students cannot achieve high levels of academic success. They need only to be given the opportunity. credo.stanford.edu 38

Related documents

MPI: A Message Passing Interface Standard

MPI: A Message Passing Interface Standard

MPI : A Message-Passing Interface Standard Version 3.0 Message Passing Interface Forum September 21, 2012

More info »
CityNT2019TentRoll 1

CityNT2019TentRoll 1

STATE OF NEW YORK 2 0 1 9 T E N T A T I V E A S S E S S M E N T R O L L PAGE 1 VALUATION DATE-JUL 01, 2018 COUNTY - Niagara T A X A B L E SECTION OF THE ROLL - 1 CITY - North Tonawanda TAX MAP NUMBER ...

More info »
CalCOFI Atlas 33

CalCOFI Atlas 33

THE EARLY STAGES IN OF THE FISHES CALIFORNIA CURRENT REGION CALIFORNIA FISHERIES COOPERATIVE OCEANIC INVESTIGATIONS ATLAS NO. 33 BY THE SPONSORED STATES OF COMMERCE DEPARTMENT UNITED OCEANIC AND ATMOS...

More info »
RIE Tenant List By Docket Number

RIE Tenant List By Docket Number

SCRIE TENANTS LIST ~ By Docket Number ~ Borough of Bronx SCRIE in the last year; it includes tenants that have a lease expiration date equal or who have received • This report displays information on ...

More info »
Fourth National Report on Human Exposure to Environmental Chemicals Update

Fourth National Report on Human Exposure to Environmental Chemicals Update

201 8 Fourth National Report on Human Exposure to Environmental Chemicals U pdated Tables, March 2018 , Volume One

More info »
JO 7400.11C   Airspace Designations and Reporting Points

JO 7400.11C Airspace Designations and Reporting Points

U.S. DEPARTMENT OF TRANSPORTATION ORDER FEDERAL AVIATION ADMINISTRATION 7400.11C JO Air Traffic Organization Policy August 13, 2018 SUBJ: Airspace Designations and Reporting Points . This O rder, publ...

More info »
H966v2

H966v2

GENERAL ASSEMBLY OF NORTH CAROLINA SESSION 2019 H 2 HOUSE BILL 966 Committee Substitute Favorable 4/30/19 2019 Appropriations Act. (Public) Short Title: Sponsors: Referred to: April 26, 2019 A BILL TO...

More info »
Eli Hertz Reply 11 6 05c

Eli Hertz Reply 11 6 05c

R Even the most sacred precepts of international law can be manipulated to e pervert the truth. In its Advisory Opinion of 9 July 2004, the UN's Interna - p l tional Court of Justice ruled that Israel...

More info »
CompleteBusBook

CompleteBusBook

February 10, 2019 $ 1 BUS BOOK EFFECTIVE THROUGH JUNE 8, 2019 OCBus.com EFECTIVO HASTA EL 8 DE JUNIO 2019 EASY JUST GOT EASIER. Upgrade to version 2.0 See back for cool new features! CHANGE HIGHLIGHTS...

More info »
Second National Report on Biochemical Indicators of Diet and Nutrition in the U.S. Population

Second National Report on Biochemical Indicators of Diet and Nutrition in the U.S. Population

Second National Report on Biochemical Indicators of Diet and Nutrition in the U.S. Population Second National Report on Biochemical Indicators of Diet and Nutrition in the U.S. Population 2012 Nationa...

More info »
Untitled

Untitled

Harmoniz ed vision 4 hedule of the United States (2019) Re Tariff Sc Annotated f poses ting Pur or Statistical Repor GN p .1 GENERAL R ATION ULES OF INTERPRET inciples: wing pr ollo y the f verned b i...

More info »
UNSCEAR 2008 Report Vol.I

UNSCEAR 2008 Report Vol.I

This publication contains: VOLUME I: SOURCES SOURCES AND EFFECTS Report of the United Nations Scientific Committee on the Effects of Atomic Radiation to the General Assembly OF IONIZING RADIATION Scie...

More info »
FY 2019 PB Green Book

FY 2019 PB Green Book

NATIONAL BUDGET DEFENSE ESTIMATES FOR FY 201 9 OFFICE OF THE UNDER SECRE TA RY OF DEFENSE (COMPTROLLER) APRIL 201 8

More info »
625137

625137

2018-19 Nebraska All-Sports Record Book - Nebraska Communications Office -

More info »
Men VAW report Feb2015 Version6

Men VAW report Feb2015 Version6

" Working with Men and Boys to End Violence Against Women an d Girls Approaches, Challenges, and Lessons February 2015 This report is made possible by the generous support of the Americ an people thro...

More info »
pisa 2012 results volume I

pisa 2012 results volume I

PISA 2012 Results: What Students Know and Can Do tICS, themA StuDent PeRfoRmAnCe In mA ReADIng AnD SCIenCe Volume I rogramme for ssessment A tudent S nternational I P

More info »
Out of Reach 2018

Out of Reach 2018

2018 of OUT REACH THE HIGH COST OF HOUSING MADE POSSIBLE BY THE GENEROSITY OF:

More info »
WEF GlobalInformationTechnology Report 2014

WEF GlobalInformationTechnology Report 2014

Insight Report The Global Information Technology Report 2014 Rewards and Risks of Big Data Beñat Bilbao-Osorio, Soumitra Dutta, and Bruno Lanvin, Editors

More info »
WJP ROLI 2017 18 Online Edition 0

WJP ROLI 2017 18 Online Edition 0

l i a a o a i z a n r a s n i a i i a a n a e r d v v z F o i g a w i B o l l o l o m s d J a I o b e r t u g e s a t a u l a n i e i m e o B u B B n i l m z a r y g h s a a B r l k a m a n i o r J l ...

More info »