A Win WIn Solution The Empirical Evidence on School Choice

Transcript

1 A WIN-WIN SOLUTION The Empirical Evidence on School Choice FOURTH EDITION Greg Forster , Ph.D. MAY 2016

2 About the Friedman Foundation for Educational Choice The Friedman Foundation for Educational Choice is a 501(c)(3) nonprofit and nonpartisan organization, solely dedicated to advancing Milton and Rose Friedman’s vision of school choice for all children. First established as the Milton and Rose D. Friedman Foundation in 1996, the Foundation promotes school choice as the most effective and equitable way to improve the quality of K–12 education in America. The Friedman Foundation is dedicated to research, education, and outreach on the vital issues and implications related to school choice.

3 A WIN-WIN SOLUTION The Empirical Evidence on School Choice FOURTH EDITION Greg Forster , Ph.D. MAY 2016

4 Table of Contents Executive Summary ... 1 3 Introduction ... ... Choice in Education 3 Why Science Matters—the “Gold Standard” and Other Methods ... 4 6 ... The Method of This Report Criteria for Study Inclusion or Exclusion ... 6 ... Defining a “Study” 7 7 ... Search Method Part I: Academic Outcomes of Choice Participants 9 ... 10 ... The Importance of Random Assignment—The Gold Standard ... What the Gold-Standard Studies Show 11 15 Part II: Academic Outcomes of Public Schools ... 16 ... What the Studies Show 20 ... Part III: Fiscal Impact on Taxpayers and Public Schools 21 Measuring Fiscal Impact ... ... 22 What the Studies Show ... Part IV: Racial Segregation in Schools 25 Measuring Racial Segregation ... 26 ... 27 What the Studies Show 29 ... Part V: Civic Values and Practices 30 ... Measuring Civic Values and Practices 30 ... What the Studies Show ... Conclusion: Remarks on School Choice 33 34 ... Universal Choice Could Deliver and Education Revolution 34 ... Yes, Choice Improves Schools 34 ... Choice Could Work Much Better If We Let It ... 35 Only Universal School Choice Can Sustain Dramatic Change 37 ... References Academic Outcomes of Participants ... 37 Academic Outcomes of Public Schools ... 37 Fiscal Impact on Taxpayers and Public Schools ... 39 39 ... Racial Segregation 40 Civic Values and Practices ... Notes ... 41 About the Author ... 41

5 List of Tables 2 ... Table 1: Empirical Studies on School Choice 14 ... Academic Outcomes of Choice Participants Table 2: 19 Table 3: Academic Outcomes of Public Schools ... Fiscal Impact on Taxpayers and Public Schools 24 Table 4: ... Table 5: Racial Segregation ... 28 Table 6: Civic Values and Practices ... 32

6 The Friedman Foundation for Educational Choice The size of the benefit provided by existing school Executive Summary choice programs is sometimes large, but is usually more modest. This is not surprising because the programs This report surveys the empirical research on private themselves are modest—curtailed by strict limits on the school choice programs. It provides a thorough students they can serve, the resources they provide, and overview of what the research has found on five key the freedom to innovate. Only a universal educational topics: choice program, accessible to all students, is likely to deliver the kind of dramatic improvement American Academic outcomes of choice participants • schools need in all five of these important areas. • Academic outcomes of public schools Key findings: • Fiscal impact on taxpayers and public schools • Eighteen empirical studies have examined • Racial segregation in schools academic outcomes for school choice participants using random assignment, the gold standard of • Civic values and practices social science. Of those, 14 find choice improves student outcomes: six find all students benefit The evidence points clearly in one direction. and eight find some benefit and some are not Opponents frequently claim school choice does visibly affected. Two studies find no visible not benefit participants, hurts public schools, effect, and two studies find Louisiana’s costs taxpayers, facilitates segregation, and even voucher program—where most of the eligible undermines democracy. However, the empirical private schools were scared away from the evidence shows that choice improves academic program by an expectation of hostile future action outcomes for participants and public schools, saves from regulators—had a negative effect. taxpayer money, moves students into more integrated classrooms, and strengthens the shared civic values empirical studies (including all • Thirty-three and practices essential to American democracy. A methods) have examined school choice’s effect few outlier cases that do not fit this pattern may get a on students’ academic outcomes in public schools. disproportionate amount of attention, but the research Of those, 31 find choice improved public schools. consensus in favor of school choice as a general policy One finds no visible effect. One finds a negative is clear and consistent. effect. The results are not difficult to explain. School choice • Twenty-eight empirical studies have examined improves academic outcomes for participants and school choice’s fiscal impact on taxpayers public schools by allowing students to find the schools and public schools. Of these, 25 find school choice that best match their needs and by introducing healthy programs save money. Three find the programs competition that keeps schools mission-focused. It they study are revenue neutral. No empirical saves money by eliminating administrative bloat and study has found a negative fiscal impact. rewarding good stewardship of resources. It breaks down the barriers of residential segregation, drawing students Ten empirical studies have examined school • together from diverse communities. And it strengthens choice and racial segregation in schools. Of democracy by accommodating diversity, de-politicizing those, nine find school choice moves students the curriculum, and allowing schools the freedom to from more segregated schools into less sustain the strong institutional cultures that are necessary segregated schools, and one finds no net effect to cultivate democratic virtues, such as honesty, diligence, on segregation. No empirical study has found achievement, responsibility, service to others, civic that choice increases racial segregation. participation, and respect for the rights of others. 1 edchoice .org

7 .org edchoice • Eleven empirical studies have examined school choice’s effect on civic values and practices, such as respect for the rights of others and civic knowledge. Of those, eight find school choice improves civic values and practices. Three find no visible effect from school choice. No empirical study has found that school choice has a negative effect on civic values and practices. TABLE 1 Empirical Studies on School Choice Any Positive Effect No Visible Effect Any Negative Effect 2 Academic Outcomes of Choice Participants 14 2 31 1 Academic Outcomes of Public Schools 1 25 0 Fiscal Impact on Taxpayers and Public Schools 3 9 1 Racial Segregation in Schools 0 3 8 Civic Values and Practices 0 Note: Shows the number of empirical studies with each type of finding. The first row includes all studies using random-assignment methods. Other rows include all studies using all types of methods. A Win-Win Solution: The Empirical Evidence on School Choice 2

8 The Friedman Foundation for Educational Choice rather than “draining” money, and sends all types of Introduction students to private schools rather than “creaming.” They also point to the benefits of allowing each student School choice programs allow parents to decide to find the right school and the healthy incentives what schools their children attend using the public created by competition. funds dedicated to their children’s educations. Some “school choice” programs are limited to a choice School choice raises other important policy questions. among government-owned schools and thus provide Opponents portray school choice as a cost to taxpayers, a “choice” of options that are all ultimately controlled while supporters say school choice saves money for by the same entity. Private school choice—the subject taxpayers through improved stewardship of resources. of this report—gives parents the option of selecting Opponents of choice frequently have charged that it a private school. Such programs are among the most will exacerbate racial segregation in schools (which is prominent and successful reforms in the education already at epidemic levels in the government monopoly field. There are now 61 such school choice programs system) whereas supporters say choice is a tool for in 30 states and Washington, D.C. More than 399,000 breaking down segregation. And opponents argue that 1 students use these programs to attend private schools. private schools will not teach students the civic values and practices upon which democracy depends, such The most well-known form of school choice is school as respect for the rights of others and civic knowledge. vouchers, which give parents the ability to redirect Meanwhile, supporters say choice strengthens those their children’s education funding to a participating same democratic values and practices. private school for tuition support. More recently, education savings accounts have introduced an A large body of empirical evidence examines these innovation to the school voucher model, allowing questions using scientific methods. Twenty years parents to use redirected funds for other educational ago, before this body of evidence existed, there was services and expenses in addition to tuition costs. some excuse for making policy based on speculation, This further incentivizes good financial stewardship anecdotal observation, and intuition. Today, the because parents can select educational providers for effects of these programs are known, and there is price as well as for quality. An alternative approach, no longer any excuse for policymakers and opinion tax-credit scholarships, gives donors a tax credit if leaders to be ignorant of the facts. they donate money to nonprofits that provide private school scholarships. Finally, some programs give This report reviews the available empirical studies on parents a direct tax credit or deduction that reimburses the five policy areas described above. For participant them for a certain amount of private school costs. effects, a large body of studies using the “gold standard” method of random assignment is available, One of the most important questions about school so this report reviews that evidence. For the other choice is how it affects academic outcomes, both questions, it reviews all available empirical studies for the students who use it and in public schools at using any quantitative method. It also discusses the large. Defenders of the government school monopoly most important methodological issues confronted claim that choice does no good for the students who by research on this subject, and some of the larger use it and harms public schools by “draining money” implications of what the research finds. or by “creaming students”—that is, skimming off the best students who rise to the top and would be most attractive to private schools. School choice Choice in Education proponents, on the other hand, argue that choice improves academic outcomes both for the Americans expect and demand the right to select participating students and for public schools. They their own goods and services in every area other than say choice saves money for public school budgets, 3 edchoice .org

9 edchoice .org K–12 education, including everything from food, politically powerful parents, which means wealthier, housing, clothing, transportation, and medical care to better-educated, and (let’s face it) whiter parents. Poor magazines, haircuts, dry cleaning, and video games. and otherwise disadvantaged families too often get the If the government tried to assign people to live in least attention from the system. And they are the least certain neighborhoods or shop at certain grocery likely to have the means to seek private schooling or stores, Americans would howl in protest. They even move. Seventy percent of black workers, for example, expect and demand choice when it comes to education make less than $50,000 per year, compared to 52 percent 2 outside of K–12 schools—everywhere from colleges to of white workers. Indeed, a decreased ability to exit the trade schools to tutoring services. But when it comes to system only reinforces the system’s tendency to deliver K–12 education, the American idea that people should poor services. They are captive clientele. have stewardship over their own lives and choose for themselves rather than have government dictate what Thus, in the absence of parental choice, schools lack the they receive is not embodied in public policy. healthy, natural environment of client empowerment that is essential to producing better performance in The arguments typically used to defend this lack of choice most other service institutions. Hospitals know they must do a good job or lose patients. Professionals are empirically false or poorly reasoned. For example, like doctors and lawyers must provide good services teachers’ unions claim that choice “drains money” from public schools. But how would Americans respond if or lose clients. Stores must provide good value or they were told that from now on they would have to lose customers. This system is so critical to keeping receive all their medical care from a doctor assigned to institutions mission-focused that we take it completely them by the government, rather than from their current for granted—everywhere but in K–12 schooling. family doctor, on grounds that their choice to seek care It is widely agreed that monopolies generally provide from their own doctor “drains money” from the budget poor quality because nothing bad will happen to them of some other doctor chosen by the government? if they do not serve their clients well. When they get bad service, customers say, “I’ll take my business Meanwhile, the idea that school choice might improve elsewhere,” because they know that is what will public schools is dismissed as ideological claptrap. In prompt better service. They do this to nonprofit fact, the empirical evidence consistently shows it is institutions the same way they do it to businesses, the case, and the reasons are not hard to explain. One because they know it is not profit that creates better reason choice would improve public schools is that it performance; it is client choice. allows parents to find the right particular school for each individual child. Every child is unique and has The failure of education policy to embrace the American unique educational needs. principle that people should have stewardship over their own lives and make their own choices is a great But probably the most important reason school choice hindrance to reform. One way opinion leaders can would improve public schools is because it gives rectify this problem is by making the public aware of parents a meaningful way to hold schools accountable the large body of empirical research that examines how for performance. Under the current system, if a school choice affects participants, public schools, and the civic is not doing a good job, the only ways to get a better community at large. school—purchase private schooling or move to a new neighborhood—are expensive and impractical. Why Science Matters—the “Gold The current school system is especially unjust to low- income and disadvantaged families. As a government Standard” and Other Methods monopoly, the system is most likely to provide good There is no such thing as a “scientifically right” services to, and be responsive to the concerns of, A Win-Win Solution: The Empirical Evidence on School Choice 4

10 The Friedman Foundation for Educational Choice education policy. Science cannot identify what every respect other than the treatment. See the next education policy is most fitting to the intrinsic section of this report for more about the importance nature of the human person, or most aligned with of this method, and its results in studies of choice. America’s ideals of freedom and democratic self-rule. To answer those questions, one needs other kinds of Though it may be the best kind of research, the gold knowledge—knowledge about the nature of human standard of random assignment is not the only kind life, the meaning of freedom and democracy, and the of research worth considering. Where it is not possible 3 historic self-understanding of the American people. to conduct a random-assignment study, other kinds of research methods can produce useful information that sheds light on important policy questions. However, abstract ideas and history are not by themselves an adequate basis for public policy. The The next best research method is to track year-to- public hears competing claims about the real-world year changes in outcomes for individual students, effects of education policy in the concrete world of especially if it is possible to track them as they switch the here and now. The public wants, and rightly so, to in and out of schools of choice. know which claims are true and which are false. That is an empirical question. Addressing such questions is 4 the special right and duty of science. Tracking individual students over time removes from the analysis most, though not all, of the influence of unmeasured factors. If a student is advantaged in When evaluating the effectiveness of an education a way that is not measurable, that advantage will policy, it is especially important to rely on empirical typically be present in the student’s outcomes for both research of high scientific quality. The student year one and year two of the study. Thus, the change outcomes that education reforms are designed to in outcomes between year one and year two will promote are affected by many different influences, mostly be from other factors. Removing the influence including demographic factors (income, race, family of unmeasured factors allows the analysis to isolate structure, etc.), school factors (type of school, teacher the effect of the factors that are being measured, such quality, etc.), and intangibles, such as the level of as the offer of school choice or the decision to switch enthusiasm parents and teachers invest in a child’s in or out of a choice school. A moderate disadvantage education. The job of social science is to disentangle of this method is that some unmeasured factors the influence exercised by each of these factors as well influencing outcomes may change over time without as possible with the available evidence. those changes being tracked. A study that uses good research methods can If it is not possible to track individual students, good overcome those problems and provide reliable - research still can be conducted by tracking year information about what is influencing student to-year changes in individual schools. By tracking outcomes. But if scientific procedures are not rigorously year-to-year changes we are tracking whether followed, or if people make judgments without first students are learning over time, rather than how examining the science, it is easy to draw the wrong much they knew when they walked into school. conclusions about what factors cause what outcomes. And we are removing much (though not all) of the influence of demographic characteristics. It is The gold standard for empirical science is the method reasonable to expect that the unmeasured advantages known as “random assignment,” in which subjects of the students in a given school will be similar from are randomly divided into a treatment group that will year to year. If a school had highly advantaged receive the treatment being studied (such as school students last year, it probably will still have highly choice) and a control group that will not receive advantaged students this year. Mobility among it. Because the two groups are separated only by a the student population will create some change in random lottery, they are likely to be very similar in 5 edchoice .org

11 edchoice .org student characteristics from year to year, but not so plausibly have been conducted by economic much that we cannot learn anything from school-level researchers or government agencies not connected to studies. the education research community.) This report is the fourth edition in a series, and the earlier editions contain methodological discussions Criteria for Study Inclusion or Exclusion that may be of interest to the reader of this report. 5 The first edition was published in 2009. That report We did not include studies of programs outside the included only the research on how school choice U.S. in this review because the education systems of other countries work very differently than those of the effects public schools; its discussion of the methodological issues involved in that research was U.S., especially in the area of school choice. We did much more detailed than the overview provided not include case studies and other qualitative studies, 6 here. The second edition was published in 2011. That whose purpose is to help us learn to ask the right questions rather than to reach broadly generalizable version added the research on participant effects and answers; we included only studies using quantitative also provided more detail on the methodological data. We also did not include statistical modeling that issues involved in those studies than is included here. The third edition was published in 2013 and examines what would happen on certain hypothetical assumptions; we only include studies of what added reviews of the research on the fiscal, racial, and 7 civic impact of school choice. has actually happened as a result of school choice Readers seeking more programs. We did not include studies that are not extended methodological discussions of the studies publicly available, because scientific validity depends reviewed here may wish to consult those editions. upon transparency and the opportunity for researchers to critique one another’s work. And we excluded Previous editions bore the title A Win-Win Solution studies of private schooling generally, and of “choice” because the evidence on academic effects showed opportunities that are limited to government-owned positive results both for participants and for public schools (charter schools, magnet schools, district schools. Starting with the third edition, we have looked choice, etc.). beyond those two constituencies to consider three ways in which school choice affects the democratic However, to avoid the possibility of “cherry picking,” polity. Thus, school choice is not merely a win-win, we included all empirical, quantitative studies of but actually a “win-win-win” solution. U.S. private school choice programs, within the specific limits of each of our five reviews. We did not exclude studies simply because they used methods The Method of This Report we found inadequate or objectionable. We did this to avoid the risk of selectivity; if we excluded every This edition brings our research review on the five study that used a method we found objectionable, covered topics up to date as of February 25, 2016. The we could bias the review. Hence, this review includes body of research on school choice is very widely and three studies—one finding no visible effect on actively discussed. The community of professional academic outcomes of participants, one finding social scientists focused on school choice research is a positive effect on academic outcomes in public large but not too large for easy communication, and schools, and one finding no effect on civic values and all of the research has occurred since the rise of the practices—using methods of highly questionable internet. Though the possibility of a study being value. These studies and their shortcomings are overlooked can never be ruled out, discovering the discussed in the text below. available research is much less difficult in this field than in most other fields of social science. (The In addition, three of the five topical reviews included exception to this is fiscal analyses, which might A Win-Win Solution: The Empirical Evidence on School Choice 6

12 The Friedman Foundation for Educational Choice in this report excluded some studies based on differences between the analytical models are method, not because we judged the methods to be sometimes subtle, but subtle differences can be wrong, but to appropriately limit the scope of our extremely important. Thus, re-analyses of the same review. These limitations are discussed in more detail data using new models (even if the differences seem in the relevant sections; for convenience, they are small) must be counted as multiple studies. If we summarized here. The review on academic outcomes tried to distinguish which changes of statistical of choice participants is limited to “gold standard” models are substantial enough to be worth counting random-assignment studies, because where a large as new studies, and which are minor enough to be body of such studies exists, it ought to be given considered trivial variations on the same study, we priority. The review of studies on racial segregation would introduce an almost unlimited opportunity for excludes methods that do not measure schools against “cherry picking” selectivity. an appropriate measurement of racial segregation. The review of studies on civic values and practices To add further protection against “cherry picking,” excludes merely descriptive studies, because they do we classify studies by whether their various analytical not address a relevant policy question. models produced any positive result or any negative result, rather than picking and choosing (or allowing the studies’ authors to pick and choose) which of a study’s analyses “really count” and which don’t. A Defining a “Study” study typically includes multiple analytical models— sometimes many of them, occasionally even more This review counts analyses as separate “studies” if than 100. Selecting (or allowing the studies’ authors to they examine separate school choice programs or if select) among these models would make it too easy to they use different sets of analytical models to examine ignore findings that fail to confirm our (or the authors’) their data. Analyses examining separate programs 8 biases. should be considered as separate studies because the Following scientific convention, we classify programs are different; every school choice program analyses that do not achieve statistical significance as is unique. If the same researcher finds that a program having found “no visible effect.” in New York has a positive effect and also that a program in Washington, D.C. has a positive effect, Search Method she has produced two different findings, not the same finding twice. Similarly, analyses that use different statistical models should be considered as separate Most of the studies added to this report since the previous edition came to the author’s attention studies because they are asking different questions informally, either through his own ongoing work in of the data. Replication is the essence of science; the school choice research field or as a result of others scientific methods will invariably produce occasional in the field bringing these studies to his attention. false positives and false negatives, which is why we (It is difficult to work in this field and not be aware of wait until something has been studied multiple times before we consider conclusions firmly supported. new studies as they come out!) One part of this replication process is the gradual improvement the analytical models we bring to However, to help ensure the review was comprehensive, the author conducted two formal any data set, as the scientific community raises new questions and develops new methodological searches. The first was a search of the Education Resources Information Center (ERIC) database. Four insight. It is therefore not at all unusual for the same searches were conducted using the search terms “school researcher to go back to the same data set and analyze choice,” “voucher,” “tax credit,” and “education it in new and different ways, even with the same 9 Since the previous edition of this savings account.” basic research question in mind (e.g. “Did this report brought the review up to date through January program have a positive effect on math scores?”). The 7 edchoice .org

13 .org edchoice 2013, we searched for all relevant studies published 10 in 2013 or later. The second search was of the Journal Storage (JSTOR) database. We again conducted four searches, using the same search terms, for all relevant 11 studies published in 2013 or later. A Win-Win Solution: The Empirical Evidence on School Choice 8

14 The Friedman Foundation for Educational Choice PART I Academic Outcomes of Choice Participants edchoice .org 9

15 edchoice .org here have been 18 studies using random- random-assignment research design. Students who T win the lottery and are offered choice can be compared assignment methods to examine how school to students who lose the lottery and were not offered choice affects the academic outcomes of participants. choice. If any systematic (i.e., non-random) differences This body of evidence shows that school choice benefits students. Fourteen studies find positive between the outcomes of the two groups are effects on school choice participants: six find choice observed, those differences can be attributed to the offer of choice because nothing separates the groups had a positive effect across all students participating but the offer of choice and randomness. and another eight find choice had a positive effect on some student groups and no visible impact on other One limitation of random-assignment research is the students. Two studies find no visible effect from choice. randomization process itself; a random-assignment Two studies on Louisiana’s voucher program find that study can be only as good as the randomization used it had a negative effect. to create the treatment and control groups. In most cases, where random assignment has been used to The Importance of Random Assignment— study school choice programs, the randomization occurs at the level of the program itself, because the Gold Standard the program is oversubscribed. This presents no difficulties. However, in some cases the randomization When examining academic effects of school choice happens at the level of the individual school—the on participants, this report focuses on studies treatment and control groups are created not because using random-assignment methods. These studies the program as such is oversubscribed, but because separate subjects into “treatment” and “control” some schools are. In those cases, the effect being groups randomly. Random assignment generates studied is not the effect of the program as such but high confidence that factors other than the one being the effect of those particular schools upon the students studied—the “treatment”—are not influencing the who choose those schools. Attrition is another results. limitation of random assignment. For instance, if significant numbers of students drop out of the The special value of random-assignment research is program (or the study!) over time, it becomes more that it removes not only the influence of observable difficult for results to achieve standard levels of factors, such as demographics, but also the vast array statistical significance, making it less likely the study of unobservable factors that researchers know will achieve a positive or negative finding and more influence education, but cannot directly measure. likely it will find no visible effect. For example, researchers agree widely that one of the key drivers of student outcomes is parental Nonetheless, because random assignment is so motivation. Parents who highly value the education preferable to other methods, it should be given of their children are an important positive influence priority whenever a large body of random-assignment on outcomes. Random assignment assures high studies exists. It would make no sense to ignore confidence that differences in factors such as this are the difference between the proven reliability of not influencing research results. gold-standard studies and studies that are more methodologically limited. Moreover, identifying Unfortunately, it usually is not possible to conduct all the non-gold-standard research that has been random-assignment research on education policy. done on this question over the years would be too However, school choice has been one of the rare cumbersome to do here. A great deal of empirical exceptions. When there are more applicants for research has compared public and private schools’ a choice program than there are available slots, a effects on test scores using methods other than random lottery is often used to determine who random assignment. This research question actually may participate. This creates a naturally occurring A Win-Win Solution: The Empirical Evidence on School Choice 10

16 The Friedman Foundation for Educational Choice goes all the way back to the origins of modern trait. A student with a black father and a Hispanic Coleman education science, with the well-known mother is arbitrarily labeled black by Krueger and Report of 1966 and James Coleman’s follow-up Zhu, but would typically be at least as likely to research. It is worth noting, however, that most of consider herself Hispanic or biracial. Krueger and the studies that rise to a reasonable level of scientific Zhu also added students with significant missing 12 quality have found in favor of private schools. data to their data set and failed to control for the students’ baseline scores (a standard step in scientific education analysis). Unsurprisingly, through these What the Gold-Standard Studies Show manipulations they were able to lower the variable for statistical significance below the conventional 13 threshold for recognizing a finding. The last edition of this report counted 12 random- assignment studies of academic effects of school choice on participants. Readers seeking descriptions Paul Peterson and William Howell, the authors of the original study, have demonstrated how badly of those studies should consult previous editions of this report. Five additional studies have been distorted Krueger and Zhu’s findings are. They published or come to our attention since then. published a series of 120 reanalyses of their data set, Moreover, a publication that previous editions of each using a different set of specifications. These this report discussed, but did not recognize as a analyses demonstrated that the positive finding for black students is robust across numerous different study, is now recognized as a study. These 18 studies collectively show a positive effect from school assumptions about racial identification. Howell and choice. Peterson show that the positive effect disappears only if the analysis incorporates Krueger and Zhu’s exact Among the 12 studies reviewed in the previous A combination of arbitrary racial redefinition, students with missing data, and exclusion of baseline scores. If Win-Win Solution report, six find a positive effect for all students. Five find a positive effect for some we leave out any two of these three, the results become 14 student groups (black students in some studies; positive. The Krueger-Zhu statistical model must be students leaving especially low-performing public regarded as discredited. schools in others) with no visible impact on other groups. Probably the most plausible hypothesis to Peterson and Howell’s publication exposing the explain the studies finding benefits for some groups deficiencies of Krueger and Zhu’s analysis was but not others is that student groups that were served discussed in earlier editions of this report but was more poorly in their public schools stood to gain not classified as a study. However, it meets all of more from the opportunity to choose a new school. this report’s criteria for being counted as a study. Therefore, it is counted here as a study finding a The remaining study, a reanalysis of data from positive effect on some participating students. a previous study, finds no visible impact from choice. However, the authors Alan Krueger and Pei Of the five random-assignment studies that have Zhu introduced two methodological changes that been released or come to our attention since the undermine the study’s validity. First, they changed previous edition of this report, two find a positive the way students were classified by race in a way effect for some participating students and no that violates the rules of social science. When student negative effects for any student group. Matthew self-identification is absent, the generally accepted Chingos and Peterson published a new analysis of method is to use the race of the mother; Krueger data from a privately funded New York City voucher and Zhu classified a student as black if either parent program in the late 1990s, which they had analyzed 15 was black. This method is indefensible because it is The using different methods in a previous study. asymmetrical, treating “black” as a dominant racial new analysis included a more sophisticated array of 11 edchoice .org

17 .org edchoice tests for subgroup effects and other factors. It finds That said, it is all the more appropriate to count the that the program had no visible effect on college study as having a positive finding given that the enrollment or attainment rates for all students, Krueger and Zhu statistical model must be regarded but did have positive effects on those rates both for as discredited for the reasons described above. If one minority students in general and for black students were to be selective among this report’s findings, in particular. The new analysis also found that the one would have much stronger grounds for selecting program had a positive effect on college enrollment its replication of Peterson and Howell’s model and and attainment rates for only one other subgroup disregarding its replication of Krueger and Zhu’s. the authors examined: children of mothers born in the U.S. The size of the effects varied according to One random-assignment study has come to our different statistical models used by the authors. attention since the previous edition of this report In the simplest model, positive effects are six was published. A study by Eric Bettinger and Robert percentage points on college enrollment rates and Slonim published in 2006 examined a privately five percentage points on bachelor’s degree funded school voucher program in Toledo, Ohio. The attainment for black students and five percentage study’s primary topic was the effect of the program points on enrollment and three percentage points on altruistic behavior, but it also contained an analysis on attainment for children of U.S.-born mothers. No of participants’ math scores. It found no visible effect 16 18 negative effects on any student group were found. from the program. Two random-assignment studies published since the Analyzing the same New York City program, previous edition of this report, both examining the Marianne Bitler and three co-authors find no visible Louisiana Scholarship Program (LSP), find a negative effect on participants as a whole. They also divide effect on academic outcomes for participating students into quintiles based on student achievement and find no visible effects on academic students. The first study was conducted by Atila Abdulkadiroglu, Parag Pathak, and Christopher outcomes in any quintile. They also replicate the Walters. It finds that in the first year, the voucher racial group analyses of both Peterson and Howell program had a negative effect on participants’ math, and Krueger and Zhu, discussed previously, using a different method of accounting for missing data. reading, science, and social studies scores. The When they replicate Peterson and Howell’s statistical negative effect on math scores was 0.4 standard model, they find a positive effect for black students; deviations—a large effect, in this case equal to a 50 percent increase in a student’s chance of receiving a when they replicate Krueger and Zhu’s, they find no 19 17 failing grade. The second study, by Jonathan Mills visible effect. and Patrick Wolf, finds that the program had a In their discussion, Bitler and her co-authors go to negative effect on math and no visible effect on great lengths to give reasons why they would prefer reading in its first two years. The negative effect in 20 to think that the program had no impact. However, math was 0.34 standard deviations over two years. this recitation of their personal preferences does not The most likely explanation for this anomalous negate the empirical findings they report. Because finding is low private school participation in the they report positive findings among their results, we program due to poor program design and fear count this study as having found a positive effect for of future action from hostile regulators. In sharp some participating students. This is consistent with the practice we have followed in classifying all studies contrast to other choice programs, only a small in this research review—where multiple statistical minority of eligible private schools in Louisiana models are reported, this report includes them all, to participate in the voucher program. Less than one- avoid accusations of selectivity (“cherry picking”). third of Louisiana private schools chose to participate 21 in the program in its first year. Survey research finds A Win-Win Solution: The Empirical Evidence on School Choice 12

18 The Friedman Foundation for Educational Choice that fear of “future regulations” was the number student populations is another—although it would one reason cited by private schools choosing not to be difficult to explain why this was a program- 22 participate. busting problem in Louisiana but not in New York, Milwaukee, Washington, D.C., Cleveland or any other Among other regulatory burdens, participating school choice program. One hypothesis that is very unlikely to find empirical validation is the possibility schools must administer the state test and can be that gains in participating private schools were removed from the program if their scores are too low. outshone by even greater gains among public school They are also subject to inspections by public school students. As Abdulkadiroglu, Pathak, and Walters officials while tests are being administered. However, note, the students who lost the voucher lottery ended in addition to these existing regulations, we should 25 also consider the survey results showing that private up in low-performing public schools. school leaders were worried about future regulations. The key issue may not have been the poor initial design of the program so much as a lack of trust that even these undesirable terms of participation would continue to be honored once schools entered the program. Moreover, an attempt by the U.S. Department of Justice to gain extensive regulatory control over participating schools through a strained reading of desegregation law, even though it was eventually struck down in court, would also have been noticed by high-quality Louisiana private schools that might 23 have been interested in participating. Schools choosing to join and remain within a choice program under such adverse conditions are likely to be the worst performing schools. Abdulkadiroglu, Pathak and, Walters note that “survey data show that LSP-eligible private schools experience rapid enrollment declines prior to entering the program, indicating that the LSP may attract private schools 24 Where a struggling to maintain enrollment.” program is unattractive to private schools, only schools desperate for cash flow can be expected to enter the program. And those schools will usually be the worst schools. If a private school is strapped for cash, that is a sign parents have decided it is not a good deal. Other possible explanations have been offered to explain the negative finding. Though they are less likely to be primary factors, they are worth noting. Lack of alignment between private school curricula and state tests is a possible factor. Schools not being ready to serve more challenging, disadvantaged 13 edchoice .org

19 edchoice .org Academic Outcomes of Choice Participants TABLE 2 Results Any Positive Effect Year Author Location No Visible Effect Any Negative Effect Some Students All Students X Louisiana 2016 Mills & Wolf X Louisiana 2016 Abdulkadiroglu et. al X New York 2015 Chingos & Peterson X New York 2015 Bitler et. al. X New York 2013 Chingos & Peterson X D.C. 2013 Wolf et. al. X New York 2010 Jin et. al. X Charlotte 2008 Cowen X Toledo 2006 Bettinger & Slonim X New York 2004 Howell & Peterson X New York 2004 Krueger & Zhu X New York 2003 Barnard et. al. X New York Howell & Peterson X D.C. 2002 Howell & Peterson X Dayton Howell & Peterson Charlotte X 2001 Greene Milwaukee X 1998 Greene et. al. Milwaukee X 1998 Rouse Note: This table shows all empirical studies using random-assignment methods. A Win-Win Solution: The Empirical Evidence on School Choice 14

20 The Friedman Foundation for Educational Choice PART II Academic Outcomes of Public Schools 15 edchoice .org

21 edchoice .org hirty-three empirical studies have been conducted was published in 2013. Readers seeking a descriptive T on how school choice programs affect academic overview of those studies should consult previous outcomes in public schools. Of these studies, 31 find editions of this report. Ten additional studies have that choice improves academic outcomes at public been published or have come to our attention since then. Of those 33 studies, 31 find school choice schools. One of the remaining studies finds that choice has no visible impact on public schools, and one finds improves public schools, one finds no visible effect, 26 a negative impact. and one finds a negative effect. Seven studies examine Milwaukee’s voucher For academic outcomes of public schools, this report looks at empirical studies using all methods. It is not program, and all seven find a positive effect on public schools. Milwaukee’s voucher program possible to conduct random-assignment research on is available to all students who meet an income how choice affects public schools. Random assignment restriction, so eligible students are spread across is possible only in studies of participants because the city. Research methods in Milwaukee therefore of the naturally occurring opportunity to conduct a random lottery when there are more applicants to a cannot compare “public schools that are not affected choice program than there are slots available. There is by vouchers” and “public schools that are affected no naturally occurring equivalent that would permit by vouchers.” Typically, they compare public schools random-assignment research methods in studying the with more students eligible for vouchers to public effects of choice on public schools. We must therefore schools with fewer students eligible for vouchers. turn to other evidence. In two cases, researchers used the number of nearby private schools participating in the voucher program, Fortunately, this question has been studied only more or the number of available seats in such schools, recently and the amount of evidence is manageable. as a measurement of voucher impact. All of these The body of research is also of good methodological methods are like testing the effectiveness of a medicine by comparing the effects of a large dose to quality, increasingly so over time. The last decade has the effects of a small dose, rather than to the effects seen major improvements in the quality of studies on this question, to the point where some approach the of not taking it at all – the research will tend to make the effect of vouchers look smaller than it really is. confidence level of gold-standard random assignment. But it is the best that can be done given the absence of a better control. It is also important to bear in mind that these studies are examining a different kind of question from those analyzing the effect on participants. The absence One of these studies has come to our attention since of random assignment is not as great a problem the last edition of this report. In an unpublished here. There is no act of parental choice that must be dissertation, Nicholas Mader confirms the overcome methodologically. “Choosers” and “non- longstanding research finding that exposure to the choosers” are not being compared; all the relevant voucher program produces moderate academic gains students are non-choosers. The only comparison is in Milwaukee public schools. He measures changes between schools exposed to choice and schools not in the intensity of competitive pressure on a public exposed, which is usually an easier methodological school by measuring the “classroom capacity” (i.e. barrier to overcome. number of empty seats, by grade level) in nearby private schools. This is a problematic measure insofar as private schools have some capacity to What the Studies Show increase or decrease the number of “seats” they provide in response to demand. Mader finds a There had been 23 studies of academic effects on positive relationship between increased capacity in 27 public schools when the last edition of this report nearby private schools and public school performance. A Win-Win Solution: The Empirical Evidence on School Choice 16

22 The Friedman Foundation for Educational Choice Fourteen empirical studies have been conducted Goldhaber, and Figlio. In a previous study, they on how two Florida voucher programs and one found low-performing schools improved when Florida tax-credit scholarship program have affected their students could become eligible for vouchers if 30 academic outcomes at public schools. Twelve find In the new study, they the schools didn’t improve. choice has improved Florida public schools and find that this accountability pressure is specifically one finds a negative effect. One of the two voucher connected to changes in schools’ institutional programs made all students at underperforming practices and that these changes are, in turn, 31 schools eligible for vouchers, so researchers were able connected to test score improvements. to measure the effect of vouchers in two ways: 1) by comparing performance at the same school before In the third study published since the last edition of and after voucher eligibility and 2) by comparing this report, Daniel Bowen and Julie Trivitt examine very similar schools that were just over or just under how schools were affected after the voucher program the threshold for voucher eligibility. Researchers also targeting failing schools was ended in 2006. They were able to measure the effect of “voucher threat” found that the designation of schools as “failing” at low-performing schools that were in danger of had a positive effect (a so-called “stigma effect” becoming eligible for vouchers. For the other two motivating schools to improve in order to get rid of programs (a voucher program for students with the failing grade) and that the removal of voucher special needs and a tax-credit scholarship program eligibility from these schools did not reduce this effect, for low-income students) researchers used methods indicating vouchers were not producing a positive similar to those used in the Milwaukee studies. effect. Public schools saw no visible change in math scores, and Bowen and Trivitt found the removal of Three of those studies were published since the vouchers increased reading scores—a negative finding 32 last edition of this report. One is a new analysis by for vouchers. David Figlio and Cassandra Hart of data they had analyzed in an earlier study. That study had used The negative finding is hard to explain given that several different measurements of the presence of nine other studies find a positive effect from the private school competition to examine the impact of same voucher program, including one study that 28 Florida’s tax-credit scholarship program. Among focuses (as this one does) on how schools were 33 the refinements introduced in the new version of affected by the removal of the vouchers in 2006. Bowen and Trivitt write: “Despite the exhaustive data the study, they use the number of nearby houses of worship as a proxy measurement for private school available, we are not currently able to explain the negative effect of the threat on reading performance competition, alongside more conventional methods 34 A change in the state’s system for of measurement—distance to the nearest private definitively.” school, number of private schools within a given grading schools did require Bowen and Trivitt to radius of the public school, number of different types make judgment calls about how to measure school exposure to the stigma effect, but there is no obvious of private schools within a given radius, and number reason to think their method deficient. of “slots per grade” (similar to Mader’s “classroom capacity,” above). The new study finds a positive effect on public schools in both reading and math for all Twelve studies have been conducted on the effect of school choice programs in other places (Maine, five measures of private school competition, though Vermont, Ohio, Louisiana, Indiana, Washington, D.C., the effect was much smaller for slots per grade and 29 and San Antonio, Texas). Eleven of those studies find houses of worship than for the other three measures. choice improves public schools’ academic outcomes, Another study that offers a new analysis of data and one finds that it made no visible difference. previously examined by the same researchers was The outlier study finding no effect is a study of the published by Cecilia Rouse, Jane Hannaway, Dan federal voucher program in Washington, D.C. This 17 edchoice .org

23 edchoice .org is the nation’s only school choice program with a within a given distance, 3) number of different types “hold harmless” provision that allocates additional of such schools within a given distance, and 4) an money to the public school system to “compensate” index that measures nearby private school types. She for the loss of students. The provision is intended to finds that in Louisiana, higher levels of competition insulate the public school system from the competitive on the last three of these measures increased both effects of school choice. Thus, the absence of a visible math and English scores at public schools exposed effect in this study does not detract from the research to vouchers. In Indiana, she finds no visible effect on consensus in favor of a positive effect on public math scores, but a positive effect on English scores schools. from the last measure (the index of types of schools) and in some cases, depending on the distance from the Six of these studies have been published, or come school used, from the third measure (the number of 38 to our attention, since the last edition of this report. types of schools). Three examine a privately funded program offering vouchers to all students in the Edgewood school Finally, Egalite published another study on how the district, near San Antonio, Texas, starting in 1998. Louisiana voucher program affects public schools. John Diamond finds the percentage of Edgewood This study uses the same four measurements of public school students passing state tests increased at exposure to private school competition, but analyzing a higher rate after the program was introduced, and them with a more complex statistical model. In this Edgewood’s graduation rate increased faster relative study, she finds positive effects in math when using 35 whole. to the state as a two of her measurements—the number of private Notably, these are very schools in a given radius and the number of types rough, low-quality methods of analysis, and readers of private schools in a given radius. In contrast to would be unwise to rely much on these findings. Egalite’s first Louisiana study, both positive findings Nonetheless, this study is included in this review to remained when either a larger or smaller radius was avoid “cherry picking” selectivity. used. However, she finds no visible effect on English 39 scores. Two other studies of this program used better methods. In the first of these studies, John Merrifield and Nathan Gray find test scores and graduation As the first studies on how school choice affects rates improved at a faster rate in Edgewood than public schools emerged, some speculated that the improvements they found in public schools might in two selected sets of control districts after the 36 The second, conducted be caused by other factors besides a positive impact introduction of the program. from school choice. Those alternate theories included by Gray, Merrifield and Kerry Adzima, confirmed a statistical phenomenon known as “regression to the this finding using more sophisticated statistical 37 mean,” and the possibility that the worst students methods and a different set of control variables. This is a better method, although still not as good as were leaving public schools. Subsequent research the methods used in most studies of school choice rigorously tested these alternative hypotheses and programs’ effect on public schools. Because all found them to be unsupported. These theories were students in the district are eligible, designing a good extensively discussed in the original 2009 edition of study to examine its effects is a challenge. this report; readers seeking a review of them should consult that edition. Notably, Bowen and Trivitt’s Two studies that examine school choice programs in Florida study is the first empirical study ever to test Louisiana and Indiana were published in a paper by one of these alternative hypotheses and find in favor Anna Egalite. She uses a variety of different ways to of it. In this case, they tested the “stigma effect” measure exposure to private school competition—1) hypothesis, which holds that improvement in schools distance to the nearest private school participating labeled “failing” and eligible for vouchers is due to in the voucher program, 2) number of such schools the stigma of the failing label rather than the effects A Win-Win Solution: The Empirical Evidence on School Choice 18

24 The Friedman Foundation for Educational Choice of school choice. However, seven other empirical studies have also specifically tested for the presence of a stigma effect (in Florida and Ohio) and found that this effect either did not exist or was not large enough to explain away the school choice effect. Also, stigma cannot explain the positive findings for Milwaukee, Florida’s two other programs, or the century-old “town tuitioning” voucher systems in Maine and 40 Vermont. TABLE 3 Academic Outcomes of Public Schools Results Author Year Location Any Negative Effect No Visible Effect Any Positive Effect X Louisiana 2016 Egalite X Louisiana 2014 Egalite X Indiana 2014 Egalite X Florida 2014 Figlio & Hart X Florida 2014 Bowen & Trivitt X San Antonio 2014 Gray et. al. X Florida 2013 Rouse et. al. X Florida 2013 Chakrabarti X Florida 2011 Figlio & Hart X Florida 2011 Winters & Greene X Ohio 2011 Carr X Milwaukee 2010 Mader X Milwaukee 2009 Greene & Marsh X San Antonio 2009 Merrifield & Gray X Ohio 2008 Forster X Florida 2008 Forster X Milwaukee 2008 Chakrabarti X Florida 2008 Chakrabarti X Milwaukee 2008 Chakrabarti X Florida 2007 Rouse et. al. X Milwaukee 2007 Carnoy et. al. X San Antonio 2007 Diamond X D.C. 2007 Greene & Winters X Florida 2006 Figlio & Rouse X Florida 2006 West & Peterson X Florida 2004 Greene & Winters X Florida 2004 Chakrabarti X Milwaukee 2002 Greene & Forster X San Antonio 2002 Greene & Forster X Maine 2002 Hammons X Vermont 2002 Hammons X Milwaukee 2001 Hoxby X Florida 2001 Greene Note: This table shows all empirical studies using all methods. 19 edchoice .org

25 edchoice .org PART III Fiscal Impact on Taxpayers and Public Schools A Win-Win Solution: The Empirical Evidence on School Choice 20

26 for Educational Choice The Friedman Foundation of savings. But a program’s fiscal impact on the public here have been 28 empirical studies examining the fiscal impact of school choice on taxpayers includes its effect on both, so each study’s results are T classified based on the total effect. and public schools. Of those, 25 find school choice programs save money, and three find the programs studied are revenue neutral. With regard to taxpayers, the public spends money on schools at all three levels of government—federal (10 percent), state (45 percent), and local (45 percent)— Measuring Fiscal Impact but direct fiscal impact on taxpayers usually occurs 41 only at the state level. School choice does not This report covers all empirical studies of actual have much immediate impact on federal and local school choice programs. It does not cover analyses taxpayer funding for schools, because funding at those levels is not highly sensitive to changes in that predict fiscal impact using economic modeling; such analyses are not empirical studies. Analyses of school enrollment. Federal funding mainly flows this kind are familiar to policymakers and opinion through Title I for low-income students and special education programs. Title I allocates funds based leaders through the widespread use of legislative on the demographics of the school district, and “notes” and comparable analyses to predict the federal special education spending was reformed impact of legislation. These analyses are legitimate and important, as legislators and the public must in 1997 specifically to disconnect funding formulas have some basis on which to evaluate specific from enrollment levels (to avoid creating a financial legislative proposals, and empirical data from the incentive to place students in special education). Meanwhile, local funding typically comes from future are, unfortunately, not available. However, property taxes. Small amounts of federal and local for the general purpose of evaluating the effect of funding may vary with enrollment, but they are too school choice, empirical research on actual program effects is far preferable to modeling. complex and too small as a percentage of education 42 spending to be worth tracking. School choice affects the finances of taxpayers and public schools. Because public schools are government By contrast, school choice has a major effect on state agencies, both represent fiscal impact on the public. funding. Spending on schools has been migrating Fiscal studies are therefore classified here based on toward the state level over time, to the point where the total effect they find on taxpayers and public education is now a very large portion of most state schools. Additionally, this method is appropriate budgets. Education spending sometimes even makes 43 because some studies report the total effect on both up a majority of the state general fund. This change without differentiating between effect on taxpayers has been driven in large part by concerns over and effect on public schools. equity in funding across districts. Because of those concerns, almost every state funds schools based on To some extent, effect on taxpayers and effect on their enrollment levels, allocating a base amount per public schools are interchangeable. Money saved in student to each district (usually with some adjustments state budgets is money that can be used for increased for local conditions). Most states have two major funding of public schools, and money saved in school systems for funding schools: a “formula funding” budgets is money taxpayers can recoup through system that distributes the majority of spending based corresponding funding cuts. This interchangeability on number of students and a separate fund for capital is greatly mitigated in practice because states have expenses, such as building costs. many budget priorities other than public schools, and funding of public schools is not strongly responsive to As a result, school choice creates savings and costs improved efficiencies in school budgeting. In the text for state budgets. When a student uses school choice below, those categories are treated as two distinct loci to leave public school for a private school, the state 21 edchoice .org

27 edchoice .org 44 number of students enrolled. must cover that student’s cost to the choice program, This means school but it also spends less on public schools by an amount choice programs produce financial windfalls for local equal to one student’s worth of funding. For schools if they redirect less than that amount per example, if the share of public school funding that student. comes out of the state budget is $5,000 per student, and the state offers vouchers equal to $4,000 per Though local taxpayers might not immediately student, a typical student using the program will save benefit much because property tax levels are not state taxpayers $1,000. For each student entering the sensitive to enrollment changes, local schools would voucher program from a public school, the state’s benefit a great deal because they would have more spending on vouchers will go up by $4,000, but its money to spend per student. This is one possible spending on public schools will go down by $5,000. explanation for the positive impact choice has on Thus, students who leave public schools because public school outcomes. Of course, it is possible that of the school choice program create savings for the as school choice generates large savings for local state budget. The net effect of the program is equal schools, local governments may eventually take to the amount of savings from students leaving notice and recover some of those savings for taxpayers public schools minus the total cost of the program. by lowering property taxes. This total cost includes costs associated with the small number of voucher participants who would have self-financed private education even in the absence What the Studies Show of the program, generating costs but no savings. The previous edition of this report counted six There is always some variation within these broad empirical studies on the fiscal effects of school choice. parameters. Some students will not use the full However, one of those “studies” actually includes voucher amount, reducing the program cost. Savings 12 separate analyses of 12 separate programs, so— in public school spending also will vary from student applying the same standard that we have consistently to student as a result of state funding formulas that applied to academic effects studies in every edition adjust spending somewhat based on local conditions of this report—it ought to have been counted as 12 45 in each district. This would have produced studies rather than as one. a total of 17 fiscal studies in the previous edition. Another issue is the fiscal impact of school choice Since then, 11 more studies have been published; on public schools. Here, as with state budgets, there 10 are included in a single report that separately are two sides to the ledger. When a student leaves a analyzes the fiscal effects of 10 school choice public school using a choice program, the school programs. Of these 28 studies, 25 find that loses all the costs associated with educating that school choice programs save money, and three student but not all the funding. As has been find the programs studied to be revenue neutral. noted already, federal and local education spending does not vary much with enrollment, The publication that studies 12 programs, released in so those funds stay when students leave. That 2007, is a comprehensive review of the fiscal effects of means public schools are left with more money all school choice programs in existence from 1990 to to serve the students who remain. 2006. Two of the programs it studies are century-old “town tuitioning” programs in Maine and Vermont, An empirical study on schools nationwide confirms which were created to cover private school tuition this. Benjamin Scafidi examines school finances in in small towns that decided not to create their own every state and finds that out of a total of $12,450 public schools when the government education spent per student in 2008–09, only 64 percent ($7,967) monopoly was first created. Because of the is made up of variable costs that change with the unique design of these programs, they are not A Win-Win Solution: The Empirical Evidence on School Choice 22

28 The Friedman Foundation for Educational Choice only revenue neutral for states (which direct savings from those 10 programs was $1.7 billion from 47 100 percent of state education spending to the 1990 through 2011. program) but generate no savings for local public schools (as the affected towns have no public Another study of a single program has come to our schools). A third program, a voucher for students attention since the previous edition of this report. with special needs in Utah, is found to be revenue- Merrifield and Gray include in their study of the neutral for the state, which directs 100 percent of its privately funded program in San Antonio, Texas, spending for those students into the program. Its an analysis of how the program affected taxpayers. impact on public schools was not assessed due to lack Because the program was privately funded, this of necessary data on local special education spending. analysis explores a different set of questions than the rest of the research in this field. Merrifield and Gray The other nine programs in this study are all found find the program attracted families to the Edgewood to have a net positive fiscal effect. The three tax-credit school district, where they would become eligible for scholarship programs studied are all found to cost vouchers. The increase in the number and value of money at the state level, on the assumption that state houses, including about a $6,500 rise in the value of legislatures do not reduce public school funding to the average house, delivered a $10.6 million benefit 48 offset lost tax revenue from the tax credits, but these to local taxpayers. costs are outweighed by savings for public schools. However, this balance is reversed in the Milwaukee voucher program, whose legislation—over the objection of local school choice advocates—includes a transfer of funding from local schools to the state that is unnecessary for the operation of the program. This “funding flaw,” as it is locally known, creates a negative fiscal impact on local public schools, but is outweighed by savings at the state level (including funds seized by the funding flaw). Fortunately, the funding flaw is now in the process of being phased out. Unfortunately, the phase-out is so gradual that it will not be complete until 2025. The other five programs are either positive or neutral at both the state and local school levels, and all have an overall 46 positive fiscal effect. Since the previous edition of this report, another large-scale evaluation of multiple programs has been published. A 2014 study by Jeff Spalding examines the fiscal effect of ten school choice programs from 1990 through 2011. Unlike the 2007 review, this study does not differentiate between savings for state budgets and local school budgets. It simply subtracts the per- student cost of providing the program from the per- student reduction in educational costs, yielding a total savings figure without examining how savings were distributed between state and school budgets. It finds that all 10 programs save money; the grand total of 23 edchoice .org

29 edchoice .org TABLE 4 Fiscal Impact on Taxpayers and Public Schools Results Location Year Author Negative Effect No Visible Effect Positive Effect Spalding 2014 D.C. X Spalding 2014 Florida X Spalding 2014 Florida X Spalding 2014 Georgia X Spalding 2014 Louisiana X Spalding 2014 Cleveland X Spalding 2014 Ohio X Spalding 2014 Ohio X Spalding 2014 Utah X Spalding 2014 Milwaukee X Wolf & McShane 2013 D.C. X LOEDR* 2012 Florida X Costrell 2010 Milwaukee X Merrifield & Gray 2009 San Antonio X OPPAGA** 2008 Florida X Aud 2007 Vermont X Aud 2007 Maine X Aud 2007 Milwaukee X Aud 2007 Cleveland X Aud 2007 Arizona X Aud 2007 Florida X Aud 2007 Florida X Aud 2007 Pennsylvania X Aud 2007 Florida X Aud 2007 D.C. X Aud 2007 Ohio X Aud 2007 Utah X Aud & Michos 2006 D.C. X Note: This table shows all empirical studies using all methods; the total fiscal effect of school choice programs is referenced. *LOEDR stands for Legislative Office of Economic and Demographic Research (State of Florida) . **OPP AGA stands for Office of Program Policy Analysis and Government Accountability (State of Florida). A Win-Win Solution: The Empirical Evidence on School Choice 24

30 The Friedman Foundation for Educational Choice PART IV Racial Segregation in Schools 25 edchoice .org

31 .org edchoice here have been 10 studies using valid empirical Jay Greene provides an instructive example that T shows how this problem undermines the validity of methods to examine school choice and racial segregation in schools. Nine of those studies find school such measures of segregation. In studies using the prevailing method, a school that is 98 percent white choice moves students into less racially segregated classrooms. The remaining study finds school choice is considered perfectly integrated if it is in a school has no visible effect on racial segregation. None finds district that also is 98 percent white. The school receives this perfect score even if the 98-percent-white choice increases racial segregation. school district is right next door to another district Public schools have been growing more racially that is 98 percent minority. Clearly, this should be segregated for some time. Paradoxically, this is considered segregation, but the prevailing method happening even as residential segregation has masks segregation when it occurs at the district level. 49 Understandably, racial segregation in declined. Greene issues a concise verdict on what studies like schools is an increasing concern. The issue of school this really are saying: “The schools are well integrated, 51 choice and racial segregation involves a number given that they are horribly segregated.” of interlocking societal concerns. Public schools are intractably segregated by race, mostly because A segregation study in the recent lawsuit challenging students are assigned to schools based on where they Louisiana’s voucher program is a case in point. live. School choice has the potential to break down Christine Rossell finds that the program reduced those residential barriers. Even so, many people racial segregation, using the Relative Exposure Index, have difficulty giving the evidence on this question which “standardizes the IEm index—the proportion a hearing. Space does not permit a discussion of the white in the average minority child’s school—by the issues here, but they are reviewed in an earlier report proportion white in the district.” In other words, the , and interested Freedom from Racial Barriers entitled index Rossell relies upon does not, in fact, measure 50 readers can consult that publication. how racially segregated schools are. It measures how evenly each school district’s level of racial segregation is distributed across its schools. A highly segregated Measuring Racial Segregation district in which the experience of segregation was evenly distributed across all its schools would look good on this measure. Rossell gives the game away Unfortunately, most research on school segregation when she repeatedly refers to the Relative Exposure is compromised by inadequate definitions of segregation. Researchers typically use the racial Index as a measure of “racial balance.” She calls it this because it is not a measure of racial segregation. makeup of a larger administrative unit—such as a Unfortunately, Rossell speaks all too truly when she school district, a municipality, or a system of private schools—as the standard against which segregation comments that “this index has been used in many cases, including every school desegregation case I in individual schools is measured. This problem 52 have been involved in.” is present, for example, in the way segregation measures such as the Index of Dissimilarity, the Another common problem in the existing research Index of Exposure, and the Gini Index are commonly on school segregation is the failure to compare used. All this approach really does is measure the similar grade levels. Elementary schools tend to be evenness of the racial distribution within the chosen more segregated than secondary schools because administrative unit. It ignores any segregation caused they draw from a smaller geographic area. Private by the structure of the administrative unit itself. Much schools are more likely than public schools to be of the segregation in the public school system occurs elementary schools, so a comparison of all public because school districts and municipal boundaries schools and all private schools will create a false themselves are segregated, so studies using this impression of greater segregation in private schools. approach effectively mask the real level of segregation. A Win-Win Solution: The Empirical Evidence on School Choice 26

32 for Educational Choice The Friedman Foundation is still an accurate snapshot; it describes what is To get an accurate picture of segregation levels, happening at that moment. These studies also researchers must compare elementary schools to provide a baseline against which popular descriptive elementary schools and secondary schools to secondary schools. It also is important not to claims can be evaluated. Widespread claims that private schools participating in choice programs are compare student populations composed of only prekindergarten or kindergarten students, as access heavily segregated should be examined against this evidence. to and voluntary participation in these grade levels is very uneven. This report reviews all available studies using What the Studies Show empirical methods that do not fall afoul of the problems described above. The best way to measure Ten empirical studies have examined segregation segregation is by comparing schools to the racial levels in public schools and choice-participating composition of the larger metropolitan area in private schools without falling afoul of the which they are situated. By looking at the whole methodological problems described previously. metropolitan area rather than a particular Eight of those studies were included in the previous administrative unit, such as a school district, edition of this report, published in 2013. One of researchers can detect levels of segregation that those studies, the only one able to use individual most studies miss. A second-best way employed by student data to examine causal effects, finds some studies is to measure the occurrence of racial the Milwaukee voucher program was having homogeneity. For example, measuring the percentage no visible effect on segregation levels in the 53 of schools that are more than 90 percent white or period studied. This was after the program more than 90 percent minority. had already been in existence for a decade. It is possible that the program had an effect on In many cases, the available evidence on school segregation during its first decade and then segregation is only descriptive rather than causal. produced a stable equilibrium, but in the absence Where researchers have access to data on individual of the necessary historical data we cannot know. school choice users matched to their public and private schools before and after the introduction of The remaining seven studies, using descriptive a school choice program, they can examine causal data, find school choice moves students from more relationships in the relevant variables. In other cases, segregated public schools into less segregated private researchers can only describe the segregation levels schools. Three studies of the voucher program in in affected public and private schools; they cannot Milwaukee used measures of racial homogeneity and examine to what extent the choice program, rather a fourth compared schools to the population of the than other factors, is causing the levels to be what surrounding metropolitan area. Two studies of the they are. voucher program in Cleveland compared schools to the surrounding metropolitan area. Additionally, one However, the descriptive evidence available in those of them used a measure of racial homogeneity. A study cases is enough to show the impact choice has on of the voucher program in Washington, D.C. also used the school environments of participating students. both types of measures. They can tell us whether school choice programs are moving students into schools that are more The two empirical studies that have been published segregated or less segregated than their assigned since the previous edition of this report both examine public schools at a given moment in time. While the Louisiana voucher program. Egalite and Mills we cannot draw causal conclusions when we only use individual student data, which were available have a “snapshot” of a single moment, the snapshot for 841 of the participating students, to perform a 27 edchoice .org

33 edchoice .org causal analysis of the effect of the program on racial reduce segregation in affected public schools, while segregation in 2012–13. Using the population of the 18 percent increase it, indicating a large net positive surrounding metropolitan area as their standard of effect. On the other hand, 45 percent of transfers comparison, they find that the student transfers from reduce segregation in affected private schools, while public to private schools under the voucher program 55 percent increase it, indicating a much smaller reduce segregation in both the public schools and net negative effect. The overall effect on racial the private schools. That is, these transfers move segregation in schools is therefore positive. They also both public and private schools closer to the racial conduct a separate examination of the 34 Louisiana 54 composition of the surrounding metropolitan area. school districts under federal desegregation orders. In those districts, 75 percent of voucher transfers reduce segregation in public schools, while no visible A second study by Egalite, Mills, and Wolf runs a 55 similar analysis on a data set including 1,741 students. effect was found on private schools. That analysis finds 82 percent of student transfers TABLE 5 Racial Segregation Results Year Author Location Positive Effect No Visible Effect Negative Effect Egalite et. al. 2016 X Louisiana Egalite & Mills Louisiana X 2014 Greene et. al. Milwaukee 2010 X Forster X Milwaukee 2006 Forster Cleveland X 2006 Greene & Winters X D.C. 2005 Fuller & Greiveldinger X Milwaukee 2002 Fuller & Mitchell X Milwaukee 2000 Fuller & Mitchell Milwaukee X 1999 Greene X Cleveland 1999 Note: This table shows all empirical studies using all methods; the total effect on segregation in all schools is referenced. A Win-Win Solution: The Empirical Evidence on School Choice 28

34 The Friedman Foundation for Educational Choice PART V Civic Values and Practices 29 edchoice .org

35 edchoice .org here have been 11 empirical studies examining a re-analysis of old data. In other words, we do not T how school choice affects civic values and have grounds to expect a significant body of random- practices, such as toleration for the rights of others. assignment studies will be built up over time, as has Eight of those studies find school choice has a positive already occurred for studies on academic effects. effect on these civic concerns. The remaining three studies find school choice has no visible effect on One study examining a privately funded voucher them. None finds choice has a negative effect on these program in San Francisco has been excluded values and practices. from this review because it is only a descriptive comparison of the voucher-using and non-voucher - using populations. That study finds no visible Measuring Civic Values and Practices difference in tolerance for the rights of others between the two populations. However, as a descriptive Research on how education affects civic values and analysis, this cannot explain much about whether the 57 voucher had an effect, as distinct from other factors. practices has measured a wide range of variables, It was appropriate to include descriptive studies in including tolerance for the rights of others, civic knowledge, civic participation, volunteerism, social the review of research on racial segregation because descriptive information about the racial makeup of capital, civic skills, and patriotism. Wolf’s 2008 article “Civics Exam,” the largest existing review of schools provides insight into an important question: the research comparing public and private schools What effect has school choice had on students’ school on these issues, finds the research overwhelmingly environments? If choice is moving students from more segregated schools to less segregated schools points to either no difference or a positive effect from (or vice versa), it is highly desirable to know that. private schooling on these measures—even in studies that use methods to compensate for the “selection Here, the descriptive information does not contribute to the relevant policy questions. bias” of families selecting into private schools. Readers interested in a thorough overview of that research 56 should consult Wolf’s systematic review. The most widely studied topic in this field is tolerance for the rights of others. Researchers generally use the same method to measure this topic, with only This report looks only at empirical studies of school small variations. Students are asked to identify their choice programs, as opposed to the broader universe least-liked group of people. Students typically name of studies that compare public and private schooling a variety of groups ranging from neo-Nazis and the generally. These studies tend to be methodologically KKK to those who disagree with them on passionate superior, as better ways of accounting for selection political issues (for example, pro-lifers name pro- bias are often available with choice programs. In four choicers and pro-choicers name pro-lifers) to disliked cases, gold-standard random-assignment methods religious minorities such as evangelical Christians. were employed. Students are then asked a battery of questions on whether their least-liked group should be permitted This report examines all empirical studies of civic to have or do certain things. Examples include voting, values and practices using all methods, not just organizing a march, or having a book in the library the random-assignment studies. This is not the sympathetic to their point of view. practice we followed for studies of academic effects. There are a smaller number of random-assignment studies available for civic values and practices than What the Studies Show for academic effects, and it is dangerous to rely on too small a universe of studies. Only two random- Of the 11 studies on this topic, seven were included in assignment studies of civic values and practices the previous edition of this report, published in 2013. have been published since 2002, one of which was A Win-Win Solution: The Empirical Evidence on School Choice 30

36 The Friedman Foundation for Educational Choice Three new studies have been published since then, modestly higher levels of political tolerance, civic and one has come to our attention. Of those 11 studies, skills, future political participation, and volunteering eight find school choice improves civic values and when compared to public school students. When practices, and three find no visible effect. None finds religious private schools are separated from other school choice has a negative effect. private schools, they find the positive effect is 59 significantly stronger in religious schools. Four of the studies included in the previous edition of this report use random-assignment methods. Of In a second new study, Mills and four co-authors those, one finds students using vouchers to attend conducted a phone survey of applicants to the private schools are more likely to show tolerance for Louisiana voucher program, including those who the rights of others they dislike. A second confirms were and those who were not ultimately offered this positive finding for tolerance, while finding no vouchers. They found that among those responding visible difference in civic knowledge. A third finds to the survey, there was no visible difference between no visible difference for tolerance. A fourth finds no those who were and were not offered a voucher in visible difference for tolerance or civic knowledge. respect for the rights of others or in several other non-cognitive outcomes, such as grit, locus of control, Of the three non-random-assignment studies and self-esteem. However, diagnostic analysis included in the previous edition of this report, one raised questions about the precision of their survey finds school choice students are more tolerant of the instrument, and the study was also undermined by a rights of those they dislike, but finds no visible effect low response rate—only 11 percent. The study authors on civic knowledge. The other two find parents of rightly caution us not to attribute too much weight to children participating in school choice programs these results. As with other studies described above, are 1) more likely to be actively involved in their we include this study in obedience to our rule of children’s schools, parent-teacher organizations, and including even methodologically challenged studies, 60 other education groups and 2) more likely to see a to avoid the possibility of “cherry-picking.” connection between education and the civic institutions of society, to say that their children are In the final new study, Corey DeAngelis and Wolf learning how government works, and to be involved examine the impact of Milwaukee’s voucher program in civic activities themselves. on students’ criminal records. They used seven measurements: 1) whether students were accused Bettinger and Slonim’s 2006 random-assignment of any crime, 2) whether they were convicted of any study of a privately funded school choice program crime, 3) whether they were convicted of a felony, in Toledo focused on altruistic behaviors. Students 4) whether they were convicted of a misdemeanor, and parents participated in six different experiments, 5) whether they were convicted of a traffic-related during which they had to decide how much out of a offence, 6) whether they were convicted of a given amount of money to keep for themselves and theft-related offence or 7) whether they were convicted how much to share with a charity or peer. The study of a drug-related offence. Their sample included finds the voucher program increased the amount of students who were in eighth or ninth grade in 2006 money participating students donated to charities. No and examined criminal outcomes as of 2015, when effect was found on students’ donations to peers or on the students would be 22–25 years old. Matching 58 parents’ donations. participating students to non-participating students with similar demographic characteristics and test In one of the three new studies published since scores then comparing the two, they found that the the previous edition of this report, David Fleming, voucher program decreased participants’ criminal William Mitchell, and Michael McNally find students activity, especially for males. The longer students participating in the Milwaukee voucher program have remained in the voucher program, the more this 31 edchoice .org

37 edchoice .org positive finding was visible across multiple measures of criminal outcomes. Males who remained in the program throughout high school had better outcomes than their public-school peers on all seven measurements. For example, this group had a 79 percent reduction in felonies (relative to the total incidence of felonies) due to vouchers, a 93 percent reduction in drug offenses, and an 87 percent reduction 61 in theft. TABLE 6 Civic Values and Practices Results Location Author Year No Visible Effect Any Negative Effect Any Positive Effect DeAngelis & Wolf 2016 X Milwaukee Mills et. al. Louisiana 2016 X Fleming et. al. Milwaukee X 2014 Campbell Nationwide X 2013 Fleming X Milwaukee 2012 Fleming Milwaukee X 2011 Bettinger & Slonim Toledo X 2006 Howell & Peterson D.C. 2002 X Campbell Nationwide X 2002 Peterson & Campbell Nationwide 2001 X Wolf et. al. D.C. X 2001 Note: This table shows all empirical studies using all methods. A Win-Win Solution: The Empirical Evidence on School Choice 32

38 The Friedman Foundation for Educational Choice CONCLUSION Remarks on School Choice 33 edchoice .org

39 edchoice .org “failing” standardized tests. Claims that choice “does Universal Choice Could Deliver an not work” directly contradict a clear consensus in the Education Revolution scientific evidence. Critics of school choice often ask: If school choice is so great, why are the public school systems in cities Choice Could Work Much Better... and states with choice still showing little to no overall If We Let It improvement? Milwaukee public schools were widely dysfunctional in 1990 when the voucher program was And yet, though it might be unreasonable to expect enacted, and they remain widely dysfunctional today. miracles, there is still an urgent need for larger There has been no “Milwaukee Miracle.” improvements than choice is now delivering. Are the results of today’s programs the best that school choice But the absence of a dramatic “miracle” is not a valid can do? Or is it reasonable to expect more? reason to conclude that choice is not helping. The government monopoly school system is so tenaciously The positive effect of school choice programs resistant to change that it is unreasonable to expect identified in the empirical research is sometimes miraculous results from any education reform. large, but it is more often modest in size. That is hardly surprising given that existing choice programs are also modest in size. If modest programs produce Yes, Choice Improves Schools modest benefits, is the logical conclusion to deny that these programs have any benefit and give up on Countless factors affect the overall performance of a them? Or is the logical conclusion to expand them— school system. Some of those factors, such as political and protect them from the kinds of overregulation policymaking, can change quickly and dramatically. experienced in Louisiana—until they are able to have Others, such as demographic factors, are highly stable. a bigger effect? As a result, the overall performance of a school Existing school choice programs are formally hindered system can never, by itself, provide a reliable guide in a number of ways, such as: to whether any one factor, such as school choice, is having a positive effect. If a man with asthma starts • limits on the number of students they may serve, taking a new medication and at the same time takes up smoking, his overall health and ability to breathe • limits on the types of students they may serve, may not improve, but this has no bearing on the • limits on the purchasing power they are allowed question of whether the medicine is helping. to provide, The only way to know whether choice is having a • limits on families’ ability to supplement that positive impact is to conduct empirical research using purchasing power, and high-quality scientific methods. That is the whole on how students may be admitted to • limits purpose of using scientific methods—to isolate the participating schools. effect of choice from the effects of all the other factors that influence academic outcomes, so effect can be The Greenfield School Revolution An earlier report, measured accurately. , discusses the significance of and School Choice 62 these limitations in more detail. And as the case of Given the remarkably consistent findings of the Louisiana shows, informal threats to choice programs research, it is clear school choice is having a positive also cannot be overlooked. effect. It is wrong to say choice must be doing no good simply because a lot of public schools are still A Win-Win Solution: The Empirical Evidence on School Choice 34

40 The Friedman Foundation for Educational Choice in new directions and do things radically differently Only Universal School Choice Can need a client base. There need to be people who will Sustain Dramatic Change benefit from the new direction and support it. And that client base must be robust on three dimensions: Ultimately, the only way to make school reform size, strength, and suffrage. There must be enough work on a large scale is to break the government supporters; they must have enough ability to provide monopoly on schooling. The monopoly is not just support; and they must have enough freedom to one powerful obstacle to reform among many; it is decide for themselves what to support. what makes all the many obstacles as powerful as they are. The monopoly ensures that no meaningful The government school monopoly crowds out this accountability for performance can occur, except client base. School choice has the potential to solve in rare cases as a result of Herculean efforts. The this problem by providing enough families (size) monopoly empowers a dense cluster of rapacious with enough dollars (strength) and enough choice special interests resisting efforts to improve schools. (suffrage) to support educational entrepreneurs. Unfortunately, existing school choice programs fall The monopoly creates an environment where the short on all three dimensions. Only universal choice urgent need for change cannot be made a tangible can open the door to the full-fledged revolution in part of the daily cultural life of the school. schooling America needs in the new century. Institutional culture in the existing system is hostile, not just to this or that reform, but also to reform in general, because the monopoly excludes the only institutional basis for making the need for change seem plausible and legitimate: the prospect of losing the institution’s client base and the funding that goes with it. When any institution has a captive client base, support for innovation vanishes. Reform requires people and institutions to do uncomfortable new things, and change will not occur until discomfort with the status quo becomes greater than the discomfort of the change. An institution with captive clients can continue to function into the foreseeable future more or less as it always has, without change. Why not just continue doing things in the way that feels comfortable and natural? Worst of all, the monopoly pushes out educational entrepreneurs who can reinvent schools from the ground up. Only a thriving marketplace that allows entrepreneurs to get the support they need by serving their clients better can produce sustainable innovation. In any field of human endeavor—whether education, medicine, politics, art, religion, manufacturing, or anything else—entrepreneurs who want to strike out 35 edchoice .org

41 edchoice .org A Win-Win Solution: The Empirical Evidence on School Choice 36

42 for Educational Choice The Friedman Foundation Howell, William G., and Paul E. Peterson. The Education Gap: Vouchers References . Rev. ed. Washington, DC: Brookings Institution, and Urban Schools 2006. Empirical Studies Covered in this Report: Jin, Hui, John Barnard, and Donald B. Rubin. “A Modified General Academic Outcomes of Participants Location Model for Noncompliance with Missing Data: Revisiting the New York City School Choice Scholarship Program using Abdulkadiroglu, Atila, Parag A. Pathak, and Christopher R. Walters. Principal Stratification.” Journal of Educational and Behavioral Statistics “School Vouchers and Student Achievement: First-Year Evidence 35, no. 2 (Apr. 2010), pp. 154-73. doi:10.3102/1076998609346968. from the Louisiana Scholarship Program.” NBER Working Paper 21839. Cambridge, MA: National Bureau of Economic Research, Krueger, Alan B., and Pei Zhu. “Another Look at the New York City 2015. http://www.nber.org/papers/w21839. American Behavioral Scientist 47, no. 5 School Voucher Experiment.” (Jan. 2004), pp. 658-98. doi:10.1177/0002764203260152. Barnard, John, Constantine E. Frangakis, Jennifer L. Hill, and Donald B. Rubin. “Principal Stratification Approach to Broken Randomized The Effects of the Louisiana Mills, Jonathan N., and Patrick J. Wolf. Experiments: A Case Study of School Choice Vouchers in New York . Scholarship Program on Student Achievement after Two Years 98, no. 462 (June City.” Journal of the American Statistical Association Louisiana Scholarship Program Evaluation Report 1. Fayetteville: 2003), pp. 299-323. doi:10.2307/30045239. Univ. of Ark., School Choice Demonstration Project; New Orleans: Tulane Univ., Education Research Alliance, 2016. http:// Bettinger, Eric and Robert Slonim. “Using Experimental Economics to educationresearchalliancenola.org/files/publications/Report-1- Measure the Effects of a Natural Educational Experiment on LSP-Y2-Achievement.pdf. Journal of Public Economics Altruism.” 90, no. 8–9 (Sept. 2006), pp. 1625-48. doi:10.1016/j.jpubeco.2005.10.006. Peterson, Paul E., and William G. Howell. “Voucher Research Controversy: New Looks at the New York City Evaluation.” Bitler, Marianne P., Thurston Domina, Emily K. Penner, and Hilary W. 4, no. 2 (Spring 2004), pp. 73-78. http://educationnext. Education Next Hoynes. “Distributional Effects of a School Voucher Program: org/files/ednext20042_73.pdf. Journal of Research on Evidence from New York City.” Educational Effectiveness 8, no. 3 (July-Sept. 2015), pp. 419-50. Rouse, Cecilia E. “Private School Vouchers and Student Achievement: doi:10.1080/19345747.2014.921259. An Evaluation of the Milwaukee Parental Choice Program.” 113, no. 2 (May 1998), pp. 553-602. Quarterly Journal of Economics Chingos, Matthew M., and Paul E. Peterson. “Experimentally doi:10.1162/003355398555685. Estimated Impacts of School Vouchers on College Enrollment and 122 (Feb. 2015), Journal of Public Economics Degree Attainment.” Wolf, Patrick J., Brian Kisida, Babette Gutmann, Michael Puma, Nada pp. 1-12. doi:10.1016/j.jpubeco.2014.11.013. Eissa, and Lou Rizzo. “School Vouchers and Student Outcomes: Experimental Evidence from Washington, DC.” Journal of Policy Chingos, Matthew M., and Paul E. Peterson. “The Impact of School 32, no. 2 (Spring 2013), pp. 246-70. Analysis and Management 13, no. 3 (Summer Education Next Vouchers on College Enrollment.” doi:10.1002/pam.21691. 2013), p. 59-64. http://educationnext.org/files/ednext_XIII_3_ chingos.pdf. Academic Outcomes of Public Schools Cowen, Joshua M. “School Choice as a Latent Variable: Estimating Bowen, Daniel, and Julie Trivitt. “Stigma without Sanctions: The (Lack the ‘Complier Average Causal Effect’ of Vouchers in Charlotte.” of) Impact of Private School Vouchers on Student Achievement.” Policy Studies Journal 36, no. 2 (May 2008), pp. 301-15. doi:10.1111/ Education Policy Analysis Archives 22, no. 87 (Aug. 2014), pp. 1-19. j.1541-0072.2008.00268.x. doi:10.14507/epaa.v22n87.2014. 1, no. 2 (Summer Greene, Jay P. “Vouchers in Charlotte,” Education Next Carnoy, Martin, Rank Adamson, Amita Chudgar, Thomas F. Luschei, 2001), pp. 55-60. http://educationnext.org/files/ednext20012_46b. and John F. Witte. Vouchers and Public School Performance: A Case Study pdf. . Washington, DC: Economic of the Milwaukee Parental Choice Program Policy Institute, 2007. Greene, Jay P., Paul E. Peterson, and Jiangtao Du. “Effectiveness of Education and Urban School Choice: The Milwaukee Experiment.” Carr, Matthew. “The Impact of Ohio’s EdChoice on Traditional Public 31, no. 2 (Jan. 1999), pp. 190-213. doi:10.1177/001312459903100 Society Cato Journal School Performance.” 31, no. 2 (Spring/Summer 2001), pp. 2005. 257-84. http://object.cato.org/sites/cato.org/files/serials/files/ cato-journal/2011/5/cj31n2-5.pdf. 37 edchoice .org

43 edchoice .org Lost Opportunity: An Empirical Analysis of How Vouchers Forster, Greg. Chakrabarti, Rajashri. “Can Increasing Private School Participation . School Choice Issues in the State. and Monetary Loss in a Voucher Program Affect Public School Affected Florida Public Schools Journal of Public Performance? Evidence from Milwaukee.” Indianapolis: Friedman Foundation for Educational Choice, 2008. 92, no. 5-6 (June 2008), pp. 1371-93. doi:10.1016/j. Economics http://www.edchoice.org/wp-content/uploads/2015/09/ jpubeco.2007.06.009. Lost-Opportunity-How-Vouchers-Affected-Florida-Public - Schools.pdf. Chakrabarti, Rajashri. “Closing the Gap.” In “Competition Passes Forster, Greg. the Test.” Education Next 4, no. 3 (Summer 2004), p. 70. http:// Promising Start: An Empirical Analysis of How EdChoice Vouchers Affect Ohio Public Schools educationnext.org/files/ednext20043_66.pdf. . School Choice Issues in the State. Indianapolis: Friedman Foundation for Educational Choice, 2008. Chakrabarti, Rajashri. “Impact of Voucher Design on Public School http://www.edchoice.org/wp-content/uploads/2015/09/ Performance: Evidence from Florida and Milwaukee Voucher Promising-Start-How-EdChoice-Vouchers-Affect-Ohio-Public- Schools.pdf. Programs.” Staff Reports 315. New York: Federal Reserve Bank of New York, 2008. http://www.newyorkfed.org/research/staff_ Greene, Jay P. An Evaluation of the Florida A-Plus Accountability and reports/sr315.html. School Choice Program . Civic Report. New York: Manhattan Chakrabarti, Rajashri. “Vouchers, Public School Response and the http://www.manhattan-institute.org/html/cr_ Institute, 2001. Economic Inquiry aplus.htm. Role of Incentives: Evidence from Florida.” 51, no. 1 (Jan. 2013), pp. 500-26. doi:10.1111/j.1465-7295.2012.00455.x. Gray, Nathan L., John D. Merrifield, and Kerry A. Adzima. “A Diamond, John W. Should Texas Adopt a School Choice Program? An Private Universal Voucher Program’s Effects on Traditional Public . Austin: 39, no. 1 (Dec. 2014), pp. Journal of Economics and Finance Schools.” Evaluation of the Horizon Scholarship Program in San Antonio 1–26. doi:10.1007/s12197-014-9309-z. http://old.texaspolicy. Texas Public Policy Foundation, 2007. com/center/education-policy/reports/should-texas-adopt- Rising to the Challenge: The Effect of Greene, Jay P., and Greg Forster. school-choice-program. . New School Choice on Public Schools in Milwaukee and San Antonio York: Manhattan Institute, 2002. http://www.manhattan-institute. Egalite, Anna. J. Competitive Impacts of Means Tested Vouchers on Public School Performance: Evidence from Louisiana and Indiana. org/html/cb_27.htm. PEPG 14-05. Cambridge, MA: Harvard Univ., Program on Education Policy and Governance, 2014. The Effect of Milwaukee’s Parental Greene, Jay P., and Ryan H. Marsh. Choice Program on Student Achievement in Milwaukee Public Schools . Egalite, Anna. J. The Competitive Effects of the Louisiana Scholarship SCDP Comprehensive Longitudinal Evaluation of the Milwaukee . Louisiana Scholarship Parental Choice Program Report 11. Fayetteville: Univ. of Program on Public School Performance Ark., School Choice Demonstration Project, 2009. http://www. Program Evaluation Report 4. Fayetteville: Univ. of Ark., School uaedreform.org/the-effect-of-milwaukees-parental-choice- Choice Demonstration Project; New Orleans: Tulane program-on-student-achievement-in-milwaukee-public-schools. Univ., Education Research Alliance, 2016. http:// educationresearchalliancenola.org/files/publications/Report-4- Greene, Jay P., and Marcus A. Winters. “Competition Passes the LSP-Competitive-Effects.pdf. Education Next 4, no. 3 (Summer 2004), pp. 66-71. http:// Test.” Figlio, David N., and Cassandra M.D. Hart. “Competitive Effects of educationnext.org/files/ednext20043_66.pdf. Means-Tested School Vouchers,” American Economic Journal: 6, no. 1 (Jan. 2014), pp. 133-56. doi:10.1257/ Greene, Jay P., and Marcus A. Winters. “An Evaluation of the Applied Economics app.6.1.133. Effects of DC’s Voucher Program on Public School Achievement and Racial Integration after One Year.” Figlio, David N., and Cassandra M.D. Hart. “Does Competition , Catholic Education: A Journal of Inquiry and Practice 11 no. 1 (Sept. 2007), pp. 83-101. http://digitalcommons.lmu.edu/ Improve Public Schools? New Evidence from the Florida ce/vol11/iss1/7. Education Next Tax-Credit Scholarship Program.” 11, no. 1 http://educationnext.org/files/ (Winter 2011), pp. 74-80. ednext_20111_Figlio.pdf. The Effects of Town Tuitioning in Vermont and Hammons, Christopher. . School Choice Issues in Depth 1. Indianapolis: Milton and Maine Figlio, David N., and Cecilia E. Rouse. “Do Accountability and Rose D. Friedman Foundation, 2002. Journal Voucher Threats Improve Low-Performing Schools?” of Public Economics 90, no. 1-2 (Jan. 2006), pp. 239-55. doi:10.1016/j 1, no. 4 (Winter Hoxby, Caroline M. “Rising Tide.” Education Next .jpubeco.2005.08.005. 2001), p. 69-74. http://educationnext.org/files/ednext20014_68.pdf. A Win-Win Solution: The Empirical Evidence on School Choice 38

44 The Friedman Foundation for Educational Choice Choice Program: 2010-2011 Update and Policy Options.” SCDP Mader, Nicholas S. “School Choice, Competition and Academic Quality: Essays on the Milwaukee Parental Choice Program” PhD Milwaukee Evaluation Report 22. Fayetteville: Univ. of Ark., School diss., Univ. of Wisconsin-Madison, 2010. ProQuest (3424049). Choice Demonstration Project, 2010. http://www.uaedreform.org/ downloads/2011/03/report-22-the-fiscal-impact-of-the- milwaukee-parental-choice-program-2010-2011-update-and-policy- Merrifield, John, and Nathan L. Gray. “An Evaluation of the CEO options.pdf. Journal of Horizon, 1998-2008, Edgewood Tuition Voucher Program.” 3, no. 4 (2009), pp. 414-15. doi:10.1080/15582150903430764. School Choice Merrifield, John, and Nathan L. Gray. “An Evaluation of the CEO Journal of Horizon, 1998-2008, Edgewood Tuition Voucher Program.” Rouse, Cecilia E., Jane Hannaway, Dan Goldhaber, and David N. Figlio. “Feeling the Florida Heat? How Low-Performing Schools Respond 3, no. 4 (2009), pp. 414-15. doi:10.1080/15582150903430764. School Choice to Voucher and Accountability Pressure.” Working Paper 13. Washington, DC: American Institutes for Research, National Center The Office of Program Policy Analysis and Government Accountability. for Analysis of Longitudinal Data in Education Research, 2007. . Report Corporate Tax Credit Scholarship Program Saves State Dollars http://files.eric.ed.gov/fulltext/ED509667.pdf. 08-68. Tallahassee: Fla. Legislature, Office of Program Policy http://www. Analysis and Government Accountability, 2008. oppaga.state.fl.us/reports/pdf/0868rpt.pdf. Rouse, Cecilia E., Jane Hannaway, Dan Goldhaber, and David N. Figlio. “Feeling the Florida Heat: How Low Performing Schools The School Voucher Audit: Do Publicly Funded Private American Respond to Voucher and Accountability Pressure.” Spalding, Jeff. 5, no. 2 (May 2013), pp. 251–81. School Choice Programs Save Money? Indianapolis: Friedman Economic Journal: Economic Policy Foundation for Educational Choice, 2014. http://www.edchoice. doi:10.1257/pol.5.2.251. org/wp-content/uploads/2015/07/The-School-Voucher-Audit- West, Martin R., and Paul E. Peterson. “The Efficacy of Choice Threats Do-Publicly-Funded-Private-School-Choice-Programs-Save-Money. pdf. Within School Accountability Systems: Results From Legislatively Induced Experiments.” 116, no. 510 (Mar. 2006), pp. Economic Journal Wolf, Patrick J., and Michael McShane. “Is the Juice Worth the Squeeze? C46-62. doi:10.1111/j.1468-0297.2006.01075.x. A Benefit/Cost Analysis of the District of Columbia Opportunity 8, no. 1 (Winter Winters, Marcus A., and Jay P. Greene. “Public School Response to Education Finance and Policy Scholarship Program.” 2013), pp. 74-99. doi:10.1162/EDFP_a_00083. Special Education Vouchers: The Impact of Florida’s McKay Scholarship Program on Disability Diagnosis and Student Educational Evaluation and Policy Racial Segregation Achievement in Public Schools.” Analysis 33, no. 2 (June 2011), pp. 138-58. doi:10.3102/0162373711404220. Egalite, Anna J., and Jonathan N. Mills. “The Louisiana Scholarship Fiscal Impact on Taxpayers and Public Schools Program: Contrary to Justice Department Claims, Student Transfers 14, no. 1 (Winter 2014), Education Next Improve Racial Integration.” pp. 66-69. http://educationnext.org/files/ednext_XIV_1_egalite.pdf. “Revenue Estimating Conference.” Fla. Legislative Office of Economic and Demographic Research, Mar. 16, 2012, p. 546, line 55. http:// The Impact Egalite, Anna J., Jonathan N. Mills, and Patrick J. Wolf. www.edr.state.fl.us/Content/conferences/revenueimpact/ archives/2012/pdf/page540-546.pdf. of the Louisiana Scholarship Program on Racial Segregation in Louisiana . Louisiana Scholarship Program Evaluation Report 3. Schools School Choice by the Numbers: The Fiscal Effect of Aud, Susan L. Fayetteville: Univ. of Ark., School Choice Demonstration Project; . School Choice Issues in Depth. New Orleans: Tulane Univ., Education Research Alliance, 2016. School Choice Programs, 1990-2006 http://educationresearchalliancenola.org/files/publications/ Indianapolis: Milton and Rose D. Friedman Foundation, 2007. http://www.edchoice.org/wp-content/uploads/2015/09 / Report-3-LSP-and-Racial-Segregation.pdf. Education-by-the-Numbers-Fiscal-Effect-of-School-Choice- Forster, Greg. Programs.pdf. Segregation Levels in Cleveland Public Schools and . School Choice Issues in the State. the Cleveland Voucher Program Indianapolis: Milton and Rose D. Friedman Foundation; Columbus, Spreading Freedom and Saving Aud., Susan Aud and Leon Michos. OH: Buckeye Institute, 2006. http://www.edchoice.org/wp- Money: The Fiscal Impact of the D.C. Voucher Program . Washington, DC:: content/uploads/2015/09/Segregation-Levels-in-Cleveland - Cato Institute; Indianapolis: Milton and Rose D. Friedman Public-Schools-and-the-Cleveland-Voucher-Program.pdf. http://www.edchoice.org/wp-content/ 2006. Foundation, uploads/2015/09/Spreading-Freedom-and-Saving-Money-The- Fiscal-Impact-of-the-DC-Voucher-Program.pdf. Forster, Greg. Segregation Levels in Milwaukee Public Schools and the Milwaukee Voucher Program . School Choice Issues in the State. Costrell, Robert M. “The Fiscal Impact of the Milwaukee Parental Indianapolis: Milton and Rose D. Friedman Foundation, 2006. 39 edchoice .org

45 edchoice .org School Vouchers and Civic Education.” Unpublished manuscript, http://www.edchoice.org/wp-content/uploads/2015/09/ - Segregation-Levels-in-Milwaukee-Public-Schools-and-the provided by the author via email on Jan. 24, 2013. Milwaukee-Voucher-Program.pdf. DeAngelis, Corey and Patrick J. Wolf. “The School Choice Voucher: A Fuller, Howard L., and Deborah Greiveldinger. “The Impact of School ‘Get Out of Jail’ Card?” EDRE Working Paper 2016-03. Fayetteville: Choice on Racial Integration in Milwaukee Private Schools.” Aug. Univ. of Ark., Dept. of Education Reform, 2016. www.uaedreform. - org/downloads/2016/03/the-school-choice-voucher-a-get-out 2002. American Education Reform Council Manuscripts. of-jail-card.pdf. The Impact of School Fuller, Howard L., and George A. Mitchell. Choice on Integration in Milwaukee Private Schools Fleming, David J. “Choice, Voice and Exit: School Vouchers in . Current Education Milwaukee.” Paper presented at the Annual Meeting of the Issues 2000-02. Milwaukee, WI: Marquette Univ., Institute for the American Political Science Association, Sept. 2011. http://papers. http://files.eric.ed.gov/ Transformation of Learning, 2000. ssrn.com/sol3/papers.cfm?abstract_id=190101. fulltext/ED443939.pdf. Fleming, David J. “Learning from Schools: School Choice, Political Fuller, Howard L., and George A. Mitchell. The Impact Policy Studies Journal 42, no. 1 Learning and Policy Feedback.” of School Choice on Racial and Ethnic Enrollment in Milwaukee Private Schools (Feb. 2014), pp. 55-78. doi:10.1111/psj.12042. . Current Education Issues 99-5. Milwaukee, WI: Marquette Univ., Institute for the Transformation Fleming, David J., William Mitchell, and Michael McNally, “Can of Learning, 1999. http://files.eric.ed.gov/fulltext/ED441903.pdf. Markets Make Citizens? School Vouchers, Political Tolerance, and Greene, Jay P. “The Racial, Economic, and Religious Context of Journal of School Choice: International Research Civic Engagement.” 8, no. 2 (2014), pp. 213-36. doi:10.1080/15582159.2014.905 and Reform Parental Choice in Cleveland.” Paper presented at the Annual 397. Meeting of the Association for Public Policy Analysis and http://www.ksg. Management, Washington, DC, Nov. 1999. Howell, William G., and Paul E. Peterson. The Education Gap: Vouchers harvard.edu/pepg/PDF/Papers/parclev.pdf. . Rev. ed. Washington, DC: Brookings Institution, and Urban Schools 2006. Greene, Jay P., Jonathan N. Mills, and Stuart Buck. The Milwaukee . SCDP Parental Choice Program’s Effect on School Integration Mills, Jonathan N., Albert Cheng, Collin E. Hitt, Patrick J. Wolf, Milwaukee Evaluation 20. Fayetteville: Univ. of Ark., School Measures of Student Non-Cognitive Skills and Choice Demonstration Project, 2010. and Jay P. Greene, http://www.uaedreform. org/downloads/2010/04/report-20-the-milwaukee-parental- Political Tolerance After Two Years of the Louisiana Scholarship choice-programs-effect-on-school-integration.pdf. . Louisiana Scholarship Program Evaluation Report 2. Program Fayetteville: Univ. of Ark., School Choice Demonstration Project; New Orleans: Tulane Univ., Education Research Alliance, 2016. Greene, Jay P., and Marcus A. Winters. “An Evaluation of the Effects of DC’s Voucher Program on Public School Achievement and Racial http://educationresearchalliancenola.org/files/publications/ Integration after One Year.” Report-2-LSP-Non-Cog-and-Political-Tolerance.pdf. Catholic Education: A Journal of Inquiry and Practice 11, no. 1 (Sept. 2007), pp. 83-101. http://digitalcommons. Peterson, Paul E., and David E. Campbell. An Evaluation of the lmu.edu/ce/vol11/iss1/7. Children’s Scholarship Fund . PEPG 01-03. Cambridge, MA: Harvard Univ., Program on Education Policy and Governance, 2001. http:// Civic Values and Practices www.hks.harvard.edu/pepg/PDF/Papers/CSF%20Report%20 Bettinger, Eric and Robert Slonim. “Using Experimental Economics 2001.pdf. to Measure the Effects of a Natural Educational Experiment Wolf, Patrick J., Paul E. Peterson, and Martin R. West. “ Results of a on Altruism.” Journal of Public Economics 90, no. 8-9 (Sept. 2006), School Voucher Experiment: The Case of Washington, D.C. after Two pp. 1625-48. doi:10.1016/j.jpubeco.2005.10.006 Years . PEPG 01-05. Cambridge, MA: Harvard Univ., Program on Campbell, David E. The Civic Side of School Reform: How Do School Education Policy and Governance, 2001. http://files.eric.ed.gov/ Vouchers Affect Civic Education? Princeton, NJ: Princeton Univ., fulltext/ED457272.pdf. Center for the Study of Democratic Politics; Cambridge, MA: Harvard Univ., Program on Education Policy and Governance, https://www.princeton.edu/csdp/events/Campbell041702/ 2002. campbell2.pdf. Campbell, David E. “The Civic Side of School Reform: Private Schools, A Win-Win Solution: The Empirical Evidence on School Choice 40

46 for Educational Choice The Friedman Foundation +“school choice,” +voucher, +vouchers, +“tax credit,” +“tax credits,” Notes +“education savings account” and +“education savings accounts.” Plural search terms yielded different results so we ran both singular 1. The Friedman Foundation for Educational Choice, The ABCs of and plural versions of search terms. School Choice: The Comprehensive Guide to Every Private School Choice Program in America , 2016 ed. (Indianapolis: Friedman Foundation for 12. See Paul E. Peterson, “A Courageous Look at the American High Educational Choice, 2016), p. 4, http://www.edch School,” Education Next 10, no. 2 (Spring 2010), pp. 25-33, http:// oice.org/wp-content/uploads/2016/02/2016-ABCs-WEB-2.pdf. educationnext.org/files/ednext_20102_24.pdf. For up-to-date data on school choice programs, see http://www. edchoice.org/school-choice/school-choice-in-america. 13. Alan Krueger and Pei Zhu, “Another Look at the New York City School Voucher Experiment,” 47, no. 5 American Behavioral Scientist Current Population Survey: 2013 Annual Social 2. US Census Bureau, (Jan. 2004), pp. 658-98, doi:10.1177/0002764203260152. (Washington, DC: US Census and Economic (ASEC) Supplement Bureau, 2013), http://www.census.gov/prod/techdoc/cps/ 14. Paul E. Peterson and William G. Howell, “Voucher Research cpsmar13.pdf. 4, no. 2 (Spring 2004), pp. 73-78, http:// Education Next Controversy,” educationnext.org/files/ednext20042_73.pdf. 3. Two illuminating books on the historic American understanding of education as it relates to the principles of human personhood, 15. Matthew M. Chingos and Paul E. Peterson, “The Impact of School The Death of character, freedom, and democracy are James D. Hunter, Vouchers on College Enrollment,” Education Next 13, no. 3 (Summer Character: Moral Education in an Age Without Good or Evil (New York: 2013), pp. 59-64, http://educationnext.org/files/ednext_XIII_3_ Basic Books, 2000); Charles L. Glenn, The Myth of the Common School chingos.pdf. (Tirana, Alb.: Institute for Contemporary Studies, 2002). 16. Matthew M. Chingos and Paul E. Peterson, “Experimentally 4. Of course, what counts as “science” and how we carry it out are Estimated Impacts of School Vouchers on College Enrollment and themselves moral and cultural questions, so science and civilization Journal of Public Economics Degree Attainment,” 122 (Feb. 2015), pp. are ultimately interdependent. But that is a topic for another day. 1-12, doi:10.1016/j.jpubeco.2014.11.013. 5. Greg Forster, A Win-Win Solution: The Empirical Evidence on 17. Marianne P. Bitler, Thurston Domina, Emily K. Penner, and Hilary How Vouchers Affect Public Schools , School Choice Issues in Depth W. Hoynes, “Distributional Effects of a School Voucher Program: (Indianapolis: Friedman Foundation for Educational Choice, 2009), Journal of Research on Educational Evidence from New York City,” http://www.edchoice.org/wp-content/uploads/2015/09/Win- Effectiveness 8, no. 3 (July-Sept. 2015), pp. 419-50, doi:10.1080/1934 Win-Solution-2009.pdf. 5747.2014.921259. A Win-Win Solution: The Empirical Evidence on School 6. Greg Forster, 18. Eric Bettinger and Robert Slonim, “Using Experimental Vouchers , 2nd ed. (Indianapolis: Foundation for Educational Choice, Economics to Measure the Effects of a Natural Educational 2011), http://www.edchoice.org/wp-content/uploads/2015/07/3- 90 (2006), pp. Experiment on Altruism,” Journal of Public Economics 2011-Win-Win-National-Study.pdf. 1625-48, doi:10.1016/j.jpubeco.2005.10.006. A Win-Win Solution: The Empirical Evidence on 7. Greg Forster, 19. Atila Abdulkadiroglu, Parag A. Pathak, and Christopher R. , 3rd ed. (Indianapolis: Friedman Foundation for School Choice Walters, “School Vouchers and Student Achievement: First-Year Educational Choice, 2013), http://www.edchoice.org/wp-content/ Evidence from the Louisiana Scholarship Program,” NBER Working uploads/2015/07/2013-4-A-Win-Win-Solution-WEB.pdf. Paper 21839 (Cambridge, MA: National Bureau of Economic Research, 2015), http://www.nber.org/papers/w21839. 8. Another option would be to consider each analytical model a separate “study” and simply report them in the aggregate as 20. Jonathan N. Mills and Patrick J. Wolf, The Effects of the Louisiana hundreds of studies. But this would give undue weight to research Scholarship Program on Student Achievement after Two Years , Louisiana that runs numerous analyses; a study using a hundred analyses (for Scholarship Program Evaluation Report 1 (Fayetteville: Univ. example, to test the robustness of a finding across many analytic of Ark., School Choice Demonstration Project; New Orleans: specifications) does not have twenty times more evidentiary weight Tulane Univ., Education Research Alliance, 2016), http:// than a study using five analyses. It would also defeat the purpose educationresearchalliancenola.org/files/publications/Report-1- of this report, which is to make the results of the research easily LSP-Y2-Achievement.pdf. understandable. 21. Ibid., p. 38. 9. Specifically, we ran the searches using the following four constructions: +“school choice,” +voucher, +“tax credit” and 22. Brian Kisida, Patrick J. Wolf, and Evan Rhinesmith, Views from +“education savings account.” Tests showed that plurals did not Private Schools (Washington, DC: American Enterprise Institute, change search results (e.g. “voucher” and “vouchers” yielded 2015), https://www.aei.org/wp-content/uploads/2015/01/Views- identical results). from-Private-Schools-7.pdf. 10. Specifically, we conducted each search multiple times, adding 23. See Lindsey M. Burke and Jonathan Butcher, “School-Voucher a date qualifier each time. We used pubdate:2013, pubdate:2014, Rules Trip Up Student Success in Louisiana,” National Review Online , pubdate:2015 and pubdate:2016 in turn. Jan. 6, 2016, http://www.nationalreview.com/article/429320/ school-vouchers-threatened-doj-over-regulation. 11. Specifically, we ran the searches for “articles” or “books” published from 2013 through 2016 using the following four constructions: 24. Abdulkadiroglu, Pathak, and Walters, “School Vouchers,” 41 edchoice .org

47 edchoice .org School Choice Issues in the Vouchers Affected Florida Public Schools, abstract. LSP stands for Louisiana Scholarship Program. State (Indianapolis: Friedman Foundation for Educational Choice, http://www.edchoice.org/wp-content/uploads/2015/09/ 2008), 25. Ibid. Granted, Louisiana’s underperforming public schools could – in theory – have suddenly started making huge gains, and this Lost-Opportunity-How-Vouchers-Affected-Florida-Public-Schools. remarkable phenomenon could – in theory – have been completely pdf. overlooked by everybody right up to the point where it was needed 34. Bowen and Trivitt, “Stigma without Sanctions,” p. 11. to explain away the empirical findings of the research on Louisiana’s voucher program, whereupon it was suddenly discovered. Evaluating the plausibility of this hypothesis is left as an exercise for 35. John W. Diamond, Should Texas Adopt a School Choice Program? An Evaluation of the Horizon Scholarship Program in San Antonio (Austin: the reader. For discussion of these alternative theories, see the articles and blog posts cited in Jason Bedrick, “The Folly of Overregulating Texas Public Policy Foundation, 2007), http://old.texaspolicy.com/ sites/default/files/documents/2007-03-RR03-education-diamond. School Choice: A Response to Critics,” Cato at Liberty (blog), Jan. http://www.cato.org/blog/folly-overregulating-school- 8, 2016, pdf. choice-response-critics. 36. John Merrifield and Nathan L. Gray, “An Evaluation of the CEO Horizon, 1998-2008, Edgewood Tuition Voucher 26. This report’s count of studies examining public school effects differs somewhat from that in Patrick J. Wolf and Anna J. Egalite, Program,” Journal of School Choice 3, no. 4 (2009), p. 414-15, Pursuing Innovation: How Can Educational Choice Transform K–12 doi:10.1080/15582150903430764. Education in the U.S.? (Indianapolis: Friedman Foundation for 37. Nathan L. Gray, John D. Merrifield, and Kerry A. Adzima, “A Educational Choice, 2016), http://www.edchoice.org/wp-content/ uploads/2016/05/2016-4-Pursuing-Innovation-WEB.pdf. The Private Universal Voucher Program’s Effects on Traditional Public main reason for the difference is a stricter screen used by Wolf and Schools,” Journal of Economics and Finance 39, no. 1 (Dec. 2014), pp. Egalite, which excludes studies included in this report. Wolf and 1–26, doi:10.1007/s12197-014-9309-z. Egalite are not just reviewing what the research finds but conducting Competitive Impacts of Means-Tested Vouchers on 38. Anna J. Egalite, an integrative evaluation of the state of knowledge in the field. A Public School Performance: Evidence from Louisiana and Indiana , PEPG stricter screen is appropriate to their integrative purpose just as a more inclusive screen is appropriate to the descriptive purpose of 14-05 (Cambridge, MA: Harvard Univ., Program on Education Policy and Governance, 2014). this report. (Integrative and evaluative comments are offered in the text of this report, but they are not its main purpose.) Also, Wolf he Competitive Effects of the Louisiana Scholarship Program 39. Egalite, T and Egalite have stated to the author of this report that one study on Public School Performance , Louisiana Scholarship Program included here (the Bowen and Trivitt study in Florida) came out too Evaluation Report 4 (Fayetteville: Univ. of Ark., School Choice late to be included in their report. Demonstration Project; New Orleans: Tulane Univ., Education 27. Nicholas Mader, “School Choice, Competition and Academic Research Alliance, 2016), http://educationresearchalliancenola.org/ files/publications/Report-4-LSP-Competitive-Effects.pdf Quality: Essays on the Milwaukee Parental Choice Program” (PhD diss., Univ. of Wisconsin-Madison, 2010) ProQuest (3424049). 40. For details see the discussions in the 2009 and 2011 editions of this report, as well as Matthew Carr’s study of Ohio’s EdChoice 28. David N. Figlio and Cassandra M. D. Hart, “Does Competition Improve Public Schools? New Evidence from the Florida Tax-Credit program, discussed in the 2013 edition of this report. Scholarship Program,” Education Next 11, no. 1 (Winter 2011), pp. 74- 41. Author’s calculations; Thomas D. Snyder, Cristobal de Brey, 80, http://educationnext.org/files/ednext_20111_Figlio.pdf. and Sally A. Dillow, Digest of Education Statistics 2014 , NCES 2016- 006 (Washington, DC: US Dept. of Education, National Center for 29. Figlio and Hart, “Competitive Effects of Means-Tested School Vouchers,” American Economic Journal: Applied Economics Education Statistics, 2016), pp. 380-81, table 235.10, http://nces. 6, no. 1 (Jan. 2014), pp. 133-56, doi:10.1257/app.6.1.133. doi:10.1257/app.6.1.133. ed.gov/pubs2016/2016006.pdf. School Choice by the Numbers: The Fiscal Effect of 42. Susan L. Aud, 30. Cecilia E. Rouse , Jane Hannaway, Dan Goldhaber, and David School Choice Issues in Depth N. Figlio, “Feeling the Florida Heat? How Low-Performing Schools School Choice Programs, 1990-2006, Respond to Voucher and Accountability Pressure,” Working Paper (Indianapolis: Milton and Rose D. Friedman Foundation, 2006), http://www.edchoice.org/wp-content/uploads/2015/09/ 13 (Washington, DC: American Institutes for Research, National Center for Analysis of Longitudinal Data in Education Research, Education-by-the-Numbers-Fiscal-Effect-of-School-Choice- 2007), http://files.eric.ed.gov/fulltext/ED509667.pdf. Programs.pdf. 31. Cecilia E. Rouse, Jane Hannaway, Dan Goldhaber, and David 43. Ibid. N. Figlio, “Feeling the Florida Heat: How Low Performing Schools 44. Benjamin Scafidi, Respond to Voucher and Accountability Pressure,” American The Fiscal Effects of School Choice Programs on 5, no. 2 (May 2013), pp. 251–81, Economic Journal: Economic Policy (Indianapolis: Friedman Foundation for Public School Districts Educational Choice, 2012), http://www.edchoice.org/wp-content/ doi:10.1257/pol.5.2.251. uploads/2015/07/The-Fiscal-Effects-of-School-Choice-Programs. 32. Daniel Bowen and Julie Trivitt, “Stigma without Sanctions: pdf. The (Lack of) Impact of Private School Vouchers on Student 22, no. 87 (Aug. Education Policy Analysis Archives Achievement,” 45. See note 42 above. 2014), pp. 1-19. doi:10.14507/epaa.v22n87.2014. 46. Ibid. Lost Opportunity: An Empirical Analysis of How 33. Greg Forster, A Win-Win Solution: The Empirical Evidence on School Choice 42

48 for Educational Choice The Friedman Foundation 59. David J. Fleming, William Mitchell and Michael McNally, “Can 47. Jeff Spalding, The School Voucher Audit: Do Publicly Funded (Indianapolis: Friedman Markets Make Citizens? School Vouchers, Political Tolerance, and Private School Choice Programs Save Money? 8, no. 2 (2014), pp. 213-36, Journal of School Choice Civic Engagement,” Foundation for Educational Choice, 2014), http://www.edchoice. doi:10.1080/15582159.2014.905397. org/wp-content/uploads/2015/07/The-School-Voucher-Audit-Do- Publicly-Funded-Private-School-Choice-Programs-Save-Money.pdf. 60. Jonathan N. Mills, Albert Cheng, Collin E. Hitt, Patrick J. Wolf, and Jay P. Greene, Measures of Student Non-Cognitive Skills 48. See note 36 above. and Political Tolerance After Two Years of the Louisiana Scholarship 49. Benjamin Scafidi, The Integration Anomaly: Comparing the Effects of , Louisiana Scholarship Program Evaluation Report 2 Program K–12 Education Delivery Models on Segregation in Schools (Fayetteville: Univ. of Ark., School Choice Demonstration Project; (Indianapolis: New Orleans: Tulane Univ., Education Research Alliance, 2016), Friedman Foundation for Educational Choice, 2015), http:// http://educationresearchalliancenola.org/files/publications/ www.edchoice.org/wp-content/uploads/2015/10/2015-10-The- Integration-Anomaly-WEB.pdf. Report-2-LSP-Non-Cog-and-Political-Tolerance.pdf. The authors suggest interpreting these results as merely descriptive, because of 50. Greg Forster, Freedom from Racial Barriers: The Empirical Evidence on the problems identified here; this raised the question of whether the study should be included in this review given that we exclude , School Choice Issues in Depth (Indianapolis: Vouchers and Segregation Milton and Rose D. Friedman Foundation, 2006), http://www. descriptive studies from our review of research on civic values and - practices. However, this study was not designed to be descriptive; edchoice.org/wp-content/uploads/2015/09/Freedom-from-Racial it was an attempt to study causal factors; the admonition to give Barriers-Vouchers-and-Segregation.pdf. the data only descriptive force is an a fortiori suggestion from its School Choice and 51. Jay P. Greene, “Choosing Integration,” in authors rather than an integral part of the study design. It therefore Diversity: What the Evidence Says , ed. Janelle Scott, (New York: seemed appropriate to include it here and let the reader draw his Teachers College Press, 2005), p. 30. own conclusions. 52. Christine H. Rossell, A Report on the Effect of the 2012-13 Scholarship 61. Corey DeAngelis and Patrick J. Wolf, “The School Choice Voucher: A ‘Get Out of Jail’ Card?,” EDRE Working Paper 2016- Program on Racial Imbalance in Louisiana School Districts Under a , special report prepared in the 03 (Fayetteville: Univ. of Ark., Dept. of Education Reform, 2016), Desegregation Court Order as of 2012-13 , Nov. 7, 2013, http://images.politico.com/ Brumfield v. Dodd case of http://www.uaedreform.org/downloads/2016/03/the-school- global/2013/11/07/lavoucheraff2.pdf. choice-voucher-a-get-out-of-jail-card.pdf. 62. Greg Forster and James L. Woodworth, The Greenfield School 53. Jay P. Greene, Jonathan N. Mills, and Stuart Buck, The Milwaukee (Indianapolis: Friedman Foundation for Parental Choice Program’s Effect on School Integration Revolution and School Choice , SCDP Milwaukee Evaluation Report 20 (Fayetteville: Univ. of Ark., Dept. of Education Educational Choice, 2012), http://www.edchoice.org/wp-content/ Reform, School Choice Demonstration Project, 2010), http://www. uploads/2015/07/Greenfield-School-Revolution-and-School- Choice.pdf. uaedreform.org/downloads/2010/04/report-20-the-milwaukee- parental-choice-programs-effect-on-school-integration.pdf. 54. Anna J. Egalite and Jonathan N. Mills, “The Louisiana Scholarship Program,” Education Next 14, no. 1 (Winter 2014), pp. 66-69, http:// educationnext.org/files/ednext_XIV_1_egalite.pdf. 55. Anna J. Egalite, Jonathan N. Mills, and Patrick J. Wolf, The Impact of the Louisiana Scholarship Program on Racial Segregation in Louisiana Schools , Louisiana Scholarship Program Evaluation Report 3 (Fayetteville: Univ. of Ark., School Choice Demonstration Project; New Orleans: Tulane Univ., Education Research Alliance, 2016), http://educationresearchalliancenola.org/files/publications/ Report-3-LSP-and-Racial-Segregation.pdf. 56. Patrick J. Wolf, “Civics Exam: Schools of Choice Boost Civic Values,” Education Next 7, no. 3 (Summer 2007), p. 66-72, http:// educationnext.org/files/ednext_20073_66.pdf. The author is aware of one additional study more recent than Wolf’s literature review; see Russell S. Sobel and Kerry A. King, “Does School Choice Increase the Economics of Education Review Rate of Youth Entrepreneurship?” 27, no. 4 (Aug. 2008), pp. 429-38, doi:10.1016/j.econedurev.2007.01.005. An 57. Paul E. Peterson, David E. Campbell, and Martin R. West, Evaluation of the BASIC Fund Scholarship Program in the San Francisco Bay Area, California , PEPG 01-01 (Cambridge, MA: Harvard Univ., Program on Education Policy and Governance, 2001), https://www. hks.harvard.edu/pepg/PDF/Papers/BasicReport.PDF. 58. See note 18 above. 43 edchoice .org

49 edchoice .org A Win-Win Solution: The Empirical Evidence on School Choice 44

50 The Friedman Foundation for Educational Choice About the Author Greg Forster, Ph.D. , is a senior fellow with the Friedman Foundation for Educational Choice. He conducts research and writes on school choice policy. Forster has conducted empirical studies on the impact of school choice programs in Milwaukee, Ohio, Florida, and Texas, as well as national empirical studies comparing public and private schools in terms of working conditions for teachers, racial segregation, and teacher and staff misconduct. He also has conducted empirical studies of other education topics, including charter schools, accountability testing, graduation rates, student demographics, and special education. Forster’s research has appeared in the peer-reviewed publications Teachers College Record and Education Working Paper Archive, and his articles on education policy have appeared in the Washington Post, Wall Street Journal, Los Angeles Times, Philadelphia Inquirer, Education Next, Chronicle of Higher Education, and numerous other publications. He is co-author of the book Education Myths: What Special-Interest Groups Want You to Believe about Our Schools and Why It Isn’t So , from Rowman & Littlefield. Forster is also a contributor to Jay P. Greene’s Blog (jaypgreene.com). The interpretations, views, and recommendations expressed in this report are solely those of the author(s). 45 edchoice .org

51 edchoice .org Commitment to Methods & Transparency The Friedman Foundation for Educational Choice is committed to research that adheres to high scientific standards, and matters of methodology and transparency are taken seriously at all levels of our organization. We are dedicated to providing high-quality information in a transparent and efficient manner. All individuals have opinions, and many organizations (like our own) have specific missions or philosophical orientations. Scientific methods, if used correctly and followed closely in well-designed studies, should neutralize these opinions and orientations. Research rules and methods minimize bias. We believe rigorous procedural rules of science prevent a researcher’s motives, and an organization’s particular orientation, from pre-determining results. If research adheres to proper scientific and methodological standards, its findings can be relied upon no matter who has conducted it. If rules and methods are neither specified nor followed, then the biases of the researcher or an organization may become relevant, because a lack of rigor opens the door for those biases to affect the results. The author welcomes any and all questions related to methods and findings. Can Competition Transform K–12 Education in the U.S.? 10 Incentivizing Innovation:

52 111 Monument Circle Suite 2650 Indianapolis, IN 46204 (317) 681-0745 edchoice.org Dr. Milton Friedman , Founder Nobel Laureate Dr. Rose D. Friedman , Founder Noted Economist BOARD OF DIRECTORS Dr. Patrick Byrne , Chairman Founder, Overstock.com Janet F. Martel , Vice Chairperson Lawrence A. O’Connor, Jr. , Treasurer J. Scott Enright , Secretary Executive Vice President and General Counsel, Emmis Communications Robert C. Enlow President & CEO, Friedman Foundation for Educational Choice Charles H. Brunie Dr. David D. Friedman Professor, Santa Clara University William J. Hume Chairman of the Board, Basic American, Inc. Fred S. Klipsch Chairman, Hoosiers for Quality Education Fred Reams Reams Asset Management Virginia Walden Ford Education Activist Dr. Michael Walker President, The Fraser Institute Foundation facebook.com/ edchoice twitter.com/ edchoice

Related documents

base.dvi

base.dvi

Discrete Choice Methods with Simulation Kenneth Train University of California, Berkeley National Economic Research Associates Version dated March 8, 2002 Publisher: Cambridge University Press Schedul...

More info »
12.09   America's Best and Worst Cities for School Choice

12.09 America's Best and Worst Cities for School Choice

Amer ica’s Amer ica’s Best (And Worst) Cities for December 2015 School Choice PRISCILLA WOHLSTETTER & DARA ZEEHANDELAAR WITH DAVID GRIFFITH Nor m ber M. J. Petrilli thern a nd Mi chael d Forewor by A

More info »
policyandchoice book

policyandchoice book

a dvance Praise F or Congdon • Kling Mullai hoice and c Policy “Policy and Choice is a must-read for students of public finance. If you want to learn N William J. Congdon is a research director in Tra...

More info »
/CT/CochraneCMS/TexRendering5/4666669.dvi

/CT/CochraneCMS/TexRendering5/4666669.dvi

Cochrane Database of Systematic Reviews Decision aids for people facing health treatment or screening decisions (Review) Stacey D, Légaré F, Lewis K, Barry MJ, Bennett CL, Eden KB, Holm es-Rovner M, L...

More info »
MPI: A Message Passing Interface Standard

MPI: A Message Passing Interface Standard

MPI : A Message-Passing Interface Standard Version 3.0 Message Passing Interface Forum September 21, 2012

More info »
networks-book

networks-book

Networks, Crowds, and Markets: Reasoning about a Highly Connected World David Easley Jon Kleinberg Dept. of Economics Dept. of Computer Science Cornell University Cornell University Cambridge Universi...

More info »
Numerical Recipes

Numerical Recipes

Sample page from NUMERICAL RECIPES IN C: THE ART OF SCIENTIFIC COMPUTING (ISBN 0-521-43108-5) Permission is granted for internet users to make one paper copy for their own personal use. Further reprod...

More info »
Bandit Algorithms

Bandit Algorithms

Bandit Algorithms ́ Tor Lattimore and Csaba Szepesv ari st Draft of Wednesday 1 May, 2019 Revision: c0525791b66f0f41db4e87204ac91f41693d4365

More info »
catalog 2019

catalog 2019

2019 ® HARLEY-DAVIDSON GENUINE MOTOR PARTS & ACCESSORIES

More info »
AndersBehringBreivikManifesto

AndersBehringBreivikManifesto

2011 , London – By Andrew Berwick

More info »
book.dvi

book.dvi

Fuzzy Control Kevin M. Passino Department of Electrical Engineering The Ohio State University Stephen Yurkovich Department of Electrical Engineering The Ohio State University ddison-Wesley Longman, In...

More info »
Microsoft Word   01 Front page.doc

Microsoft Word 01 Front page.doc

Final Reports of Technical Review Of Costing Tools Commissioned by an Inter-agency Steering Committee and the Partnership for Maternal, Newborn and Child Health This package contains the final reports...

More info »
Kentucky Academic Standards Visual and Performing Arts

Kentucky Academic Standards Visual and Performing Arts

KENTUCKY ACADEMIC STANDARDS VISUAL AND PERFORMING ARTS Kentucky Academic Standards – June 2015

More info »
Kentucky Academic Standards Visual and Performing Arts

Kentucky Academic Standards Visual and Performing Arts

KENTUCKY ACADEMIC STANDARDS VISUAL AND PE RFORMING AR TS Kentucky Academic Standards – June 2015

More info »
Hossler/Web

Hossler/Web

™ EW ERIES S GENDA A N Fifty Years of College Choice: Social, Political and Institutional Influences on the Decision- making Process Jillian Kinzie, Megan Palmer, John Hayek, Don Hossler, Stacy A. Jac...

More info »
IOM Dying in America

IOM Dying in America

This PDF is available from The National Academies Press at http://www.nap.edu/catalog.php?record_id=18748 Dying in America: Improving Quality and Honoring Individual Preferences Near the End of Life C...

More info »