What We Know About Developmental Education Outcomes

Transcript

1 RESEARCH OVERVIEW / JANUARY 2014 What We Know About Developmental Education Outcomes What Is Developmental Education? Many recent high school graduates who enter community college are required to take remedial or developmental education courses before enrolling in college-level courses. Developmental courses essentially reteach high school– and junior high school–level content in reading, writing, and math. In some cases, students are referred to two or even three courses of developmental education in a single subject area. The annual cost of providing remediation to all college students nationwide has 1 been estimated at approximately $7 billion. This overview is part of CCRC’s practitioner packet on developmental education. For more information on the effectiveness of community college assessment and placement tests, accelerated approaches to developmental education, and overcoming challenges inherent in the developmental education reform process, please see our complete packet, . Designing Meaningful Developmental Reform How Many Students Need Developmental Education? Federal data indicate that develop - need While there is no way to gauge with perfect accuracy how many students actually 68 percent of community mental education, recent federal data indicate that 68 percent of community college students and 40 2 college students and 40 take at least one remedial course. percent of students at public four-year colleges Research suggests percent of students at 3 that many more students are referred to developmental courses but never enroll in them. public four-year colleges take at least one remedial What Do We Know About the Effects of course. Developmental Education? Only 28 percent of community college students who take a developmental education course 4 go on to earn a degree within eight years, and many students assigned to developmental 5 A courses drop out before completing their sequence and enrolling in college-level courses. number of rigorous studies have been undertaken to assess the extent to which the traditional system of developmental education helps students into and through college-level coursework. These studies are discussed below.

2 COMMUNITY COLLEGE RESEARCH CENTER / TEACHERS COLLEGE, COLUMBIA UNIVERSITY What the Research Tells Us About the Methodology In order to understand the impact of developmental education, it is necessary to compare students who are academically similar, some of whom are assigned to remedial education and some of whom - are not. An important methodological strategy that can accurately make this comparison is the re gression discontinuity (RD) approach. This approach compares students who scored just below the cutoff for assignment to college-level courses with those who scored at or just above the cutoff. For example, if a cutoff score were 45, an RD analysis with a bandwidth of +/- 5 points would compare students who scored from 40 to 44 (assigned to remediation) with students who scored from 45 to 49 (assigned to college-level courses). Within such a narrow range, where differences in assessment scores are insignificant in terms Within a narrow range around the cutoff score, of predicting success in college-level courses (as shown in the figure below), the assignment of assignment to remediation students to remedial or college-level courses is effectively random. Thus, if remedial courses help is effectively random. students who scored near the cutoff succeed in college, then the just-below-the-cutoff students - who were assigned to remediation should have better outcomes than the just-above-the-cutoff stu dents, who are virtually identical but who were assigned to enroll directly in college-level courses. Success Rates by Assessment Score Among Students Enrolling Directly in College-Level Math 80 76% 70% 69% 69% 70 66% 65% 65% 64% 64% 60% 60 57% 50 Passed With C or Better 0 40 41 42 43 44 45 46 47 48 49 50 Score on COMPASS Algebra Exam Note. In the sample from which the above data are drawn, the actual cutoff score used was 30; thus, none of the students represented in the figure underwent remediation. The vertical line represents a hypothetical cutoff score of 45, which is a 6 more typical college-level cutoff. Findings on Developmental Education Student Outcomes In the tables below, we summarize findings from eight studies, all but one of which used an RD 7 to evaluate the effectiveness of community college remedial courses across a large college approach system or state. It is important to note that the colleges examined in these studies used a wide range of cutoff scores to determine college readiness and that some of these studies compared students above and below the cutoff scores not just for developmental versus college-level courses but also for higher 8 versus lower level remedial courses. 2

3 RESEARCH OVERVIEW / JANUARY 2014 / DEVELOPMENTAL EDUCATION OUTCOMES 9 indings on Outcomes for Developmental Students Overview of F Positive Negative Null DEVELOPMENTAL MATH STUDENTS Medium- & Long-Term Impacts Short-Term Impacts Grade in College- Passed College- College-Level Credential and/or Persistence Study Level Persistence Level Math Level Math Credits Earned Transfer 10 UPPER TENNESSEE NULL (conditional) NEG NULL NULL (conditional) NEG (credential) 11 UPPER TEXAS NULL NULL 12 UPPER OHIO NULL POS (transfer) 13 UPPER LUCCS NEG NULL NULL NULL NEG 14 UPPER FLORIDA NULL NULL NULL NULL LOWER 15 VIRGINIA 1 NEG (credential) NULL vs. MIDDLE LOWER TENNESSEE NULL NULL (conditional) NULL POS (credential) NULL (conditional) vs. MIDDLE DEVELOPMENTAL READING STUDENTS Medium- & Long-Term Impacts Short-Term Impacts Grade in College- Passed College- College-Level Credential and/or Study Level Persistence Persistence Level English Level English Credits Earned Transfer UPPER TENNESSEE POS NULL NULL (conditional) NULL (conditional) NULL (credential) UPPER TEXAS NULL NULL UPPER OHIO NULL NULL UPPER LUCCS NEG NEG NEG (credential) NEG NEG UPPER FLORIDA NULL NEG NULL NULL 16 UPPER VIRGINIA 2 NULL NEG NULL NULL (conditional) LOWER VIRGINIA 2 NULL (conditional) NEG NEG NEG vs. UPPER LOWER TENNESSEE NULL POS (conditional) NULL (credential) POS NULL (conditional) vs. MIDDLE DEVELOPMENTAL WRITING STUDENTS Medium- & Long-Term Impacts Short-Term Impacts Passed College- Grade in College- College-Level Credential and/or Level Persistence Study Persistence Level English Level English Transfer Credits Earned UPPER TENNESSEE NULL (conditional) NEG NEG (credential) NEG (conditional) NULL UPPER VIRGINIA 2 NULL (conditional) NULL NULL NULL Writing & Reading vs. LUCCS NULL NULL NULL NULL NULL Reading Only LOWER vs. VIRGINIA 2 NULL NEG NULL (conditional) NEG UPPER LOWER vs. TENNESSEE POS (conditional) NULL NULL (conditional) NULL (credential) POS UPPER Note. “Conditional” signifies that only outcomes for students who enrolled in college-level courses, or persisted in college, were compared. LUCCS stands for large urban community college system. 3

4 COMMUNITY COLLEGE RESEARCH CENTER / TEACHERS COLLEGE, COLUMBIA UNIVERSITY The RD analyses show that, with a few notable exceptions, developmental education has mostly null and sometimes negative effects on student outcomes for students near the cutoffs. A null result indicates that no statistically significant effect was found in the analysis, suggesting that students spent time and tuition on courses that may have made no discernable difference in their 17 ability to succeed in college. Differential Effects of Remediation on Subgroups The research suggests that the impact of remediation may vary depending on student demograph - ics and level of academic preparation. Combining the results from all the studies, students who - scored near the college-level cutoffs and were placed in developmental courses appear to have expe rienced substantially more negative or null than positive effects (two positive vs. 15 negative and 32 null). On the other hand, students who scored near the cutoffs between upper- and lower-level (or middle- and lower-level) developmental courses and were placed into the lower course experi - enced a higher proportion of positive effects (five positive vs. six negative and 19 null). Research suggests the One RD study that looked at the effects of remediation on particular subgroups of students found impact of remediation large differences in how student populations were impacted. For instance, assignment to remedia - varies depending on tion tended to have significant and large negative impacts on students who attended colleges with student demographics a high proportion of remedial students, on female students, on students who were younger than and level of academic 25, and on Black students. Conversely, assignment to remediation had nonsignificant effects on preparation. students who attended colleges with a low proportion of remedial students, on male students, on 18 students who were 25 or older, and on White students. - Another study found positive effects on persistence and college-level credit accumulation for Eng - lish language learners who were required to take both reading and writing developmental educa tion instead of just reading remediation. Native English speakers, on the other hand, experienced 19 no benefits from placement into both courses versus placement into just reading remediation. Student Progression Through the Remedial Sequence One reason why developmental education may not be very effective is because of high attrition from the remedial sequence. A CCRC analysis involving data from 57 community colleges found that lengthy remedial sequences are extremely “leaky.” The necessity of completing several courses before enrolling in college-level courses creates multiple points at which students can exit the sequence, thus forgoing any chance of completing the first college-level (or “gatekeeper”) 20 course in the same subject area. - For instance, among the 63,650 students in the study who were assigned to three levels of devel opmental math, only 11 percent ever successfully completed college-level introductory algebra. More than one fourth never enrolled in their first remedial course. And even among the students - who had the tenacity to complete all three levels of remedial math, 2,500 (4 percent of the origi nal cohort, or almost one fourth of those who completed all three developmental courses) failed to enroll in the gatekeeper math course. 4

5 RESEARCH OVERVIEW / JANUARY 2014 / DEVELOPMENTAL EDUCATION OUTCOMES 21 Student Progression Through the Developmental Math Sequence Did Not Enroll in Next Course 26% 15% 7% 4% Referred to 100% 11% Level 2 Course Level 3+ Course Level 1 Course Gatekeeper 3+ Levels of Passed Remediation (63,650) Gatekeeper Math 2% 4% 9% 22% Did Not Pass/Complete Course 22 Student Progression Through the Developmental Reading Sequence 29% Did Not Enroll in Next Course 9% 4% 5% Referred to 29% 100% Gatekeeper Level 1 Course Level 3+ Course Level 2 Course 3+ Levels of Passed Remediation (11,210) Gatekeeper English 2% 2% 4% Did Not Pass/Complete Course 17% Inaccurate Placement Into Developmental Courses The negative and null effects of developmental education may also be explained in part by inaccurate placement into developmental courses. Research indicates that some students who 23 do not need it are placed into developmental courses. Underplaced students may experience negative effects from developmental courses that overshadow potential positive effects experi - 24 enced by accurately placed students. Conclusion Research evidence suggests that, for the most part, the traditional system of developmental educa - tion is not achieving its intended purpose: to improve outcomes for underprepared students. These - findings do not mean that developmental education should be discarded; large numbers of commu nity college students need support to succeed academically. The findings do suggest, however, that the system could benefit from thoughtful reform. In part two of this practitioner packet, Designing , we lay out some of the challenges to reforming developmental Meaning ful Developmental Reform education, review relevant reform research, and describe case studies of colleges that successfully resolved tensions that are often obstacles in the developmental reform process. 5

6 COMMUNITY COLLEGE RESEARCH CENTER / TEACHERS COLLEGE, COLUMBIA UNIVERSITY Endnotes Scott-Clayton, Crosta, & Belfield (2012). 1. 2. Scott-Clayton, personal communication, NCES QuickStats (2013). 3. Bailey (2009). 4. Based on calculations using the National Educational Longitudinal Study (NELS:88). The comparison figure for nonremedial students is 43 percent (Attewell, Lavin, Domina, & Levey, 2006). 5. Bailey, Jeong, & Cho (2010). 6. Fields & Parsad (2012). 7. We include one study (Bettinger & Long, 2005) that is similar to the other seven studies in conceptual approach and rigor but which used an instrumental variable (I V) approach to compare outcomes of students in Ohio, where various colleges differed in how they assigned students to remediation. For further information as to why we include these studies and not others, see Bailey, Jaggars, and Scott-Clayton (2013). 8. Math cutoff scores range from 27– 40 out of 100 on the COMPASS Algebra test and from 28 – 40 out of 100 on the COMPASS Pre-Algebra/Arithmetic tests. Cutoff scores on the COMPASS reading test range from 46 – 81 out of 100. Cutoff scores on the COMPASS writing test range from 28 –59 out of 100. In all tables, nonsignificant findings are shown as NULL; significant findings are 9. significant at or below the 10-percent level. 10. R D I V analyses based on a sample limited to students under the age of 21 who began full- time at a Tennessee two-year public college in the fall of 2000 and whose assignment to remediation was based on a COMPASS math, reading, or writing exam. Students were tracked for eight years. Covariates include gender, race, age, high school GPA, college financial aid, and postsecondary institution attended. College credits completed are those for six years (Boatman & Long, 2013). 11. R D I V analyses based on sample of 255,878 degree-seeking freshman entering Texas public two-year colleges between 1991–1992 and 1999 –2000 and tracked until 200 4 – 2005; analysis limited to students who took the Texas Academic Skills Program (TASP) test used for remedial placement at Texas community colleges (Martorell & McFarlin, 2 011) . 12 . I V analyses based on 13,000 first-time, degree-seeking, traditional-age students who took the AC T and enrolled in one of 19 public two-year Ohio colleges in 1998. Students were tracked for five years (Bettinger & Long, 2005). 13. R D analyses based on a sample of 100,250 first-time, degree-seeking students admitted to one of six community colleges in a large urban community college system (LUCCS) between fall 2001 and fall 2007. Students were tracked for at least three years. From 2001–200 4, LUCCS used an in-house math exam; after 200 4, it used the COMPASS pre-algebra and algebra tests. For reading ,it used the COMPASS test, and for writing, it used a customized writing test. Main specification results are shown (Scott-Clayton & Rodríguez, 2012). 14. R D analyses based on a sample of 100,000 students who enrolled in one of 28 Florida community colleges in 1997 and took the Florida College Placement Test (CP T). Students were tracked through 2002. Math results are based on Regression 8: “R D I V full no- retesting sample.” Reading results are based on Regression 10:“R D I V, no retesting and narrow band sample” (Calcagno & Long, 2008). 15. Full sample consists of 24,66 4 first-time community college students who enrolled in one of 23 Virginia community colleges in summer or fall of 200 4. R D I V analyses limited to 5,4 40 students who took the pre-algebra section of the COMPASS test. Covariates include gender, age, intent, and dual enrollment status. Results are from baseline model (Dadgar, 2012). 6

7 RESEARCH OVERVIEW / JANUARY 2014 / DEVELOPMENTAL EDUCATION OUTCOMES 16 . R D analyses based on sample of 46,000 students who enrolled at one of Virginia’s 23 community colleges in 200 4 –2006 who took a COMPASS reading or writing exam. Students were tracked until 2011. Covariates include gender, race, cohort, financial aid, transfer program, and dual enrollment prior to college (Xu, 2013). Among the 52 null results shown in the three tables, 32 trend in a negative direction, and 1 7. 17 trend in a positive direction (and in three cases the direction of the null effect was not indicated in the original study). None of these results are statistically significant, so we cannot say with any certainty whether they indicate a real effect or are just due to chance. 18 . Xu (2013). Hodara (2012). 19. Bailey et al. (2010). 20. 21. Analysis tracked for three years 63,650 first-time, credential-seeking students at 35 Achieving the Dream community colleges who began their enrollment from fall 2006 to fall 2008 and were referred to at least three levels of developmental education. The figure on student progression through the math developmental sequence is updated from analyses originally presented in Bailey et al. (2010). 22. Analysis tracked for three years 11,210 first-time, credential-seeking students at 16 Achieving the Dream community colleges who began their enrollment from fall 2006 to fall 2008 and were referred to at least three levels of developmental education. The figure on student progression through the reading developmental sequence is updated from analyses originally presented in Bailey et al. (2010). 23. See part two of this practitioner packet, Designing Meaning ful Developmental Reform . 24. Scott-Clayton & Rodríguez (2012). References Attewell, P. A., Lavin, D. E., Domina, T., & Levey, T. (2006). New evidence on college remediation. , 77(5), 886–924. Journal of Higher Education Bailey, T. (2009). Challenge and opportunity: Rethinking the role and function of developmental New Directions for Community Colleges , 145 , 11–30. education in community college. Characterizing the effectiveness of developmental Bailey, T., Jaggars, S. S., & Scott-Clayton, J. (2013). education: A response to recent criticism . New York, NY: Columbia University, Teachers College, Community College Research Center. Bailey, T., Jeong, D. W., & Cho, S. W. (2010). Referral, enrollment, and completion in developmental education sequences in community colleges. Economics of Education Review , 29 (2), 255–270. Bettinger, E., & Long, B. T. (2005). Remediation at the community college: Student participation New Directions for Community Colleges, Spring (129), 17–26. and outcomes. Boatman, A., & Long, B. T. (2013). Does remediation work for all students? How the effects of postsecondary remedial and developmental courses vary by level of academic preparation: Additional data and updated analysis (NCPR Working Paper). New York, NY: National Center for Postsecond - ary Research. Calcagno, J. C., & Long, B. T. (2008). The impact of postsecondary remediation using a regression discontinuity approach: Addressing endogenous sorting and noncompliance (NCPR Working Paper). New York, NY: National Center for Postsecondary Research. 7

8 Dadgar, M. (2012). Essays on the economics of community college students’ academic and labor (Doctoral dissertation). Retrieved from ProQuest Dissertations and Theses database. market success (Order No. 1014168985) - Fields, R., & Parsad, B. (2012). Tests and cut scores used for student placement in postsecondary edu cation: Fall 2011 . Washington, DC: National Assessment Governing Board. Hodara, M. (2012). Language minority students at community college: How do developmental edu - cation and English as a second language affect their educational outcomes? (Doctoral dissertation). Retrieved from ProQuest Dissertations and Theses database. (Order No. 1014026879) Martorell, P., & McFarlin, I., Jr. (2011). Help or hindrance? The effects of college remediation on academic and labor market outcomes. Review of Economics and Statistics , 93 (2), 436–454. Scott-Clayton, J., Crosta, P. M., & Belfield, C. R. (2012). Improving the targeting of treatment: Evidence from college remediation (NBER Working Paper No. 18457). Cambridge, MA: National Bureau of Economic Research. Scott-Clayton, J., & Rodríguez, O. (2012). Development, discouragement, or diversion? New (NBER Working Paper No. 18328). Cambridge, MA: evidence on the effects of college remediation National Bureau of Economic Research. Assistance or obstacle? The impact of different levels of English remediation on un - Xu, D. (2013). derprepared students in community colleges (Unpublished doctoral dissertation). Teachers College, Columbia University, New York, NY. This research overview was prepared by Shanna Smith Jaggars and Georgia West Stacey, Community College Research Center, Teachers College, Columbia University. Funding was provided by the Bill & Melinda Gates Foundation. Community College Research Center Teachers College, Columbia University 525 West 120th Street, Box 174 New York, New York 10027 Tel: 212.678.3091 Fax: 212.678.3699 [email protected] http://ccrc.tc.columbia.edu

Related documents

NCDOT Current STIP

NCDOT Current STIP

May 2019 NCDOT Current STIP

More info »
June2018CUR

June2018CUR

CHANCELLOR'S UNIVERSITY REPORT JUNE 25 2018

More info »
iBudget Waiver Handbook June 2018

iBudget Waiver Handbook June 2018

Florida Medicaid DEVELOPMENTAL DISABILITIES INDIVIDUAL BUDGETING WAIVER SERVICES COVERAGE AND LIMITATIONS HANDBOOK Agency for Health Care Administration June 2018

More info »
Microsoft Word   Responsibilities in Action Massachusetts Curriculum FINAL 7 28 17.docx

Microsoft Word Responsibilities in Action Massachusetts Curriculum FINAL 7 28 17.docx

Responsibilities in Action Understanding the Connections The Massachusetts Medication Administration Program Certification Training

More info »
PRINCIPLES FOR EVALUATING HEALTH RISKS IN CHILDREN ASSOCIATED WITH EXPOSURE TO CHEMICALS: Environmental Health Criteria 237

PRINCIPLES FOR EVALUATING HEALTH RISKS IN CHILDREN ASSOCIATED WITH EXPOSURE TO CHEMICALS: Environmental Health Criteria 237

This report contains the collective views of an international group of experts and does not necessarily represent the decisions or the stated policy of the United Nations Environment Programme, the In...

More info »
CFOC4 pdf  FINAL

CFOC4 pdf FINAL

National Health and Safety Performance Standards • Guidelines for Early Care and Education Programs, 4th Edition Caring for Our Children Caring for Our Children Caring for Our National Health and Safe...

More info »
CFOC3 updated final

CFOC3 updated final

Caring for Our Children: National Health and Safety Performance Standards; Guidelines for Early Care and Education Programs, Third Edition A Joint Collaborative Project of American Academy of Pediatri...

More info »
Microsoft Word   1. Dev Ed Lit Review Title embedded

Microsoft Word 1. Dev Ed Lit Review Title embedded

unlocking ThE GATE What We Know About Improving Developmental Education Elizabeth Zachry Rutschow Emily Schneider June 2011

More info »
Wisconsin Model Early Learning Standards Fifth Edition

Wisconsin Model Early Learning Standards Fifth Edition

Wisconsin Model Early Learning Standards Fifth Edition The Wisconsin Model Early Learning Standards Steering Committee

More info »
Foundations for Young Adult Success: A Developmental Framework

Foundations for Young Adult Success: A Developmental Framework

CONCEPT PAPER FOR RESEARCH AND PRACTICE JUNE 2015 Foundations for Young Adult Success A Developmental Framework I N T E G R S D E N T A I S M Y T C E N D WORK FAMILY E I G D E A V E G S A D N L L E L ...

More info »
doubling graduation rates fr

doubling graduation rates fr

DOUBLING GRADUATION R ATES Three-Year Effects of CUNY’s Accelerated Study in Associate Programs (ASAP) for Developmental Education Students Susan Scrivener Michael J. Weiss Alyssa Ratledge Timothy Rud...

More info »
Evidence Based Practices for Children, Youth, and Young Adults with Autism Spectrum Disorder

Evidence Based Practices for Children, Youth, and Young Adults with Autism Spectrum Disorder

Evidence-Based Practices for Children, Youth, and Young Adults with Autism Spectrum Disorder Connie Wong, Samuel L. Odom, Kara Hume, Ann W. Cox, Angel Fettig, Suzanne Kucharczyk, Matthew E. Brock, Jos...

More info »
DEC CUR

DEC CUR

CHANCELLOR'S UNIVERSITY REPORT DECEMBER 10, 2018

More info »
1 leslie haley wasserman debby zambo

1 leslie haley wasserman debby zambo

Educating the Young Child 7 Advances in Theory and Research, Implications for Practice Leslie Haley Wasserman Debby Zambo Editors Early Childhood and Neuroscience - Links to Development and Learning

More info »
Navigating Social and Emotional Learning from the Inside Out: Looking Inside & Across 25 leading SEL Programs: A Practical Resource for Schools and OST Providers

Navigating Social and Emotional Learning from the Inside Out: Looking Inside & Across 25 leading SEL Programs: A Practical Resource for Schools and OST Providers

NAVIGATING SEL FROM THE INSIDE OUT LOOKING INSIDE & ACROSS 25 LEADING SEL PROGRAMS: A PRACTICAL RESOURCE FOR SCHOOLS AND OST PROVIDERS (ELEMENTARY SCHOOL FOCUS) MARCH 2017 Stephanie Jones, Katharine B...

More info »
OctoberCUR2018

OctoberCUR2018

CHANCELLOR'S UNIVERSITY REPORT OCTOBER 29 2018

More info »
Ready By Design 2016

Ready By Design 2016

READY BY DESIGN THE SCIENCE (AND ART) OF YOUTH READINESS Stephanie Malia Krauss, Karen J. Pittman and Caitlin Johnson March 2016

More info »
on the brink of collapse

on the brink of collapse

On the Brink of Collapse The Consequences of Underfunding California’s Developmental Services System Prepared by the Association of Regional Center Agencies February 2015

More info »
DoD7045.7H

DoD7045.7H

DoD 7045.7-H EPARTMENT OF D EFENSE D F UTURE Y EARS D EFENSE P ROGRAM (FYDP) S TRUCTURE Codes and Definitions for All DoD Components Office of the Director, Program Analysis and Evaluation A pril 2004

More info »
NTP Monograph on Health Effects of Low Level Lead

NTP Monograph on Health Effects of Low Level Lead

National Toxicology Program NTP U.S. Department of Health and Human Services NTP Monograph Health Effects of Low-Level Lead June 2012

More info »