1 What Community College Developmental Mathematics Students Understand about Mathematics James W. Stigler, Karen B. Givvin, and Belinda J. Thompson University of California, Los Angeles schools, are not able to perform basic arithmetic, pre- The nation is facing a crisis in its community algebra, and algebra, shows the real cost of our failure colleges: more and more students are attending to teach mathematics in a deep and meaningful way in community colleges, but most of them are not prepared our elementary, middle, and high schools. Although our for college-level work. The problem may be most dire focus here is on the community college students, it is in mathematics. By most accounts, the majority of important to acknowledge that the methods used to teach students entering community colleges are placed (based mathematics in K-12 schools are not succeeding, and that on placement test performance) into “developmental” the limitations of students’ mathematical proficiency are (or remedial) mathematics courses (e.g., Adelman, 1985; cumulative and increasingly obvious over time. Bailey et al., 2005). The organization of developmental The limitations in K-12 teaching methods have been mathematics differs from school to school, but most well-documented in the research literature. The Trends in colleges have a sequence of developmental mathematics International Mathematics and Science Study (TIMSS) courses that starts with basic arithmetic, then goes on to video studies (Stigler & Hiebert, 1999; Hiebert et al., pre-algebra, elementary algebra, and finally intermediate 2003) showed that the most common teaching methods algebra, all of which must be passed before a student can used in the U.S. focus almost entirely on practicing enroll in a transfer-level college mathematics course. routine procedures, with virtually no emphasis on Because the way mathematics has traditionally understanding of core mathematics concepts that might been taught is sequential, the implications for students help students forge connections among the numerous who are placed in the lower-level courses can be quite mathematical procedures that make up the mathematics severe. A student placed in basic arithmetic may face two curriculum in the U.S. The high-achieving countries full years of mathematics classes before he or she can in TIMSS, in contrast, use instructional methods that take a college-level course. This might not be so bad if focus on actively engaging students with understanding they succeeded in the two-year endeavor. But the data mathematical concepts. Procedures are taught, of course, show that most do not: students either get discouraged but are connected with the concepts on which they are and drop out all together, or they get weeded out at based. In the U.S., procedures are more often presented each articulation point, failing to pass from one course as step-by-step actions that students must memorize, not to the next (Bailey, 2009). In this way, developmental as moves that make sense mathematically. mathematics becomes a primary barrier for students ever Given that U.S. students are taught mathematics as being able to complete a post-secondary degree, which a large number of apparently-unrelated procedures that has significant consequences for their future employment. must be memorized, it is not surprising that they forget One thing not often emphasized in the literature is most of them by the time they enter the community the role that our K-12 education system plays in this college. It is true that some students figure out, on their problem. We know from international studies that U.S. own, that mathematics makes sense and that procedures mathematics education produces student achievement forgotten can be reconstructed based on a relatively scores that fall below the scores of students in most small number of core concepts. And even a few students other industrialized nations. But the fact that community who don’t figure this out are smart enough to actually college students, most of whom graduate from U.S. high Math ~ Vol. 1, No. 3 ~ May 2010 Educator AMATYC 4

2 remember the procedures they are taught in school. But exploring the hypothesis that these students who have many students don’t figure this out, and these are the ones failed to learn mathematics in a deep and lasting way that swell the ranks of students who fail the placement up to this point might be able to do so if we can first tests and end up in developmental mathematics. convince them that mathematics makes sense, and then Sadly, all the evidence we have (which is not much) provide them with the tools and opportunities to think shows that although community college faculty are far and reason. In other words, if we can teach mathematics more knowledgeable about mathematics than are their as a coherent and tightly related system of ideas and K-12 counterparts (Lutzer et al., 2007), their teaching procedures that are logically linked, might it not be methods may not differ much from those used in K-12 possible to accelerate and deepen students’ learning and schools (Grubb, 1999). “Drill- create in them the disposition and-skill” is still thought to to reason about fundamental ...developmental dominate most instruction concepts? Might this approach mathematics becomes a Thus, (Goldrick-Rab, 2007). reach those students who students who failed to learn have not benefited from the primary barrier for students how to divide fractions in way they have been taught ever being able to complete elementary school, and who mathematics up to this point? also probably did not benefit (English & Halford, 1995). a post-secondary degree... from attempts to re-teach the Consideration of this algorithm in middle and high hypothesis led us to inquire school, are basically presented the same material in the into what we actually know about the mathematics same way yet again. It should be no surprise that the knowledge and understanding of students who are methods that failed to work the first time also don’t work placed into developmental math courses. Surprisingly, an in community college. And yet that is the best we have extensive search of the literature revealed that we know been able to do thus far. almost nothing about these aspects of community college Currently there is great interest in reforming students. Grubb (2005) made a similar point: we know developmental mathematics education at the community quite a bit about community college teachers and about college. Yet, it is worth noting that almost none of the the institutions in which they work reforms have focused on actually changing the teaching ...but our knowledge of students and their methods and routines that define the teaching and attitudes toward learning is sorely lack - learning of mathematics in community colleges. Many ing. ...The conventional descriptions of schools have instituted courses that teach students how developmental students stress demographic to study, how to organize their time, and how to have a characteristics (for example, first-generation more productive motivational stance towards academic college status and ethnicity) and external pursuits (Zachry, 2008; Zeidenberg et al, 2007). They - demands (such as employment and fam have tried to make it easier for students burdened with ily), but aside from finding evidence of low families and full-time jobs to find time to devote to self-esteem and external locus of control, their studies. They have created forms of supplemental there has been little effort to understand how instruction (Blanc et al., 1983; Martin & Arendale, developmental students think about their 1994) and learning assistance centers (Perin, 2004). education. (Grubb & Cox, 2005, p. 95). They have tried to break down bureaucratic barriers that Most of what we know about the mathematical make it difficult for students to navigate the complex knowledge of community college students we learn pathways through myriad courses that must be followed from placement tests (Accuplacer, Compass, MDTP). if students are ever to emerge from developmental math But placement test data is almost impossible to come by and pass a transfer-level course. Some have redesigned due to the high-stakes nature of the tests and the need the curriculum; they’ve accelerated it, slowed it down, to keep items protected. Further, the most commonly or tried to weed out unnecessary topics (e.g., Lucas & used tests (Accuplacer and Compass) are adaptive tests, McCormick, 2007). Yet few have questioned the methods meaning that students take only the minimum number used to teach mathematics (Zachry, 2008). of items needed to determine their final score, and so An assumption we make in this report is that they don’t take items that might give a fuller picture substantive improvements in mathematics learning will of their mathematical knowledge. Finally, most of the not occur unless we can succeed in transforming the way items on the placement tests are procedural in nature: mathematics is taught. In particular, we are interested in they are designed to assess what students are able to do, www.amatyc.org 5

3 but not what students understand about fundamental gather information that might help us in the design one- on-one interviews with students (the results of which are mathematical concepts. forthcoming). Because of this gap in the literature, we undertook More details on methods will be presented together the study reported here. Our aim was to gather information about what students actually understand with results in the following sections. about the mathematics that underlie the topics they’ve Placement Test Data been taught, including their understanding of the reasons for using known procedures. We also sought, specifically, Participants and Tests evidence that students used reasoning in answering All Santa Barbara City College students who took 1 mathematical questions. the Mathematics Diagnostic Testing Project (MDTP) We investigated these questions using two sources placement tests during the 2008-2009 school year were of data. The first data source was one collection of the included in the study. Tests were administered at three placement tests to which we’ve referred, those developed time points during the year: summer and fall of 2008, and by the Mathematics Diagnostic Testing Project (MDTP). spring of 2009. In all, 5830 tests were administered. The purpose of examining it was to see what we could There were four different tests: Algebra Readiness, glean about student understanding from an existing Elementary Algebra, Intermediate Algebra, and Pre- measure. The MDTP tests are unusual in that they are Calculus. Although the majority of students took only not a commercially designed or administered test, and one test, some took more than one in order to determine are not adaptive. They were developed back in the their placement in the mathematics sequence. The gender early 1980s as a joint CSU/UC project whose members of participants was relatively stable across tests, with included mathematics faculty from all segments of public slightly more males than females in each case. Ethnicity education in California. Specifically, the group included varied somewhat depending on test form, with the mathematics professors from public institutions of higher Hispanic and Black populations decreasing as test level education and high school mathematics teachers, as increased. The Asian population increased as test level well as some physical sciences higher education faculty. increased. Age decreased slightly with increase in test The goal of the first test was not only placement of level. entering University students, but also to give feedback There are 50 multiple choice items on the Algebra to high schools on how well prepared their students Readiness and Elementary Algebra assessments, 45 were in mathematics areas critical for later success in on the Intermediate Algebra assessment, and 60 on the college. But many community colleges do use the MDTP Pre-Calculus assessment. The items on each assessment for placement purposes, including more than 50 in are grouped into multiple subscales, defined by the test California. Interestingly, the tests used by most California writers. For the Algebra Readiness, Elementary Algebra, community colleges—in particular, the test used in this and Intermediate Algebra assessments, students had 45 study—have not changed since 1986. For this study we minutes to complete the test. For the Pre-Calculus test have been able to get access to all placement test data students were allowed 90 minutes. given by Santa Barbara City College for the past nearly Student Difficulties 20 years. For the present report, we will present findings The examination of standardized test results often from the tests administered during the 2008-2009 begins and ends with an analysis of mean scores. Our academic year. primary interest in the MDTP, however, lay not in The second data source was a survey of math the percent of items students got correct on the test questions that we administered to a convenience sample or on a subscale of it, but rather in what their answer of 748 community college developmental mathematics selections could tell us about their thinking. A correctly students. There were a total of twelve questions, and chosen response on a multiple choice test indicate may each student answered four. The purpose of this survey understanding. (That’s an issue to be pursued with the was to delve more deeply into students’ thinking and to forthcoming interviews.) The selection of a wrong answer can sometimes be even more telling. Students 1 occasionally answer questions randomly, but more often We differentiate between students’ understanding of the reasons for executing particular procedures from their ability than not, they make their selection with some thought. to reason. The former reflects their understanding of the Exploring the patterns of students selections of wrong link between procedures and the mathematical concepts that answers was therefore our starting point in identifying underlie them and the latter reflects their ability to reach a student difficulties. logical conclusion based on what they know. Math ~ Vol. 1, No. 3 ~ May 2010 Educator AMATYC 6

4 Our examination of incorrect answers has focused thus far on the Algebra Readiness and Elementary Algebra assessments. For each we determined which items on the test proved most difficult for students. There were three criteria upon which our definition of difficulty was based. First, we included all items for which fewer than 25 percent of participants marked the correct answer. We also included items for which more students selected one incorrect answer than selected the correct answer. Finally, we counted those items for which there were two incorrect answer options, each of which was selected by at least 20 percent of students. The result was a collection of 13 difficult items for Algebra Readiness and 10 difficult items for Elementary Algebra. Those items and the common errors made on them are described in Tables 1 and 2, respectively. It is important to note that the table describes common procedural errors. Errors in reasoning are described in a subsequent section. Table 1. Difficult items on the Algebra Readiness form of the MDTP (in ascending order of percent correct), 2008-2009. Item description % of Common error(s) % of students who students answered who made correctly common error Add a simple fraction and Found GCF 28 19 a decimal. 21 Converted decimal to a fraction, then added 59 Find LCM of two numbers. numerators and added denominators 22 Converted fractions to decimals and ordered 36 Order four numbers (two simple fractions and two by number of digits decimals). Represented 1/3 as 0.3 and ordered decimals 24 by number of digits Add two squares under a 23 Added two squares, but failed to take the 31 square root, stopping short of solving radical. 2 2 2 25 = ( a + b ) b or that + a Assumed 2 2 2 2 + ab ab += 24 Approximated ratio 25 Find a missing length for one of two similar Multiplied two bases of one triangle and 23 triangles. divided by a base of the second triangle 24 Added numerators and added denominators 41 Add two improper fractions. 45 26 Added numerators and denominators of the Find the missing value of two fractions provided, stopping short of a portion of a circle that solving [other option also involved stopping has two portions labeled short] with simple fractions Find the diameter of a 26 Found radius and failed to cancel �, stopping 37 short of solving circle, given the area. Find the percent increase 27 Found dollar amount increase and labeled it as 43 between two dollar a percentage, stopping short of solving amounts. 23 Used larger of the two amounts as denominator when calculating increase www.amatyc.org 7

5 Find area of half of Found area of the square, stopping short of 28 33 a square drawn on a solving coordinate plane. Found smallest fraction or converted to Find the largest of four 33 44 simple fractions. decimals and chose the only fraction that didn’t repeat Multiply two simple 37 Simplified incorrectly before multiplying 22 fractions. Simplified incorrectly before multiplying 20 Divide one decimal by 41 Misplaced decimal (omitted zero as a 23 placeholder) another. Divided denominator by numerator and 20 misplaced decimal Table 2. Difficult items on the Elementary Algebra form of the MDTP (in ascending order of percent correct), 2008-2009. % of Common error(s) % of Item description students students who who made answered common correctly error 34 Added numerators and added denominators Add two fractions that 15 include variables Multiply two fractions 24 Simplified incorrectly before multiplying and 16 misplaced negative sign that include variables. Simplified incorrectly before multiplying 23 17 Factored the quadratic equation incorrectly 21 x in a quadratic Solve for x incorrectly and perhaps also solved for equation. Simply a fraction that Simplified incorrectly 19 31 includes variables. Find the percent that a 26 Divided the smaller number by the larger and 25 converted the quotient to a decimal larger number is of a smaller. 24 Divided the larger number by the smaller number, but failed to move the decimal in converting to a percent, stopping short of solving 26 Ignored the negative sign in the exponent 50 Find the value of a number with a negative exponent. 23 Found the area of a triangle different from the Find the area of a triangle 31 one asked within a square that shares a common base. 22 Multiplied the two lengths provided Math ~ Vol. 1, No. 3 ~ May 2010 Educator AMATYC 8

6 Square a binomial Squared each term in the expression and made 32 38 an error with the negative sign difference. 21 Squared each term in the expression and xy omitted the ‘ ’ term 34 24 Find the difference Subtracted one number from the other and between two square roots. took square root of the difference Factored the numbers provided and placed 23 the common factor outside the radical without taking its square root Identify the smallest of 46 Constructed equation incorrectly 23 three consecutive integers given the sum of those integers Answer choices related to decimals lead us think that When we examined the errors students made on the students may not have a firm grasp of place value. For most difficult test items, a few core themes emerged. instance, two frequently chosen answer options suggested Students’ tendencies to make those errors were quite that students believed that the size of a value written in consistent: when they could make these errors, they decimal form was determined by the number of digits in did. We then looked at the ten items on each of the two it (e.g., 0.53 < 0.333). tests that were answered correctly by the most students. Another emergent theme suggested that students None of these items provided opportunities for making do not know how to operate with exponents and square the kinds of errors we saw among the difficult items. roots. For example, some students added terms that Several of the most common errors involved working 2 2 2 shared a common exponent (e.g., 4 ). Others = 9 + 5 with fractions. Across the two placement tests, the most treated the sum of two numbers as the same as the square common mistake was to simplify incorrectly. On the of those numbers. Algebra Readiness assessment, two of the frequent errors Two final themes were related not as much to on difficult problems were caused by simplifying proper 2 procedural misunderstanding as they were to problem On fractions incorrectly (e.g., simplifying 9/16 as 3/4). solving. It was common, particularly on the Algebra the Elementary Algebra assessment, three of the frequent Readiness assessment, for students to respond to a errors on difficult problems were made when simplifying 2 multi-step problem by completing only the first step. + 5) as x + 1)/( x terms with variables (e.g., simplifying ( It was as if they knew the steps to take, but when they + 4). In these cases the option chosen showed that x 1/( saw an intermediate response as an answer option, they either the students factored expressions incorrectly or aborted the solution process. This may provide evidence made no attempt to use factoring. that students have a disposition to treat the goal of It was also the case, as is common with younger mathematical problems as getting answers quickly rather students, that our community college sample frequently than correctly and with understanding. Another possible added across the numerator and across the denominator interpretation is that the student knew the first step, when adding fractions (e.g., 1/2 + 2/3 = 3/5). Three of and then knew there was some next step, but couldn’t the commonly chosen wrong answers we examined were remember it and chose the option matching what s/he caused by that mistake on the Algebra Readiness test knew was correct. “Stopping short” could be used to and the process presented itself also on the Elementary explain five of the common errors on difficult Algebra Algebra assessment. Finally, the Algebra Readiness test Readiness items and one error on a difficult Elementary also showed multiple instances of converting a fraction to Algebra item. a decimal by dividing the denominator by the numerator Lastly, it appeared as though students sometimes (e.g., 5/8 = 8 ÷ 5). These errors reveal that rather than fell back on their knowledge of how math questions are using number sense, students rely on a memorized typically posed. It was as if the item (or answer options) procedure, only to carry out the procedure incorrectly or prompted their approach to it. For instance, when asked inappropriately. to find the least common multiple of two numbers that 2 In order to protect the items that appear on the MDTP, items also had a greatest common factor other than one, they are discussed in general terms and numbers have been changed. www.amatyc.org 9

7 selected the answer that represented the greatest common two sides. Nearly a quarter of students selected an answer factor. For example, if asked for the least common that was geometrically impossible. They selected lengths multiple of 6 and 9, students answered 3 (the greatest that could not have made a triangle, given the two lengths common factor) instead of 18 (the correct answer). provided. Two of their answer choices yielded triangles Rarely do students practice finding least common with two sides whose sum was equal to the length of the multiples on anything but numbers without common third side. The third choice produced a triangle with a factors, so they assumed in this case that the question was base longer than the hypotenuse. actually seeking the greatest common factor. Another geometry problem provided a diagram of Students also fell back on what they’re typically similar triangles and asked students to identify a missing asked to do when they were presented with a percentage AB length. The base of the larger triangle ( in Figure 1) to calculate. Instead of finding what percentage 21 is was indicated to be 28. Students were to use the values of 14 (as was asked), they calculated the percentage 14 of the other sides to find the length of AC . One of the is of 21. The latter, with a result less than 100 percent, Thirteen answer options was strikingly out of range. is the more frequent form of the question. Finally, on a was 84. AC percent of students said that the length of geometry problem that prompted students to find the area What they did was notice that the length of one of the of a figure, they operated on the values provided in the bases was three times the other and therefore multiplied problem without regard to whether the values were the 28 (i.e., the length of AB ) by 3 to get their answer. appropriate ones. They simply took familiar operations Presumably, they didn’t check to see if their answer made and applied them to what was there. logical sense. Do We See Evidence of Reasoning? As with many standardized mathematics tests, the B A C items of the MDTP focus on procedural knowledge; very little reasoning is called for. Because of that, it is difficult and AC Figure 1. Line segments AB represent the bases of to assess reasoning from test scores. When we examine two similar triangles. frequent procedural errors though, we can see many cases where, had students reasoned at all about their answer choice, they wouldn’t have made the error. This lack of So is it the case that students are incapable of reasoning was pervasive. It was apparent on both the reasoning? Are they lacking the skills necessary to Algebra Readiness and the Elementary Algebra tests, estimate or check their answers? In at least one case, we across math subtopics, and on both “easy” and “difficult” have evidence that community college students have items. We will provide a number of specific examples. the skills they need. On one Elementary Algebra test On the Elementary Algebra test, students were asked x and y and were item, students were provided values for to find the decimal equivalent of an improper fraction. asked to find the value of an expression in which they Only one of the available answer options was greater than were used. (Though the expression included a fraction, 1, yet nearly a third of students (32 percent) selected a there was no need for either simplification or division, wrong answer. If students had had a sense for the value The item proved to be the third two error-prone tasks.) of the improper fraction (simply that it represented a easiest on the test, with nearly three quarters of students number greater than 1) and then scanned the options, they answering correctly. Their performance on the item could have eliminated all four distractors immediately demonstrates that they are capable of substituting values and without doing a calculation of any kind. and using basic operations to solve. That skill would have Another item prompted students to subtract a proper eliminated a great number of frequently chosen wrong fraction (a value nearly 1) from an improper fraction (a answers if students had thought to use it . If students familiar form of one and a half). Again, if students had had only chosen a value for the variable and substituted examined the fractions and developed a sense of what the this value into both the original expression and their answer should be, they would have known that it would answer choice, they could have caught the mistakes be slightly more than a half. Surprisingly, 13 percent they’d made doing such things as executing operations of students chose a negative number as their answer, and simplifying. Some people may think of substituting revealing that they could not detect that the first fraction answer options into an item prompt as purely a test- was greater than the second. taking strategy, but we argue that verification is a form of A geometry problem asked students to find one of reasoning. In this case, it shows that the student knows the bases of a right triangle, given the lengths of the other the two expressions are meant to be equivalent, and Educator AMATYC Math ~ Vol. 1, No. 3 ~ May 2010 10

8 should therefore have the same value. Survey Items We noted in the introduction that students are taught To construct the survey, we began by listing key mathematics as a large number of apparently-unrelated concepts in the mathematics curriculum, from arithmetic procedures that must be memorized. It appears from the through elementary algebra. They included comparisons MDTP that the procedures are memorized in isolated of fractions, placement of fractions on a number line, contexts. The result is that a memorized procedure isn’t operations with fractions, equivalence of fractions/ necessarily called upon in a novel situation. Procedures decimals/percents, ratio, evaluation of algebraic aren’t seen as flexible tools – tools useful not only in expressions, and graphing linear equations. Survey items finding, but also in checking answers. What do students were created to assess each of those concepts. To better think they are doing when they simplify an algebraic understand students’ thinking, several of the items also expression, or for that matter simplify a fraction? Do included the question, “How do you know?” they understand that they are generating an equivalent The initial survey consisted of 12 questions divided expression or do they think they are merely carrying out a into three forms of four questions each. Each student was procedure from algebra class? randomly given one of the three forms. We cannot know from the MDTP the degree to which Understanding of Numbers and Operations students are capable of reasoning, but we do know that The first items we will examine tried to get at their reasoning skills are being underutilized and that students’ basic understanding of numbers, operations, their test scores would be greatly improved if they had a and representations of numbers. We focused on fractions, disposition to reason. decimals, and percents. In one question students were instructed: “Circle Survey the numbers that are equivalent to 0.03. There is more Study Participants than one correct response.” The eight choices they were Students were recruited from four community asked to evaluate are shown in Table 4, along with the colleges in the Los Angeles metropolitan area. All were percentage of students who selected each option. (The enrolled in 2009 summer session classes, and all were order of choices has been re-arranged according to their taking a developmental mathematics class. There were frequency of selection.) 748 participants (82 in Arithmetic, 334 in Pre-Algebra, Table 4. Survey question: Circle the Numbers Equivalent to 319 in Elementary Algebra, and 13 for whom class 0.03. information was missing). We collected no data from Intermediate Algebra students, even though it, too, is not Percent of Students Who Response a college-credit-bearing class. Our sample lies mainly in Option Marked It as Equivalent to 0.03 the two most common developmental placements: Pre- 67* 3/100 Algebra and Elementary Algebra. 53* 3% We asked students to tell us how long it had been 38* 0.030 since their last math class and the results are shown in 23 3/10 Table 3. 12 0.30% 9* 30/1000 Table 3. Length of time since survey study participants’ 6 0.30 most recent math class. 3 3/1000 N Time Since Last Math Class *indicates a correct option 1 year or less 346 Only 4 percent of the students got all answers 2 years 118 correct. The easiest two options (3/100 and 3%) 83 3-5 years were correctly identified by only 67 percent and 53 More than 5 years 149 percent of the students, respectively. It appeared that Missing Data 52 as the answers departed further from the original form (0.03) students were less likely to see the equivalence. Although the modal student in our sample was 20 Interestingly, only 9 percent of students correctly years old (M = 22.6, SD = 7.3), the histogram of ages identified 30/1000 as equivalent, even though 38 percent has a rather long tail out to the right, with a number of correctly identified 0.030. It appears that some students students in their 30s and 40s. learned a rule (adding a zero to the end of a decimal www.amatyc.org 11

9 doesn’t change the value), yet only some of these saw Do We See Evidence of Reasoning? that 0.030 was the same as 30/1000. Students clearly As we analyzed the students’ responses, we started are lacking a basic fluency with the representations of to feel that, first, students will whenever possible decimals, fractions, and percents. The students enrolled just fire off some procedure that they seem to have in Elementary Algebra did significantly better on the remembered from before, and, second, that they item than those enrolled in Pre-Algebra or Arithmetic generally don’t engage in reasoning at all, unless ( = .001). Yet, even of the students p (156, 2) = 7.290, F there is just no option. When they do reason they have in Algebra, only 17 percent correctly chose 30/1000 as difficulty. No doubt this is due in part to the fragile equivalent to 0.03. understanding of fundamental concepts that they bring Another question asked students to mark the to the task. But also it indicates a conception of what approximate position of two numbers (– 0.7 and 1 it means to do mathematics that is largely procedural, 3/8) on a number line that ranged from – 2 to +2 (see and thus a lack of experience reasoning about Figure 2). mathematical ideas. We asked students, “Which is larger, 4/5 or 5/8? Figure 2. Number line on which students were to place How do you know?” Seventy-one percent correctly -0.7 and 1 3/8. selected 4/5 and 24 percent selected 5/8. (Four percent did not choose either answer.) Twenty-four 2 0 –2 percent of the students did not provide any answer to the question, “How do you know?” Those who did Only 21 percent of students were able to place answer the question, for the most part, tried whatever both numbers correctly. Thirty-nine percent correctly procedure they could think of that could be done with placed – 0.7, and 32 percent correctly placed 1 3/8. two fractions. For example, students did everything Algebra students performed significantly better than from using division to convert the fraction to a < .01), the Arithmetic students ( F (362, 2) = 5.056, p decimal, to drawing a picture of the two fractions, to but still, only 30 percent of Algebra students marked finding a common denominator. What was fascinating both positions correctly. was that although any of these procedures could be On another question students were asked, “If n is used to help answer the question, students using the × n a positive whole number, is the product 1/3 greater procedures were almost equally split between choosing than n , or is it impossible to n , less than n , equal to 4/5 or choosing 5/8. This was often because they tell?” Only 30 percent of students selected the correct weren’t able to carry out the procedure correctly, or answer (i.e., less than n ). Thirty-four percent said that because they weren’t able to interpret the result of the the product would be greater than n (assuming, we procedure in relation to the question they were asked. think, that multiplication would always results in a Only 6 percent of the students produced an explanation larger number). Eleven percent said the product would that did not require execution of a procedure: they be equal to n , and 26 percent said that they could not simply reasoned that 5/8 is closer to half, and 4/5 tell (presumably because they think it would depend is closer to one. No one who reasoned in this way ). on what value is assigned to n incorrectly chose 5/8 as the larger number. Interestingly, students in Algebra were no more We asked a related question to a different group successful on this question than were students in either a “If of students: is a positive whole number, which of the other two classes ( < 0.136). F (176, 2) = 2.020, p a is greater: a /8?” If one is reasoning, then this /5 or Furthermore, students who reported longer time since should be an easier question than the previous one. their last math class (i.e., 2 years ago) actually did better Yet, it proved harder, perhaps because many of the than students who had studied mathematics more recently procedures students used to answer the previous p = 0.027). (i.e., a year or less ago; F (166, 3) = 3.139, question could not be immediately executed without This kind of question is not typical of what students having a value for . Only 53 percent of our sample a would confront in a mathematics class; they are not asked correctly chose a /5 as the larger number. Twenty-five to calculate anything, but just to think through what the /8, and 22 percent did not answer. a percent chose answer might be. Perhaps the longer students have been We followed up this question by asking, “How do away from formal mathematics classes, the less likely you know?” This time, 36 percent were not able to they are to remember what they are supposed to do, and answer this question. (Interestingly, this percentage the more they must rely on their own understanding to was approximately the same for students who chose figure out how to answer a question like this one. a /5 as for those who chose a /8.) Of those who did ~ Vol. 1, No. 3 ~ May 2010 Math Educator AMATYC 12

10 Table 5. Response options and percent of students produce an answer, most could be divided into three choosing them in answer to the question, “If , c = b + a categories. which of the following equations are also true?” Some students simply cited some single aspect of the two fractions as a sufficient explanation. For Percent Students Response example, 5 percent simply said that “8 is bigger” Choosing Option or “8 is the larger number.” All of these students 91* b + = c a a /8 as the larger number. In a related incorrectly chose 89* b + a = c explanation, 17 percent mentioned the denominator 45* a = b – c as being important - which it is, of course - but half 41* a – c b = /8 as the larger a of these students incorrectly chose 17 b c = a – number. 28* – a = 0 c b + Another group of students (10 percent) used a 9 b – a + c = 0 procedure, something they had learned to do. For *indicates a correct option a example, some of them substituted a number for and then divided to find a decimal, but not always Most of the students knew that the first two options the correct decimal. Others cross multiplied, ending b were equivalent to a + . They knew that the order = c , or found a common denominator up with 8 a and 5 a didn’t matter ( a + a) and they knew that you b = b + (40ths). Approximately half the students who executed could switch what was on each side of the equals sign /5 as larger, and half a one of these procedures chose without affecting the truth of the equation. Still, 10 chose /8. They would execute a procedure, but had a percent of students did not know these two things. a hard time linking the procedure to the question they It proved much harder for students to recognize that had been asked to answer. = b – c (or, b = a – c , then c = b + a if ), with only 45 a The most successful students (15 percent) percent and 41 percent of the students choosing each of produced a more conceptual explanation. Some of these options. Students could have arrived at these two these students interpreted the fractions as division. answers either by executing a procedure (e.g., subtracting For example, they pointed out that when you divide from both sides of the equation) or by understanding b a number by five you get a larger number than if you the inverse relationship between addition and subtraction. divide it by eight. Others drew pictures, or talked It is illuminating to look at the patterns of response about the number of “pieces” or “parts” a was divided students gave to the following three options: into. Some said that if you “think about a pizza” cut – b = a c into five pieces vs. eight pieces, the five pieces would – a = c b be larger. Significantly, all of the students who used b = a c – /5 a these more conceptual explanations correctly chose Even though 40+ percent of students correctly chose as the larger number. the first two options, fully 13 percent chose all three Four percent of students said that it was impossible options as correct. This finding suggests that students to know which fraction was larger “because we don’t are examining each option in comparison to the original a know what is.” + ), but not necessarily looking at the c = b a equation ( We know from previous research that it is difficult options compared with each other. It is hard to imagine for students to make the transition to algebra, to how someone could believe that the latter two options learn to think with variables about quantities. These are simultaneously true, unless they mistakenly think that results, of much older students, suggest that the lack of b ) is not important, c – b – c the order of subtraction ( vs. experience thinking algebraically may actually impede overgeneralizing the commutative property of addition students’ understanding of basic arithmetic. to apply to subtraction, as well. Only 25 percent of the Another test item that revealed students’ ability to sample correctly chose both of the first two options but c reason was the following: “If , which of the a + b = not the third. following equations are also true? There may be more A similar analysis can be done with the last two than one correct response.” The possible responses, options: together with the percentage of students who chose b a + c – = 0 each response, are presented in Table 5. = 0 c – a + b Although 28 percent of the students correctly selected the first option as true, only 19 percent selected www.amatyc.org 13

11 explanations included: “Because if you add 1 to anything only the second. Nine percent of the first option and not or any number, the answer has to be different than the the students selected both options as true. Interestingly, letter in the question or equation;” or, “ x can’t equal for both of these last two pattern analyses, there was itself + 1.” For the second equation, correct explanations no significant effect of which class students are in on included: “A squared number should be positive since the their ability to produce the correct pattern of responses: first number was multiplied by itself;” or, “not possible Elementary Algebra students were no more successful because positive times positive will always be positive than Pre-Algebra or Arithmetic students. This is a very and negative times negative will always be positive.” intriguing result. It suggests that students who place Two more questions help to round out our into algebra may not really differ all that much in terms exploration of students’ reasoning about quantitative of their conceptual understanding from students placed relations. The first provided the equation = 0 and x a – into basic arithmetic or pre-algebra classes. The main is positive, if a increases, a x would: asked, “Assuming difference may simply be in the ability to correctly (a) increase, (b) decrease, (c) remain the same, or (d) remember and execute procedures, a kind of knowledge can’t tell. Only 25 percent of students correctly chose that is fragile without a deeper conceptual understanding increase. Thirty-four percent chose decrease, 23 percent, of fundamental mathematical ideas. remain the same, and 11 percent said that you can’t tell. In fact, none of the other items presented in this = 1 and asked the The second provided the equation ax section showed significant differences in performance same question. Only 15 percent of students got this item across the different classes. Clearly, there must be correct (decrease). Thirty-two percent said increase, 33 something different across these three classes of students percent, remain the same, and 14 percent said that you - hence their placement into the different classes. Yet, in can’t tell. terms of reasoning and understanding in the context of As with the previous items in this section, there was non-standard questions, we could find few differences. no significant difference in performance between students For the next two questions we told students the taking Arithmetic, Pre-Algebra, and Algebra. answer, but asked them to explain why it must be true. The first question was, “Given that x is a real number, Conclusions neither of these equations has a real solution. Can you In this study we drew from two sources of data to explain why that would be the case?” The equations paint a picture of what community college developmental were: mathematics students know about mathematics. Three + 1 = x x 2 findings are particularly worthy of note and jointly they = – 9 x have implications for the kind of instruction that may Forty-seven percent of the students could not think prove beneficial for community college students. First, of any explanation for why there would be no real our examination of responses on the MDTP showed that solution to the first equation. For the second equation, 50 the most common errors made by students were made percent could not generate an explanation. An additional whenever the possibility to do so was present. Students’ 8 percent of students for the first equation (7 percent for routines for calling upon procedures to solve problems the second equation) said that it would not be possible to appears to be well-established. Although their knowledge know if the equations were true or not unless they could of mathematical concepts may be fragile, their knowledge is. x know what of procedures is firmly rooted—albeit in faulty notions For the first equation, 23 percent of students tried of when and how procedures should be applied. Second, to solve it with an algebraic manipulation. For example, students can apply appropriate reasoning under the right , subtracted x they started with from both sides, x x + 1 = conditions, but that form of knowledge is rarely accessed. and then wrote down on their paper 1 = 0. Or, they Finally, when students are able to provide conceptual x – 1. Once subtracted 1 from each side and wrote: x = explanations, they also produce correct answers. they had obtained these results they did not know what (Whether the relationship is causal is deserving of further to do or say next. Similarly, for the second equation, exploration.) Together, the results suggest that students 20 percent launched into an algebraic manipulation. 2 should be encouraged to draw more extensively on Starting with x = – 9, for example, these students tried their extant conceptual reasoning and, where necessary, taking the square root of both sides, subtracting x from be provided with the skills to do so. Instructors should both sides, and so on. be prepared though, to aid students in breaking flawed Only 10 percent of students were able to give a good procedural habits, many of which have spent years explanation for the first equation, and only 9 percent for in the making. We argue here that these suggestions the second equation. For the first equation, these correct Math ~ Vol. 1, No. 3 ~ May 2010 Educator AMATYC 14

12 for instruction could improve learning outcomes for mathematics students. Those interviews will be used to developmental students, but our findings beg the dig deeper in each case, trying to discern what, precisely, question: might these students’ difficulties in math have underlies each student’s difficulties with mathematics. been prevented if the same suggestions were to have been Is it simply failure to remember the conventions of applied to their earlier grades? mathematics? Is it a deficiency in basic knowledge Our findings about students’ thinking suggest of number and quantity? Is it a lack of conceptual that there are positive consequences to understanding understanding? What do these students understand about mathematical concepts. Together with prior research on basic mathematics concepts (regardless of their ability to K-12 teaching practices, our findings are also consistent solve school-like mathematics problems)? Beyond this, with the hypothesis that there are long-term, negative we have two goals unique to the interviews. Through consequences of an almost exclusive focus on teaching them we will examine what students think it means to mathematics as a large number of procedures that must do mathematics. Is is just remembering, or is reasoning be remembered, step-by-step, over time. As the number also required? Finally, we will examine whether students of procedures to be remembered grows – as it does can reason if provided an opportunity and pressed to do through the K-12 curriculum – it becomes harder and it. Can they discover some new mathematical fact based harder for most students to remember them. Perhaps only on making effective use of other facts they know? most disturbing is that the students in community college A full report of the results of the interviews will be developmental mathematics courses did, for the most forthcoming, but there is already some good news to part, pass high school algebra. They were able, at one share. We have found that it is often possible to coax point, to remember enough to pass the tests they were students into reasoning by first asking them questions given in high school. But as they moved into community that could be answered by reasoning, and second, by college, many of the procedures were forgotten, or partly giving them permission to reason (instead of doing it the forgotten, and the fragile nature of their knowledge is way they were taught). Furthermore, the thirty students revealed. Because the procedures were never connected we have interviewed uniformly find the interview with conceptual understanding of fundamental interesting, even after spending well over an hour mathematics concepts, they have little to fall back on with the interviewer, thinking hard about fundamental when the procedures fade. mathematics concepts. This gives us further cause to The placement tests we examined provide ample believe that developmental math students might respond evidence that students entering community colleges well to a reason-focused mathematics class in which they have difficulty with the procedures of mathematics. are given opportunities to reason, and tools to support What is suggested by our data is that the reason for these their reasoning. procedural difficulties might be tied to a condition we are References calling conceptual atrophy: students enter school with The New College Course Map and Adelman, C. (1985). basic intuitive ideas about mathematics. They know, for . Washington, DC: US Dept of Transcript Files example, that when you combine two quantities you get Education. a larger quantity, that when you take half of something Bailey, Thomas (2009). Challenge and opportunity: you get a smaller quantity. But because our educational Rethinking the role and function of practices have not tried to connect these intuitive ideas developmental education in community college. to mathematical notation and mathematical procedures, New Directions for Community Colleges , Volume the willingness and ability to bring reason to bear on 2009, Issue 145,11-30. mathematical problems lies dormant. The fact that the Bailey, T., Jenkins, D., & Leinbach, T. (2005). community college students have so much difficulty with Community college low-income and mathematical notation is significant, for mathematical minority student completion study: Descriptive notation plays a major part in mathematical reasoning. statistics from the 1992 high school cohort. Because these students have not been asked to reason, New York: Columbia University, Teachers they also have not needed the rigors of mathematical College, Community College Research Center. notation, and so have not learned it. Blanc, R., DeBuhr, L., & Martin, D. (1983). Breaking Though we have learned a great deal from data the attrition cycle: The effects of supplemental reported here, we remain unsatisfied with the richness instruction on undergraduate performance and of the picture we’ve painted and our work on the topic , 54(1), Education attrition. Journal of Higher therefore continues. We alluded earlier to our intent to 80-90. carry out one-on-one interviews with developmental www.amatyc.org 15

13 Deil-Amen, R., & Rosenbaum, J.E. (2002). The unintended Stigler, J. W., & Hiebert, J. (1999). The teaching Sociology consequences of stigma-free remediation. gap: Best ideas from the world’s teachers for , 75(3), 249-268. of Education improving education in the classroom . New York: English, L.D. & Halford, G.S. (1995). Mathematics Free Press. . Mahwah, NJ: education: Models and processes Zachry, E.M. (2008). Promising instructional reforms Lawrence Erlbaum Associates. in developmental education: A case study of Goldrick-Rab, S. (2007). Promoting academic momentum . New York: three Achieving the Dream colleges at community colleges: Challenges and MDRC. opportunities . New York: Columbia university, Zeidenberg, M., Jenkins, D., & Calcagno, J. C. Teachers College, Community College Research Do student success courses actually (2007). Center. help community college students succeed? Grubb, N.W. & Cox, R.D. (2005). Pedagogical alignment New York: Columbia University, Teachers and curricular consistency: The challenges for College, Community College Research Center. developmental education. New Directions for Community Colleges, Volume 2005, Issue 129, Acknowledgements 93-103. We wish to thank Alfred Manaster, MaryAnne Honored but Grubb, N. W., & Associates. (1999). Anthony, and Robert Elmore for arranging access to the invisible: An inside look at teaching in MDTP data and for the trust they showed in doing so. We . New York: Routledge. community colleges are grateful also to Bruce Arnold and James Hiebert for Hiebert, J., Gallimore, R., Garnier, H., Givvin, K. B., their thoughtful comments on an earlier draft. This study Hollingsworth, H., Jacobs, J., Chui, A. M-Y., was funded by grants from the Carnegie Foundation for Wearne, D., Smith, M., Kersting, N., Manaster, the Advancement of Teaching, the Hewlett Foundation, A., Tseng, E., Etterbeek, W., Manaster, C., and the Bill and Melinda Gates Foundation. The views Gonzales, P., & Stigler, J. W. (2003). Teaching expressed in this article are part of ongoing research and mathematics in seven countries: Results from analysis and do not necessarily reflect the views of the the TIMSS 1999 Video Study (NCES 2003-013). funding agencies. Washington, DC: U.S. Department of Education, National Center for Education Statistics. Lucas, M.S., & McCormick, N.J. (2007). Redesigning Lucky Larry mathematics curriculum for underprepared John Savage college students. The Journal of Effective Montana State University Teaching , 7(2), 36-50. COT in Bozeman Lutzer, D.J., Rodi, S.B., Kirkman, E.E., & Maxwell, Statistical abstract of J.W. (2007). This ingenious solution was provided as a test undergraduate programs in the mathematical solution by one of my students, who seems to be sciences in the United States: Fall 2005 on a mission to stamp out the distributive rule. His CBMS Survey . Washington, DC: American process seems to work well on any proportion with Mathematical Society. number patterns similar to this example. Marcotte, D.E., Bailey, T., Boroski, C., & Kienzl, G.S. (2005). The returns of a community college 25 − − x x 52 education: Evidence from the National Education Solve: = 12 12 Longitudinal Study. Educational Evaluation and , 27(2), 157-175. Policy Analysis − − x x 25 52 Martin, D., & Arendale, D. (1994). Supplemental = 12 12 instruction: Increasing achievement − =⋅ −⋅ x x 25 12 12 52 New Directions in Teaching and and retention. Learning , 60(4), 1-91. − −= x x 260602 Perin, D. (2004). Remediation beyond developmental 2 2 − −= x 58 60 education: The use of learning assistance centers 58 58 −= x to increase academic preparedness in community colleges. Community College Journal of 1 =− x , 28(7), 559-582. Research and Practice Math ~ Vol. 1, No. 3 ~ May 2010 Educator AMATYC 16