Cognitive Complexity of FCAT Test Items

Transcript

1 OMPLEXITY C LASSIFICATION OGNITIVE C C T EST I TEMS FCAT OF The benchmarks in the Sunshine State Standards (SSS) identify knowledge and skills students are expected to on that students also demonstrate critical thinking. Goal acquire at each grade level, with the underlying expectati 3, Standard 4, of Florida’s System of School Improvement and Accountability makes this expectation clear: Florida students use creative thinking skills to gene rate new ideas, make the best decisions, recognize and solve problems through reasoning, interpret sy mbolic data, and develop efficient techniques for lifelong learning. ® Florida Comprehensive Assessment Test (FCAT) test items, while assessing SSS benchmarks, must also reflect this goal and standard. It is important lexity of knowledge and skills required to develop items that elicit the comp to meet these objectives. items is currently categorized in two ways: The degree of challenge of FCAT multiple-choice item difficulty and . Item difficulty has two meanings, depending on the stage of item development. Before cognitive complexity testing, item difficulty is a prediction of the percentage of students who will choose the correct answer. After testing, item difficulty refers to the percentage of students who actually chose the correct answer. Items for which the correct answer is chosen by more than 70 percent of the students are considered easy. Items for which nts are considered average. Items for which the correct the correct answer is chosen by 40–70 percent of the stude answer is chosen by less than 40 percent of the students are considered challenging. Cognitive complexity refers to the cognitive demand associat ed with an item. In the early years of the FCAT 1 program, the DOE used Bloom’s Taxonomy to classify test items; however, Bloom’s Taxonomy is difficult to use because it requires an inference a bout the skill, knowledge, and background of the students responding to the item. Beginning in 2004, the DOE implemented a new cognitive classification system based upon Dr. Norman L. 2 Webb’s Depth of Knowledge (DOK) levels. The rationale for classifying items by their level of complexity is to of the student. The demands on thinking that an item focus on the expectations of the item, not the ability makes—what the item requires the student to recall, understand, analyze, and do—are made with the assumption that the student is familiar with the basic concepts of the task. Items are chosen for the FCAT based on the SSS and their grade-level appropriateness, but the complexity of the items remains independent of the particular curriculum a student has experienced . The cognitive complexity of a multiple-choice item is generally NOT dependent on the distractors (answer choices). These answer choice options may affect the difficulty of the item, but not the complexity of the item. nd high complexity—form an ordered description of the The categories—low complexity, moderate complexity, a demands an item may make on a student. For example, low complexity items may require a student to solve a multiple steps. High complexity items may require a one-step problem. Moderate complexity items may require student to analyze and synthesize information. The distinctions made in item complexity ensure that items will assess the depth of student knowledge at each benchmark. The intent of the item writer weighs heavily in determining the complexity of an item. The pages that follow illustrate some, but not all, of the varying complexity demands that FCAT items might more descriptions, but should be classified in the highest make at each level. Note that an item may fit one or level of complexity demanded by the item. The final pa ge breaks down the percentages of points by cognitive complexity level for each content area. 1 Bloom, B.S., et al. Taxonomy of Educational Objectiv es, Handbook I: Cognitive Domain . New York: McKay, 1956. 2 Wisconsin Center of Educati onal Research. University of Webb, Norman L. and others. “Webb Alignment Tool” 24 July 2005. http://www.wcer.wisc.edu/WAT/index.aspx Wisconsin-Madison. 2 Feb. 2006. . ___________________________________________________________________________ Cognitive Complexity Classification of FCAT Test Items © July 2008 Florida Department of Education Page 1 of 6

2 OMPLEXITY C LASSIFICATION C OGNITIVE C T EST I TEMS OF FCAT FCAT Reading Low Complexity This category requires students to recall, observe, question, or represent basic facts. For a low complexity item, the student would be expected to demonstrate simple skills or abilities. A low complexity item requires only a or simple understanding of a single word or phrase. basic understanding of text—often verbatim recall from text to low complexity items include Skills required to respond correctly • identifying the correct meanings of grade-appropriate words; • locating details in a text; • locating details on a graph, chart, or diagram; • recognizing the correct order of events in a text; or • identifying figurative language in a text. Moderate Complexity Items may require a two-step process: first, comprehensi on, and then subsequent processing of text. Students are expected to make simple inferences within the text and may encounter items that include words such as summarize, infer, classify, gather, organize, compare, and display . Depending on the objective of a particular moderate level item, students may also be required to explain, describe, or interpret. Skills required to respond correctly to moderate complexity items include • using context clues to identify the meanings of unfamiliar words; determining how details support the main idea; • • interpreting the information in graphs, charts, and diagrams; • identifying cause-and-effect relationships; determining an author’s main purpose or point of view; • • identifying similarities and differences; • demonstrating an understandi ng of plot development; • recognizing elements of plot; • recognizing patterns of organization; • summarizing the major points of a text; or • comparing word meanings. High Complexity thinking. Students may be encouraged to explain, High complexity items make heavy demands on student xity items require several steps involving abstract generalize, or make multiple connections. High comple reasoning and planning. Students must be able to support their thinking. Items may involve identifying theme and implicit main idea and making complex inferences within or across text. Students may also be asked to take information from at least one portion of the text and apply this information to a new t ask. They may be asked to perform complex analyses of the connections among texts. Skills required to respond correctly to high complexity items include • analyzing the use of figurative language in a text; • showing how graphs, charts, and diagrams contribute to a text; • determining an author’s purpose and/or point of view and describing how it affects the text; • evaluating strong vs. weak arguments in a text; • analyzing similarities and differences; describing and analyzing the characteristics of various types of literature; • • describing and illustrating how common themes are found across texts; or • analyzing cause-and-effect relationships. ___________________________________________________________________________ Cognitive Complexity Classification of FCAT Test Items © July 2008 Florida Department of Education Page 2 of 6

3 OGNITIVE C LASSIFICATION C C OMPLEXITY EST I TEMS FCAT OF T FCAT Mathematics Low Complexity This category relies heavily on the recall and recognition of previously learned concepts and principles. Items typically specify what the student is to do, which is of ten to carry out some procedure that can be performed mechanically. It is not left to the student to come up w ith a low complexity original method or solution. Skills required to respond to low complexity items include solving a one-step problem; • • computing a sum, difference, product, or quotient; • evaluating a variable expression, given specific values for the variables; recognizing or constructing an equivalent representation; • recalling or recognizing a fact, term, or property; • • retrieving information from a graph, table, or figure; • identifying appropriate units or tools for common measurements; or • performing a single-unit conversion. Moderate Complexity Items in the moderate complexity category involve more flexible thinking and choice among alternatives than low complexity items. They require a response that goes beyond the habitual, is not specified, and ordinarily has more than a single step. The student is expected to decide what to do — using informal methods of reasoning and and knowledge from various problem-solving strategies—and to bring together skill domains. Skills required to respond to moderate complexity items include solving a problem requiring multiple operations; • • solving a problem involving spatial visualization and/or reasoning; • tions, depending on s ituation and purpose; selecting and/or using different representa • retrieving information from a graph, table, or figure and using it to solve a problem; determining a reasonable estimate; • extending an algebraic or geometric pattern; • providing a justification for steps in a solution process; • • comparing figures or statements; • representing a situation mathematically in more than one way; or • formulating a routine problem , given data and conditions. High Complexity High complexity items make heavy demands on student thinking. Students must engage in more abstract ght. The high-complexity item requires that the student reasoning, planning, analysis, judgment, and creative thou ed to respond correctly to high complexity items include think in an abstract and sophisticated way. Skills requir performing a procedure having multiple steps and multiple decision points; • • solving a non-routine problem (as determ ined by grade-level appropriateness); • solving a problem in more than one way; • describing how different representations can be used for different purposes; • generalizing an algebraic or geometric pattern; • explaining and justifying a solution to a problem; • describing, comparing, and contrasting solution methods; • providing a mathematical justification; • analyzing similarities and differences between procedures and concepts; • formulating an original problem, given a situation; formulating a mathematical mode l for a complex situation; or • analyzing or producing a deductive argument. • ___________________________________________________________________________ Cognitive Complexity Classification of FCAT Test Items © July 2008 Florida Department of Education Page 3 of 6

4 OMPLEXITY C LASSIFICATION C OGNITIVE C T EST I TEMS OF FCAT FCAT Science Low Complexity This category relies heavily on the recall and recognition of previously learned concepts and principles. Items typically specify what the student is to do, which is of ten to carry out some procedure that can be performed mechanically. It is not left to the student to come up with an original method or solution. Skills required to respond to low complexity items may include, but are not limited to, • identifying a common example or recognizing a concept; • retrieving information from a chart, table, diagram, or graph; • recognizing a standard scientific represe ntation of a simple phenomenon; or • calculating or completing a familiar single-step pr ocedure or equation using a reference sheet. Moderate Complexity Items in the moderate complexity category involve more flexible thinking and choice among alternatives than low complexity items. They require a response that goes beyond the habitual, is not specified, and ordinarily has more than a single step or thought process. The student is exp ected to decide what to do—using informal methods of together skill and knowle reasoning and problem-solving strategies—and to bring dge from various domains. Skills required to respond to m oderate complexity items may include, but are not limited to, • applying or inferring relationships among f acts, terms, properties, or variables; • describing examples and nonexamples of scientific processes or concepts; • predicting or determining the logical next step or outcome; • comparing or contrasting structures or func tions of different organisms or systems; • choosing the appropriate formula or equation to solve a problem and then solving it; or • applying and using concepts from a standard scientific model or theory. High Complexity High complexity items make heavy demands on student thinking. Students must engage in more abstract reasoning, planning, analysis, judgment, and creative thought. The items require that the student think in an eps. Skills required to respond to high complexity items abstract and sophisticated way often involving multiple st may include, but are not limited to, • constructing models for research; • generalizing or drawing conclusions; • designing an experiment, given data and conditions; • explaining or solving a problem in more than one way; • providing a justification for steps in a solution or process; • analyzing an experiment to identify a fl aw and propose a method for correcting it; oblem involving spatial relationships; or interpreting, explaining, or solving a pr • • predicting a long-term effect, outcome, or result of a change within a system. ___________________________________________________________________________ Cognitive Complexity Classification of FCAT Test Items © July 2008 Florida Department of Education Page 4 of 6

5 C LASSIFICATION OMPLEXITY C C OGNITIVE FCAT I EST T OF TEMS FCAT Writing High Complexity For FCAT Writing, this is not a multiple-choice item. This category makes heavy demands on students’ thinking. Instead, students are asked to produce an extended, writte n response to an assigned topic for a designated purpose. The response should integrate the writing elements of focus, organization, support , and conventions. Students may explain, generalize, or make multip le connections. Inherent in such an item is the expectation that students with the topic and purpose. This item type requires respond in a way that indicates insight and involvement students to support their thinking as they engage in developing a written composition. The writing task calls for a mature command of language and complex sentence structures, showing synthesis and analysis of compositional elements. The student is provided an extended time period to work on a response to this type of item. Some of the skills required for such a high complex ity item include, but are not limited to, the following: • establishing a relationship between a written response and the intended audience and purpose; using the prewriting and drafting processes; and • capitalization, punctuation, and sentence structure. applying complex conventions for spelling, usage, • ___________________________________________________________________________ Cognitive Complexity Classification of FCAT Test Items © July 2008 Florida Department of Education Page 5 of 6

6 OGNITIVE OMPLEXITY C LASSIFICATION C C EST I TEMS FCAT OF T Percentage of Points by Cognitive Complexity Level for FCAT Content Areas The tables below show the target range for the percen tage of points by cognitive complexity level for each FCAT content grade-level test. FCAT Reading Moderate Le Grades Low Level High Level vel 5 50–7 0 5–15 25–3 3 0 20–3 0 50–7 0 10–2 4* 5–7 15–2 5 50–7 0 15–2 5 0 8* 10–2 0 50–7 0 20–3 20–3 0 50–7 0 10–2 9 0 10–2 0 45–6 5 25–3 5 10* FCAT Mathematics Grades Low Level Moderate Level High Level 3–4 25–35 50–70 5–15 20–30 10–20 50–70 5* 10–20 10–20 60–80 6–7 10–20 20–30 50–70 8* 10–20 9 10–20 60–80 10* 20–30 10–20 50–70 FCAT Science Grades Low Level Moderate Level High Level 5* 25–35 15–25 40–60 25–35 15–25 8* 40–60 25–35 40–60 15–25 11* high complexity points due to the nature of *These grades have a greater percentage of performance tasks. FCAT Writing cognitive complexity performance The FCAT Writing prompt is a high task administered at Grades 4, 8, and 10. ___________________________________________________________________________ Cognitive Complexity Classification of FCAT Test Items July 2008 Florida Department of Education © Page 6 of 6

Related documents