pisa 2012 results volume I

Transcript

1 PISA 2012 Results: What Students Know and Can Do tICS, themA StuDent PeRfoRmAnCe In mA ReADIng AnD SCIenCe Volume I rogramme for ssessment A tudent S nternational I P

2

3 PISA 2012 Results: What Students Know and Can Do Student Performance in m athematic S , r eading and Science i ) (Volume r evised edition, f e bruary 2014

4 This work is published on the responsibility of the Secretary - General of the OECD. The opinions expressed and arguments employed herein do not necessarily reflect the official views of the Organisation or of the governments of its member countries. This document and any map included herein are without prejudice to the status of or sovereignty over any territory, to the delimitation of international frontiers and boundaries and to the name of any territory, city or area. Please cite this publication as: OECD (2014), PISA 2012 Results: What Students Know and Can Do – Student Performance in Mathematics, Reading and Science (Volume I, Revised edition, February 2014 ), PISA, OECD Publishing. http://dx.doi.org/10.1787/9789264201118-en ISBN 978-92-64-20877-3 (print) ISBN 978-92-64-20878-0 (PDF) Note by Turkey: The information in this document with reference to “Cyprus” relates to the southern part of the Island. There is no single authority representing both Turkish and Greek Cypriot people on the Island. Turkey recognises the Turkish Republic of Northern Cyprus (TRNC). Until a lasting and equitable solution is found within the context of the United Nations, Turkey shall preserve its position concerning the “Cyprus issue”. Note by all the European Union Member States of the OECD and the European Union: The Republic of Cyprus is recognised by all members of the United Nations with the exception of Turkey. The information in this document relates to the area under the effective control of the Government of the Republic of Cyprus. The statistical data for Israel are supplied by and under the responsibility of the relevant Israeli authorities. The use of such data by the OECD is without prejudice to the status of the Golan Heights, East Jerusalem and Israeli settlements in the West Bank under the terms of international law. Revised edition, February 2014 Photo credits: © Flying Colours Ltd ges /Getty Ima /Kzenon © Jacobs Stock Photography © khoa vu ges /Flic kr /Getty Ima /Corbis © Mel Curtis Shutterstock /Kzenon © © Simon Jarratt /Corbis Corrigenda to OECD publications may be found on line at: www.oecd.org/publishing/corrigenda. © OECD 2014 You can copy, download or print OECD content for your own use, and you can include excerpts from OECD publications, databases and multimedia products in your own documents, presentations, blogs, websites and teaching materials, provided that suitable acknowledgement of OECD as source and copyright owner is given. All requests for public or commercial use and translation rights Requests for permission to photocopy portions of this material for public or commercial use [email protected] should be submitted to shall be addressed directly to the Copyright Clearance Center (CCC) at [email protected] or the Centre français d’exploitation du droit de copie (CFC) at [email protected]

5 Foreword Equipping citizens with the skills necessary to achieve their full potential, participate in an increasingly interconnected global economy, and ultimately convert better jobs into better lives is a central preoccupation of policy makers Results from the OECD’s recent Survey of Adult Skills show that highly skilled adults are twice as likely around the world. to be employed and almost three times more likely to earn an above-median salary than poorly skilled adults. In other words, poor skills severely limit people’s access to better-paying and more rewarding jobs. Highly skilled people are also more likely to volunteer, see themselves as actors rather than as objects of political processes, and are more likely to trust others. Fairness, integrity and inclusiveness in public policy thus all hinge on the skills of citizens. The ongoing economic crisis has only increased the urgency of investing in the acquisition and development of citizens’ skills – both through the education system and in the workplace . At a time when public budgets are tight and there is little room for further monetary and fiscal stimulus, investing in structural reforms to boost productivity, such as education and skills development, is key to future growth. Indeed, investment in these areas is essential to support the recovery, as well as to address long-standing issues such as youth unemployment and gender inequality. In this context, more and more countries are looking beyond their own borders for evidence of the most successful and efficient policies and practices. Indeed, in a global economy, success is no longer measured against national standards alone, but against the best-performing and most rapidly improving education systems. Over the past decade, the OECD Programme for International Student Assessment, PISA, has become the world’s premier yardstick for evaluating the quality, equity and efficiency of school systems. But the evidence base that PISA has produced goes well beyond statistical benchmarking. By identifying the characteristics of high-performing education systems PISA allows governments and educators to identify effective policies that they can then adapt to their local contexts. The results from the PISA 2012 assessment, which was conducted at a time when many of the 65 participating countries and economies were grappling with the effects of the crisis, reveal wide differences in education outcomes, both within and across countries. Using the data collected in previous PISA rounds, we have been able to track the evolution of student performance over time and across subjects. Of the 64 countries and economies with comparable data, 40 improved their average performance in at least one subject. Top performers such as Shanghai in China or Singapore were able to further extend their lead, while countries like Brazil, Mexico, Tunisia and Turkey achieved major improvements from previously low levels of performance. Some education systems have demonstrated that it is possible to secure strong and equitable learning outcomes at the same time as achieving rapid improvements. Of the 13 countries and economies that significantly improved their mathematics performance between 2003 and 2012, three also show improvements in equity in education during the same period, and another nine improved their performance while maintaining an already high level of equity – proving that countries do not have to sacrifice high performance to achieve equity in education opportunities. Nonetheless, PISA 2012 results show wide differences between countries in mathematics performance. The equivalent of almost six years of schooling, 245 score points, separates the highest and lowest average performances C e – Volume i © OECD 2014 3 m e in C o: Student Performan d and Can W Kno S athemati , r eading and S CS C ien What Student

6 Foreword of the countries that took part in the PISA 2012 mathematics assessment. The difference in mathematics performances within countries is even greater, with over 300 points – the equivalent of more than seven years of schooling – often separating the highest- and the lowest-achieving students in a country. Clearly, all countries and economies have excellent students, but few have enabled all students to excel. The report also reveals worrying gender differences in students’ attitudes towards mathematics: even when girls perform as well as boys in mathematics, they report less perseverance, less motivation to learn mathematics, less belief in their own mathematics skills, and higher levels of anxiety about mathematics. While the average girl underperforms in mathematics compared with the average boy, the gender gap in favour of boys is even wider among the highest-achieving students. These findings have serious implications not only for higher education, where young women are already under- represented in the science, technology, engineering and mathematics fields of study, but also later on, when these young women enter the labour market. This confirms the findings of the OECD Gender Strategy, which identifies some of the factors that create – and widen – the gender gap in education, labour and entrepreneurship. Supporting girls’ positive attitudes towards and investment in learning mathematics will go a long way towards narrowing this gap. PISA 2012 also finds that the highest-performing school systems are those that allocate educational resources more equitably among advantaged and disadvantaged schools and that grant more autonomy over curricula and A belief that all students can achieve at a high level and a willingness to engage assessments to individual schools. all stakeholders in education – including students, through such channels as seeking student feedback on teaching practices – are hallmarks of successful school systems. PISA is not only an accurate indicator of students’ abilities to participate fully in society after compulsory school, . There is no single but also a powerful tool that countries and economies can use to fine-tune their education policies combination of policies and practices that will work for everyone, everywhere. Every country has room for improvement, even the top performers. That’s why the OECD produces this triennial report on the state of education across the globe: to share evidence of the best policies and practices and to offer our timely and targeted support to help countries provide the best education possible for all of their students. With high levels of youth unemployment, rising inequality, a significant gender gap, and an urgent need to boost growth in many countries, we have no time to lose. The OECD stands ready to support policy makers in this challenging and crucial endeavour. Angel Gurría OECD Secretary-General CS © i e – Volume C ien C eading and S r , What Student athemati m e in C o: Student Performan d and Can W Kno S OECD 2014 4

7 Acknowledgements This report is the product of a collaborative effort between the countries participating in PISA, the experts and institutions working within the framework of the PISA Consortium, and the OECD Secretariat. The report was drafted by Andreas Schleicher, Francesco Avvisati, Francesca Borgonovi, Miyako Ikeda, Hiromichi Katayama, Flore-Anne Messy, Chiara Monticone, Guillermo Montt, Sophie Vayssettes and Pablo Zoido of the OECD Directorate for Education and Skills and the Directorate for Financial Affairs, with statistical support from Simone Bloem and Giannina Rech and editorial oversight by Marilyn Achiron. Additional analytical and editorial support was provided by Adele Atkinson, Jonas Bertling, Marika Boiron, Célia Braga-Schich, Tracey Burns, Michael Davidson, Cassandra Davis, Elizabeth Del Bourgo, Joachim Funke, Samuel Greiff, Tue Halgreen, Ben Jensen, Eckhard Klieme, André Laboul, Henry Levin, John A. Dossey, Juliette Mendelovits, Tadakazu Miki, Christian Monseur, Simon Normandeau, Mathilde Overduin, Elodie Pools, Dara Ramalingam, William H. Schmidt (whose work was supported by the Thomas J. Alexander fellowship programme), Kaye Stacey, Lazar Stankov, Ross Turner, Elisabeth Villoutreix and Allan Wigfield. The system level data collection was - conducted by the OECD NESLI (INES Network for the Collection and Adjudication of System-Level Descriptive Information on Educational Structures, Policies and Practices) team: Bonifacio Agapin, Estelle Herbaut and Jean Yip. Volume II also draws on the analytic work undertaken by Jaap Scheerens and Douglas Willms in the context of PISA 2000. Administrative support was provided by Claire Chetcuti, Juliet Evans, Jennah Huxley and Diana Tramontano. The OECD contracted the Australian Council for Educational Research (ACER) to manage the development of the mathematics, problem solving and financial literacy frameworks for PISA 2012. Achieve was also contracted by the OECD to develop the mathematics framework with ACER. The expert group that guided the preparation of the mathematics assessment framework and instruments was chaired by Kaye Stacey; Joachim Funke chaired the expert group that guided the preparation of the problem-solving assessment framework and instruments; and Annamaria Lusardi led the expert group that guided the preparation of the financial literacy assessment framework and instruments. The PISA assessment instruments and the data underlying the report were prepared by the PISA Consortium, under the direction of Raymond Adams at ACER. The development of the report was steered by the PISA Governing Board, which is chaired by Lorna Bertrand (United Kingdom), with Ben ő Csapó (Hungary), Daniel McGrath (United States) and Ryo Watanabe (Japan) as vice chairs. Annex C of the volumes lists the members of the various PISA bodies, as well as the individual experts and consultants who have contributed to this report and to PISA in general. CS , r eading and S C ien C e – Volume i © OECD 2014 5 o: Student Performan d and Can W Kno S C m e in athemati What Student

8

9 Table of Contents 17 ... Summary E cutiv E Ex Guid S r’ E ad rE E ... 21 CHAPTER 1 ? t i S P i S a Wha ... 23 2012 survey measure? a S i What does the P ... 25 S Who are the P students? i a ... 26 What is the test like? ... 27 ow is the test conducted? h ... 29 What kinds of results does the test provide? ... 29 Where can you find the results? 29 ... ath E a S CHAPTER 2 Profil E of Stud E nt P E rformanc E in m matic 31 ... context for comparing the mathematics performance of countries and economies a ... 34 approach to assessing student performance in mathematics t he P i a S 37 ... • The PISA definition of mathematical literacy ... 37 • The PISA 2012 framework for assessing mathematics 37 ... • Example 1: WHICH CAR? 41 ... Example 2: CLIMBING MOUNT FUJI • ... 42 • How the PISA 2012 mathematics results are reported ... 44 How mathematics proficiency levels are defined in PISA 2012 • 46 ... Student performance in mathematics ... 46 Average performance in mathematics • 46 ... • Trends in average mathematics performance ... 51 Trends in mathematics performance adjusted for sampling and demographic changes • ... 58 • Students at the different levels of proficiency in mathematics ... 60 • Trends in the percentage of low- and top-performers in mathematics ... 69 • Variation in student performance in mathematics ... 71 Gender differences in mathematics performance • ... 71 • Trends in gender differences in mathematics performance ... 74 Student performance in different areas of mathematics 79 ... Process subscales • ... 79 • Content subscales ... 95 S a mathematics units Examples of P i ... 125 a mE CHAPTER 3 matic S E ath m arn lE to ES ortuniti G urin S oPP 145 ... pportunity to learn and student achievement o ... 150 d ifferences in opportunities to learn ... 156 Questions used for the construction of the three opportunity to learn indices 170 ... he thr t ee opportunity to learn indices ... 172 CS OECD 2014 Kno What Student W and Can d o: Student Performan C e in m athemati S , r eading and S C ien C e – Volume i © 7

10 Table of con s T en T 175 ... G adin rE in E rformanc E nt P E of Stud E Profil a CHAPTER 4 Student performance in reading ... 176 • Average performance in reading 176 ... Trends in average reading performance • ... 181 • Trends in reading performance adjusted for sampling and demographic changes 187 ... Students at the different levels of proficiency in reading • ... 190 • Trends in the percentage of low- and top-performers in reading ... 197 Variation in student performance in reading • ... 199 • Gender differences in reading performance ... 199 Trends in gender difference in reading performance • ... 201 S a reading units i Examples of P ... 203 Profil a E CHAPTER 5 of Stud E nt P E rformanc E in Sci E nc E 215 ... Student performance in science 216 ... Average performance in science • 216 ... Trends in average science performance • ... 218 Trends in science performance adjusted for sampling and demographic changes • ... 229 • Students at the different levels of proficiency in science 230 ... • Trends in the percentage of low- and top-performers in science 235 ... • Variation in student performance in science 239 ... • Gender differences in science performance ... 239 • Trends in gender difference in science performance 241 ... a i science units Examples of P S 242 ... a 2012 S lication P m i Policy CHAPTER 6 E E rformanc nt P E in P i S of Stud ... 251 i mproving average performance ... 252 Pursuing excellence 253 ... kling low performance t ac ... 254 ssessing strengths and weaknesses in different kinds of mathematics a 254 ... Providing equal opportunities for boys and girls ... 255 P i S a 2012 tE ANNEX A G round chnical back ... 257 nnex a a 1 Indices from the student, school and parent context questionnaires ... 258 265 ... he PISA target population, the PISA samples and the definition of schools T 2 a nnex a T a 3 a nnex echnical notes on analyses in this volume ... 277 a nnex a 4 Quality assur ance ... 279 T a a 5 nnex echnical details of trends analyses ... 280 Dev a nnex a 6 elopment of the PISA assessment instruments ... 294 a nnex a 7 T echnical note on Brazil ... 295 d ANNEX B i S a 2012 P a ta ... 297 298 b 1 Results for countries and economies ... nnex a nnex b 2 a Results for regions within countries ... 405 a nnex b 3 Results for the computer -based and combined scales for mathematics and reading ... 491 T a b 4 nnex rends in mathematics, reading and science performance ... 537 of ntation E m E l P im and 555 ... ffort E E collaborativ a – a S i P ANNEX C th E d E v E lo P m E nt o: Student Performan e – Volume © OECD 2014 What Student S Kno W and Can d C C e in m athemati CS , r eading and S C ien i 8

11 Table of con T s T en BOXES Box I.1.1 24 ... hole world can take A test the w 26 ... ey features of PISA 2012 Box I.1.2 K ... 32 y about readiness for further education and a career? What does performance in PISA sa Box I.2.1 ... 52 Measuring trends in PISA Box I.2.2 64 ... op performers and all-rounders in PISA T Box I.2.3 ... 76 Box I.2.4 Impro ving in PISA: Brazil 122 ... Impro ving in PISA: Turkey Box I.2.5 188 ... ving in PISA: Korea Box I.4.1 Impro ... 238 Impro ving in PISA: Estonia Box I.5.1 FIGURES Map of PISA countries and economies 25 Figure I.1.1 ... 28 ... Summary of the assessment areas in PISA 2012 Figure I.1.2 Figure I.2.1 Mathematics performance and Gross Domestic Product ... 35 35 ... Mathematics performance and spending on education Figure I.2.2 ... 35 Mathematics performance and parents’ education Figure I.2.3 35 ... vantaged students Mathematics performance and share of socio-economically disad Figure I.2.4 ... 35 Mathematics performance and proportion of students from an immigr ant background Figure I.2.5 ... 35 Equi valence of the PISA assessment across cultures and languages Figure I.2.6 Main features of the PISA 2012 mathematics framework ... 37 Figure I.2.7 40 Categories describing the items constructed for the PISA 2012 mathematics assessment ... Figure I.2.8 ... Classification of sample items, by process, context and content categories and response type 41 Figure I.2.9 ... ? – a unit from the PISA 2012 main survey AR C HICH W 42 Figure I.2.10 LIMBING 43 M OUNT F C – a unit from the field trial ... UJI Figure I.2.11 The relationship between questions and student performance on a scale 45 ... Figure I.2.12 Comparing countries’ and economies’ performance in mathematics ... 47 Figure I.2.13 48 ... Mathematics performance among PISA 2012 participants, at national and regional lev Figure I.2.14 els ... 52 Annualised change in mathematics performance throughout participation in PISA Figure I.2.15 Curvilinear trajectories of average mathematics performance across PISA assessments ... 55 Figure I.2.16 56 ... Figure I.2.17 Multiple comparisons of mathematics performance between 2003 and 2012 ... 58 Relationship between annualised change in performance and average PISA 2003 mathematics scores Figure I.2.18 Adjusted and observed annualised performance change in average PISA mathematics scores ... 59 Figure I.2.19 60 Map of selected mathematics questions, by proficiency level ... Figure I.2.20 61 ... Summary descriptions for the six levels of proficiency in mathematics Figure I.2.21 Proficiency in mathematics ... 62 Figure I.2.22 Overlapping of top performers in mathematics, reading and science on average across OECD countries ... 64 Figure I.2.a 65 Top performers in mathematics, reading and science ... Figure I.2.b ... Percentage of low-performing students and top performers in mathematics in 2003 and 2012 70 Figure I.2.23 ... 72 Relationship between performance in mathematics and variation in performance Figure I.2.24 ... Gender differences in mathematics performance 73 Figure I.2.25 9 Kno W and Can d o: Student Performan C e in m athemati CS What Student r eading and S C ien C OECD 2014 S e – Volume i © ,

12 Table of con T en T s Figure I.2.26 ... Proficiency in mathematics among boys and girls 74 Figure I.2.27 75 ... Change between 2003 and 2012 in gender differences in mathematics performance Figure I.2.c Observed and expected trends in mathematics performance for Brazil (2003-12) ... 77 Figure I.2.28 80 Comparing countries’ and economies’ performance on the mathematics subscale formulating ... Figure I.2.29 formulating Summary descriptions of the six proficiency levels for the mathematical subscale 81 ... Figure I.2.30 ... 82 formulating Proficiency in the mathematics subscale Figure I.2.31 84 ... employing Comparing countries’ and economies’ performance on the mathematics subscale Figure I.2.32 employing Summary descriptions of the six proficiency levels for the mathematical subscale 85 ... Figure I.2.33 ... Proficiency in the mathematics subscale employing 86 Figure I.2.34 87 Comparing countries’ and economies’ performance on the mathematics subscale interpreting ... Figure I.2.35 Summary descriptions of the six proficiency levels for the mathematical subscale interpreting ... 88 Figure I.2.36 90 ... interpreting Proficiency in the mathematics subscale Figure I.2.37 91 ... Comparing countries and economies on the different mathematics process subscales ank on the different mathematics process subscales Where countries and economies r Figure I.2.38 ... 92 Figure I.2.39a 96 formulating subscale ... Gender differences in performance on the Figure I.2.39b employing Gender differences in performance on the ... 97 subscale Figure I.2.39c subscale interpreting 98 ... Gender differences in performance on the Figure I.2.40 99 change and relationships Comparing countries’ and economies’ performance on the mathematics subscale ... Figure I.2.41 Summary descriptions of the six proficiency levels for the mathematical subscale change and relationships ... 100 Figure I.2.42 101 Proficiency in the mathematics subscale change and relationships ... Figure I.2.43 space and shape Comparing countries’ and economies’ performance on the mathematics subscale 102 ... Figure I.2.44 Summary descriptions of the six proficiency levels for the mathematical subscale space and shape ... 103 Figure I.2.45 Proficiency in the mathematics subscale space and shape ... 104 Figure I.2.46 106 Comparing countries’ and economies’ performance on the mathematics subscale quantity ... Figure I.2.47 quantity Summary descriptions of the six proficiency levels on the mathematical subscale 107 ... Figure I.2.48 ... quantity Proficiency in the mathematics subscale 108 Figure I.2.49 109 ... uncertainty and data Comparing countries’ and economies’ performance on the mathematics subscale Figure I.2.50 uncertainty and data 110 ... Summary descriptions of the six proficiency levels on the mathematical subscale Figure I.2.51 111 ... uncertainty and data Proficiency in the mathematics subscale Figure I.2.52 ... Comparing countries and economies on the different mathematics content subscales 113 Where countries and economies r ank on the different mathematics content subscales Figure I.2.53 ... 114 Figure I.2.54a ... subscale change and relationships Gender differences in performance on the 118 Figure I.2.54b 119 ... subscale space and shape Gender differences in performance on the Figure I.2.54c Gender differences in performance on the quantity subscale ... 120 Figure I.2.54d Gender differences in performance on the 121 ... subscale uncertainty and data Figure I.2.55 ... HELEN THE CYCLIST 125 Figure I.2.56 128 ... CLIMBING MOUNT FUJI Figure I.2.57 131 ... REVOLVING DOOR Figure I.2.58 ? ... 134 W CAR HICH Figure I.2.59 ... HARTS 137 C Figure I.2.60 140 ... GARAGE 147 Students’ exposure to word problems ... Figure I.3.1a Figure I.3.1b Students’ exposure to formal mathematics ... 148 Figure I.3.1c 149 ... Students’ exposure to applied mathematics o: Student Performan e – Volume © OECD 2014 What Student S Kno W and Can d C C e in m athemati CS , r eading and S C ien i 10

13 Table of con T en T s Figure I.3.2 Relationship between mathematics performance and students’ exposure to applied mathematics ... 150 Figure I.3.3 Country-level regressions between opportunity to learn variables and mathematics performance at the student and school levels ... 151 Figure I.3.4a ... Relationship between the index of exposure to word problems and students’ mathematics performance 152 Figure I.3.4b Relationship between the index of exposure to formal mathematics and students’ mathematics performance 153 ... Figure I.3.4c Relationship between the index of exposure to applied mathematics and students’ mathematics performance ... 154 Figure I.3.5 ... Significance of exposure to applied mathematics 155 Figure I.3.6 Percentage of students who reported having seen applied mathematics problems like “calculating the power consumption of an electric appliance per week” frequently or sometimes ... 157 Figure I.3.7 Percentage of students who reported having seen applied mathematics problems like “calculating how many square metres of tiles you need to cover a floor” frequently or sometimes ... 158 Figure I.3.8 Percentage of students who reported having seen formal mathematics problems in their mathematics lessons frequently or sometimes ... 159 Figure I.3.9 ... 160 Percentage of students who reported having seen word problems in their mathematics lessons frequently or sometimes Figure I.3.10 Percentage of students who reported having seen applied problems in mathematics in their mathematics lessons frequently or sometimes 162 ... Figure I.3.11 Percentage of students who reported having seen real-world problems in their mathematics lessons frequently or sometimes ... 163 Figure I.3.12 Student exposure to mathematics problems 164 ... Figure I.3.13 Percentage of students who reported having seen linear equations often or knowing the concept well and understanding it ... 165 Figure I.3.14 Percentage of students who reported having seen complex numbers often or knowing the concept well and understanding it ... 166 Figure I.3.15 Percentage of students who reported having seen exponential functions often or knowing the concept well and understanding it ... 167 Figure I.3.16 Percentage of students who reported having seen quadratic functions often or knowing the concept well and understanding it ... 168 Figure I.3.17 169 ... Exposure to applied mathematics vs. exposure to formal mathematics Figure I.4.1 Comparing countries’ and economies’ performance in reading ... 177 els Reading performance among PISA 2012 participants, at national and regional lev Figure I.4.2 178 ... Figure I.4.3 182 Annualised change in reading performance throughout participation in PISA ... Figure I.4.4 Curvilinear trajectories of average reading performance across PISA assessments 183 ... Multiple comparisons of reading performance between 2000 and 2012 Figure I.4.5 184 ... Figure I.4.6 ... Relationship between annualised change in performance and average PISA 2000 reading scores 186 Figure I.4.7 188 Adjusted and observed annualised performance change in average PISA reading scores ... Figure I.4.8 Summary description for the seven levels of proficiency in print reading in PISA 2012 ... 191 Figure I.4.9 192 ... Map of selected reading questions, by proficiency level Figure I.4.10 194 ... Proficiency in reading Figure I.4.11 198 ... Percentage of low-performing students and top performers in reading in 2000 and 2012 Figure I.4.12 Gender differences in reading performance ... 200 Figure I.4.13 202 ... Change between 2000 and 2012 in gender differences in reading performance Figure I.4.14 203 ... THING THE LAY T ’ HE P S Figure I.4.15 ABOUR 206 ... L Figure I.4.16 ... ALLOON 207 B Figure I.4.17 ISER 211 ... M Figure I.5.1 217 Comparing countries’ and economies’ performance in science ... Figure I.5.2 els Science performance among PISA 2012 participants, at national and regional lev 219 ... Figure I.5.3 ... Annualised change in science performance throughout participation in PISA 222 11 OECD 2014 S Kno W and Can d o: Student Performan C e in m athemati What Student , r eading and S C ien C e – Volume i © CS

14 Table of con T T s en ... 223 Figure I.5.4 Curvilinear trajectories of average science performance across PISA assessments ... 224 Multiple comparisons of science performance between 2006 and 2012 Figure I.5.5 228 Relationship between annualised change in science performance and average PISA 2006 science scores ... Figure I.5.6 Adjusted and observed annualised performance change in average PISA science scores ... 230 Figure I.5.7 Summary description for the six levels of proficiency in science in PISA 2012 231 ... Figure I.5.8 Map of selected science questions, by proficiency level ... 231 Figure I.5.9 232 Proficiency in science ... Figure I.5.10 237 Percentage of low-performing students and top performers in science in 2006 and 2012 ... Figure I.5.11 Gender differences in science performance ... 240 Figure I.5.12 241 ... Change between 2006 and 2012 in gender differences in science performance Figure I.5.13 G ... REENHOUSE 242 Figure I.5.14 C LOTHES ... 245 Figure I.5.15 246 ... M ARY M ONTAGU Figure I.5.16 248 ... G ENETICALLY M ODIFIED CROPS Figure I.5.17 249 P HYSICAL Ex ERCISE ... Figure I.5.18 Annualised change in mathematics performance since PISA 2003 and observed difference in performance Figure A5.1 between PISA 2012 and PISA 2003 ... 286 Figure A5.2 Annualised c hange in reading performance since PISA 2000 and observed difference in performance ... 287 between PISA 2012 and PISA 2000 Annualised change in science performance since PISA 2006 and observed difference in performance Figure A5.3 between PISA 2012 and PISA 2006 ... 287 537 ... Trends in mathematics, reading and science performance: OECD countries Figure B4.1 ... 546 rends in mathematics, reading and science performance: Partner countries and economies Figure B4.2 T TABLES Table A1.1 Lev els of parental education converted into years of schooling ... 260 ... 262 el model to estimate grade effects in mathematics accounting for some background variables A multilev Table A1.2 264 ... Student questionnaire rotation design Table A1.3 267 ... PISA target populations and samples Table A2.1 269 ... Exclusions Table A2.2 271 ... ... rates Table A2.3 Response ... 274 ercentage of students at each grade level P Table A2.4a ... 275 Table A2.4b ercentage of students at each grade level, by gender P ... 281 Link error for comparisons of performance between PISA 2012 and previous assessments Table A5.1 ... 282 Link error for comparisons of proficienc Table A5.2 y levels between PISA 2012 and previous assessments ... 285 hange between PISA 2012 and previous assessments Link error for comparisons of annualised and curvilinear c Table A5.3 ... 289 Table A5.4 ve statistics for variables used to adjust mathematics, reading and science scores to the PISA 2012 samples Descripti P Table A7.1 ercentage of Brazilian students at each proficiency level on the mathematics scale 295 ... and mathematics subscales ... 295 P Table A7.2 ercentage of Brazilian students at each proficiency level on the reading scale ... 296 Table A7.3 ercentage of Brazilian students at each proficiency level on the science scale P 296 ... T Table A7.4 op performers in mathematics, reading and science in Brazil 296 ... Mean score, v ariation and gender differences in student performance in Brazil Table A7.5 Kno eading and S , CS athemati m C C o: Student Performan d and Can W r S What Student OECD 2014 ien C e – Volume i © e in 12

15 Table of con T en T s P ercentage of students at each proficiency level in mathematics Table I.2.1a 298 ... P ercentage of students below Level 2 and at Level 5 or above in mathematics in PISA 2003 through 2012 Table I.2.1b 299 ... P Table I.2.2a ercentage of students at each proficiency level in mathematics, by gender 301 ... ercentage of students below Level 2 and at Level 5 or above in mathematics in PISA 2003 and 2012, by gender P Table I.2.2b ... 303 Mean score, v Table I.2.3a ariation and gender differences in student performance in mathematics ... 305 Mean mathematics performance in PISA 2003 through 2012 Table I.2.3b 306 ... Table I.2.3c Gender differences in mathematics performance in PISA 2003 and 2012 307 ... Table I.2.3d Distribution of scores in mathematics in PISA 2003 through 2012, b y percentiles 308 ... Table I.2.4 rends in mathematics performance adjusted for demographic changes T ... 311 Table I.2.5 formulating ercentage of students at each proficiency level on the mathematics subscale P 312 ... P , by gender formulating ercentage of students at each proficiency level on the mathematics subscale Table I.2.6 ... 313 formulating Table I.2.7 Mean score, v ariation and gender differences in student performance on the mathematics subscale 315 ... Table I.2.8 employing ercentage of students at each proficiency level on the mathematics subscale P ... 316 P , by gender employing ercentage of students at each proficiency level on the mathematics subscale Table I.2.9 ... 317 Table I.2.10 ariation and gender differences in student performance on the mathematics subscale employing Mean score, v ... 319 Table I.2.11 ercentage of students at each proficiency level on the mathematics subscale P interpreting ... 320 ercentage of students at each proficiency level on the mathematics subscale , by gender interpreting Table I.2.12 P 321 ... interpreting Table I.2.13 ariation and gender differences in student performance on the mathematics subscale Mean score, v ... 323 P Table I.2.14 change and relationships ercentage of students at each proficiency level on the mathematics subscale ... 324 , by gender P ercentage of students at each proficiency level on the mathematics subscale change and relationships Table I.2.15 ... 325 Table I.2.16 ariation and gender differences in student performance on the mathematics subscale Mean score, v change and relationships 327 ... ... P space and shape ercentage of students at each proficiency level on the mathematics subscale Table I.2.17 ... 328 , by gender Table I.2.18 ercentage of students at each proficiency level on the mathematics subscale space and shape P 329 ... ariation and gender differences in student performance on the mathematics subscale ... space and shape Table I.2.19 Mean score, v 331 quantity ercentage of students at each proficiency level on the mathematics subscale P Table I.2.20 332 ... Table I.2.21 quantity ercentage of students at each proficiency level on the mathematics subscale P , by gender 333 ... Table I.2.22 quantity ariation and gender differences in student performance on the mathematics subscale Mean score, v ... 335 ercentage of students at each proficiency level on the mathematics subscale uncertainty and data Table I.2.23 P ... 336 P uncertainty and data ercentage of students at each proficiency level on the mathematics subscale , by gender Table I.2.24 ... 337 Table I.2.25 ariation and gender differences in student performance on the mathematics subscale Mean score, v uncertainty and data 339 ... Gender differences in performance in mathematics after taking student progr Table I.2.26 ammes into account 340 ... Socio-economic indicators and the relationship with performance in mathematics Table I.2.27 341 ... Country r ankings on preferred questions Table I.2.28 ... 343 Table I.2.29 T op performers in mathematics, reading and science 344 ... T Table I.2.30 op performers in mathematics, reading and science, by gender 345 ... 347 ... Table I.3.1 ariables Index of opportunity to learn v Estimated regression coefficients for student and sc hool opportunity to learn variables related to achievement Table I.3.2 ... 348 Students’ exposure to the mathematics task “using a tr ain timetable” Table I.3.3 ... 349 Students’ exposure to the mathematics task “calculating ho w much more expensive a computer Table I.3.4 350 would be after adding tax” ... Table I.3.5 w many square metres of tiles you need to cover a floor” Students’ exposure to the mathematics task “calculating ho ... 351 Table I.3.6 Students’ exposure to the mathematics task “understanding scientific tables presented in an article” ... 352 2 Table I.3.7 Students’ exposure to the mathematics task “solving an equation like 6x ... + 5 = 29” 353 o places on a map Table I.3.8 Students’ exposure to the mathematics task “finding the actual distance between tw with a 1:10,000 scale” ... 354 ... 13 OECD 2014 S Kno W and Can d o: Student Performan C e in m athemati What Student , r eading and S C ien C e – Volume i © CS

16 Table of con s T en T 355 ... Students’ exposure to the mathematics task “solving an equation like 2(x+3) = (x+3)(x-3)” Table I.3.9 Table I.3.10 wer consumption of an electronic appliance per week” Students’ exposure to the mathematics task “calculating the po ... 356 e equation; find volume” Students’ exposure to the mathematics problem “solv Table I.3.11 ... 357 Students’ exposure to the mathematics problem “w Table I.3.12 ord problems” ... 358 Table I.3.13 Students’ exposure to the mathematics problem “geometrical theorems; prime number” 359 ... Students’ exposure to mathematics problem requiring a real-life context (data) Table I.3.14 360 ... Students’ exposure to the mathematics concept “exponential function” Table I.3.15 361 ... visor” Students’ exposure to the mathematics concept “di Table I.3.16 ... 362 atic function” Students’ exposure to the mathematics concept “quadr Table I.3.17 ... 363 Students’ exposure to the mathematics concept “linear equation” Table I.3.18 ... 364 Students’ exposure to the mathematics concept “v Table I.3.19 ectors” ... 365 Table I.3.20 Students’ exposure to the mathematics concept “complex number” 366 ... Table I.3.21 ational number” Students’ exposure to the mathematics concept “r ... 367 Table I.3.22 Students’ exposure to the mathematics concept “r adicals” ... 368 Table I.3.23 Students’ exposure to the mathematics concept “polygon” 369 ... Students’ exposure to the mathematics concept “congruent figure” Table I.3.24 370 ... Students’ exposure to the mathematics concept “cosine” Table I.3.25 371 ... Table I.3.26 Students’ exposure to the mathematics concept “arithmetic mean” ... 372 Table I.3.27 Students’ exposure to the mathematics concept “probability” ... 373 amiliarity with mathematics topics Table I.3.28 F 374 ... ... P ercentage of students at each proficiency level in reading Table I.4.1a ... 375 Table I.4.1b ercentage of students below Level 2 and at Level 5 or above in reading in PISA 2000 through 2012 P 376 ... P Table I.4.2a ercentage of students at each proficiency level in reading, by gender ... 378 ercentage of students below Level 2 and at Level 5 or above in reading in PISA 2000 and 2012, by gender Table I.4.2b P ... 380 ariation and gender differences in student performance in reading Mean score, v Table I.4.3a 382 ... Mean reading performance in PISA 2000 through 2012 Table I.4.3b ... 383 Gender differences in reading performance in PISA 2000 and 2012 Table I.4.3c 385 ... Distribution of scores in reading in PISA 2000 through 2012, b Table I.4.3d y percentiles 386 ... T rends in reading performance adjusted for demographic changes Table I.4.4 ... 390 P ercentage of students at each proficiency level in science Table I.5.1a 392 ... P Table I.5.1b ercentage of students below Level 2 and at Level 5 or above in science in PISA 2006 through 2012 ... 393 Table I.5.2a ercentage of students at each proficiency level in science, by gender P ... 394 Table I.5.2b P ercentage of students below Level 2 and at Level 5 or above in science in PISA 2006 and 2012, by gender 396 ... ariation and gender differences in student performance in science Table I.5.3a Mean score, v ... 398 Mean science performance in PISA 2006 through 2012 Table I.5.3b ... 399 Table I.5.3c Gender differences in science performance in PISA 2006 and 2012 ... 400 Table I.5.3d y percentiles Distribution of scores in science in PISA 2006 through 2012, b 401 ... T rends in science performance adjusted for demographic changes Table I.5.4 ... 404 ercentage of students at each proficiency level in mathematics, by region Table B2.I.1 P 405 ... ercentage of students at each proficiency level in mathematics, by gender and region P Table B2.I.2 ... 407 Table B2.I.3 ariation and gender differences in student performance in mathematics, by region Mean score, v 411 ... ercentage of students at each proficiency level on the mathematics subscale P Table B2.I.4 , by region formulating ... 413 Table B2.I.5 P ercentage of students at each proficiency level on the mathematics subscale formulating , by gender and region 415 ... Mean score, v formulating Table B2.I.6 , ariation and gender differences in student performance on the mathematics subscale by region ... 419 o: Student Performan e – Volume © OECD 2014 What Student S Kno W and Can d C C e in m athemati CS , r eading and S C ien i 14

17 Table of con T T en s 421 , by region employing ercentage of students at each proficiency level on the mathematics subscale P Table B2.I.7 ... 423 ... Table B2.I.8 P ercentage of students at each proficiency level on the mathematics subscale employing , by gender and region ... 427 employing , by region ariation and gender differences in student performance on the mathematics subscale Table B2.I.9 Mean score, v 429 ... Table B2.I.10 P ercentage of students at each proficiency level on the mathematics subscale interpreting , by region ... 431 Table B2.I.11 , by gender and region interpreting ercentage of students at each proficiency level on the mathematics subscale P ariation and gender differences in student performance on the mathematics subscale , Table B2.I.12 interpreting Mean score, v 435 ... by region 437 ... change and relationships ercentage of students at each proficiency level on the mathematics subscale P Table B2.I.13 , by region Table B2.I.14 ercentage of students at each proficiency level on the mathematics subscale change and relationships , P 439 ... by gender and region Table B2.I.15 ariation and gender differences in student performance on the mathematics subscale Mean score, v ... 443 change and relationships , by region 445 ... , by region space and shape ercentage of students at each proficiency level on the mathematics subscale Table B2.I.16 P ... 447 Table B2.I.17 , by gender and region space and shape ercentage of students at each proficiency level on the mathematics subscale P , Table B2.I.18 ariation and gender differences in student performance on the mathematics subscale space and shape Mean score, v ... 451 by region 453 ... , by region quantity ercentage of students at each proficiency level on the mathematics subscale P Table B2.I.19 455 ... P Table B2.I.20 quantity , by gender and region ercentage of students at each proficiency level on the mathematics subscale 459 ... Mean score, v ariation and gender differences in student performance on the mathematics subscale Table B2.I.21 quantity , by region 461 ... , by region uncertainty and data P Table B2.I.22 ercentage of students at each proficiency level on the mathematics subscale uncertainty and data ercentage of students at each proficiency level on the mathematics subscale P Table B2.I.23 , 463 ... by gender and region Mean score, v Table B2.I.24 ariation and gender differences in student performance on the mathematics subscale 467 ... uncertainty and data , by region ... 469 Table B2.I.25 P ercentage of students at each proficiency level in reading, by region 471 ... P Table B2.I.26 ercentage of students at each proficiency level in reading, by gender and region ... 475 Table B2.I.27 Mean score, v ariation and gender differences in student performance in reading, by region ... 477 ercentage of students at each proficiency level in science, by region P Table B2.I.28 ... 479 ercentage of students at each proficiency level in science, by gender and region P Table B2.I.29 ... 483 ariation and gender differences in student performance in science, by region Mean score, v Table B2.I.30 ... 485 T op performers in mathematics, reading and science, by region Table B2.I.31 487 ... T op performers in mathematics, reading and science, by gender and region Table B2.I.32 ... 493 Table B3.I.1 P ercentage of students at each proficiency level on the computer-based mathematics scale ... 494 Table B3.I.2 ercentage of students at each proficiency level on the computer-based mathematics scale, by gender P 495 ... ariation and gender differences in student performance on the computer-based mathematics scale Table B3.I.3 Mean score, v ... 496 Table B3.I.4 P ercentage of students at each proficiency level on the combined mathematics scale 497 ... ercentage of students at each proficiency level on the combined mathematics scale, by gender Table B3.I.5 P ... 498 ariation and gender differences in student performance on the combined mathematics scale Table B3.I.6 Mean score, v 499 ... Table B3.I.7 ercentage of students at each proficiency level on the digital reading scale P 500 ... ercentage of students at each proficiency level on the digital reading scale, by gender P Table B3.I.8 501 ... Table B3.I.9 Mean score, v ariation and gender differences in student performance on the digital reading scale ... 502 Table B3.I.10 P ercentage of students at each proficiency level on the combined reading scale ... 503 ercentage of students at each proficiency level on the combined reading scale, by gender P Table B3.I.11 ... 504 Mean score, v Table B3.I.12 ariation and gender differences in student performance on the combined reading scale ... 505 ercentage of students at each proficiency level on the computer-based mathematics scale, by region P Table B3.I.13 ercentage of students at each proficiency level on the computer-based mathematics scale, Table B3.I.14 P ... 507 by gender and region 15 S Kno W and Can d o: Student Performan C e in m athemati What Student , r eading and S C ien C e – Volume OECD 2014 i © CS

18 Table of con en s T T ariation and gender differences in student performance on the computer-based mathematics scale, Mean score, v Table B3.I.15 ... 511 by region 513 ... ercentage of students at each proficiency level on the combined mathematics scale, by region P Table B3.I.16 515 ... P Table B3.I.17 ercentage of students at each proficiency level on the combined mathematics scale, by gender and region ... 519 ariation and gender differences in student performance on the combined mathematics scale, by region Table B3.I.18 Mean score, v ... 521 P Table B3.I.19 ercentage of students at each proficiency level on the digital reading scale, by region 523 ... Table B3.I.20 ercentage of students at each proficiency level on the digital reading scale, by gender and region P ... 527 Table B3.I.21 ariation and gender differences in student performance on the digital reading scale, by region Mean score, v 529 ... Table B3.I.22 P ercentage of students at each proficiency level on the combined reading scale, by region 531 ... ercentage of students at each proficiency level on the combined reading scale, by gender and region P Table B3.I.23 535 ... Table B3.I.24 ariation and gender differences in student performance on the combined reading scale, by region Mean score, v This book has... StatLinks 2 ® A service that delivers Excel files from the printed page! at the bottom left-hand corner of the tables or graphs in this book. Look for the StatLinks ® To download the matching Excel spreadsheet, just type the link into your Internet browser, prefix. http://dx.doi.org starting with the If you’re reading the PDF e-book edition, and your PC is connected to the Internet, simply appearing in more OECD books. StatLinks click on the link. You’ll find o: Student Performan e – Volume © OECD 2014 What Student S Kno W and Can d C C e in m athemati CS , r eading and S C ien i 16

19 Executive Summary Nearly all adults, not just those with technical or scientific careers, now need to have adequate proficiency in mathematics – as well as reading and science – for personal fulfilment, employment and full participation in society. With mathematics as its primary focus, the PISA 2012 assessment measured 15-year-olds’ capacity to reason mathematically and use mathematical concepts, procedures, facts and tools to describe, explain and predict phenomena, and to make the well- founded judgements and decisions needed by constructive, engaged and reflective citizens. Literacy in mathematics defined this way is not an attribute that an individual has or does not have; rather, it is a skill that can be acquired and used, to a greater or lesser extent, throughout a lifetime. Shanghai-China has the highest scores in mathematics, with a mean score of 613 points – 119 points above the OECD average, or the equivalent of nearly 3 years of schooling. Singapore, Hong Kong-China, Chinese Taipei, Korea, Macao-China, Japan, Liechtenstein, Switzerland and the Netherlands, in descending order of their scores, round out the top 10 performers in mathematics. Of all countries and economies with trend data between 2003 and 2012, 25 improved in mathematics performance, 25 show no change, and 14 deteriorated. Among countries that participated in every assessment since 2003, Brazil, Italy, Mexico, Poland, Portugal, Tunisia and Turkey show an average improvement in mathematics performance of more than 2.5 points per year since 2003. Although countries and economies that improved the most are more likely to be those that had lower performance in 2003, some with average or high performance in 2003 – such as Germany, Hong Kong-China and Macao-China – also improved during this period. Shanghai-China and Singapore, which began their participation in PISA after the 2003 assessment, also improved their already-high performance. On average across OECD countries, 12.6% of students are top performers in mathematics, meaning that they are proficient at Level 5 or 6. The partner economy Shanghai-China has the largest proportion of students performing at Level 5 or 6 (55.4%), followed by Singapore (40.0%), Chinese Taipei (37.2%) and Hong Kong-China (33.7 %). In Korea, 30.9% of students are top performers in mathematics; and between 15% and 25% of students in Belgium, Canada, Finland, Germany, Japan, Liechtenstein, Macao-China, the Netherlands, New Zealand, Poland and Switzerland are top performers in mathematics. Between 2003 and 2012 Italy, Poland and Portugal increased the share of top performers and simultaneously reduced the share of low performers in mathematics. Israel, Qatar and Romania saw similar improvements between 2006 and 2012 as did Ireland, Malaysia and the Russian Federation between 2009 and 2012. Boys perform better than girls in mathematics in only 38 out of the 65 countries and economies that participated in PISA 2012, and girls outperform boys in 5 countries. In only six countries is the gender gap in mathematics scores larger than the equivalent of half a year of formal schooling. , S OECD 2014 © i e – Volume C ien C eading and S r 17 CS athemati m e in C o: Student Performan d and Can W Kno What Student

20 Ex E Summary E cutiv Shanghai-China, Hong Kong-China, Singapore, Japan and Korea are the five highest-performing countries and economies in reading. Shanghai-China had a mean score of 570 points in reading – the equivalent of more than a year-and-a-half of schooling above the OECD average of 496 score points, and 25 score points above the second best-performing participant, Hong Kong-China. Of the 64 countries and economies with comparable data in reading performance throughout their participation in PISA, 32 improved their reading performance, 22 show no change, and 10 deteriorated in reading performance. Among OECD countries, Chile, Estonia, Germany, Hungary, Israel, Japan, Korea, Luxembourg, Mexico, Poland, Portugal, Switzerland and Turkey improved their reading performance across successive PISA assessments. Across OECD countries, 8.4% of students are top performers in reading, meaning that they are proficient at Level 5 or 6. Shanghai-China has the largest proportion of top performers – 25.1% – among all participating countries and economies. More than 15% of students in Hong Kong-China, Japan and Singapore are top performers in reading, as are more than 10% of students in Australia, Belgium, Canada, Finland, France, Ireland, Korea, Liechtenstein, New Zealand, Norway, Poland and Chinese Taipei. Between the 2000 and 2012 PISA assessments, Albania, Israel and Poland increased the share of top performers and simultaneously reduced the share of low performers in reading. The same trend was observed in Hong Kong-China, Japan and the Russian Federation since PISA 2003; in Bulgaria, Qatar, Serbia, Spain and Chinese Taipei since PISA 2006; and in Ireland, Luxembourg, Macao-China and Singapore since PISA 2009. Between 2000 and 2012 the gender gap in reading performance – favouring girls – widened in 11 countries and economies. In Bulgaria, France and Romania, the gender gap in reading performance widened by more than 15 score points during that period. Only in Albania did the gap narrow as a result of a greater improvement in reading performance among boys than among girls. Shanghai-China, Hong Kong-China, Singapore, Japan and Finland are the top five performers in science in PISA 2012. Shanghai-China’s mean score in science (580 points) is more than three-quarters of a proficiency level above the OECD average of 501 score points. Estonia, Korea, Viet Nam, Poland, Canada, Liechtenstein, Germany, Chinese Taipei, the Netherlands, Ireland, Australia, Macao-China, New Zealand, Switzerland, Slovenia, the United Kingdom and the Czech Republic also score above the OECD average in science, while Austria, Belgium, Latvia, France, Denmark and the United States scored around the OECD average. Across OECD countries, 8.4% of students are top performers in science and score at proficiency Level 5 or 6. More than 15% of students in Shanghai-China (27.2%), Singapore (22.7%), Japan (18.2%), Finland (17.1%) and Hong Kong China (16.7%) are top performers. - Between 2006 and 2012, Italy, Poland and Qatar, and between 2009 and 2012, Estonia, Israel and Singapore increased the share of top performers and simultaneously reduced the share of low performers in science. Brazil, Hong Kong-China, Ireland, Japan, Korea, Latvia, Lithuania, Portugal, Romania, Spain, Switzerland, Thailand, Tunisia, Turkey and the United States saw a significant reduction in the share of students performing below proficiency Level 2 between 2006 and 2012. Boys and girls perform similarly in science and, on average, that remained true in 2012. However, in Finland, Montenegro, the Russian Federation and Sweden, while there was no gender gap in science performance in 2006, a gender gap in favour of girls was observed in 2012. ien and Can © OECD 2014 What Student S Kno i e – Volume C d C eading and S r , CS athemati m e in C o: Student Performan W 18

21 Ex cutiv Summary E E • • Table I.A , reading and S hot of performance in mathematic S Snap S cience Countries/economies with a mean performance/share of top performers above the OECD average Countries/economies with a share of low achievers below the OECD average Countries/economies with a mean performance/share of low achievers/share of top performers not statistically significantly different from the OECD average Countries/economies with a mean performance/share of top performers below the OECD average Countries/economies with a share of low achievers above the OECD average eading m athematics r Science Share of top performers Share m ean score nnualised a a nnualised a m nnualised ean score in mathematics of low achievers ean score m 2012 a S a S change i l evel 5 or 6) change 2012 change i in P in P ( l b evel 2) elow ( 2012 S i a in P OECD average 494 23.0 12.6 -0.3 496 0.3 501 0.5 Shanghai-China 613 3.8 55.4 4.2 570 4.6 580 1.8 3.3 551 5.4 542 3.8 40.0 Singapore 573 8.3 Hong Kong-China 8.5 33.7 1.3 545 2.3 555 2.1 561 523 560 12.8 37.2 1.7 523 4.5 -1.5 Chinese Taipei Korea 554 9.1 30.9 1.1 536 0.9 538 2.6 Macao-China 538 10.8 24.3 1.0 509 0.8 521 1.6 538 0.4 23.7 2.6 547 1.5 Japan 536 11.1 0.3 24.8 14.1 535 Liechtenstein 0.4 525 1.3 516 Switzerland 21.4 0.6 509 1.0 515 0.6 12.4 531 523 14.8 19.3 -1.6 511 -0.1 522 -0.5 Netherlands 521 10.5 14.6 0.9 516 2.4 541 1.5 Estonia Finland 519 12.3 15.3 -2.8 524 -1.7 545 -3.0 Canada 13.8 16.4 -1.4 523 -0.9 525 -1.5 518 4.6 518 14.4 16.7 2.6 518 2.8 526 Poland -0.9 Belgium 515 19.0 19.5 -1.6 509 0.1 505 508 524 1.4 1.8 1.4 17.5 17.7 514 Germany Viet Nam 511 13.3 m 508 m 528 m 14.2 Austria 18.7 14.3 0.0 490 -0.2 506 -0.8 506 Australia 504 19.7 14.8 -2.2 512 -1.4 521 -0.9 Ireland 501 16.9 10.7 -0.6 523 522 2.3 -0.9 Slovenia 501 20.1 13.7 -0.6 481 -2.2 514 -0.8 Denmark 16.8 10.0 -1.8 496 0.1 498 0.4 500 New Zealand 500 22.6 15.0 -2.5 512 -1.1 516 -2.5 Czech Republic 499 21.0 12.9 -2.5 493 -0.5 508 -1.0 22.4 -1.5 505 12.9 0.6 0.0 499 France 495 514 -0.1 494 United Kingdom -0.3 11.8 499 0.7 21.8 -2.0 478 -1.3 483 -2.2 11.2 21.5 493 Iceland 0.5 8.0 19.9 491 Latvia 2.0 502 1.9 489 Luxembourg 490 11.2 -0.3 488 0.7 491 0.9 24.3 489 504 9.4 -0.3 Norway 0.1 495 1.3 22.3 Portugal 24.9 10.6 2.8 488 1.6 489 2.5 487 Italy 485 24.7 9.9 2.7 490 0.5 494 3.0 Spain 23.6 8.0 0.1 488 -0.3 496 1.3 484 Russian Federation 482 24.0 7.8 1.1 475 1.1 486 1.0 Slovak Republic 482 27.5 11.0 -1.4 463 -0.1 471 -2.7 8.8 United States 481 25.8 0.3 498 -0.3 497 1.4 Lithuania 479 8.1 -1.4 477 1.1 496 1.3 26.0 478 483 8.0 -3.3 Sweden -2.8 485 -3.1 27.1 Hungary 28.1 9.3 -1.3 488 1.0 494 -1.6 477 Croatia 471 29.9 7.0 0.6 485 1.2 491 -0.3 33.5 9.4 4.2 486 3.7 470 2.8 466 Israel -1.1 eece 453 35.7 3.9 1.1 477 0.5 467 Gr 7.6 1.5 Serbia 449 38.9 4.6 2.2 446 445 463 475 3.2 5.9 42.0 448 Turkey 4.1 6.4 3.4 Romania 445 40.8 3.2 438 1.1 439 4.9 449 m 3.7 42.0 440 * Cyprus m m 438 Bulgaria 43.8 4.1 4.2 436 0.4 446 2.0 439 United Arab Emirates 434 46.3 3.5 m 442 m 448 m Kazakhstan 432 45.2 0.9 9.0 393 0.8 425 8.1 Thailand 427 49.7 2.6 1.0 441 1.1 444 3.9 Chile 51.5 1.6 1.9 441 3.1 445 1.1 423 -1.4 421 51.8 1.3 8.1 398 -7.8 420 Malaysia 0.9 1.1 415 424 3.1 Mexico 413 54.7 0.6 Montenegro 1.7 -0.3 410 5.0 422 1.0 56.6 410 55.8 1.4 -1.4 411 -1.8 416 -2.1 409 Uruguay Costa Rica 407 59.9 0.6 -1.2 441 -1.0 429 -0.6 397 Albania 394 60.7 0.8 5.6 394 4.1 2.2 2.3 405 1.2 410 4.1 Brazil 391 67.1 0.8 2.4 388 66.5 0.3 1.2 396 -1.6 406 Argentina 2.2 388 67.7 0.8 3.1 404 3.8 398 Tunisia 0.2 -2.1 399 -0.3 409 Jordan 386 68.6 0.6 0.3 3.0 73.8 376 Colombia 403 399 1.8 1.1 384 12.0 388 9.2 2.0 69.6 376 Qatar 5.4 0.3 Indonesia 0.7 2.3 382 -1.9 375 396 75.7 384 Peru 368 74.6 1.3 373 5.2 0.6 1.0 Note: Countries/economies in which the annualised change in performance is statistically significant are marked in bold. * See notes in the Reader’s Guide. Countries and economies are ranked in descending order of the mathematics mean score in PISA 2012. OECD, PISA 2012 Database, Tables I.2.1a, I.2.1b, I.2.3a, I.2.3b, I.4.3a, I.4.3b, I.5.3a and I.5.3b. Source: 2 http://dx.doi.org/10.1787/888932937035 1 , Kno W and Can d o: Student Performan C e in m athemati CS What Student r eading and S C ien C e – Volume i OECD 2014 S © 19

22

23 Reader’s Guide Data underlying the figures The data referred to in this volume are presented in Annex B and, in greater detail, including some additional tables, on the PISA website ( www.pisa.oecd.org ) . Four symbols are used to denote missing data: a T he category does not apply in the country concerned. Data are therefore missing. T c here are too few observations or no observation to provide reliable estimates (i.e. there are fewer than 30 students or fewer than 5 schools with valid data). vailable. These data were not submitted by the country or were collected but subsequently Data are not a m removed from the publication for technical reasons. w Data ha ve been withdrawn or have not been collected at the request of the country concerned. Country coverage This publication features data on 65 countries and economies, including all 34 OECD countries and 31 partner countries and economies (see Figure I.1.1). The statistical data for Israel are supplied by and under the responsibility of the relevant Israeli authorities. The use of such data by the OECD is without prejudice to the status of the Golan Heights, East Jerusalem and Israeli settlements in the West Bank under the terms of international law. Two notes were added to the statistical data related to Cyprus: 1. Note by Turkey: The information in this document with reference to “Cyprus” relates to the southern part of the Island. There is no single authority representing both Turkish and Greek Cypriot people on the Island. Turkey recognises the Turkish Republic of Northern Cyprus (TRNC). Until a lasting and equitable solution is found within the context of the United Nations, Turkey shall preserve its position concerning the “Cyprus issue”. 2. Note by all the European Union Member States of the OECD and the European Union: The Republic of Cyprus is recognised by all members of the United Nations with the exception of Turkey. The information in this document relates to the area under the effective control of the Government of the Republic of Cyprus. Calculating international averages An OECD average was calculated for most indicators presented in this report. In the case of some indicators, a total representing the OECD area as a whole was also calculated: • Th e OECD average corresponds to the arithmetic mean of the respective country estimates. The OECD t • otal takes the OECD countries as a single entity, to which each country contributes in proportion to the number of 15-year-olds enrolled in its schools (see Annex B for data). It illustrates how a country compares with the OECD area as a whole. In this publication, the OECD total is generally used when references are made to the overall situation in the OECD area. Where the focus is on comparing performance across education systems, the OECD average is used. In the case of some countries, data may not be available for specific indicators, or specific categories may not apply. Readers should, therefore, keep in mind that the terms “OECD average” and “OECD total” refer to the OECD countries included in the respective comparisons. Rounding figures Because of rounding, some figures in tables may not exactly add up to the totals. Totals, differences and averages are always calculated on the basis of exact numbers and are rounded only after calculation. , S OECD 2014 © i e – Volume C ien C eading and S r 21 CS athemati m e in C o: Student Performan d and Can W Kno What Student

24 Reade R ’s Guide All standard errors in this publication have been rounded to one or two decimal places. Where the value 0.0 or 0.00 is shown, this does not imply that the standard error is zero, but that it is smaller than 0.05 or 0.005, respectively. Reporting student data The report uses “15-year-olds” as shorthand for the PISA target population. PISA covers students who are aged between 15 years 3 months and 16 years 2 months at the time of assessment and who are enrolled in school and have completed at least 6 years of formal schooling, regardless of the type of institution in which they are enrolled and of whether they are in full-time or part-time education, of whether they attend academic or vocational programmes, and of whether they attend public or private schools or foreign schools within the country. Reporting school data The principals of the schools in which students were assessed provided information on their schools’ characteristics by completing a school questionnaire. Where responses from school principals are presented in this publication, they are weighted so that they are proportionate to the number of 15-year-olds enrolled in the school. Focusing on statistically significant differences This volume discusses only statistically significant differences or changes. These are denoted in darker colours in figures and in bold font in tables. See Annex A3 for further information. Categorising student performance This report uses a shorthand to describe students’ levels of proficiency in the subjects assessed by PISA: t are those students proficient at Level 5 or 6 of the assessment. op performers Strong performers are those students proficient at Level 4 of the assessment. m are those students proficient at Level 2 or 3 of the assessment. oderate performers are those students proficient at or below Level 1 of the assessment. owest performers l are those students who perform at or above the 90th percentile in their own country/economy. ighest achievers h h who perform at or above the 75th percentile in their own country/economy. are those students igh achievers l ow achievers are those students who perform below the 25th percentile in their own country/economy. l owest achievers are those students who perform below the 10th percentile in their own country/economy. Abbreviations used in this report ESCS PISA index of economic, social and cultural status PPP Purchasing power parity Gross domestic product S.D. Standard deviation GDP ISCED International Standard Classification of Education S.E. Standard error STEM Science, Technology, Engineering International Standard Classification ISCO of Occupations and Mathematics Further documentation For further information on the PISA assessment instruments and the methods used in PISA, see the PISA 2012 Technical Report (OECD, forthcoming). This report uses the OECD StatLinks service. Below each table and chart is a url leading to a corresponding TM Excel workbook containing the underlying data. These urls are stable and will remain unchanged over time. In addition, readers of the e-books will be able to click directly on these links and the workbook will open in a separate window, if their internet browser is open and running. o: Student Performan i © OECD 2014 What Student S Kno W and Can d C C e in m athemati CS , r eading and S C ien e – Volume 22

25 1 What is PISA? The Programme for International Student Assessment (PISA) reviews the extent to which students near the end of compulsory education have acquired some of the knowledge and skills that are essential for full participation in modern society, particularly in mathematics, reading and science. This section offers an overview of the Programme, including which countries and economies participate and which students are assessed, what types of skills are measured, and how PISA 2012 differs from previous PISA assessments. m CS , r eading and S C What Student C e – Volume i © athemati OECD 2014 23 e in C o: Student Performan d and Can W Kno S ien

26 1 ? What is P isa “What is important for citizens to know and be able to do?” That is the question that underlies the triennial survey of 15-year-old students around the world known as the Programme for International Student Assessment (PISA). PISA assesses the extent to which students near the end of compulsory education have acquired key knowledge and skills that are essential for full participation in modern societies. The assessment, which focuses on reading, mathematics, science and problem solving, does not just ascertain whether students can reproduce knowledge; it also examines how well students can extrapolate from what they have learned and apply that knowledge in unfamiliar settings, both in and outside of school. This approach reflects the fact that modern economies reward individuals not for what they know, but for what they can do with what they know. PISA is an ongoing programme that offers insights for education policy and practice, and that helps monitor trends in students’ acquisition of knowledge and skills across countries and in different demographic subgroups within each country. PISA results reveal what is possible in education by showing what students in the highest-performing and most rapidly improving education systems can do. The findings allow policy makers around the world to gauge the knowledge and skills of students in their own countries in comparison with those in other countries, set policy targets against measurable goals achieved by other education systems, and learn from policies and practices applied elsewhere. While PISA cannot identify cause-and-effect relationships between policies/practices and student outcomes, it can show educators, policy makers and the interested public how education systems are similar and different – and what that means for students. PISA’s unique features include its: policy or • , which links data on student learning outcomes with data on students’ backgrounds and attitudes ientation towards learning and on key factors that shape their learning, in and outside of school, in order to highlight differences in performance and identify the characteristics of students, schools and education systems that perform well; • inno vative concept of “literacy” , which refers to students’ capacity to apply knowledge and skills in key subjects, and to analyse, reason and communicate effectively as they identify, interpret and solve problems in a variety of situations; • r , as PISA asks students to report on their motivation to learn, their beliefs about elevance to lifelong learning themselves, and their learning strategies; • r egularity , which enables countries to monitor their progress in meeting key learning objectives; and • br , which, in PISA 2012, encompasses the 34 OECD member countries and 31 partner countries eadth of coverage and economies. test the whole world can take a Box I.1.1. PISA is now used as an assessment tool in many regions around the world. It was implemented in 43 countries and economies in the first assessment (32 in 2000 and 11 in 2002), 41 in the second assessment (2003), 57 in the third assessment (2006) and 75 in the fourth assessment (65 in 2009 and 10 in 2010). So far, 65 countries and economies have participated in PISA 2012. In addition to OECD member countries, the survey has been or is being conducted in: East, South and Southeast Asia: Himachal Pradesh-India, Hong Kong-China, Indonesia, Macao-China, Malaysia, Shanghai-China, Singapore, Chinese Taipei, Tamil Nadu-India, Thailand and Viet Nam. Albania, Azerbaijan, Bulgaria, Croatia, Georgia, Central, Mediterranean and Eastern Europe, and Central Asia: Kazakhstan, Kyrgyzstan, Latvia, Liechtenstein, Lithuania, the former Yugoslav Republic of Macedonia, Malta, Moldova, Montenegro, Romania, the Russian Federation and Serbia. The Middle East: Jordan, Qatar and the United Arab Emirates. Central and South America: Argentina, Brazil, Colombia, Costa Rica, Netherlands-Antilles, Panama, Peru, Trinidad and Tobago, Uruguay and Miranda-Venezuela. Mauritius and Tunisia. Africa: ... o: Student Performan e – Volume © OECD 2014 What Student S Kno W and Can d i C e in m athemati CS , r eading and S C ien C 24

27 1 ? What is P isa Decisions about the scope and nature of the PISA assessments and the background information to be collected are made by leading experts in participating countries. Considerable efforts and resources are devoted to achieving cultural and linguistic breadth and balance in assessment materials. Since the design and translation of the test, as well as sampling and data collection, are subject to strict quality controls, PISA findings are considered to be highly valid and reliable. • Figure I.1.1 • pi a ap of m S countries and economies i cd countries P artner countries and economies in P E S a 2012 Partner countries and economies in previous cycles o Albania Japan Australia Azerbaijan Montenegro Peru Austria Korea Argentina Georgia Belgium Luxembourg Brazil Qatar Himachal Pradesh-India Bulgaria Kyrgyzstan Canada Romania Mexico Colombia Netherlands Chile Former Yugoslav Republic of Macedonia Russian Federation Costa Rica Malta Serbia New Zealand Czech Republic Norway Croatia Denmark Shanghai-China Mauritius 1, 2 Estonia Cyprus Poland Miranda-Venezuela Singapore Finland Portugal Hong Kong-China Chinese Taipei Moldova France Slovak Republic Indonesia Thailand Panama Slovenia Tamil Nadu-India Tunisia Jordan Germany Greece Spain Kazakhstan United Arab Emirates Trinidad and Tobago Hungary Sweden Latvia Uruguay Iceland Viet Nam Liechtenstein Switzerland Ireland Turkey Lithuania Israel Macao-China United Kingdom Malaysia Italy United States 1. Note by Turkey: The information in this document with reference to “Cyprus” relates to the southern part of the Island. There is no single authority representing both Turkish and Greek Cypriot people on the Island. Turkey recognises the Turkish Republic of Northern Cyprus (TRNC). Until a lasting and equitable solution is found within the context of the United Nations, Turkey shall preserve its position concerning the “Cyprus issue”. 2. Note by all the European Union Member States of the OECD and the European Union: The Republic of Cyprus is recognised by all members of the United Nations with the exception of Turkey. The information in this document relates to the area under the effective control of the Government of the Republic of Cyprus. 2012 urvey mea What doe S the ure? S pi S S a The PISA 2012 survey focuses on mathematics, with reading, science and problem solving as minor areas of assessment. For the first time, PISA 2012 also included an assessment of the financial literacy of young people, which was optional for countries. proficiency means the capacity of individuals to formulate, employ and interpret mathematics in For PISA, mathematics a variety of contexts. The term describes the capacities of individuals to reason mathematically and use mathematical concepts, procedures, facts and tools to describe, explain and predict phenomena. Mathematics literacy is not an attribute that an individual either has or does not have; rather, it is a skill that can be developed over a lifetime. CS OECD 2014 S Kno W and Can d o: Student Performan C e in m athemati What Student , r eading and S C ien C e – Volume i © 25

28 1 ? What is P isa The 2012 survey is the fifth round of assessments since PISA began in 2000, and the second, after the 2003 survey, that focuses on mathematics. As such, PISA 2012 provides an opportunity to evaluate changes in student performance in mathematics since 2003, and to view those changes in the context of policies and other factors. For the first time, PISA 2012 includes an optional computer-based assessment of mathematics. Specially designed PISA questions are presented on a computer, and students respond on the computer, although they can also use pencil and paper as they think through the test questions. es of 2012 a S pi Box I.1.2. Key featur The content he PISA 2012 survey focused on mathematics, with reading, science and problem solving as minor areas of • T assessment. For the first time, PISA 2012 also included an assessment of the financial literacy of young people, which was optional for countries and economies. hether students can reproduce knowledge, but also whether they can extrapolate from PISA assesses not only w • what they have learned and apply their knowledge in new situations. It emphasises the mastery of processes, the understanding of concepts, and the ability to function in various types of situations. The students ear-olds in the Around 510 000 students completed the assessment in 2012, representing about 28 million 15-y • schools of the 65 participating countries and economies. The assessment P • aper-based tests were used, with assessments lasting a total of two hours for each student. In a range of countries and economies, an additional 40 minutes were devoted to the computer-based assessment of mathematics, reading and problem solving. • T est items were a mixture of multiple-choice items and questions requiring students to construct their own responses. The items were organised in groups based on a passage setting out a real-life situation. A total of about 390 minutes of test items were covered, with different students taking different combinations of test items. Students ans wered a background questionnaire, which took 30 minutes to complete, that sought information • about themselves, their homes and their school and learning experiences. School principals were given a questionnaire, to complete in 30 minutes, that covered the school system and the learning environment. In some countries and economies, optional questionnaires were distributed to parents, who were asked to provide information on their perceptions of and involvement in their child’s school, their support for learning in the home, and their child’s career expectations, particularly in mathematics. Countries could choose two other optional questionnaires for students: one asked students about their familiarity with and use of information and communication technologies, and the second sought information about their education to date, including any interruptions in their schooling and whether and how they are preparing for a future career. pi Who are the S tudent S ? a S Differences between countries in the nature and extent of pre-primary education and care, in the age of entry into formal schooling, in the structure of the education system, and in the prevalence of grade repetition mean that school grade levels are often not good indicators of where students are in their cognitive development. To better compare student performance internationally, PISA targets a specific age of students. PISA students are aged between 15 years 3 months and 16 years 2 months at the time of the assessment, and have completed at least 6 years of formal schooling. They can be enrolled in any type of institution, participate in full-time or part-time education, in academic or vocational programmes, and attend public or private schools or foreign schools within the country. (For an operational definition of this target population, see Annex A2.) Using this age across countries and over time allows PISA to compare consistently the knowledge and skills of individuals born in the same year who are still in school at age 15, despite the diversity of their education histories in and outside of school. o: Student Performan e – Volume © OECD 2014 What Student S Kno W and Can d C C e in m athemati CS , r eading and S C ien i 26

29 1 ? What is P isa The population of participating students is defined by strict technical standards, as are the students who are excluded from participating (see Annex A2). The overall exclusion rate within a country was required to be below 5% to ensure that, under reasonable assumptions, any distortions in national mean scores would remain within plus or minus 5 score points, i.e. typically within the order of magnitude of 2 standard errors of sampling. Exclusion could take place either through the schools that participated or the students who participated within schools (see Annex A2, Tables A2.1 and A2.2). There are several reasons why a school or a student could be excluded from PISA. Schools might be excluded because they are situated in remote regions and are inaccessible, because they are very small, or because of organisational or operational factors that precluded participation. Students might be excluded because of intellectual disability or limited proficiency in the language of the assessment. In 28 out of the 65 countries and economies participating in PISA 2012, the percentage of school-level exclusions amounted to less than 1%; it was less than 4% in all countries and economies. When the exclusion of students who met the internationally established exclusion criteria is also taken into account, the exclusion rates increase slightly. However, the overall exclusion rate remains below 2% in 30 participating countries and economies, below 5% in 57 participating countries, and below 7% in all countries except Luxembourg (8.4%). In 11 out of the 34 OECD countries, the percentage of school-level exclusions amounted to less than 1% and was less than 3% in 31 OECD countries. When student exclusions within schools were also taken into account, there were 11 OECD countries below 2% and 26 OECD countries below 5%. Restrictions on the level of exclusions in PISA 2012: • Sc hool-level exclusions for inaccessibility, feasibility or other reasons were required not to exceed 0.5% of the total number of students in the international PISA target population. Schools on the sampling frame that had only one or two eligible students were not allowed to be excluded from the frame. However, if, based on the frame, it was clear that the percentage of students in these schools would not cause a breach of the allowable limit, then those schools could be excluded from the field, if at that time, they still had only one or two students who were eligible for PISA. • Sc hool-level exclusions for students with intellectual or functional disabilities, or students with limited proficiency in the language of the PISA assessment, were required not to exceed 2% of students. W ithin-school exclusions for students with intellectual or functional disabilities, or students with limited language • proficiency were required not to exceed 2.5% of students. Students who could be excluded from PISA 2012 were: • Intellectually disabled students, defined as students who are considered, in the professional opinion of the school principal, or by other qualified staff members, to be intellectually disabled, or who have been assessed psychologically as such. This category includes students who are emotionally or mentally unable to follow even the general instructions of the assessment. Students were not to be excluded solely because of poor academic performance or common discipline problems. ho are permanently physically disabled in such a way that Students with functional disabilities, defined as students w • they cannot perform in the PISA testing situation. Students with functional disabilities who could perform were to be included in the testing. • with limited proficiency in the language of the PISA assessment, defined as students who had received less Students than one year of instruction in the language of the assessment. (For more detailed information about the restrictions on the level of exclusions in PISA 2012, see Annex A2.) S What i the te S t li K e? For each round of PISA, one subject is tested in detail, taking up nearly two-thirds of the total testing time. The major subject was reading in 2000 and 2009, mathematics in 2003 and 2012, and science in 2006. As in previous PISA assessments, the paper-based assessment was designed as a two-hour test comprising four 30-minute clusters of test material from one or more subjects. Information was obtained from about 390 minutes worth of test items. For each country, the total set of questions was packaged into 13 linked test booklets. Financial literacy, an option in the paper- based assessment, was allocated two clusters (that is, 60 minutes of testing time) in the 2012 survey. CS © S Kno W and Can d o: Student Performan C e in m athemati What Student , r eading and S C ien C e – Volume 27 OECD 2014 i

30 1 What is P isa ? Each booklet was completed by a sufficient number of students so that reliable estimates could be made of the level of achievement among students in each country and in relevant subgroups – such as boys and girls, and students with different socio-economic status – within a country. Students also spent 30 minutes answering a background questionnaire. Some questions were answered by all students, as in previous assessments; some were answered by subsamples of students. In addition to this core assessment, 44 countries and economies participated in a computer-based assessment of problem solving; 32 of them also participated in a computer-based assessment of reading and mathematics. The PISA 2012 computer-delivered assessment lasted 40 minutes. A total of 80 minutes of problem-solving material was organised into four 20-minute clusters. Students from countries not participating in the optional computer-based assessment of mathematics and digital reading completed two of the clusters. Students from countries that did participate in the optional computer-based assessment of mathematics and digital reading completed two, one or none of the four problem- solving clusters. The optional computer-based component contained a total of 80 minutes of mathematics material and 80 minutes of reading material. Figure I.1.2 • • Summary of the assessment areas in pi S a 2012 Sci G adin rE m matic E ath E nc E S An individual’s scientific knowledge An individual’s capacity to understand, efinitions An individuals’ capacity d and use of that knowledge to identify use, reflect on and engage with written to formulate, employ, and questions, to acquire new knowledge, texts, in order to achieve one’s goals, interpret mathematics in a to explain scientific phenomena, and to develop one’s knowledge and variety of contexts. It includes to draw evidence-based conclusions potential, and to participate in society. reasoning mathematically and about science-related issues. It includes using mathematical concepts, understanding the characteristic procedures, facts and tools to features of science as a form of human describe, explain and predict knowledge and enquiry, awareness phenomena. It assists individuals of how science and technology shape in recognising the role that our material, intellectual, and cultural mathematics plays in the world environments, and willingness to and to make the well-founded engage in science-related issues, judgements and decisions needed and with the ideas of science, by constructive, engaged and as a reflective citizen. reflective citizens. c ontents Four overarching ideas that relate The form of reading materials includes: Scientific knowledge or concepts are related to physics, chemistry, biological to numbers, algebra and geometry: • continuous texts or prose organised sciences and earth and space sciences, quantity • in sentences and paragraphs (e.g. but they are applied to the content of space and shape • narration, exposition, argumentation, the items and not just recalled. • change and relationships description, instruction) • uncertainty and data non-continuous texts that present • information in other ways, such as in lists, forms, graphs, or diagrams Processes • describing, explaining and • accessing and retrieving information formulating situations • forming a broad general • mathematically predicting scientific phenomena understanding of the text • understanding scientific employing mathematical • concepts, facts, procedures and • interpreting the text investigation • • reflecting on the content and the interpreting scientific evidence and reasoning • form and features of the text conclusions interpreting, applying and evaluating mathematical outcomes (referred to in abbreviated form as “formulate, employ and interpret”) The situations in which science The situations in which ontexts c The use for which a text is constructed: literacy is applied: mathematics literacy is applied: personal • • personal • educational • personal • occupational occupational • social • societal • public • • global scientific • For some applications of science: life and health • • earth and environment • technology o: Student Performan e – Volume © OECD 2014 What Student S Kno W and Can d C C e in m athemati CS , r eading and S C ien i 28

31 1 ? What is P isa The material for each subject was arranged in four clusters of items, with each cluster representing 20 minutes of testing time. All material that was presented on a computer was arranged in a number of test forms, with each form containing two clusters. Each student did one form, representing a total testing time of 40 minutes. S h o W i t conducted? the te S When a school participates in PISA, a school co-ordinator is appointed. The school co-ordinator compiles a list of all 15-year-olds in the school and sends this list to the PISA National Centre in the country, which randomly selects 35 students to participate. The school co-ordinator then contacts the students who have been selected and obtains the necessary permission from parents. The testing session is usually conducted by a test administrator who is trained and employed by the National Centre. The test administrator contacts the school co-ordinator to schedule administration of the assessment. The school co-ordinator ensures that the students, who may come from different grades and different classes, attend the testing sessions. The test administrator’s primary tasks are to ensure that each test booklet is distributed to the correct student and to introduce the tests to the students. After the test is over, the test administrator collects the test booklets and sends them to the National Centre for coding. In PISA 2012, at least 13 different test booklets were used in each country. With 13 different booklets for each group of 35 students, no more than 3 students were given the same booklet. Booklets were allocated to individual students according to a random selection process. The test administrator’s introduction came from a prescribed text so that all students in different schools and countries received exactly the same instructions. Before starting the test, the students were asked to do a practice question from their booklets. The testing session was divided into two parts: the two-hour test to assess their knowledge and skills, and the 30-minute questionnaire session to collect data on their personal background. Students were usually given a short break half-way through the test and again before they completed the questionnaire. K the te S doe S ult S of re S ind What t provide? S The PISA assessment provides three main types of outcomes: vide a baseline profile of students’ knowledge and skills; • basic indicators that pro w how skills relate to important demographic, social, economic and educational variables; and • indicators that sho indicators on trends that sho • w changes in student performance and in the relationships between student-level and school-level variables and outcomes. Although indicators can highlight important issues, they do not provide answers to policy questions. To respond to this, PISA also developed a policy-oriented analysis plan that uses the indicators as a basis for policy discussion. S ult Where can you find the re ? S This is the first of six volumes that presents the results from PISA 2012. It begins by discussing student performance in mathematics in PISA 2012 and examines how that performance has changed over previous PISA assessments. Chapter 3 examines how opportunities to learn are associated with mathematics performance. Chapters 4 and 5 provide an overview of student performance in reading and science, respectively, and describe the evolution of performance in these subjects over previous PISA assessments. Chapter 6 discusses the policy implications based on analyses of the results of the preceding chapters and on the policy-reform experience of some countries that have improved during the participation in PISA. The other five volumes cover the following issues: defines and measures equity in Excellence through Equity: Giving Every Student the Chance to Succeed, Volume II, education and analyses how equity in education has evolved across countries between PISA 2003 and 2012. The volume examines the relationship between student performance and socio-economic status, and describes how other individual student characteristics, such as immigrant background and family structure, and school characteristics, such as school location, are associated with socio-economic status and performance. The volume also reveals differences in how equitably countries allocate resources and opportunities to learn to schools with different socio-economic profiles. Case studies, examining the policy reforms adopted by countries that have improved in PISA, are highlighted throughout the volume. explores students’ engagement with and Volume III, Ready to Learn: Students’ Engagement, Drive and Self-Beliefs, at school, their drive and motivation to succeed, and the beliefs they hold about themselves as mathematics learners. CS OECD 2014 S Kno W and Can d o: Student Performan C e in m athemati What Student , r eading and S C ien C e – Volume i © 29

32 1 ? What is P isa The volume identifies the students who are at particular risk of having low levels of engagement in, and holding negative dispositions towards, school in general and mathematics in particular, and how engagement, drive, motivation and self-beliefs are related to mathematics performance. The volume identifies the roles schools can play in shaping the well-being of students and the role parents can play in promoting their children’s engagement with and dispositions towards learning. Changes in students’ engagement, drive, motivation and self-beliefs between 2003 and 2012, and how those dispositions have changed during the period among particular subgroups of students, notably socio-economically advantaged and disadvantaged students, boys and girls, and students at different levels of mathematics proficiency, are examined when comparable data are available. Throughout the volume, case studies examine in greater detail the policy reforms adopted by countries that have improved in PISA. examines how student performance is What Makes Schools Successful? Resources, Policies and Practices, Volume IV, associated with various characteristics of individual schools and of concerned school systems. It discusses how 15-year- old students are selected and grouped into different schools, programmes, and education levels, and how human, financial, educational and time resources are allocated to different schools. The volume also examines how school systems balance autonomy with collaboration, and how the learning environment in school shapes student performance. Trends in these variables between 2003 and 2012 are examined when comparable data are available, and case studies, examining the policy reforms adopted by countries that have improved in PISA, are highlighted throughout the volume. presents student performance in the PISA 2012 Skills for Life: Student Performance in Problem Solving, Volume V, assessment of problem solving, which measures students’ capacity to respond to non-routine situations in order to achieve their potential as constructive and reflective citizens. It provides the rationale for assessing problem-solving skills and describes performance within and across countries. In addition, the volume highlights the relative strengths and weaknesses of each school system and examines how they are related to individual student characteristics, such as gender, immigrant background and socio-economic status. The volume also explores the role of education in fostering problem-solving skills. Volume VI, Students and Money: Financial Literacy Skills for the 21st Century, examines 15-year-old students’ performance in financial literacy in the 18 countries and economies that participated in this optional assessment. It also discusses the relationship of financial literacy to students’ and their families’ background and to students’ mathematics and reading skills. The volume also explores students’ access to money and their experience with financial matters. In addition, it provides an overview of the current status of financial education in schools and highlights relevant case studies. The frameworks for assessing mathematics, reading and science in 2012 are described in PISA 2012 Assessment and Analytical Framework: Mathematics, Reading, Science, Problem Solving and Financial Literacy (OECD, 2013). They are also summarised in this volume. Technical annexes at the end of this report describe how questionnaire indices were constructed and discuss sampling issues, quality-assurance procedures, the reliability of coding, and the process followed for developing the assessment instruments. Many of the issues covered in the technical annexes are elaborated in greater detail in the PISA 2012 (OECD, forthcoming). Technical Report All data tables referred to in the analysis are included at the end of the respective volume in Annex B1, and a set of additional data tables is available on line ( www.pisa.oecd.org ). A Reader’s Guide is also provided in each volume to aid in interpreting the tables and figures that accompany the report. Data from regions within the participating countries are included in Annex B2. Results from the computer-based assessment of mathematics and reading are presented in Annex B3. References OECD (forthcoming), PISA 2012 Technical Report , PISA, OECD Publishing. OECD (2013), PISA 2012 Assessment and Analytical Framework: Mathematics, Reading, Science, Problem Solving and Financial Literacy , PISA, OECD Publishing. http://dx.doi.org/10.1787/9789264190511-en o: Student Performan C © OECD 2014 What Student S Kno W and Can d e – Volume C e in m athemati CS , r eading and S C i ien 30

33 2 A Profile of Student Performance in Mathematics This chapter compares student performance in mathematics across and within countries and economies. It discusses the PISA definition of literacy in mathematics and describes the tasks associated with each PISA proficiency level. The chapter then digs deep into the results of the mathematics assessment, showing gender differences in performance, trends in mathematics performance up to 2012, and differences in handle certain mathematics processes, such as students’ abilities to formulating situations mathematically, and certain mathematics contents, uncertainty and data such as . space and shape , and CS e – Volume What Student OECD 2014 31 C ien C eading and S r , i athemati m e in C o: Student Performan d and Can W Kno S ©

34 2 f Student Perform A nce i n mA them A tic S o A Profile All adults, not just those with technical or scientific careers, now require adequate mathematics proficiency for personal fulfilment, employment and full participation in society. To one degree or another, mathematical concepts and processes are intrinsic to many daily tasks: from buying and selling goods and services, to cooking or planning a vacation, to explaining highly complex phenomena. Students about to leave compulsory education should thus have a solid understanding of these concepts and be able to apply them to solve problems that they encounter in their daily lives. This chapter summarises the mathematics performance of students in PISA 2012. It describes how performance is defined, measured and reported, and then provides results from the paper-based assessment, showing what students are able to do in mathematics. After a summary of mathematics performance, it examines the ways in which this performance varies on subscales representing different aspects of mathematics. Annex B3 provides further results for 32 countries and economies that participated in the computer-based assessment, supplementing the paper-based scale with two others: the computer-based scale and the combined paper- and computer-based scale. What the data tell us • w an average annual improvement in Of the 64 countries and economies with trend data up to 2012, 25 sho mathematics performance, 25 show no change, and 14 show a deterioration in performance. ve participated in every assessment since 2003, Brazil, Italy, Mexico, Among countries and economies that ha • Poland, Portugal, Tunisia and Turkey show an average improvement in mathematics performance of more than 2.5 points per year. y, Hong Kong-China, Macao-China, Shanghai-China and Singapore improved in mathematics performance German • and their previous scores placed them at or above the OECD average. • Between 2003 and 2012 Italy , Poland and Portugal reduced the proportion of low performers and increased the proportion of high performers. This was also observed in Israel, Qatar and Romania between 2006 and 2012, and in Ireland, Malaysia and the Russian Federation between 2009 and 2012. Bo • ys perform better than girls in mathematics in 38 out of the 65 countries and economies that participated in PISA 2012, and girls outperform boys in 5 countries. Box I.2.1. say about readiness a S pi What does performance in for further education and a career? To what extent is the performance of 15-year-olds in PISA predictive of further education and career readiness and success later in life? The transition from adolescence to early adulthood is a critical time in the social and intellectual development of young people. Once compulsory education is completed, adolescents have to make important decisions about post-secondary education, employment and other life choices that will have a major impact on their future learning and employment prospects as well as on their overall well-being. A decade- long study undertaken in Canada coupled data collected from the PISA assessment of 15-year-olds in 2000 with follow-ups conducted every two years through a national survey of those same students and parents (the Youth in Transition Survey). The results from this study show that having a solid foundation in the kinds of skills that PISA measures makes it much easier to advance in post-compulsory education. Reading scores in PISA, for example, are associated with the likelihood of students progressing from one grade level to another across grades 10 to 16. Some 37% of boys with a high reading score, i.e. in the top quintile of reading proficiency, attained grade 16 compared to just 3.4% of boys with low reading scores (bottom quintile). Similarly, 52.4% of girls with high reading scores attained grade 16 compared to 14.9% of girls with low reading scores. The results show that reading scores had a stronger association with grade progression during the post-secondary school years than with schooling up to grade 12, particularly for boys. Equally important, the results also show that introducing a uniform increase of one standard deviation in reading scores results in a 17.4% reduction in the proportion of young men who leave formal education before completing secondary school and a 12.6% increase in the proportion of young men who attend post-secondary education. ... o: Student Performan e in © OECD 2014 What Student S Kno W and Can d m i e – Volume C ien C eading and S r , CS athemati C 32

35 2 o A nce i n mA them A f Student Perform S A Profile tic For girls, the effects of increased reading scores are also substantial. A one standard deviation increase in reading scores is associated with a 31.5% reduction in the proportion of girls who leave formal education before completing secondary school and an 11.4% increase in the share of young women who complete at least some post-secondary education. Even after adjusting for socio-economic status, both achievement in PISA and educational attainment are associated with a higher likelihood of continuing in education and a lower likelihood of proceeding to work or to a period of inactivity (OECD, 2010a). To what extent are the differences in the performance of school systems, as observed in PISA, reflected in the skills of adults who have recently completed initial education and training? The Survey of Adult Skills, a product of the OECD Programme for the International Assessment of Adult Competencies (PIAAC), provides a way to assess this. Most adults aged 27 or under in participating countries correspond to the cohorts assessed in PISA in 2000, 2003, 2006 and 2009, when they were 15 years old. The results from the Survey of Adult Skills show that, overall, there is a reasonably close correlation between countries’ performance across the successive PISA assessments and the proficiency of the corresponding age cohorts in literacy and numeracy in the Skills Survey. Countries performing well in PISA in a given year (e.g. 2000) tend to show high performance among the corresponding age cohort (e.g. 27-year-olds) in the Survey of Adult Skills (PIAAC) and vice versa. This suggests that, at the country level, the reading and mathematics proficiency of an age cohort in PISA is a reasonably good predictor of the cohort’s subsequent performance in literacy and numeracy as it moves through post-compulsory education and into the labour market. By implication, much of the difference in the literacy and numeracy proficiency of young adults today is likely related to the effectiveness of the instruction they received in primary and lower secondary school. Of course, some caution is advised in comparing results of the two studies. The overlap between the target populations of the Survey of Adult Skills (PIAAC) and PISA is not complete; and while the concepts of literacy in the Skills Survey and reading literacy in PISA, and the concepts of numeracy in the Skills Survey and mathematical literacy in PISA are closely related, the measurement scales are not the same. In addition, the skills of 15-27 year - olds are subject to influences that vary across individuals and countries, including participation in post-secondary and tertiary education and the quality of these programmes, second-chance opportunities for low-skilled young adults, and characteristics of the labour market (OECD, 2013a and b). athemati 33 © e – Volume C ien C eading and S r , CS OECD 2014 m e in C o: Student Performan d and Can W Kno S What Student i

36 2 o A nce i n f Student Perform them A tic S A Profile mA a context for comparing the mathematic S performance of countrie S S and economie Comparing mathematics performance, and educational performance more generally, poses numerous challenges. When teachers give a mathematics test in a classroom, students with varying abilities, attitudes and social backgrounds are required to respond to the same set of tasks. When educators compare the performance of schools, the same test is used across schools that may differ significantly in the structure and sequencing of their curricula, in the pedagogical emphases and instructional methods applied, and in the demographic and social contexts of their student populations. Comparing the performance of education systems across countries adds more layers of complexity, because students are given tests in different languages, and because the social, economic and cultural context of the countries that are being compared are often very different. However, while students within a country may learn in different contexts according to their home background and the school that they attend, their performance is measured against common standards, since, when they become adults, they will all face common challenges and have to compete for the same jobs. Similarly, in a global economy, the benchmark for success in education is no longer improvement by national standards alone, but increasingly, in relation to the best-performing education systems internationally. As difficult as international comparisons are, they are important for educators, and PISA goes to considerable lengths to ensure that such comparisons are valid and fair. This section discusses countries’ mathematics performance in the context of important economic, demographic and social factors that can influence assessment results. It provides a framework for interpreting the results that are presented later in the chapter. As shown in Volume II, , a family’s wealth influences children’s performance in school, but that Excellence through Equity influence varies markedly across countries. Similarly, the relative prosperity of some countries allows them to spend more on education, while other countries find themselves constrained by a lower national income. It is therefore important to keep the national income of countries in mind when comparing the performance of education systems across countries. Figure I.2.1 displays the relationship between national income as measured by per capita Gross Domestic Product (GDP) 1 2 and students’ average mathematics performance. The figure also shows a trend line that summarises the relationship between per capita GDP and mean student performance in mathematics among OECD countries. The relationship suggests that 21% of the variation in countries’ mean scores can be predicted on the basis of their per capita GDP (12% of the variation in OECD countries). Countries with higher national incomes are thus at a relative advantage, even if the chart provides no indications about the causal nature of this relationship. This should be taken into account particularly when interpreting the performance of countries with comparatively low levels of national income, such as Viet Nam and Indonesia (Mexico and Turkey among OECD countries). Table I.2.27 shows an “adjusted” score that would be expected if the country had all of its present characteristics except that per capita GDP was equal to the average for OECD countries (Table I.2.27). While per capita GDP reflects the potential resources available for education in each country, it does not directly measure the financial resources actually invested in education. Figure I.2.2 compares countries’ actual spending per 3 student, on average, from the age of 6 up to the age of 15, with average student performance in mathematics. The results are expressed in USD using purchasing power parities (PPP). Figure I.2.2 shows a positive relationship between spending per student and mean mathematics performance among OECD countries. As expenditure on educational institutions per student increases, so does a country’s mean performance. Expenditure per student explains 30% of the variation in mean performance between countries (17% of the variation in OECD countries). Relatively low spending per student needs to be taken into account when interpreting the performance of countries such as Viet Nam and Jordan (Turkey and Mexico among OECD countries). (For more details, see Figure IV.1.7 in Volume IV). At the same time, deviations from the trend line suggest that moderate spending per student cannot automatically be equated with poor performance. For example, the Slovak Republic, which spends around USD 53 000 per student, performs at the same level as the United States, which spends over USD 115 000 per student. Similarly, Korea, the highest-performing OECD country in mathematics, spends well below the average per-student expenditure (Table I.2.27). Given the close interrelationship between a student’s performance and his or her parents’ level of education, it is also important to bear in mind the educational attainment of adult populations when comparing the performance of OECD countries, as countries with more highly educated adults are at an advantage over countries where parents have less education. Figure I.2.3 shows the percentage of 35-44 year-olds who have attained tertiary education. This group corresponds roughly to the age group of parents of the 15-year-olds assessed in PISA. Parents’ level of education explains 27% of the variation in mean performance between countries (23% of the variation among OECD countries). o: Student Performan CS © OECD 2014 What Student S Kno W and Can d , C e in m i e – Volume C ien C eading and S r athemati 34

37 2 o A nce i n mA them A tic S f Student Perform A Profile Figure I.2.1 • • • • Figure I.2.2 m athematics performance athematics performance m and g ross d omestic and spending on education p roduct Score Score 625 625 600 600 y = 0.0015x + 429.69 575 575 y = 0.7167x + 431.06 R² = 0.21 R² = 0.30 550 550 525 525 500 500 475 475 450 450 425 425 400 400 375 375 350 350 80 200180 160 0 10 20 30 40 50 60 70 80 140 120 0 20 40 60 90 100 GDP per capita (in thousand USD converted using PPPs) Cumulative expenditure (in thousand USD converted using PPPs) OECD, PISA 2012 Database, Table I.2.27. Source: OECD, PISA 2012 Database, Table I.2.27. Source: 2 http://dx.doi.org/10.1787/888932935572 1 2 http://dx.doi.org/10.1787/888932935572 1 • Figure I.2.4 • • m athematics performance and share • Figure I.2.3 athematics performance and parents’ education m of socio-economically disadvantaged students Score Score 625 625 600 600 575 575 550 550 525 525 500 500 y = 1.3836x + 443.47 y = -1.3296x + 508.21 475 475 R² = 0.27 R² = 0.24 450 450 425 425 400 400 375 375 350 350 0 70 60 90 50 40 30 20 10 80 60 50 40 30 20 10 0 Share of students whose Percentage of the population PISA index of economic, social and cultural status is below -1 in the age group 35-44 with tertiary education OECD, PISA 2012 Database, Table I.2.27. Source: OECD, PISA 2012 Database, Table I.2.27. Source: http://dx.doi.org/10.1787/888932935572 1 2 1 2 http://dx.doi.org/10.1787/888932935572 Figure I.2.6 Figure I.2.5 • • • • e m assessment quivalence of the pi S a athematics performance and proportion of students from an immigrant background across cultures and languages Rank based on own preferred Score new PISA 2009 items 625 60 Countries would have higher ranking if their 600 preferred questions 50 575 were used 550 40 525 y = 0.7714x + 464.39 500 30 R² = 0.04 475 450 20 425 Countries would have 400 10 lower ranking if their preferred questions 375 were used 0 350 10 40 30 20 10 70 0 80 50 0 20 30 40 60 50 60 Percent-correct rank based on Proportion of 15-year-olds new PISA 2009 items with an immigrant background OECD, PISA 2012 Database, Table I.2.27. Source: OECD, PISA 2009 Database, Table I.2.28. Source: 1 http://dx.doi.org/10.1787/888932935572 2 1 http://dx.doi.org/10.1787/888932935572 2 CS athemati m © C o: Student Performan d and Can W Kno S What Student 35 i e – Volume C ien OECD 2014 C eading and S r , e in

38 2 S f Student Perform A nce i n mA them A tic o A Profile Socio-economic heterogeneity in student populations poses another major challenge for teachers and education systems. , teachers instructing socio-economically disadvantaged children are Excellence through Equity As shown in Volume II, likely to face greater challenges than teachers teaching students from more advantaged backgrounds. Similarly, countries with larger proportions of disadvantaged children face greater challenges than countries with smaller proportions of these students. Figure I.2.4 shows the proportion of students at the lower end of an international scale of the economic, social and cultural status of students, which is described in detail in Volume II, and how this relates to mathematics performance. The relationship explains 24% of the performance variation among countries (46% of the variation among OECD countries). Among OECD countries, Turkey and Mexico, where 69% and 56% of students, respectively, belong to the most disadvantaged group, and Portugal, Chile, Hungary and Spain, where more than 20% of students belong to this group, face much greater challenges than, for example, Iceland, Norway, Finland and Denmark, where fewer than 5% of students are disadvantaged (Table I.2.27). These challenges are even greater in some partner countries like Viet Nam and Indonesia where 79% and 77% of students, respectively, are socio-economically disadvantaged. Integrating students with an immigrant background can also be challenging, and the level of performance of students who immigrated to the country in which they were assessed can be only partially attributed to their host country’s education system. Figure I.2.5 shows the proportion of 15-year-olds from an immigrant background and how this relates to student performance. This proportion explains only 4% of the variation in mean performance among countries. Despite having large proportions of immigrant students, some countries, like Canada, perform above the OECD average (Table I.2.27). When examining the results for individual countries, as shown in Table I.2.27, it is apparent that countries vary in their 4 demographic, social and economic contexts. Table I.2.27 summarises in an index the different factors discussed above. Among the countries with available data, the index shows Luxembourg, Norway, Japan, Finland, Iceland, Denmark, Ireland and the United States with the most advantaged demographic, social and economic contexts, and Turkey, Brazil, Mexico, Chile, Portugal, Hungary, the Slovak Republic, Poland and the Czech Republic with the most challenging contexts. These differences need to be considered when interpreting PISA results. At the same time, the future economic and social prospects of both individuals and countries depend on the results they actually achieve, not on the performance they might have achieved under different social and economic conditions. That is why the results that are actually achieved by students, schools and countries are the focus of this volume. Even after accounting for the demographic, economic and social context of education systems, the question remains: to what extent is an international test meaningful when differences in languages and cultures lead to very different ways in which subjects such as language, mathematics and science are taught and learned? It is inevitable that not all tasks on the PISA assessments are equally appropriate in different cultural contexts and equally relevant in different curricular and instructional contexts. To gauge this, in 2009 PISA asked every country to identify those tasks from the PISA tests that it considered most appropriate for an international test. Countries were advised to give an on-balance rating for each task with regard to its usefulness in indicating “preparedness for life”, its authenticity, and its relevance for 15-year-olds. Tasks given a high rating by a country are referred to as that country’s most preferred questions for PISA. PISA then scored every country on its own most preferred questions and compared the resulting performance with the performance on the entire set of PISA tasks (Figure I.2.6). It is clear that, generally, the proportion of questions answered correctly by students does not depend significantly on whether countries were only scored on their preferred questions or on the overall set of PISA tasks. This provides robust evidence that the results of the PISA assessments would not change markedly if countries had more influence in selecting texts that they thought might be “fairer” to their students. Finally, when comparing student performance across countries, the extent to which student performance on international tests might be influenced by the effort that students in different countries invest in the assessment must be considered. In PISA 2003, students were asked to imagine an actual situation that was highly important to them, so that they could try their very best and invest as much effort as they could into doing well. They were then asked to report how much effort they had put into doing the PISA test compared to the situation they had just imagined; and how much effort they would have invested if their marks from PISA had been counted in their school marks. The students generally answered realistically, saying that they would expend more effort if the test results were to count towards their school marks; but the analysis also established that the reported expenditure of effort by students was fairly stable across countries. This finding counters the claim that systematic cultural differences in the effort expended by students invalidate international comparisons. The analysis also showed that within countries, the amount of effort invested was related to student achievement, with an effect 5 size similar to variables such as single-parent family structure, gender and socio-economic background. e – Volume o: Student Performan © OECD 2014 What Student S Kno W and Can i C C ien C eading and S r , CS athemati m e in d 36

39 2 o A nce i n mA f Student Perform A tic S A Profile them he pi S t a approach to a SS S tudent performance in mathematic S ing SS e The PISA definition of mathematical literacy The focus of the PISA 2012 assessment was on measuring an individual’s capacity to formulate, employ and interpret mathematics in a variety of contexts. It includes reasoning mathematically and using mathematical concepts, procedures, facts, and tools to describe, explain and predict phenomena. It assists individuals in recognising the role that mathematics plays in the world and to make the well-founded judgements and decisions needed by constructive, engaged and reflective citizens. The definition asserts the importance of mathematics for full participation in society and it stipulates that this importance arises from the way in which mathematics can be used to describe, explain and predict phenomena of many types. The resulting insight into phenomena is the basis for informed decision making and judgements. Literacy in mathematics described in this way is not an attribute that an individual has or does not have; rather, it can be acquired to a greater or lesser extent, and it is required in varying degrees in society. PISA seeks to measure not just the extent to which students can reproduce mathematical content knowledge, but also how well they can extrapolate from what they know and apply their knowledge of mathematics, in both new and unfamiliar situations. This is a reflection of modern societies and workplaces, which value success not by what people know, but by what people can do with what they know. The focus on real-life contexts is also reflected in the reference to using “tools” that appears in the PISA 2012 definition of mathematical literacy. The word “tools” here refers to physical and digital equipment, software and calculation devices that have become ubiquitous in 21st century workplaces. Examples for this assessment include a ruler, a calculator, a spreadsheet, an online currency converter and specific mathematics software, such as dynamic geometry. Using these tools require a degree of mathematical reasoning that the PISA assessment is well-equipped to measure. The PISA 2012 framework for assessing mathematics Figure I.2.7 presents an overview of the main constructs of the PISA 2012 mathematics framework that was established and agreed by the participating countries, and how the constructs relate to each other. The largest box shows that mathematical literacy is assessed in the context of a challenge or problem that arises in the real world. The middle box highlights the nature of mathematical thought and action that can be used to solve the problem. The smallest box describes the processes that the problem solver uses to construct a solution. • Figure I.2.7 • ain features of the m 2012 mathematics framework a S pi c hallenge in real world context Mathemat ical content categories: Quantity; Uncertainty and data; Change and relationships; Space and shape Real world context categories: Personal; Societal; Occupational; Scientific m athematical thought and action Mathematical concepts, knowledge and skills Fundamental mathematical capabilities: Communication; Representation; Devising strategies; Mathematisation; Reasoning and argument; Using symbolic, formal and technical language and operations; Using mathematical tools Formulate; Employ; Interpret/Evaluate Processes: Problem Mathematical Formulate in context problem Employ Evaluate Results Mathematical Interpret results in context and Can d o: Student Performan C e in m athemati CS OECD 2014 What Student S Kno W , r eading and S C ien C e – Volume i © 37

40 2 S f Student Perform A nce i n mA them A tic o A Profile Context categories Real-world challenges or situations are categorised in two ways: their context and the domain of mathematics involved. The four context categories identify the broad areas of life in which the problems may arise: personal, which is related to individuals’ and families’ daily lives; societal, which is related to the community – local, national or global – in which an individual lives; occupational, which is related to the world of work; or scientific, which is related to the use of mathematics in science and technology. According to the framework, these four categories are represented by equal numbers of items. Content categories As seen in Figure I.2.7, the PISA items also reflect four categories of mathematical content that are related to the problems posed. The four content categories are represented by approximately equal proportions of items. For the assessment of 15-year-olds, age-appropriate content was developed. The content category quantity incorporates the quantification of attributes of objects, relationships, situations, and entities in the world, which requires an understanding of various representations of those quantifications, and judging interpretations and arguments based on quantity. It involves understanding measurements, counts, magnitudes, units, indicators, relative size, and numerical trends and patterns, and employing number sense, multiple representations of numbers, mental calculation, estimation, and assessment of reasonableness of results. covers two closely related sets of issues: how to identify and summarise The content category uncertainty and data the messages that are embedded in sets of data presented in different ways, and how to appreciate the likely impact of the variability that is inherent in many real processes. Uncertainty is part of scientific predictions, poll results, weather forecasts and economic models; variation occurs in manufacturing processes, test scores and survey findings; and chance is part of many recreational activities that individuals enjoy. Probability and statistics, taught as part of mathematics, address these issues. The content category change and relationships focuses on the multitude of temporary and permanent relationships among objects and circumstances, where changes occur within systems of interrelated objects or in circumstances where the elements influence one another. Some of these changes occur over time; some are related to changes in other objects or quantities. Being more literate in this content category involves understanding fundamental types of change and recognising when change occurs so that suitable mathematical models can be employed to describe and predict change. The content category encompasses a wide range of phenomena that are encountered everywhere: space and shape patterns, properties of objects, positions and orientations, representations of objects, decoding and encoding of visual information, navigation, and dynamic interaction with real shapes and their representations. Geometry is essential to space and shape, but the category extends beyond traditional geometry in content, meaning and method, drawing on elements of other mathematical areas, such as spatial visualisation, measurement and algebra. Mathematical literacy in space and shape involves understanding perspective, creating and reading maps, transforming shapes with and without technology, interpreting views of three-dimensional scenes from various perspectives, and constructing representations of shapes. Process categories The smallest box of Figure I.2.7 shows a schema of the stages through which a problem-solver may move when solving PISA tasks. The action begins with the “problem in context.” The problem-solver tries to identify the mathematics relevant to the problem situation, formulates the situation mathematically according to the concepts and relationships identified, and makes assumptions to simplify the situation. The problem-solver thus transforms the “problem in context” into a “mathematical problem” that can be solved using mathematics. The downward- pointing arrow in Figure I.2.7 represents the work undertaken as the problem-solver employs mathematical concepts, facts, procedures and reasoning to obtain the “mathematical results”. This stage usually involves mathematical manipulation, transformation and computation, with and without tools. The “mathematical results” then need to be interpreted in terms of the original problem to obtain the “results in context”. The problem solver thus must interpret, apply and evaluate mathematical outcomes and their reasonableness in the context of a real-world problem. The three processes – formulate, employ and interpret – each draw on fundamental mathematical capabilities, which, in turn, draw on the problem-solver’s detailed mathematical knowledge. e – Volume o: Student Performan © OECD 2014 What Student S Kno W and Can i C C ien C eading and S r , CS athemati m e in d 38

41 2 A Profile f Student Perform A nce i n mA them A tic S o However, not all PISA tasks engage students in every stage of the modelling cycle. Items are classified according to the dominant process and results are reported by these processes, formally named as: • F ormulating situations mathematically. Emplo • ying mathematical concepts, facts, procedures and reasoning. aluating mathematical outcomes. • Interpreting, applying and ev Fundamental mathematical capabilities Through a decade of experience in developing PISA items and analysing the ways in which students respond to them, a set of fundamental mathematical capabilities has been established that underpins performance in mathematics. These cognitive capabilities can be learned by individuals in order to understand and engage with the world in a mathematical way. Since the PISA 2003 framework was written, researchers (e.g. Turner, 2013) have examined the extent to which the difficulty of a PISA item can be understood, and even predicted, from how each of the fundamental mathematical capabilities is used to solve the item. Four levels describe the ways in which each of the capabilities is used, from simple to complex. For example, an item involving a low level of communication would be simple to read and require only a simple response (e.g. a word); an item involving a high level of communication might require the student to assemble information from various different sources to understand the problem, and the student might have to write a response that explains several steps of thinking through a problem. This research has resulted in sharper definitions of the fundamental mathematical capabilities at each of four levels. A composite score has been shown to be a strong predictor of PISA item difficulty. These fundamental mathematical capabilities are evident across the content categories, and are used to varying degrees in each of the three mathematical processes used in the reporting. The PISA framework (OECD, 2013c) describes this in detail. The seven fundamental mathematical capabilities used in the PISA 2012 assessment are described as follows: Communication is both receptive and expressive. Reading, decoding and interpreting statements, questions, tasks or objects enables the individual to form a mental model of the situation. Later, the problem-solver may need to present or explain the solution. Mathematising involves moving between the real world and the mathematical world. It has two parts: formulating and interpreting. Formulating a problem as a mathematical problem can include structuring, conceptualising, making assumptions and/or constructing a model. Interpreting involves determining whether and how the results of mathematical work are related to the original problem and judging their adequacy. It directly relates to the formulate interpret and processes of the framework. Representation entails selecting, interpreting, translating between and using a variety of representations to capture a situation, interact with a problem, or present one’s work. The representations referred to include graphs, tables, diagrams, pictures, equations, formulae, textual descriptions and concrete materials. is required throughout the different stages and activities associated with mathematical literacy. Reasoning and argument This capability involves thought processes rooted in logic that explore and link problem elements so as to be able to make inferences from them, check a justification that is given, or provide a justification of statements or solutions to problems. Devising strategies for solving problems is characterised as selecting or devising a plan or strategy to use mathematics to solve problems arising from a task or context, and guiding and monitoring its implementation. It involves seeking links between diverse data presented so that the information can be combined to reach a solution efficiently. involves understanding, interpreting, manipulating and Using symbolic, formal and technical language and operations making use of symbolic and arithmetic expressions and operations, using formal constructs based on definitions, rules and formal systems, and using algorithms with these entities. involves knowing about and being able to use various tools (physical or digital) that may Using mathematical tools assist mathematical activity, and knowing about the limitations of such tools. The optional computer-based component of the PISA 2012 mathematics assessment has expanded the opportunities for students to demonstrate their ability to use mathematical tools. CS eading and S S Kno W and Can d o: Student Performan C e in m athemati What Student , 39 OECD 2014 © i e – Volume C ien C r

42 2 o A nce i n mA them A tic f Student Perform A Profile S Paper-based and computer-based media PISA 2012 supplemented the paper-based assessment with an optional computer-based assessment, in which specially designed PISA units were presented on a computer and students responded on the computer. Thirty-two of the 65 participating countries and economies participated in this computer-based assessment. For these countries and economies, results are reported for the paper-based assessment scale and supplemented with a computer-based scale and a combined paper-and-computer scale (see Annex B3). The design of the computer-based assessment ensures that mathematical reasoning and processes take precedence over mastery of using the computer as a tool. Each computer-based item involves three aspects: • the mathematical demand (as for paper -based items); • the gener al knowledge and skills related to information and communication technologies (ICT) that are required (e.g. using keyboard and mouse, and knowing common conventions, such as arrows to move forward). These are intentionally kept to a minimum; • competencies related to the inter action of mathematics and ICT, such as making a pie chart from data using a simple “wizard”, or planning and implementing a sorting strategy to locate and collect desired data in a spreadsheet. Response types The response types distinguish between selected response items and constructed response items. Selected response items include simple multiple choice, complex multiple choice, in which students must select correct answers to a series of multiple-choice items, and, for computer-based items, “selected response variations”, such as selecting from options in a drop-down box. Constructed response items include those that can be scored routinely (such as a single number or simple phrase, or, for computer-based items, those for which the response can be captured and processed automatically), and others that need expert scoring (e.g. responses that include an explanation or a long calculation). Examples of items representing the different framework categories Figure I.2.8 summarises the six categories constructed to create a balanced assessment. Three of the six – process, content and medium – are reporting categories. As noted before, PISA 2012 reports scores separately for the three process categories. Since PISA questions are set in real contexts, they usually involve multiple processes, contents and contexts. It is necessary to make judgements about the major source of demand in order to allocate items to just one of the categories for process, content and context, even though the items are multi-faceted. The items are allocated to the category that reflects the highest cognitive focus of the item. • Figure I.2.8 • a S 2012 mathematics assessment ategories describing the items constructed for the c pi urther categories to ensure balanced assessment r eporting categories f esponse types ontext categories c edium categories m ontent categories c Process categories r ognitive demand c Quantity Personal Formulating situations Multiple choice Empirical difficulty mathematically Paper-based (continuum) Uncertainty and data Societal Employing mathematical concepts, Complex multiple facts, procedures, and choice Change and Occupational Across reasoning relationships fundamental Computer-based Interpreting, applying mathematical Constructed and evaluating capabilities response (simple, mathematical Space and shape Scientific elaborated) outcomes The PISA 2012 mathematics assessment includes the same proportion of items from each of the categories content, , half reflect the context and response type. A quarter of the items in the assessment reflect the process formulating interpreting employing process . To measure the full range of student performance, the , and a quarter reflect the process set of items reflects all levels of difficulty. Figure I.2.9 summarises how several sample items (see at the end of this chapter) are categorised. o: Student Performan e – Volume © OECD 2014 What Student S Kno W and Can d C C e in m athemati CS , r eading and S C ien i 40

43 2 nce i n mA them A tic S A A Profile o f Student Perform Figure I.2.9 • • c lassification of sample items, by process, context and content categories and response type tem/Question i ontext category c ontent category c Process category scale) a S i (position on P esponse type r W HICH CAR ? Interpret Uncertainty and data – Personal Simple Multiple Choice Question 01 (327.8) Quantity – ? CAR Simple Multiple Choice HICH W Personal Employ Question 02 (490.9) HICH CAR ? – W Employ Quantity Personal Constructed Response Manual Question 03 (552.6) HARTS – Interpret Uncertainty and data Societal Simple Multiple Choice C Question 01 (347.7) C – Interpret Uncertainty and data Societal Simple Multiple Choice HARTS Question 02 (415.0) C Simple Multiple Choice Societal Uncertainty and data Employ – HARTS Question 05 (428.2) Space and shape Interpret – ARAGE G Occupational Simple Multiple Choice Question 01 (419.6) ARAGE Occupational – Employ Space and shape Constructed Response Expert G Question 02 (687.3) CLIS Simple Multiple Choice C y ELEN H T – Personal Employ Change and relationships THE Question 01 (440.5) Employ – CLIS y C THE ELEN H T Simple Multiple Choice Personal Change and relationships Question 02 (510.6) Personal ELEN H THE C y CLIS T – Employ Constructed Response Manual Change and relationships Question 03 (696.6) UJI Simple Multiple Choice LIMBING M OUNT F C – Societal Formulate Quantity Question 01 (464.0) – UJI Formulate Change and relationships Societal Constructed Response Expert F OUNT M LIMBING C Question 02 (641.6) Constructed Response Manual LIMBING M OUNT F UJI – Employ Quantity Societal C Question 03 (610.0) R DOOR – EVOLVING Employ Space and shape Scientific Constructed Response Manual Question 01 (512.3) EVOLVING Constructed Response Expert R Space and shape Formulate Scientific – DOOR Question 02 (840.3) EVOLVING R Simple Multiple Choice Scientific Quantity Formulate – DOOR Question 03 (561.3) Example 1: WHICH CAR? The unit, “W HICH CAR ?”, (Figure I.2.10) consists of three questions. It presents a table of data that a person might use to c hoose a car and make sure that she can afford it. Context: Because buying a car is an experience that many people might have during their lifetimes, all three questions were allocated to the personal context category. Response type: Question 1 and Question 2 are simple multiple-choice questions; Question 3, which asks for a single number, is a constructed response item that does not require expert scoring. content category. The item requires knowledge of the basic uncertainty and data Content: Question 1 was allocated to the row-column conventions of a table, as well as co-ordinated data-handling ability to identify where the three conditions are simultaneously satisfied. While the solution also requires basic knowledge of large whole numbers, that knowledge quantity is unlikely to be the main source of difficulty in the item. In contrast, Question 2 has been allocated to the content category because it is well known that even at age 15, many students have misconceptions about the base ten and place value ideas required to order “ragged” decimal numbers. Question 3 is also allocated to the quantity content category because the calculation of 2.5% is expected to require more cognitive effort from students than identifying the correct data in the table. The difficulty for this age group in dealing with decimal numbers and percentages is reflected in the empirical results: Question 1 is considered an easy item, Question 2 is close to the international average, and Question 3 is of above-average difficulty. What Student OECD 2014 © i e – Volume C ien C eading and S r 41 CS athemati m e in C o: Student Performan d and Can W Kno S ,

44 2 f Student Perform nce i n mA them A tic S A Profile o A Process: In allocating the items to process categories, their relation to “real-world” problems has been taken into consideration. The primary demand in items in the formulate category is the transition from the real-world problem to the mathematical problem; in the employ category, the primary demand is within the mathematical world; and in the interpret category, an item’s primary demand is in using mathematical information to provide a real-world solution. category. This is because in both of these items, the main cognitive effort Questions 2 and 3 are allocated to the employ is made within mathematics: decimal notation and the calculation of a percentage. In Question 1, the construction of a table of data, including the need to identify key variables, is a mathematisation of a real situation. Question 1 is allocated category because it requires these mathematical entities to be interpreted in relation to the real world. to the interpret • Figure I.2.10 • pi ar? – a unit from the c Which 2012 main survey a S which car? Chris has just received her car driving licence and wants to buy her first car. This table below shows the details of four cars she finds at a local car dealer. Dezal Alpha Castel m odel: Bolte 2000 1999 2001 y ear 2003 4 450 4 800 3 990 4 250 dvertised price (zeds) a istance travelled d 105 000 115 000 128 000 109 000 (kilometres) 1.796 1.82 1.783 Engine capacity (litres) 1.79 WHICH CAR? ESTI – Qu on 2 WHICH CAR? – Qu on 1 ESTI Chris wants a car that meets all of these conditions: Which car’s engine capacity is the smallest? A. Alpha higher than 120 000 • The distance travelled is not kilometres. B. Bolte • It was made in the year 2000 or a later year. C. Castel • The advertised price is not higher than 4 500 zeds. Dezal D. • Which car meets Chris’s conditions? on 3 WHICH CAR? – Qu ESTI Alpha A. Chris will have to pay an extra 2.5% of the advertised B. Bolte cost of the car as taxes. Castel C. How much are the extra taxes for the Alpha? D. Dezal Extra taxes in zeds: ... Example 2: CLIMBING MOUNT FUJI M LIMBING OUNT F UJI ”, containing three questions, as shown in Figure I.2.11, was allocated to Context: The unit “C . Question 1 goes beyond the personal concerns of a walker to wider community issues – the societal context category in this case, concerns about use of the public trail. Items classified as societal involve such things as voting systems, public transport, government, public policies, demographics, advertising, national statistics and economics. Although individuals can be personally involved in these, the focus of the problem is more on the community perspective. Response: Question 1 is simple multiple choice (choose one out of four). Question 2 requires the answer 11 a.m. and as such, is a constructed response with expert scoring to ensure that all equivalent ways of writing the time are considered. Question 3 requires the number 40 for full score, or the number 0.4 (answering in metres) for partial credit. It, too, is a constructed response with expert scoring. Content: Question 1 requires calculating the number of days open using the given dates, and then calculating an quantity content category because it involves quantification of time and of an average. The question was allocated to the average. While the formula for average is required, and this is indeed a relationship, since this question requires use of an average to calculate the number of people per day, rather than focus on the relationship, this question is not allocated to the change and relationships category. Question 3 has similar characteristics, involving units of length. Question 2 is category because the relationship between distance and time, encapsulated as change and relationships allocated to the e in © OECD 2014 What Student S Kno W and Can d o: Student Performan C m athemati CS , r eading and S C ien C e – Volume i 42

45 2 o nce i n mA them A tic S A A Profile f Student Perform speed, is paramount. From information about distances and speed, the time to go up and the time to come down have to be quantified, and then used in combination with the finishing time to get the starting time. Had the time needed to go up and down been given directly, rather than indirectly through distance and speed, then the question could have been allocated to the quantity category. Figure I.2.11 • • m c ount f uji – a unit from the field trial limbing c i L i NG MOUNT FUJ i MB Mount Fuji is a famous dormant v olcano in Japan on 3 ESTI – Qu CLIMBING MOUNT FUJI CLIMBING MOUNT FUJI on 1 ESTI – Qu Toshi wore a pedometer to count his steps on his walk Mount Fuji is only open to the public for climbing from 1 July to 27 August each year. About 200 000 along the Gotemba trail. people climb Mount Fuji during this time. His pedometer showed that he walked 22 500 steps on the way up. On average, about how many people climb Mount Fuji Estimate Toshi’s average step length for his walk up each day? the 9 km Gotemba trail. Give your answer A. 340 in centimetres (cm). B. 710 Answer: ... cm 3 400 C. 7 100 D. E. 7 400 on 2 – Qu CLIMBING MOUNT FUJI ESTI The Gotemba walking trail up Mount Fuji is about 9 kilometres (km) long. Walkers need to return from the 18 km walk by 8 p.m. Toshi estimates that he can walk up the mountain at 1.5 kilometres per hour on average, and down at twice that speed. These speeds take into account meal breaks and rest times. Using Toshi’s estimated speeds, what is the latest time he can begin his walk so that he can return by 8 p.m.? ... Process: Question 1 was allocated to the formulating category because most of the cognitive effort in this relatively easy item requires taking two pieces of real-world information (open season and total number of climbers) and establishing a mathematical problem to be solved: find the length of the open season from the dates and use it with the information about the total number of climbers to find the average number of climbers each day. Expert judgement is that the major cognitive demand for 15-year-olds lies in this movement from the real world problem to the mathematical relationships, rather than in the ensuing whole number calculations. Question 2 was also allocated to the formulating process category for the same reason: the main cognitive effort required is to translate real-world data into a mathematical problem and identify all the relationships involved, rather than calculate or interpret the answer as a starting time of 11 a.m. In this difficult item, the mathematical structure involves multiple relationships: starting time = finishing time – duration; duration = time up + time down; time up (down) = distance/speed (or equivalent proportional reasoning); time down = half time up; and appreciating the simplifying assumptions that average speeds already include consideration of variable speed during the day and that no further allowance is required for breaks. C o: Student Performan d and Can W Kno S What Student i e – Volume C ien OECD 2014 eading and S r , CS athemati © 43 m e in C

46 2 A Profile f Student Perform A nce i n mA them A tic S o By contrast, Question 3 was allocated to the employing category. There is one main relationship involved: the distance walked = number of steps × average step length. There are two obstacles to using this relationship to solve the problem: rearranging the formula (which is probably done by students informally rather than formally using the written relationship) so that the average step length can be found from distance and number of steps; and making appropriate unit conversions. The main cognitive effort required for this question is in carrying out these steps, rather than identifying the answer in real-world terms. the relationships and assumptions to be made (the formulating process) or interpreting How the PISA 2012 mathematics results are reported How the PISA 2012 mathematics tests were designed, analysed and scaled The test material had to meet several requirements: • T est items had to meet the requirements and specifications of the framework for PISA 2012 that was established and agreed upon by the participating countries. The content, processes and contexts of the items had to be deemed appropriate for a test of 15-year-olds. ance for 15-year-olds in participating countries and economies. • Items had to be of interest and of curricular relev hnical quality and international comparability. • Items had to meet stringent standards of tec Items for the assessment were selected from a pool of diverse material with a diverse range of sources (authors in almost 30 different countries, with the contributions from national teams, members of the PISA mathematics expert group and the PISA Project Consortium) that reflected content, context and approaches relevant to a large number of PISA-participating countries and economies. Wordings and other features of the items were reviewed by experts, then the items were tested among classes of 15-year-old students, and finally the items underwent extensive field trials in all countries and economies that would ultimately use the material. Each participating country and economy provided detailed feedback on the curricular relevance, appropriateness and potential interest for 15-year-olds, by local mathematics experts. At each development stage, material was considered for rejecting, revising or keeping in the pool of potential items. Finally, the international mathematics expert group formulated recommendations as to which items should be included in the survey instruments and those recommendations were considered by the PISA Governing Board, in which governments of all participating countries are represented. The final selection of test items was balanced across the various categories specified in the mathematics framework and spanned a range of levels of difficulty, so that the entire pool of items could measure performance across a broad range of content, processes and contexts, and across PISA 2012 Technical Report [OECD, forthcoming]). a wide range of student abilities (for further details, see the Test items were generally developed within “units” that included some stimulus material and one or more questions related to the stimulus. In many cases, students were required to construct a response to questions, based on their analysis, calculations and mathematical thinking. Some constructed-response items were relatively open-ended, requiring students to present an extended response that may have included presenting the steps of their solution or some explanation of their result, which thus revealed aspects of the methods and thought processes they had used to answer the question. In general, these items could not be machine scored; rather they required the professional judgement of trained coders to assign the responses to defined response categories. To ensure that the response coding process yielded reliable and cross-nationally comparable results, detailed guidelines and training were provided. All the procedures PISA 2012 Technical Report ensuring the consistency of the coding within and between countries are detailed in (OECD, forthcoming). In other cases requiring students to construct their response, only a very simple response was required, such as a value read from a graph or table, or writing a word, short phrase or the numerical result of a calculation. The evaluation of these answers was restricted to the response itself and did not take into account an explanation of how the response was derived. Responses could often be processed without the intervention of a coding expert. The use of computer-delivered test forms also allowed for a number of response formats such that responses could be captured relatively easily by computer without any additional intervention. Other items were presented in a format that required students to select one or more responses from a set of given response options. This format category includes both standard multiple-choice items, for which students were required to select one correct response from a number of given response options; and complex multiple choice items, for which students were required to select a response from given optional responses to each of a number of propositions or questions. Responses to these items could be processed automatically, with no intervention by an expert coder needed. o: Student Performan athemati © OECD 2014 What Student S Kno W and Can d CS C e in i e – Volume C ien C eading and S r , m 44

47 2 o A nce i n mA f Student Perform A tic S A Profile them The final PISA 2012 survey included 36 paper-based items linking to previous PISA survey instruments, 74 new paper- based items and 41 new computer-based items. Each student completed a fraction of the paper-based items – a minimum of 12 items, up to a maximum of 37 items, depending on which test booklet they were randomly assigned from the booklet rotation design. The mathematics questions selected for inclusion in the paper-based component of the survey were arranged into half-hour clusters of 12-13 items. These, along with clusters of reading and science questions, were assembled into test booklets, each containing four clusters. Each participating student was assigned a test booklet to be completed in two hours. In the computer-based survey, students completed a one-hour test composed of two half-hour components selected from a rotated design of mathematics, reading and problem-solving item clusters. The test design, similar to those used in previous PISA assessments, makes it possible to construct a single scale of proficiency in mathematics, so that each question is associated with a particular point on the scale that indicates its difficulty, and each test-taker’s performance is associated with a particular point on the same scale that indicates his or her estimated mathematical proficiency. A description of the modelling technique used to construct this scale can be found in the (OECD, forthcoming). PISA 2012 Technical Report The relative difficulty of tasks in a test is estimated by considering the proportion of test-takers who answer each question correctly; and the relative proficiency of individuals taking a particular test can be estimated by considering the proportion of test questions they answer correctly. A single continuous scale shows the relationship between the difficulty of questions and the proficiency of test-takers. By constructing a scale that shows the difficulty of each question, it is possible to locate the level of mathematics that the question demands. By showing the proficiency of each test-taker that each test taker possesses. on the same scale, it is possible to describe the level of mathematics The location of different described levels of mathematical proficiency on this scale is set in relation to the particular group of questions used in the assessment; but just as the sample of students who sat the PISA test in 2012 was drawn to represent all 15-year-old students in the participating countries and economies, so the individual test questions used in the assessment were designed to represent the definition of literacy in mathematics adequately. Estimates of student proficiency reflect the kinds of tasks students would be expected to perform successfully. This means that students are likely to be able to successfully complete questions located at or below the difficulty level associated with their own position on the scale. Conversely, they are unlikely to be able to successfully complete questions above the difficulty level associated with their position on the scale. Figure I.2.12 illustrates how this probabilistic model works. The higher an individual’s proficiency level is located above a given test question, the more likely is he or she to successfully complete the question (and other questions of similar difficulty); the further the individual’s proficiency is located below a given question, the less likely is he or she to be able to successfully complete the question and other questions of similar difficulty. Figure I.2.12 • • t he relationship between questions and student performance on a scale Mathematical literacy scale Item VI We expect student A to successfully Student A, with Items with complete items I to V, and probably relatively high relatively high difculty item VI as well. prociency Item V Item IV We expect student B to successfully Student B, complete items I and II, and probably Items with with moderate item III as well; but not items V and VI, moderate difculty prociency and probably not item IV either. Item III Item II We expect student C to be unable to Student C, Items with successfully complete any of items II to VI, with relatively relatively low difculty low prociency and probably not item I either. Item I m athemati CS , r eading and S © C ien C e – Volume 45 OECD 2014 What Student Kno W and Can d S o: Student Performan C e in i

48 2 o A nce i n mA f Student Perform A tic S A Profile them How mathematics proficiency levels are defined in PISA 2012 PISA 2012 provides an overall mathematics scale, which draws on all of the mathematics questions in the assessment, as well as scales for the three mathematical processes and the four mathematical content categories defined above. The metric for the overall mathematics scale is based on a mean for OECD countries of 500 points and a standard deviation of 100 points that were set in PISA 2003 when the first PISA mathematics scale was first developed. The items that were common to both the 2003 and 2012 test instruments enable a link to be made with the earlier scale. To help users interpret what student scores mean in substantive terms, the scale is divided into proficiency levels. For PISA 2012, the range of difficulty of the tasks is represented by six levels of mathematical proficiency that are aligned with the levels used in describing the outcomes of PISA 2003. The levels range from the lowest, Level 1, to the highest, Level 6. Descriptions of each of these levels have been generated, based on the framework-related cognitive demands imposed by tasks that are located within each level, to describe the kinds of knowledge and skills needed to successfully complete those tasks, and which can then be used as characterisations of the substantive meaning of each level. Individuals with proficiency within the range of Level 1 are likely to be able to complete Level 1 tasks, but are unlikely to be able to complete tasks at higher levels. Level 6 reflects tasks that pose the greatest challenge in terms of the mathematical knowledge and skills needed to complete them successfully. Individuals with scores in this range are likely to be able to complete tasks located at that level, as well as all the other PISA mathematics tasks (see section Students for a detailed description of the proficiency levels in mathematics). at the different levels of proficiency in mathematics S Student performance in mathematic PISA outcomes are reported in a variety of ways. This section gives the country results and shows the location of items on the overall PISA mathematics scale described above, how the different levels of proficiency in PISA mathematics can be characterised, and how these proficiency levels are represented by mathematics questions used in the survey. In subsequent sections, mathematical performance will be examined in more detail in relation to: the process categories employing , formulating referred to as and change , quantity , space and shape ; and the content categories of interpreting . uncertainty and data , and and relationships Average in mathematics performance This section compares the countries and economies on the basis of their average mathematics scores. In addition, changes in the relative standing of countries since the 2003 survey – the most recent assessment in which mathematics was the major PISA domain – are presented. The country results are estimates because they are obtained from samples of students, rather than from a census of all students, and they are obtained using a limited set of assessment tasks, not a population of all possible assessment tasks. When the sampling and assessment are done with scientific rigour it is possible to determine the magnitude of the probable uncertainty associated with the estimates. This uncertainty needs to be taken into account when making comparisons so that differences that could reasonably arise simply due to the sampling of students and items are not interpreted as differences that actually hold for the populations. A difference is called statistically significant if it is very unlikely that such a difference could be observed by chance, when in fact no true difference exists. When interpreting mean performance, only those differences among countries and economies that are statistically significant should be taken into account. Figure I.2.13 shows each country’s/economy’s mean score and also for which groups of countries/economies the differences between the means are statistically significant. For each country/economy shown in the middle column, the countries/economies whose mean scores are not statistically significantly different are listed in the right column. In all other cases, country/economy A scores higher than country/economy B if country/ economy A is situated above country/economy B in the middle column, and scores lower if country/economy A is situated below country/economy B. Figure I.2.13 lists each participating country and economy in descending order of its mean mathematics score (left column). The values range from a high of 613 points for the partner economy - China to a low of 368 points for the partner country Peru. Shanghai Countries and economies are also divided into three broad groups: those whose mean scores are statistically around the OECD mean (highlighted in dark blue), those whose mean scores are above the OECD mean (highlighted in pale blue), and those whose mean scores are below the OECD mean (highlighted in medium blue). Across OECD countries, the average score in mathematics is 494 points (see Table I.2.3a). To gauge the magnitude of score differences, 41 score points corresponds to the equivalent of one year of formal schooling (see Annex A1, Table A1.2). o: Student Performan r © OECD 2014 What Student S Kno W and Can d eading and S C e in m athemati CS i e – Volume C ien C , 46

49 2 f Student Perform nce i n mA them A tic S A Profile o A Figure I.2.13 • • omparing countries’ and economies’ performance in mathematics c above the OECD average Statistically significantly Not statistically significantly different from the OECD average the OECD average below Statistically significantly m ean c omparison c score statistically significantly different from that comparison country’s/economy’s score country/economy not ountries/economies whose mean score is 613 Shanghai-China 573 Singapore Hong Kong-China 561 Chinese Taipei, Korea 560 Chinese Taipei Hong Kong-China, Korea Hong Kong-China, Chinese Taipei 554 Korea 538 Macao-China Japan, Liechtenstein 536 Japan Macao-China, Liechtenstein, Switzerland 535 Liechtenstein Macao-China, Japan, Switzerland Japan, Liechtenstein, Netherlands Switzerland 531 523 Netherlands Switzerland, Estonia, Finland, Canada, Poland, Viet Nam Estonia 521 Netherlands, Finland, Canada, Poland, Viet Nam Finland Netherlands, Estonia, Canada, Poland, Belgium, Germany, Viet Nam 519 518 Canada Netherlands, Estonia, Finland, Poland, Belgium, Germany, Viet Nam Netherlands, Estonia, Finland, Canada, Belgium, Germany, Viet Nam 518 Poland 515 Belgium Finland, Canada, Poland, Germany, Viet Nam 514 Germany Finland, Canada, Poland, Belgium, Viet Nam 511 Netherlands, Estonia, Finland, Canada, Poland, Belgium, Germany, Austria, Australia, Ireland Viet Nam 506 Viet Nam, Australia, Ireland, Slovenia, Denmark, New Zealand, Czech Republic Austria Australia 504 Viet Nam, Austria, Ireland, Slovenia, Denmark, New Zealand, Czech Republic 501 Ireland Viet Nam, Austria, Australia, Slovenia, Denmark, New Zealand, Czech Republic, France, United Kingdom 501 Slovenia Austria, Australia, Ireland, Denmark, New Zealand, Czech Republic 500 Austria, Australia, Ireland, Slovenia, New Zealand, Czech Republic, France, United Kingdom Denmark 500 New Zealand Austria, Australia, Ireland, Slovenia, Denmark, Czech Republic, France, United Kingdom Austria, Australia, Ireland, Slovenia, Denmark, New Zealand, France, United Kingdom, Iceland 499 Czech Republic Ireland, Denmark, New Zealand, Czech Republic, United Kingdom, Iceland, Latvia, Luxembourg, Norway, Portugal France 495 United Kingdom 494 Ireland, Denmark, New Zealand, Czech Republic, France, Iceland, Latvia, Luxembourg, Norway, Portugal Czech Republic, France, United Kingdom, Latvia, Luxembourg, Norway, Portugal Iceland 493 France, United Kingdom, Iceland, Luxembourg, Norway, Portugal, Italy, Spain Latvia 491 France, United Kingdom, Iceland, Latvia, Norway, Portugal Luxembourg 490 489 France, United Kingdom, Iceland, Latvia, Luxembourg, Portugal, Italy, Spain, Russian Federation, Slovak Republic, United States Norway 487 France, United Kingdom, Iceland, Latvia, Luxembourg, Norway, Italy, Spain, Russian Federation, Slovak Republic, United States, Lithuania Portugal 485 Italy Latvia, Norway, Portugal, Spain, Russian Federation, Slovak Republic, United States, Lithuania Spain Latvia, Norway, Portugal, Italy, Russian Federation, Slovak Republic, United States, Lithuania, Hungary 484 482 Norway, Portugal, Italy, Spain, Slovak Republic, United States, Lithuania, Sweden, Hungary Russian Federation 482 Slovak Republic Norway, Portugal, Italy, Spain, Russian Federation, United States, Lithuania, Sweden, Hungary Norway, Portugal, Italy, Spain, Russian Federation, Slovak Republic, Lithuania, Sweden, Hungary 481 United States 479 Lithuania Portugal, Italy, Spain, Russian Federation, Slovak Republic, United States, Sweden, Hungary, Croatia Russian Federation, Slovak Republic, United States, Lithuania, Hungary, Croatia Sweden 478 477 Hungary Spain, Russian Federation, Slovak Republic, United States, Lithuania, Sweden, Croatia, Israel 471 Croatia Lithuania, Sweden, Hungary, Israel 466 Israel Hungary, Croatia 453 Greece Serbia, Turkey, Romania 449 Serbia Greece, Turkey, Romania, Bulgaria 1, 2 , Bulgaria 448 Turkey Greece, Serbia, Romania, Cyprus 1, 2 , Bulgaria Greece, Serbia, Turkey, Cyprus Romania 445 1, 2 Turkey, Romania, Bulgaria Cyprus 440 1, 2 , United Arab Emirates, Kazakhstan Serbia, Turkey, Romania, Cyprus Bulgaria 439 Bulgaria, Kazakhstan, Thailand United Arab Emirates 434 Kazakhstan 432 Bulgaria, United Arab Emirates, Thailand Thailand United Arab Emirates, Kazakhstan, Chile, Malaysia 427 Chile Thailand, Malaysia 423 Malaysia 421 Thailand, Chile Mexico Uruguay, Costa Rica 413 410 Montenegro Uruguay, Costa Rica 409 Uruguay Mexico, Montenegro, Costa Rica Mexico, Montenegro, Uruguay Costa Rica 407 Albania Brazil, Argentina, Tunisia 394 Brazil 391 Albania, Argentina, Tunisia, Jordan 388 Albania, Brazil, Tunisia, Jordan Argentina Tunisia 388 Albania, Brazil, Argentina, Jordan Brazil, Argentina, Tunisia Jordan 386 376 Colombia Qatar, Indonesia, Peru 376 Qatar Colombia, Indonesia Colombia, Qatar, Peru Indonesia 375 368 Peru Colombia, Indonesia 1. Note by Turkey: The information in this document with reference to ”Cyprus” relates to the southern part of the Island. There is no single authority representing both Turkish and Greek Cypriot people on the Island. Turkey recognises the Turkish Republic of Northern Cyprus (TRNC). Until a lasting and equitable solution is found within the context of the United Nations, Turkey shall preserve its position concerning the “Cyprus issue”. 2. Note by all the European Union Member States of the OECD and the European Union: The Republic of Cyprus is recognised by all members of the United Nations with the exception of Turkey. The information in this document relates to the area under the effective control of the Government of the Republic of Cyprus. Source: OECD, PISA 2012 Database. http://dx.doi.org/10.1787/888932935572 12 CS OECD 2014 S Kno W and Can d o: Student Performan C e in m athemati What Student , r eading and S C ien C e – Volume i © 47

50 2 A Profile i n mA them A tic S nce o A f Student Perform • ] Part 1/3 • Figure I.2.14 [ pi athematics performance among m 2012 participants, at national and regional levels a S Mathematics scale r ange of ranks E cd countries a ll countries/economies o m ean score ower rank u pper rank l ower rank u pper rank l 1 613 1 Shanghai-China Singapore 2 2 573 561 3 Hong Kong-China 5 3 560 Chinese Taipei 5 554 Korea 1 1 3 5 8 538 6 Macao-China 2 536 Japan 9 6 3 Liechtenstein 6 9 535 531 2 3 7 9 Switzerland Flemish community (Belgium) 531 Trento (Italy) 524 Friuli Venezia Giulia (Italy) 523 14 9 7 3 523 Netherlands Veneto (Italy) 523 Estonia 521 14 10 8 4 15 519 4 9 10 Finland 16 Canada 518 5 9 11 Australian Capital Territory (Australia) 518 10 518 4 10 17 Poland 517 Lombardia (Italy) avarre (Spain) 517 n 516 Western Australia (Australia) 17 515 7 10 13 Belgium Germany 13 514 6 17 10 514 nited States) u Massachusetts ( 11 Viet Nam 511 19 German-speaking community (Belgium) 511 509 ew South Wales (Australia) n Castile and Leon (Spain) 509 506 Bolzano (Italy) u nited States) 506 Connecticut ( Austria 506 10 14 17 22 505 Basque Country (Spain) 21 17 14 11 504 Australia Madrid (Spain) 504 503 Queensland (Australia) La Rioja (Spain) 503 24 18 17 Ireland 11 501 501 19 16 23 12 Slovenia 501 Victoria (Australia) 500 Emilia Romagna (Italy) 18 12 500 Denmark 25 19 25 500 12 18 19 New Zealand 500 Asturias (Spain) 19 19 12 499 Czech Republic 26 499 Piemonte (Italy) 498 nited Kingdom) u Scotland ( 496 Marche (Italy) 496 Aragon (Spain) Toscana (Italy) 495 495 nited Kingdom) u England ( 29 France 495 16 21 23 31 23 23 16 United Kingdom 494 493 French community (Belgium) Catalonia (Spain) 493 22 25 29 Iceland 493 18 mbria (Italy) 493 u 492 Valle d’Aosta (Italy) 491 Cantabria (Spain) Latvia 32 25 491 27 31 Luxembourg 490 20 23 33 25 489 19 26 Norway South Australia (Australia) 489 OECD countries are shown in bold black. Partner countries are shown in bold blue. Participating economies and subnational entities that are not included in national results Notes: are shown in bold blue italics. Regions are shown in black italics (OECD countries) or blue italics (partner countries). 1. Note by Turkey: The information in this document with reference to ”Cyprus” relates to the southern part of the Island. There is no single authority representing both Turkish and Greek Cypriot people on the Island. Turkey recognises the Turkish Republic of Northern Cyprus (TRNC). Until a lasting and equitable solution is found within the context of the United Nations, Turkey shall preserve its position concerning the “Cyprus issue”. 2. Note by all the European Union Member States of the OECD and the European Union: The Republic of Cyprus is recognised by all members of the United Nations with the exception of Turkey. The information in this document relates to the area under the effective control of the Government of the Republic of Cyprus. Countries, economies and subnational entities are ranked in descending order of mean mathematics performance. OECD, PISA 2012 Database. Source: http://dx.doi.org/10.1787/888932935572 12 o: Student Performan e – Volume © OECD 2014 What Student S Kno W and Can d C C e in m athemati CS , r eading and S C ien i 48

51 2 A i n mA them A nce S A Profile o f Student Perform tic ] Part 2/3 [ Figure I.2.14 • • m athematics performance among pi S a 2012 participants, at national and regional levels Mathematics scale ange of ranks r ll countries/economies a o E cd countries ean score m u pper rank l ower rank u pper rank l ower rank Alentejo (Portugal) 489 489 Galicia (Spain) Liguria (Italy) 488 26 27 36 Portugal 487 19 u orthern Ireland ( n 487 nited Kingdom) 35 30 27 22 485 Italy 36 31 27 23 484 Spain Perm Territory region (Russian Federation) 484 482 31 39 Russian Federation Slovak Republic 482 23 29 31 39 United States 481 23 29 31 39 34 Lithuania 479 40 40 35 29 26 478 Sweden Puglia (Italy) 478 478 Tasmania (Australia) 40 477 26 30 35 Hungary Abruzzo (Italy) 476 Balearic Islands (Spain) 475 475 Lazio (Italy) 472 Andalusia (Spain) Croatia 471 38 41 Wales ( u nited Kingdom) 468 467 nited States) u Florida ( 40 41 466 30 29 Israel Molise (Italy) 466 Basilicata (Italy) 466 464 nited Arab Emirates) u Dubai ( 462 Murcia (Spain) Extremadura (Spain) 461 Sardegna (Italy) 458 44 453 31 32 42 Greece Campania (Italy) 453 orthern Territory (Australia) 452 n 45 449 42 Serbia 46 Turkey 448 31 32 42 Sicilia (Italy) 447 43 Romania 445 47 1, 2 Cyprus 45 47 440 nited Arab Emirates) u Sharjah ( 439 45 439 49 Bulgaria Aguascalientes (Mexico) 437 uevo León (Mexico) 436 n 435 Jalisco (Mexico) 434 Querétaro (Mexico) United Arab Emirates 434 47 49 Kazakhstan 50 47 432 Calabria (Italy) 430 429 Colima (Mexico) Chihuahua (Mexico) 428 428 Distrito Federal (Mexico) 52 427 49 Thailand Durango (Mexico) 424 52 50 33 33 423 Chile Morelos (Mexico) 421 u nited Arab Emirates) 421 Abu Dhabi ( 52 50 421 Malaysia 418 Coahuila (Mexico) Ciudad Autónoma de Buenos Aires (Argentina) 418 Mexico (Mexico) 417 Federal District (Brazil) 416 Ras Al Khaimah ( u nited Arab Emirates) 416 Santa Catarina (Brazil) 415 Puebla (Mexico) 415 OECD countries are shown in bold black. Partner countries are shown in bold blue. Participating economies and subnational entities that are not included in national results Notes: are shown in bold blue italics. Regions are shown in black italics (OECD countries) or blue italics (partner countries). 1. Note by Turkey: The information in this document with reference to ”Cyprus” relates to the southern part of the Island. There is no single authority representing both Turkish and Greek Cypriot people on the Island. Turkey recognises the Turkish Republic of Northern Cyprus (TRNC). Until a lasting and equitable solution is found within the context of the United Nations, Turkey shall preserve its position concerning the “Cyprus issue”. 2. Note by all the European Union Member States of the OECD and the European Union: The Republic of Cyprus is recognised by all members of the United Nations with the exception of Turkey. The information in this document relates to the area under the effective control of the Government of the Republic of Cyprus. Countries, economies and subnational entities are ranked in descending order of mean mathematics performance. OECD, PISA 2012 Database. Source: http://dx.doi.org/10.1787/888932935572 12 CS OECD 2014 S Kno W and Can d o: Student Performan C e in m athemati What Student , r eading and S C ien C e – Volume i © 49

52 2 A i n mA them A nce S A Profile o f Student Perform tic ] Part 3/3 [ Figure I.2.14 • • m athematics performance among pi S a 2012 participants, at national and regional levels Mathematics scale ange of ranks r a o E cd countries ll countries/economies m ean score l u pper rank l ower rank u pper rank ower rank 415 Baja California (Mexico) 414 Baja California Sur (Mexico) 414 Espírito Santo (Brazil) 414 n ayarit (Mexico) 34 54 Mexico 34 53 413 412 San Luis Potosí (Mexico) 412 Guanajuato (Mexico) Tlaxcala (Mexico) 411 Tamaulipas (Mexico) 411 Sinaloa (Mexico) 411 nited Arab Emirates) 411 u Fujairah ( Quintana Roo (Mexico) 411 410 Yucatán (Mexico) 410 Montenegro 56 54 53 56 409 Uruguay Zacatecas (Mexico) 408 Mato Grosso do Sul (Brazil) 408 Rio Grande do Sul (Brazil) 407 56 407 54 Costa Rica 406 Hidalgo (Mexico) Manizales (Colombia) 404 404 São Paulo (Brazil) 403 Paraná (Brazil) nited Arab Emirates) u Ajman ( 403 Minas Gerais (Brazil) 403 Veracruz (Mexico) 402 nited Arab Emirates) u mm Al Quwain ( u 398 Campeche (Mexico) 396 395 Paraíba (Brazil) Albania 394 57 59 Medellin (Colombia) 393 Bogota (Colombia) 393 Brazil 60 57 391 Rio de Janeiro (Brazil) 389 57 388 Argentina 61 57 388 61 Tunisia 62 386 59 Jordan 385 Piauí (Brazil) 384 Sergipe (Brazil) 382 Rondônia (Brazil) Rio Grande do 380 n orte (Brazil) 379 Goiás (Brazil) 379 Cali (Colombia) 378 Tabasco Ceará (Brazil) 378 Colombia 64 62 376 Qatar 64 62 376 Indonesia 62 375 65 373 Bahia (Brazil) Chiapas (Mexico) 373 370 Mato Grosso (Brazil) 65 64 368 Peru Guerrero (Mexico) 367 366 Tocantins (Brazil) Pernambuco (Brazil) 363 362 Roraima (Brazil) 360 Amapá (Brazil) Pará (Brazil) 360 Acre (Brazil) 359 Amazonas (Brazil) 356 343 Maranhão (Brazil) Alagoas (Brazil) 342 OECD countries are shown in bold black. Partner countries are shown in bold blue. Participating economies and subnational entities that are not included in national results Notes: are shown in bold blue italics. Regions are shown in black italics (OECD countries) or blue italics (partner countries). 1. Note by Turkey: The information in this document with reference to ”Cyprus” relates to the southern part of the Island. There is no single authority representing both Turkish and Greek Cypriot people on the Island. Turkey recognises the Turkish Republic of Northern Cyprus (TRNC). Until a lasting and equitable solution is found within the context of the United Nations, Turkey shall preserve its position concerning the “Cyprus issue”. 2. Note by all the European Union Member States of the OECD and the European Union: The Republic of Cyprus is recognised by all members of the United Nations with the exception of Turkey. The information in this document relates to the area under the effective control of the Government of the Republic of Cyprus. Countries, economies and subnational entities are ranked in descending order of mean mathematics performance. OECD, PISA 2012 Database. Source: http://dx.doi.org/10.1787/888932935572 12 o: Student Performan e – Volume © OECD 2014 What Student S Kno W and Can d C C e in m athemati CS , r eading and S C ien i 50

53 2 f Student Perform A nce i n mA them A tic S o A Profile Figure I.2.14 shows how participating countries and economies compare in mathematics performance. Since a country’s score is based on an estimate of scores obtained from a sample of students, there is some degree of uncertainty associated with the estimates. Thus countries/economies are shown with the range of ranks they could occupy given this uncertainty. A number of countries designed their PISA samples so that it is possible to calculate performance averages for subnational entities as well. These subnational averages are also included in Figure I.2.14. Shanghai-China ranks first in mathematics performance followed by Singapore. Given the uncertainty inherent in the score estimates, Hong Kong-China could rank third, fourth or fifth among all participating countries and economies. Korea is the top ranking OECD country, but when all participating countries are taken into consideration, it could rank either third, fourth or fifth. Japan is the second listed OECD country (seventh among all countries and economies) with a rank of 2 or 3 among OECD countries (from 6 to 9 among all countries and economies); and Switzerland is the third listed OECD country (ninth among all countries and economies) with a rank also of 2 or 3 among OECD countries (and from 7 to 9 among all countries and economies). For entities other than those for which full samples were drawn, namely Chinese Taipei, Hong Kong-China, Macao-China and Shanghai-China, it is not possible to calculate a rank order; but the mean score provides the possibility of comparing subnational entities against the performance of countries and economies. For example, the Flemish Community of Belgium matches the performance of top-performer Switzerland. Similarly, the performance of the Italian provinces of Trento and Friuli Venezia Giulia, which is similar to that of the Netherlands, a high performer, is higher than the performance of the Italian province of Sicilia, which is similar to Turkey’s performance, by the equivalent of almost two full years of schooling. Trends in average mathematics performance Trends in average performance provide an indicator of how school systems are improving. Trends in mathematics are available for 64 countries and economies that participated in PISA 2012. Thirty-eight of these have mathematics performance for 2012 and the three remaining PISA assessments (2003, 2006 and 2009); seventeen have information for 2012 and two additional assessments and nine countries and economies have information for 2012 and one 6 previous assessment. To better understand a country or economy’s trend and maximise the number of countries in the comparisons, this report focuses on the annualised change in student performance . The annualised change is the average annual change in the observed period, taking into account all observations. For countries and economies that have participated in all four PISA assessments, the annualised change takes into account all four time points, and for those countries that have valid data for fewer assessments it only takes into account the valid and available information. The annualised change is a more robust measure of trends in performance because it is based on all the available information (as opposed to the difference between one particular year and 2012). It is scaled by years, so it is interpreted as the average annual change in performance over the observed period and allows for comparisons of mathematics performance of countries that have participated in at least two PISA assessments since 2003 (for further details on the 7 estimation of the annualised change, see Box I.2.2 and Annex A5). On average across OECD countries with comparable data in PISA 2003 and PISA 2012, performance has remained broadly similar, but there have been markedly more countries with increasing than with declining mathematics performance (see Box I.2.2 for details on interpreting trends in PISA). Of the 64 countries and economies with trend data up to 2012, 25 show an average annual improvement in mathematics performance; by contrast, 14 countries and economies show an average deterioration in performance between 2003 and 2012. For the remaining 25 countries and economies, there is no change in mathematics performance during the period. Figure I.2.15 illustrates that Albania, Kazakhstan, Malaysia, Qatar and the United Arab Emirates, except Dubai (United Arab Emirates, excluding Dubai), show an average improvement in mathematics performance of more than five score points per year. Among OECD countries, improvements in mathematics performance are observed in Israel (with an average improvement of more than four score points per year), Mexico, Turkey (more than three score points per year), Italy, Poland, Portugal (more than two score points per year), and Chile, Germany and Greece (more than one score point per year). Among countries that have participated in every assessment since 2003, Brazil, Italy, Mexico, Poland, Portugal, Tunisia and Turkey, show an average improvement in mathematics performance of more than 2.5 points per year. Box I.2.4 and Box I.2.5 highlight Brazil’s and Turkey’s improvement in PISA, and provides insight on the education policies and programmes implemented in the last decade. Other chapters of this volume and other volumes of this series highlight other country’s improvements in PISA and outline their recent policy trajectories (e.g. Estonia and Korea in Chapters 4 and 5 of this volume, Mexico and Germany in Volume II, Japan and Portugal in Volume III, and Colombia, Israel, Poland and Tunisia in Volume IV). 51 , S Kno W and Can d o: Student Performan C e in m athemati What Student OECD 2014 © i e – Volume C ien C eading and S r CS

54 2 A Profile A nce i n mA them A tic S o f Student Perform • Figure I.2.15 • a S a nnualised change in mathematics performance throughout participation in pi e-point difference associated with one calendar year Mathematics scor 10 8 6 4 2 0 -2 Annualised change in mathematics performance -4 2 4 3 4 4 4 3 3 4 3 4 4 4 4 4 3 4 3 2 4 4 4 3 4 4 3 4 4 3 4 4 4 4 4 4 4 4 4 4 4 3 4 4 4 3 3 3 4 4 4 4 3 2 4 4 2 2 4 3 2 2 3 2 3 2 Italy Peru Chile Israel Brazil Spain Japan Qatar Latvia Korea Serbia Jordan France Turkey Tunisia Poland Austria Ireland Estonia Greece Finland Iceland Mexico Croatia Albania Canada Norway Sweden Bulgaria Belgium Portugal Slovenia Uruguay Malaysia Thailand Hungary Australia Romania Lithuania Germany Denmark Argentina Indonesia Colombia Singapore Costa Rica Kazakhstan Switzerland Dubai (UAE) Netherlands Montenegro Luxembourg New Zealand Liechtenstein United States Macao-China Chinese Taipei Czech Republic Slovak Republic Shanghai-China United Kingdom Hong Kong-China Russian Federation OECD average 2003 United Arab Emirates* * United Arab Emirates excluding Dubai. Statistically signicant score point changes are marked in a darker tone (see Annex A3). Notes: The number of comparable mathematics scores used to calculate the annualised change is shown next to the country/economy name. The annualised change is the average annual change in PISA score points from a country’s/economy’s earliest participation in PISA to PISA 2012. It is calculated taking into account all of a country’s/economy’s participation in PISA. For more details on the calculation of the annualised change, see Annex A5. OECD average 2003 compares only OECD countries with comparable mathematics scores since 2003. Countries and economies are ranked in descending order of the annualised change in mathematics performance. Source: OECD, PISA 2012 Database, Table I.2.3b. 2 1 http://dx.doi.org/10.1787/888932935572 Box I.2.2. a S pi easuring trends in m PISA 2012 is the fifth round of PISA since the programme was launched in 2000. Every PISA assessment assesses students’ reading, mathematics and science literacy, and in each round, one of these subjects is the main domain and the other two are minor domains. The first full assessment of reading was conducted in 2000 (when it was a major domain), while the first full assessment of mathematics was conducted in 2003 and science in 2006. In 2009, the assessment returned to reading as a major domain, which allowed for observations of trends in reading performance since PISA 2000. Mathematics is the major domain of PISA 2012, as it was in PISA 2003, allowing for observations of trends in mathematics performance since PISA 2003. The first full assessment of each domain sets the scale for future comparisons. The methodologies underpinning performance trends in international studies of education are complex (Gebhardt and Adams, 2007). In order to ensure the comparability of successive PISA results, a number of conditions must be met. First, while successive assessments include a number of common assessment items, the limited number of such items increases measurement errors. Therefore, the confidence band for comparisons over time is wider than for single-year data, and only changes that are indicated as statistically significant should be considered 8 robust. Second, the sample of students must represent an equivalent population (that of 15-year-olds enrolled in school), and only results from samples that meet the strict standards set by PISA can be compared over time. Third, the conditions in which the assessment is conducted must also remain constant across the rounds that are to be compared. ... athemati W Kno S What Student OECD 2014 © m , r eading and S C ien C e – Volume i d e in C o: Student Performan CS and Can 52

55 2 f Student Perform A nce i n mA them A tic S o A Profile Even though they participate in successive PISA assessment, some countries and economies cannot compare all their PISA results over time. For example, the PISA 2000 sample for the Netherlands did not meet the PISA response- rate standards, so the Netherland’s PISA 2000 results are not comparable to those of subsequent assessments. In Luxembourg, the testing conditions changed substantially between 2000 and 2003, so PISA 2000 results are not comparable with those of subsequent assessments. The PISA 2000 and 2003 samples for the United Kingdom did not meet the PISA response-rate standards, so data from the United Kingdom cannot be used for comparisons including these years. In the United States, no results for reading literacy are available for 2006. In 2009, a dispute between teachers’ unions and the education minister of Austria led to a boycott of PISA, which was only lifted after the first week of testing. The boycott required the OECD to remove identifiable cases from the dataset. Although the Austrian dataset met the PISA 2009 technical standards after these cases were removed, the negative reaction to education assessments has affected the conditions under which the PISA survey was conducted and could have adversely affected student motivation to respond to the PISA tasks. Therefore, the comparability of 2009 data with data from earlier PISA assessments cannot be ensured, and data for Austria have been excluded from trend comparisons. In addition, not all countries have participated in all PISA assessments. Among OECD countries, the Slovak Republic and Turkey joined PISA in 2003. Chile and Israel did not participate in the PISA 2003 assessment, and Estonia and Slovenia began participation in 2006. When comparing trends in mathematics, reading and science, only those countries with valid data to compare between assessments are included. As a result, comparisons between the 2000 and 2012 assessments use data on reading performance and include only 38 countries and economies. Comparisons between the 2003 and 2012 assessments use data on reading and mathematics performance and include 39 countries and economies. Comparisons between the 2006 and 2012 assessments use data on reading, mathematics and science performance and include 55 countries and economies (54 countries in the case of reading). Comparisons between 2009 and 2012 use data on all domains and include 63 countries and economies. In all, 64 countries and economies have valid trend information when their PISA 2012 data and all their previous valid data are used. t he annualised c hange in performance Trends in a country’s/economy’s average mathematics, reading and science performance are presented as the annualised change. The annualised change is the average rate of change at which a country’s/economy’s average mathematics, reading and science scores has changed throughout their participation in PISA assessments. Thus, a x x points indicates that the country/economy has improved in performance by positive annualised change of points per year since its earliest comparable PISA results. For countries and economies that have participated in only two assessments, the annualised change is equal to the difference between the two assessments, divided by the number of years that passed between the assessments. The annualised change is a more robust measure of a country’s/economy’s progress in education outcomes as it is based on information available from all assessments. It is thus less sensitive to abnormal measurements that may alter a country’s/economy’s PISA trends if results are compared only between two assessments. The annualised change is calculated as the best-fitting line throughout a country’s/economy’s participation in PISA. The year that individual students participated in PISA is regressed on their PISA scores, yielding the annualised change. The annualised change also takes into account the fact that, for some countries and economies, the period between PISA assessments is less than three years. This is the case for those countries and economies that participated in PISA 2000 or PISA 2009 as part of PISA+: they conducted the assessment in 2001, 2002 or 2010 instead of 2000 or 2009. Annex B4 presents the average performance in mathematics, reading and science (circles) for each country and economy as well as the annualised change (slope of the dotted / Tables I.2.3b, I.4.3b and I.5.3b present solid line). the annualised change in average mathematics, reading and science performance, respectively. Tables I.2.3d, I.4.3d and I.5.3d present the annualised change for the 10th, 25th, 75th and 90th percentile in mathematics, reading and science performance. Annex A5 provides further details on the calculation of the annualised change and other trends measures. 53 , S Kno W and Can d o: Student Performan C e in m athemati What Student OECD 2014 © i e – Volume C ien C eading and S r CS

56 2 tic f Student Perform A nce i n mA them A o S A Profile The average improvement over time shows only one aspect of a country’s/economy’s trajectory; it does not indicate whether a country’s/economy’s improvement is steady, accelerating or decelerating. To evaluate the degree to which a country’s improvement is accelerating or decelerating, only the 55 countries and economies that have participated in PISA 2012 and at least two other assessments have been considered. Annualised linear improvement in mathematics is observed for 18 countries and economies that have participated in PISA 2012 as well as two other assessments. The rate of improvement in the mathematics performance of the average student has accelerated in Macao-China and Poland, meaning that the rate of improvement observed in the 2009 to 2012 period is higher than that observed in the 2003 to 2006 period, for example. In Poland, this means that while scores improved by five score points (not statistically significant) between 2003 and 2006 and maintained that level between 2006 and 2009, between 2009 and 2012 there is a much faster improvement, at 23 points. Similarly, while mathematics scores in Macao-China did not change between 2003 and 2009, they improved by 13 score points between 2009 and 2012. The rate of improvement has remained steady in 13 countries and economies (Brazil, Bulgaria, Chile, Germany, Hong Kong-China, Israel, Italy, Montenegro, Portugal, Romania, Serbia, Tunisia and Turkey); the observed linear annualised change is similar to the rate of change observed throughout a country’s/economy’s participation in successive PISA assessments. By contrast, Qatar, Mexico and Greece show decelerating rates of improvement: the rate of improvement observed in the first assessments of PISA is slower in the later assessments. In Mexico, for example, between 2003 and 2006 the average mathematics score improved from 385 to 406 score points (a change of more than 20 points), then improved again in 2009 to 419 points, but decreased (not significantly) to 413 points in 2012 (Figure I.2.16 and Table I.2.3b). Among the 25 countries that have no positive annualised change, 23 have participated in at least two assessments in addition to PISA 2012, and all those that show deteriorating performance participated in at least two assessments prior to PISA 2012. Among these, Chinese Taipei, Croatia, Ireland and Japan show signs of moving from no change to improvement, or from initial deterioration towards no change in mathematics performance. Although Chinese Taipei, Croatia, Ireland and Japan showed no change in mathematics performance during their participation in earlier rounds of PISA, there are signs of improvement in more recent years. Between PISA 2003 and 2006 assessments, France showed a deterioration in its average annual performance, but later assessments did not show any further deterioration (Figure I.2.16 and Table I.2.3b). At any point in time, countries and economies share similar performance levels with other countries and economies. But as time passes and school systems evolve, some countries and economies improve their performance changing the group of countries with which they share similar performance levels. Figure I.2.17 shows, for each country and economy with comparable results in 2003 and 2012, those other countries and economies with similar performance in 2003 but higher or lower level performance in 2012. In 2003, Poland, for example, was similar in performance to the United States, Latvia, the Slovak Republic, Luxembourg, Hungary, Spain and Norway; but as a result of improvements during the period, it performed better than all those countries in 2012. In 2003, Poland scored below Finland, Germany, Austria, Canada, Belgium and the Netherlands; but by 2012, its performance was similar to this group of countries. Turkey was similar in performance to Uruguay and Thailand in 2003 but, in 2012, its score was higher than those of these two countries, and was at the same level as that of Greece. In 2003, Portugal scored lower than the United States, Latvia, the Slovak Republic, Luxembourg, the Czech Republic, France, Sweden, Hungary, Spain, Iceland and Norway; but by 2012 the country had caught up to those countries. Figure I.2.18 shows the relationship between each country and economy’s average mathematics performance in 2003 and their average rate of change over the 2003 to 2012 period. Countries and economies that show the strongest improvement throughout the various assessments (top half of the graph) are more likely to be those that had comparatively low performance in the initial years. The correlation between a country’s/economy’s earliest comparable mathematics score and the annualised rate of change is -0.60; this means that 35% of the variance in the rate of change can be explained by a country’s/economy’s initial score and that countries with a lower initial score tend to improve at a faster rate. But this relationship is, by no means, a given. Although countries that improve the most are more likely to be those that had lower performance in 2003, some countries and economies that had average or high performance in 2003 saw improvements in their students’ performance over time. Such was the case in the high-performing countries and economies of Hong Kong-China, Macao-China and Germany, all of which saw annualised improvements in mathematics performance even after PISA 2003 mathematics scores placed them at or above the OECD average (results for countries and economies that began their participation in PISA after PISA 2003 are in Table I.2.3b). C W © OECD 2014 What Student S i e – Volume C ien and Can eading and S r , CS athemati m e in C o: Student Performan d Kno 54

57 2 o A nce i n mA them A tic S f Student Perform A Profile • Figure I.2.16 • a c pi assessments urvilinear trajectories of average mathematics performance across S Rate of acceleration or deceleration in per formance (quadratic term) Decelerating Accelerating Steadily changing PISA mathematics score PISA mathematics score PISA mathematics score 2003 2006 2009 2012 2003 2006 2009 2012 2003 2006 2009 2012 Countries/economies Brazil Macao-China Greece Israel Serbia Bulgaria Poland Mexico Italy Tunisia with positive annualised change Chile Qatar Montenegro Turkey Germany Portugal Hong Kong-China Romania PISA mathematics score PISA mathematics score PISA mathematics score 2006 2009 2012 2009 2006 2012 2003 2006 2003 2012 2003 2009 Countries/economies Argentina Indonesia Croatia Latvia Slovenia Austria Ireland Liechtenstein Spain Colombia Japan Lithuania Switzerland Estonia Chinese Taipei Luxembourg Thailand with no signicant annualised change Jordan Norway United Kingdom Korea Russian Federation United States PISA mathematics score PISA mathematics score PISA mathematics score 2006 2003 2012 2012 2009 2009 2006 2003 2003 2006 2009 2012 Countries/economies Finland Australia France Netherlands Belgium Hungary Slovak Republic with negative annualised change New Zealand Canada Sweden Czech Republic Uruguay Denmark Iceland Figures are for illustrative purposes only. Countries and economies are grouped according to the direction and signicance of their annualised Notes: change and their rate of acceleration. Countries and economies with data from only one PISA assessments other than 2012 are excluded. OECD, PISA 2012 Database, Table I.2.3b. Source: 2 1 http://dx.doi.org/10.1787/888932935572 o: Student Performan e – Volume ien C eading and S i , CS athemati m e in C C d and Can W Kno S What Student © OECD 2014 55 r

58 2 nce n mA them A tic i A Profile o A f Student Perform S • • Figure I.2.17 [ Part 1/2 ] m ultiple comparisons of mathematics performance between 2003 and 2012 countries/economies with countries/economies with countries/economies with countries/economies with mathematics mathematics ountries/economies with similar c higher performance in 2003 ountries/economies with similar c higher performance in 2003 c ountries/economies with similar lower performance in 2003 m athematics lower performance in 2003 m athematics performance performance performance in 2003 but lower performance performance in 2003 but with similar performance performance in 2003 but higher performance performance but similar performance performance in 2003 in 2012 but higher performance in 2012 in 2012 and similar performance in 2012 in 2012 but lower performance in 2012 in 2012 in 2012 in 2012 in 2003 Hong Kong-China 550 561 Korea Hong Kong-China 550 561 Finland, Japan, Netherlands, Liechtenstein Hong Kong-China Korea 542 554 Korea 542 554 Finland, Japan, Canada, Netherlands, Liechtenstein Japan, Switzerland, Liechtenstein Finland 538 527 Macao-China 527 538 New Zealand, Czech Republic, Australia, Macao-China Canada, Belgium, Netherlands New Zealand, Finland, Australia, Canada, 536 534 Japan Japan 534 536 Hong Kong-China, Korea Macao-China, Netherlands, Switzerland, Belgium Liechtenstein 536 Japan, Macao-China, Netherlands, New Zealand, Finland, Australia, Canada, Liechtenstein Liechtenstein 535 Hong Kong-China, Korea 535 536 Belgium Switzerland Switzerland 527 531 New Zealand, Czech Republic, Australia, Switzerland Japan, Macao-China, Netherlands, 527 Finland 531 Liechtenstein Canada, Belgium 538 523 Finland, Japan, Canada, Belgium, Netherlands Poland, Germany 523 538 Netherlands Hong Kong-China, Macao-China, Korea Switzerland, Liechtenstein Finland 544 519 Netherlands Hong Kong-China, Japan, Liechtenstein, Finland Poland, Germany, Canada, 544 Macao-China, Switzerland 519 Belgium Korea Poland, Germany 518 Canada Canada 532 518 Belgium, Netherlands Japan, Macao-China, Switzerland, Finland 532 Liechtenstein, Korea 490 518 United States, Latvia, Slovak Republic, Finland, Germany, Austria, Poland New Zealand, Czech Republic, 490 Poland 518 France, Sweden, Australia, Luxembourg, Hungary, Spain, Norway Canada, Belgium, Netherlands Ireland, Denmark, Iceland Belgium 515 529 Belgium 529 515 New Zealand, Australia Canada, Netherlands Japan, Macao-China, Switzerland, Finland Poland, Germany, Austria Liechtenstein Germany 503 514 Slovak Republic, France, Sweden, Austria Poland Finland, Canada, Belgium, New Zealand, 503 Germany 514 Netherlands Czech Republic, Australia, Ireland, Denmark, Norway Iceland Austria Iceland 506 Austria 506 506 Slovak Republic, France, Sweden, 506 Germany, Czech Republic, Ireland, New Zealand, Australia, Poland Denmark Belgium Norway 524 Australia 504 Poland, Germany Austria, Ireland, Denmark Japan, Macao-China, Belgium, New Zealand, Czech Republic Australia 504 524 Switzerland, Liechtenstein Poland 503 501 Iceland 501 New Zealand, Czech Republic, Ireland 503 Germany Austria, France Slovak Republic, Sweden, Norway Ireland Australia, Denmark 500 Australia Poland Latvia, Ireland Germany New Zealand, Austria, Czech Republic, Sweden 500 514 Denmark Denmark 514 France, Iceland New Zealand 523 500 Czech Republic, Australia, Denmark Japan, Macao-China, Belgium, New Zealand Latvia, Austria, France, Ireland, 523 Poland, Germany 500 Iceland Switzerland, Liechtenstein Czech Republic Czech Republic 516 499 Sweden New Zealand, Austria, France, Australia, 516 Macao-China, Switzerland Latvia, Ireland, Portugal, 499 Poland, Germany Norway Denmark, Iceland Latvia, Luxembourg, Portugal, Poland Germany, Austria Czech Republic, Ireland, Denmark, France 511 495 New Zealand France 511 495 Sweden Iceland Norway Iceland Iceland 515 493 Sweden Czech Republic, France, Denmark Latvia, Luxembourg, Portugal, 515 Poland, Germany, Austria, 493 New Zealand Ireland Norway New Zealand, Latvia 483 491 Hungary United States, Spain, Norway, Portugal, Italy Poland Latvia Sweden 491 483 Russian Federation Slovak Republic, Luxembourg, Czech Republic, France, Denmark, Iceland Hungary Poland 490 493 United States, Latvia, Spain, Luxembourg Slovak Republic, Norway 493 490 Sweden France, Iceland Luxembourg Portugal, Russian Federation, Italy Czech Republic, France, United States, Spain, Portugal, Poland, Germany, Austria, Ireland Latvia, Slovak Republic, Luxembourg Hungary 489 495 Norway 495 489 Norway Sweden Russian Federation, Italy Iceland 487 Russian Federation, Italy United States, Latvia, 466 Portugal Portugal 487 466 Slovak Republic, Luxembourg, Czech Republic, France, Sweden, Hungary, Spain, Iceland, Norway 485 466 Portugal, Russian Federation Italy United States, Latvia, 466 485 Italy Slovak Republic, Luxembourg, Sweden, Hungary, Spain, Norway United States, Latvia, Hungary 485 484 484 Poland Portugal, Russian Federation, Slovak Republic, Luxembourg, Spain 485 Spain Sweden, Norway Italy Russian Federation 468 482 Latvia, Portugal, Italy United States, Slovak Republic, 482 468 Russian Federation Luxembourg, Sweden, Hungary, Spain, Norway Poland, Germany, Austria, Ireland 482 498 Luxembourg, Sweden, Hungary, Slovak Republic United States, Latvia, Spain, 498 482 Slovak Republic Norway Portugal, Russian Federation, Italy 481 United States 481 Slovak Republic, Luxembourg, Portugal, Russian Federation, Poland Latvia, Hungary, Spain 483 United States 483 Sweden, Norway Italy Germany, Austria, Czech Republic, Sweden 509 478 Slovak Republic United States, Hungary, Spain, Sweden 509 Poland, Latvia, Luxembourg, 478 Portugal, Russian Federation, France, Ireland, Denmark, Iceland Norway Italy Hungary Poland, Latvia, Luxembourg, Norway United States, Slovak Republic, Spain 490 477 477 Sweden Hungary Portugal, Russian Federation, 490 Italy Greece 453 Turkey 453 445 445 Greece Turkey 423 Greece Uruguay, Thailand 448 423 Turkey 448 Thailand Thailand 427 Turkey Uruguay 427 417 417 Mexico 385 413 Uruguay 413 385 Mexico 422 Uruguay 422 409 Thailand, Turkey Mexico 409 Uruguay Brazil Indonesia 391 391 356 Brazil 356 Tunisia Tunisia 359 388 Brazil, Indonesia 388 359 Tunisia Brazil 375 360 Indonesia Tunisia Indonesia 375 360 Only countries and economies that participated in the PISA 2003 and PISA 2012 assessments are shown. Note: Countries and economies are ranked in descending order of their mean mathematics performance in PISA 2012. Source: OECD, PISA 2012 Database, Table I.2.3b. http://dx.doi.org/10.1787/888932935572 12 o: Student Performan e – Volume © OECD 2014 What Student S Kno W and Can d C C e in m athemati CS , r eading and S C ien i 56

59 2 nce n mA them A tic i A Profile o A f Student Perform S • • Figure I.2.17 [ Part 2/2 ] ultiple comparisons of mathematics performance between 2003 and 2012 m ountries/economies with c c ountries/economies with c ountries/economies with c ountries/economies with athematics m m athematics countries/economies with similar higher performance in 2003 countries/economies with similar higher performance in 2003 countries/economies with similar lower performance in 2003 mathematics wer performance in 2003 lo mathematics performance performance performance in 2003 but lower performance performance in 2003 but with similar performance performance in 2003 but higher performance performance but similar performance performance in 2003 in 2012 but higher performance in 2012 in 2012 and similar performance in 2012 in 2012 but lower performance in 2012 in 2012 in 2012 in 2012 in 2003 Hong Kong-China 550 561 Finland, Japan, Netherlands, Korea 561 550 Hong Kong-China Liechtenstein 542 Korea 542 Korea 554 554 Finland, Japan, Canada, Netherlands, Hong Kong-China Liechtenstein Finland Macao-China 527 Macao-China 538 538 New Zealand, Czech Republic, Australia, Japan, Switzerland, Liechtenstein 527 Canada, Belgium, Netherlands 536 534 Japan Japan 534 536 Hong Kong-China, Korea Macao-China, Netherlands, Switzerland, New Zealand, Finland, Australia, Canada, Belgium Liechtenstein 535 535 Japan, Macao-China, Netherlands, New Zealand, Finland, Australia, Canada, Hong Kong-China, Korea 536 536 Liechtenstein Liechtenstein Belgium Switzerland 531 New Zealand, Czech Republic, Australia, Switzerland Japan, Macao-China, Netherlands, Finland 531 527 Switzerland 527 Liechtenstein Canada, Belgium 538 523 Finland, Japan, Canada, Belgium, Netherlands Hong Kong-China, Macao-China, Korea Poland, Germany 523 538 Netherlands Switzerland, Liechtenstein Finland Finland 519 Netherlands Hong Kong-China, Japan, Liechtenstein, 544 Poland, Germany, Canada, 544 Macao-China, Switzerland 519 Belgium Korea Canada 532 518 Belgium, Netherlands 532 Canada Poland, Germany Finland 518 Japan, Macao-China, Switzerland, Liechtenstein, Korea United States, Latvia, Slovak Republic, Poland 490 518 New Zealand, Czech Republic, Finland, Germany, Austria, 518 490 Poland Luxembourg, Hungary, Spain, Norway France, Sweden, Australia, Canada, Belgium, Netherlands Ireland, Denmark, Iceland Belgium 529 Belgium New Zealand, Australia Canada, Netherlands Japan, Macao-China, Switzerland, 529 Poland, Germany, Austria Finland 515 515 Liechtenstein New Zealand, Germany 503 514 Slovak Republic, France, Sweden, Finland, Canada, Belgium, Austria Poland 514 503 Germany Ireland, Denmark, Norway Czech Republic, Australia, Netherlands Iceland New Zealand, Australia, Austria 506 506 Slovak Republic, France, Sweden, 506 Germany, Czech Republic, Ireland, 506 Poland Austria Iceland Belgium Norway Denmark Australia Australia 524 504 524 Japan, Macao-China, Belgium, 504 Austria, Ireland, Denmark Poland, Germany New Zealand, Czech Republic Switzerland, Liechtenstein 501 503 Slovak Republic, Sweden, Norway Austria, France Iceland 501 503 Ireland Germany Poland New Zealand, Czech Republic, Ireland Australia, Denmark Australia Poland Latvia, Ireland Germany New Zealand, Austria, Czech Republic, Sweden 500 514 Denmark Denmark 514 500 France, Iceland New Zealand 500 Czech Republic, Australia, Denmark Japan, Macao-China, Belgium, New Zealand 523 523 Poland, Germany 500 Latvia, Austria, France, Ireland, Iceland Switzerland, Liechtenstein Czech Republic 516 499 Sweden New Zealand, Austria, France, Australia, 516 Macao-China, Switzerland Latvia, Ireland, Portugal, 499 Poland, Germany Czech Republic Norway Denmark, Iceland France 495 New Zealand Poland Latvia, Luxembourg, Portugal, 511 France 511 495 Sweden Czech Republic, Ireland, Denmark, Germany, Austria Iceland Norway Poland, Germany, Austria, Iceland 493 Sweden Czech Republic, France, Denmark Latvia, Luxembourg, Portugal, Iceland 515 515 New Zealand 493 Ireland Norway Latvia 483 491 Hungary United States, Spain, Norway, New Zealand, Poland Portugal, Italy Sweden 491 483 Latvia Russian Federation Slovak Republic, Luxembourg, Czech Republic, France, Denmark, Iceland Slovak Republic, Norway Hungary 490 493 United States, Latvia, Spain, Poland Luxembourg Luxembourg 493 490 Sweden France, Iceland Portugal, Russian Federation, Italy 489 Sweden Czech Republic, France, United States, Spain, Portugal, Poland, Germany, Austria, Ireland Latvia, Slovak Republic, Luxembourg Hungary Norway 495 489 495 Norway Russian Federation, Italy Iceland 487 Russian Federation, Italy United States, Latvia, 466 Portugal Portugal 487 466 Slovak Republic, Luxembourg, Czech Republic, France, Sweden, Hungary, Spain, Iceland, Norway 466 485 Italy Portugal, Russian Federation United States, Latvia, 466 485 Italy Slovak Republic, Luxembourg, Sweden, Hungary, Spain, Norway United States, Latvia, Hungary 485 484 485 Poland Portugal, Russian Federation, Slovak Republic, Luxembourg, Spain 484 Spain Sweden, Norway Italy Russian Federation 468 482 Latvia, Portugal, Italy United States, Slovak Republic, 482 468 Russian Federation Luxembourg, Sweden, Hungary, Spain, Norway Slovak Republic United States, Latvia, Spain, Poland, Germany, Austria, Ireland Luxembourg, Sweden, Hungary, 482 498 Slovak Republic 482 498 Portugal, Russian Federation, Norway Italy Poland 483 481 Slovak Republic, Luxembourg, Portugal, Russian Federation, Latvia, Hungary, Spain 481 483 United States United States Sweden, Norway Italy Sweden 509 478 Slovak Republic Germany, Austria, Czech Republic, United States, Hungary, Spain, 509 Sweden 478 Poland, Latvia, Luxembourg, Portugal, Russian Federation, France, Ireland, Denmark, Iceland Norway Italy 490 Hungary 477 Sweden Portugal, Russian Federation, Poland, Latvia, Luxembourg, Norway 490 Hungary United States, Slovak Republic, Spain 477 Italy Greece 445 453 Turkey 453 445 Greece Turkey 448 423 Uruguay, Thailand Greece 448 423 Turkey 417 427 Turkey Uruguay 427 417 Thailand Thailand 413 385 Uruguay 413 385 Mexico Mexico Uruguay 422 Uruguay Thailand, Turkey Mexico 409 422 409 Indonesia 356 391 356 Brazil Brazil Tunisia 391 Tunisia 359 388 Brazil, Indonesia 388 359 Tunisia 375 Brazil Indonesia 375 360 Indonesia Tunisia 360 Note: Only countries and economies that participated in the PISA 2003 and PISA 2012 assessments are shown. Countries and economies are ranked in descending order of their mean mathematics performance in PISA 2012. Source: OECD, PISA 2012 Database, Table I.2.3b. http://dx.doi.org/10.1787/888932935572 12 CS OECD 2014 S Kno W and Can d o: Student Performan C e in m athemati What Student , r eading and S C ien C e – Volume i © 57

60 2 o A nce i n mA them A tic S A Profile f Student Perform • Figure I.2.18 • elationship between annualised change in performance r and average a pi S 2003 mathematics scores PISA 2003 performance PISA 2003 performance OECD average OECD average below above 5 Brazil 4 Performance improved Turkey Mexico Tunisia 3 Portugal Italy Poland 2 Germany Russian Federation Greece Hong Kong-China Thailand Macao-China 1 Korea Annualised change in mathematics performance Switzerland Latvia Japan Indonesia United States Austria Liechtenstein 0 Spain Luxembourg OECD average 2003 Performance deteriorated Ireland Norway -1 Belgium Hungary Canada France Uruguay Slovak Republic Netherlands Denmark -2 Iceland Australia New Zealand Czech Republic Finland -3 Sweden -4 400 450 500 550 600 375 425 475 525 570 350 Mean score in mathematics in PISA 2003 Annualised score point change in mathematics that are statistically signicant are indicated in a darker tone (see Annex A3). Notes: The annualised change is the average annual change in PISA score points from a country’s/economy’s earliest participation in PISA to PISA 2012. It is calculated taking into account all of a country’s/economy‘s participation in PISA. For more details on the calculation of the annualised change, see Annex A5. The correlation between a country’s/economy‘s mean score in 2003 and its annualised performance is -0.60. OECD average 2003 considers only those countries with comparable data since PISA 2003. Source: OECD, PISA 2012 Database, Tables I.2.3b. 2 1 http://dx.doi.org/10.1787/888932935572 Other high-performing countries and economies that began their participation in PISA after the 2003 assessment, like Shanghai-China and Singapore, also show improvements in performance. In addition, there are many countries and economies that performed similarly in 2003 but evolved differently. As shown in Table I.2.3b, Bulgaria, Chile, Romania and Thailand began their participation in PISA with a mathematics performance of around 410 score points; but while Thailand showed no annual improvement between 2003 and 2012, Chile, Bulgaria and Romania showed an annual improvement between 2006 and 2012 of 1.9, 4.2 and 4.9 score points, respectively (Figure I.2.18 and Table I.2.3b). Trends in mathematics performance adjusted for sampling and demographic changes Changes in a country’s or economy’s mathematics performance can have many sources. While improvements may result from improved education services, they can also result from demographic changes that have shifted the country’s population profile. By following strict sampling and methodological standards PISA ensures that all countries and economies are measuring the mathematics performance of their 15-year-olds enrolled in school; but because of eading and S r , CS athemati m i C o: Student Performan d and Can W Kno S What Student OECD 2014 © e – Volume C ien C e in 58

61 2 A Profile A nce i n mA them A tic S f Student Perform o migration or other demographic and social trends, the characteristics of this reference population may change. Annex A5 provides details on the calculation of the adjusted trends. Figure I.2.19 presents annualised changes after adjusting for changes in the age, gender, socio-economic status, 9 migration background and language spoken at home of the population of students in each country or economy. On average across OECD countries, and assuming that the 2003, 2006 and 2009 population of 15-year-old students had the same demographic profile as the population in 2012, scores in mathematics dropped by around one point per year. The observed trend shows no change since 2006. This difference in trends before and after accounting for demographic changes means that were it not for these demographic and socio-economic changes, average mathematics performance across OECD countries would have deteriorated since 2006. • Figure I.2.19 • mathematics scores pi djusted and observed annualised performance change in average a S a After accounting for social and demographic changes Before accounting for social and demographic changes 10 9 8 7 6 5 4 3 2 1 0 -1 -2 Annualised change in mathematics performance -3 -4 -5 Italy Peru Chile Israel Brazil Spain Japan Qatar Latvia Korea Serbia Jordan France Turkey Tunisia Poland Austria Ireland Estonia Greece Finland Iceland Mexico Croatia Canada Norway Sweden Bulgaria Belgium Portugal Slovenia Uruguay Malaysia Thailand Hungary Australia Romania Lithuania Germany Denmark Argentina Indonesia Colombia Singapore Costa Rica Kazakhstan Switzerland Dubai (UAE) Netherlands Montenegro Luxembourg New Zealand Liechtenstein United States Macao-China Chinese Taipei Czech Republic Slovak Republic Shanghai-China United Kingdom Hong Kong-China Russian Federation OECD average 2003 United Arab Emirates* * United Arab Emirates excluding Dubai. Statistically signicant values are marked in a darker tone (see Annex A3). Notes: The annualised change is the average annual change in PISA score points. It is calculated taking into account all of a country’s/economy’s participation in PISA. For more details on the calculation of the annualised change, see Annex A5. PISA index of social, cultural and economic status, The annualised change adjusted for demographic changes assumes that the average age and as well as the percentage of female students, those with an immigrant background and those who speak a language other than the assessment at home is the same in previous assessments as those observed in 2012. For more details on the calculation of the adjusted annualised change, see Annex A5. OECD average 2003 considers only those countries with comparable mathematics scores since PISA 2003. Countries and economies are ranked in descending order of the annualised change after accounting for demographic changes. Source: OECD, PISA 2012 Database, Tables I.2.3b and I.2.4. http://dx.doi.org/10.1787/888932935572 1 2 As shown in Figure I.2.19, of the 25 countries and economies that saw an overall improvement in mathematics 10 performance, 16 show this improvement after accounting for demographic changes in their student population. In these countries and economies, changes in the age, immigrant background and language spoken at home of the student population do not explain all of the observed improvement in mathematics performance. Of the 14 countries and economies that show deteriorating performance during their participation in PISA, in no country or economy does this trend lose statistical significance after accounting for demographic changes in the student population. Of the 25 countries and economies that did not see an annualised change in mathematics performance, 9 would show a deterioration in performance had their student populations in previous assessments shared the same profile as students who were assessed in PISA 2012. , What Student Kno W and Can d o: Student Performan C e in m athemati CS S eading and S C ien C e – Volume i © OECD 2014 59 r

62 2 A nce i n mA them A tic S f Student Perform A Profile o Comparing the results of the adjusted and unadjusted trends in mathematics performance, shown in Figure I.2.19, Costa Rica, the Czech Republic, Dubai (United Arab Emirates), Israel, Kazakhstan, Malaysia and Mexico, have less than a 20% difference between unadjusted and adjusted annualised trends, meaning that the characteristics of the student population have not changed much between 2003 and 2012, that changes in the characteristics of the student population are unrelated to average student performance, or that education services have adapted to the changes in the student population so that any of those changes that may have an impact on student performance have been compensated for by adaptations made in education service. Similarly, in Colombia, Hungary, Jordan, Latvia, Luxembourg and the Slovak Republic, the difference between the unadjusted and adjusted annualised trends is less than 0.5 score points per year. Large differences in adjusted and unadjusted performance are observed in Chile, Liechtenstein, Montenegro, Qatar, Slovenia and the United Arab Emirates, excluding Dubai. In these countries and economies, the difference between adjusted and unadjusted annualised trends is greater than two score points, signalling that demographic changes have had a considerable impact on trends in mathematics performance. Informative as they may be, adjusted trends are merely hypothetical scenarios that help to understand the source of changes in students’ performance over time. Observed (unadjusted) trends depicted in Figure I.2.19 and throughout this chapter summarise the overall evolution of a school system, highlighting the challenges that countries and economies face in improving students’ and schools’ mathematics performance. To better understand the observed trends in performance, Chapters 2 and 3 of Volume II analyses in greater detail, how the student population has changed through migration and in socio-economic background, and how these characteristics are related to mathematics performance. Volume III explores students’ engagement with and at school, drive and self-beliefs towards learning and mathematics. Volume IV, in turn, explores how attributes of school organisation and educational resources are related to changes in performance, providing further insight into the policies and practices that may explain the trends observed in mathematics performance. Students at the different levels of proficiency in mathematics Figure I.2.20 shows the location of some of these items on the PISA 2012 scale. A selection of items used in the 2012 survey is presented at the end of the chapter. Since PISA is a triennial assessment, it is useful to retain a sufficient number of questions over successive PISA assessments in order to generate trend data over time. • Figure I.2.20 • m ap of selected mathematics questions, by proficiency level ower l e scor limit Questions (position on P i S a evel l scale) 6 EVOLVING R – Question 2 (840.3) DOOR 669 – Question 3 (696.6) y C THE CLIS ELEN H T G ARAGE – Question 2, FULL CREDIT (687.3) 5 – Question 2, ARAGE G (663.2) CREDIT PARTIAL 607 OUNT F C LIMBING M UJI – Question 2 (641.6) OUNT (610.0) LIMBING M CREDIT F UJI – Question 3, FULL C 4 PARTIAL – Question 3, UJI F OUNT C M (591.3) LIMBING CREDIT 545 – Question 3 (561.3) EVOLVING DOOR R – Question 3 (552.6) ? CAR HICH W 3 DOOR R EVOLVING – Question 1 (512.3) 482 ELEN H THE – Question 2 (510.6) T CLIS y C CAR ? – Question 2 (490.9) W HICH 2 – Question 1 (464.0) OUNT F UJI C LIMBING M 420 y T THE ELEN H – Question 1 (440.5) CLIS C HARTS C – Question 5 (428.2) 1 G – Question 1 (419.6) ARAGE 358 C – Question 2 (415.0) HARTS Below HARTS – Question 1 (347.7) C Level W – Question 1 (327.8) ? CAR HICH 1 C © OECD 2014 What Student S Kno W and Can d o: Student Performan C e in e – Volume m athemati CS , r eading and S C ien i 60

63 2 A A nce i n mA them f Student Perform tic S A Profile o The six mathematics proficiency levels are defined in the same way as the corresponding levels of the PISA 2003 scale, with the highest level labelled “Level 6”, and the lowest labelled “Level 1”. However, their descriptions have been updated to reflect the new mathematical process categories in the PISA 2012 framework and the large number of new items developed for PISA 2012. Figure I.2.21 provides descriptions of the mathematical skills, knowledge and understanding required at each level of the mathematical literacy scale and the average proportion of students at each of these proficiency levels across OECD countries. Figure I.2.22 shows the distribution of students on each of these six proficiency levels. The percentage of students performing below Level 2 is shown on the left side of the vertical axis. • Figure I.2.21 • Summary descriptions for the six levels of proficiency in mathematics Percentage of students ower l able to perform tasks e at each level or above scor E o ( What students can typically do evel l average) cd limit 6 At Level 6, students can conceptualise, generalise and utilise information based on 669 3.3% their investigations and modelling of complex problem situations, and can use their knowledge in relatively non-standard contexts. They can link different information sources and representations and flexibly translate among them. Students at this level are capable of advanced mathematical thinking and reasoning. These students can apply this insight and understanding, along with a mastery of symbolic and formal mathematical operations and relationships, to develop new approaches and strategies for attacking novel situations. Students at this level can reflect on their actions, and can formulate and precisely communicate their actions and reflections regarding their findings, interpretations, arguments, and the appropriateness of these to the original situation. 5 At Level 5, students can develop and work with models for complex situations, 607 12.6% identifying constraints and specifying assumptions. They can select, compare, and evaluate appropriate problem-solving strategies for dealing with complex problems related to these models. Students at this level can work strategically using broad, well-developed thinking and reasoning skills, appropriate linked representations, symbolic and formal characterisations, and insight pertaining to these situations. They begin to reflect on their work and can formulate and communicate their interpretations and reasoning. 4 At Level 4, students can work effectively with explicit models for complex concrete 30.8% 545 situations that may involve constraints or call for making assumptions. They can select and integrate different representations, including symbolic, linking them directly to aspects of real-world situations. Students at this level can utilise their limited range of skills and can reason with some insight, in straightforward contexts. They can construct and communicate explanations and arguments based on their interpretations, arguments, and actions. 3 At Level 3, students can execute clearly described procedures, including those that 482 54.5% require sequential decisions. Their interpretations are sufficiently sound to be a base for building a simple model or for selecting and applying simple problem- solving strategies. Students at this level can interpret and use representations based on different information sources and reason directly from them. They typically show some ability to handle percentages, fractions and decimal numbers, and to work with proportional relationships. Their solutions reflect that they have engaged in basic interpretation and reasoning. 2 At Level 2, students can interpret and recognise situations in contexts that require 77.0% 420 no more than direct inference. They can extract relevant information from a single source and make use of a single representational mode. Students at this level can employ basic algorithms, formulae, procedures, or conventions to solve problems involving whole numbers. They are capable of making literal interpretations of the results. 1 At Level 1, students can answer questions involving familiar contexts where all 92.0% 358 relevant information is present and the questions are clearly defined. They are able to identify information and to carry out routine procedures according to direct instructions in explicit situations. They can perform actions that are almost always obvious and follow immediately from the given stimuli. CS OECD 2014 S Kno W and Can d o: Student Performan C e in m athemati What Student , r eading and S C ien C e – Volume i © 61

64 2 f Student Perform nce i n mA them A tic S A A Profile o • Figure I.2.22 • p roficiency in mathematics ercentage of students at each level of mathematics proficiency P Level 2 Level 3 Level 1 Level 4 Level 5 Level 6 Below Level 1 Shanghai-China Shanghai-China Singapore Singapore Hong Kong-China Hong Kong-China Korea Korea Estonia Estonia Macao-China Macao-China Japan Japan Finland Finland Students at Level 1 Switzerland Switzerland or below Chinese Taipei Chinese Taipei Canada Canada Liechtenstein Liechtenstein Viet Nam Viet Nam Poland Poland Netherlands Netherlands Denmark Denmark Ireland Ireland Germany Germany Austria Austria Belgium Belgium Australia Australia Latvia Latvia Slovenia Slovenia Czech Republic Czech Republic Iceland Iceland United Kingdom United Kingdom Norway Norway France France New Zealand New Zealand OECD average OECD average Spain Spain Russian Federation Russian Federation Luxembourg Luxembourg Italy Italy Portugal Portugal United States United States Lithuania Lithuania Sweden Sweden Slovak Republic Slovak Republic Hungary Hungary Croatia Croatia Israel Israel Greece Greece Serbia Serbia Romania Romania Turkey Turkey Bulgaria Bulgaria Kazakhstan Kazakhstan United Arab Emirates United Arab Emirates Thailand Thailand Chile Chile Malaysia Malaysia Mexico Mexico Uruguay Uruguay Montenegro Montenegro Costa Rica Costa Rica Albania Albania Argentina Argentina Brazil Brazil Tunisia Tunisia Jordan Jordan Qatar Qatar Students at Level 2 Colombia Colombia or above Peru Peru Indonesia Indonesia % % 100 40 20 0 80 60 80 60 40 20 100 Countries and economies are ranked in descending order of the percentage of students at Levels 2, 3, 4, 5 and 6. Source: OECD, PISA 2012 Database, Table I.2.1a. 2 1 http://dx.doi.org/10.1787/888932935572 m W S What Student OECD 2014 and Can d o: Student Performan C e in Kno athemati , r © i eading and S C ien C e – Volume CS 62

65 2 A Profile f Student Perform A nce i n mA them A tic S o Proficiency at Level 6 (scores higher than 669 points) Students at Level 6 of the PISA mathematics assessment are able to successfully complete the most difficult PISA items. At Level 6, students can conceptualise, generalise and use information based on their investigations and modelling of complex problem situations, and can use their knowledge in relatively non-standard contexts. They can link different information sources and representations and move flexibly among them. Students at this level are capable of advanced mathematical thinking and reasoning. These students can apply this insight and understanding, along with a mastery of symbolic and formal mathematical operations and relationships, to develop new approaches and strategies for addressing novel situations. Students at this level can reflect on their actions, and can formulate and precisely communicate their actions and reflections regarding their findings, interpretations and arguments, and can explain why they were applied to the original situation. Question 3 in the example HELEN THE CYCLIST (Figure I.2.55) requires Level 6 proficiency. It requires a deeper understanding of the meaning of average speed, appreciating the importance of linking total time with total distance. Average speed cannot be obtained just by averaging the speeds, even though in this specific case the incorrect answer (28.3 km/hr) obtained by averaging the speeds (26.67 km/hr and 30 km/hr) is not much different from the correct answer of 28 km/hr. There are both mathematical and real world understandings of this phenomenon, leading to high demands mathematisation on the fundamental mathematical capabilities of and using reasoning and argumentation and also symbolic, formal and technical language and operations . For students who know to work from total time (9 + 6 = 15 minutes) and total distance (4 + 3 = 7 km), the answer can be obtained simply by proportional reasoning (7 km in ¼ hour is 28 km in 1 hour), or by more complicated formula approaches (e.g. distance / time = 7 / (15/60) = 420 / 15 = 28). This question has been classified as an employing process because the greatest part of the demand arises from the mathematical definition of average speed and possibly also the unit conversion, especially for students using speed–distance–time formulas. It is one of the more difficult tasks of the item pool, and sits in Level 6 on the proficiency scale. On average across OECD countries, 3.3% of students attain Level 6. The partner economy Shanghai-China has by far the largest proportion of students (30.8%) who score at this level in mathematics. Indeed, Shanghai-China has more students at this level of mathematics proficiency than at any other level, and is the only PISA participant where this is the case. Between 10% and 20% of students in four other Asian countries and economies – the three partner countries and economies Singapore (19.0%), Chinese Taipei (18.0%), Hong Kong-China (12.3%) and the OECD country Korea (12.1%) score at this level. Between 5% and 10% of students in Japan (7.6%), the partner economy Macao-China (7.6%), the partner country Liechtenstein (7.4%), Switzerland (6.8%) and Belgium (6.1%) attain Level 6 in mathematics. Thirty-three participating countries and economies show between 1% and 5% of their students at this level, while in 22 others, fewer than 1% of students score at the highest level, including the three OECD countries Mexico, Chile and Greece (Figure I.2.20 and Table I.2.1a). Proficiency at Level 5 (scores higher than 607 but lower than or equal to 669 points) At Level 5, students can develop and work with models for complex situations, identifying constraints and specifying assumptions. They can select, compare and evaluate appropriate problem-solving strategies for dealing with complex problems related to these models. Students at this level can work strategically using broad, well-developed thinking and reasoning skills, appropriate linked representations, symbolic and formal characterisations, and insights pertaining to these situations. They begin to reflect on their work and can formulate and communicate their interpretations and reasoning. Typical questions for Level 5 are exemplified by Question 3 from the unit CLIMBING MOUNT FUJI (Figure I.2.56). This question has been allocated to the category. There is one main relationship involved: the distance walked = employing number of steps x average step length. To use this relationship to solve the problem, there are two obstacles: rearranging the formula (which is probably done by students informally rather than formally using the written relationship) so that the average step length can be found from distance and number of steps, and making appropriate unit conversions. For this question, it was judged that the major cognitive demand comes from carrying out these steps; hence it has process, rather than identifying the relationships and assumptions to be made (the employing been categorised in the interpreting process) or formulating the answer in real world terms. CS eading and S S Kno W and Can d o: Student Performan C e in m athemati What Student , 63 OECD 2014 © i e – Volume C ien C r

66 2 A Profile f Student Perform A nce i n o them A tic S mA a S pi ounders in op performers and all-r t Box I.2.3. Performance in PISA refers to particular and increasingly complex tasks students are able to complete. A small proportion of students attains the highest levels and can be called top performers in mathematics, reading or science. Even fewer are the academic all-rounders, those students who achieve proficiency Level 5 or higher in mathematics, reading and science simultaneously. These students will be at the forefront of a competitive, knowledge-based global economy. They are able to draw on and use information from multiple and indirect sources to solve complex problems. Results from the PISA 2012 assessment show that nurturing top performance and tackling low performance need not be mutually exclusive. Some high-performing countries in PISA 2012, like Estonia and Finland, have also low variation in student scores. Equally important, since their first participation in PISA, France, Hong Kong - China, Italy, Japan, Korea, Luxembourg, Macao-China, Poland, Portugal and the Russian Federation have been able to increase the share of top performers in mathematics, reading or science. Figure I.2.a shows the proportion of top performers and all-rounders across OECD countries. Parts in the diagram shaded blue represent the percentage of 15-year-old students who are top performers in just one of the three subject areas assessed, that is, either in mathematics, reading or science. The parts in blue show the percentage of students who are top performers in two of the subject areas, while the grey part in the centre of the diagram shows the percentage of 15-year-old students who are top performers in all three subject areas. Figure I.2.a • • o verlapping of top performers in mathematics, reading and science on average oss acr oecd countries Reading and science 0.6% Science only 1.1% Reading only Mathematics and science 2.3% 1.9% Reading, mathematics and science 4.4% Mathematics and reading 1.5% Mathematics only 4.4% Note: Non-top performers in any of the three domains: 83.8%. OECD, PISA 2012 Database, Table I.2.29. Source: On average across OECD countries, 16.2% of students are top performers in at least one of the three subject areas; but only 4.4% of 15-year-old students are top performers in all three. This shows that excellence is not simply strong performance in all areas, but rather that it can be found among a wide range of students in various subjects. About 1.5% of students are top performers in both mathematics and reading but not in science, 2.3% are top performers in both mathematics and science but not in reading, and fewer than 1% of students (0.6%) are top performers in both reading and science but not in mathematics. The percentage of students who are top performers in both mathematics and science is greater than the percentages who are top performers in mathematics and reading or in reading and science. There is substantial variation among countries in the percentages of top performers in the three subjects (Table I.2.29). ... eading and S © r , CS athemati m e in What Student W S Kno C o: Student Performan d i OECD 2014 e – Volume and Can C ien C 64

67 2 o A nce i n mA f Student Perform A tic S A Profile them Figure I.2.b • • op performers in mathematics, r eading and science t Percentage of students reaching the two highest levels of proficiency Level 6 Level 5 Mean Mean Mean 55.4% Reading Mathematics Science score score score Shanghai-China Shanghai-China Shanghai-China 580 570 613 Singapore Singapore Singapore 542 551 573 Japan Japan Chinese Taipei 538 547 560 Hong Kong-China Finland Hong Kong-China 561 545 545 Korea Korea Hong Kong-China 555 536 554 Liechtenstein New Zealand Australia 521 535 512 New Zealand Finland Macao-China 538 524 516 Japan Estonia France 536 541 505 Germany Switzerland Canada 523 524 531 Chinese Taipei Belgium Netherlands 515 523 522 Korea Netherlands Belgium 538 523 509 Australia Germany Canada 512 514 525 United Kingdom Ireland Poland 518 523 514 Canada Poland Liechtenstein 526 518 516 Finland Ireland Norway 522 504 519 New Zealand Liechtenstein Poland 518 525 500 Slovenia Netherlands Australia 504 514 511 Estonia Israel Switzerland 515 486 521 Austria Switzerland Belgium 505 506 509 OECD average Slovenia Germany 501 508 501 Luxembourg Viet Nam Chinese Taipei 523 511 488 Luxembourg United Kingdom France 495 491 499 Czech Republic OECD average Viet Nam 528 496 499 OECD average Estonia France 499 516 494 United States United Kingdom Austria 498 494 506 Czech Republic Sweden Luxembourg 490 483 508 Iceland Macao-China Norway 493 495 509 Italy Slovak Republic United States 482 490 497 Czech Republic Denmark Ireland 498 501 493 Portugal Iceland Macao-China 487 521 483 Portugal Denmark Sweden 485 500 488 Italy Italy Hungary 485 494 488 Norway Spain Hungary 488 489 494 Austria Israel Israel 490 466 470 Hungary Iceland Denmark 478 477 496 Greece United States Lithuania 496 481 477 Slovak Republic Slovenia Lithuania 481 471 479 Spain Russian Federation Sweden 478 475 496 Viet Nam Croatia Spain 491 508 484 Latvia Portugal Croatia 485 491 489 Slovak Republic Russian Federation Latvia 482 463 502 Russian Federation Turkey Croatia 471 475 486 Turkey Bulgaria Bulgaria 436 448 446 Latvia Serbia United Arab Emirates 489 449 448 Lithuania Bulgaria Greece 439 477 467 Greece Serbia Turkey 463 453 446 United Arab Emirates Serbia United Arab Emirates 445 434 442 Qatar Romania Qatar 384 445 388 Romania Uruguay Thailand 416 427 438 Albania Chile Qatar 376 394 445 Thailand Chile Montenegro 422 444 423 Uruguay Romania Uruguay 411 439 409 Albania Malaysia Thailand 421 397 441 Montenegro Montenegro Chile 410 441 410 Malaysia Kazakhstan Costa Rica 441 432 420 Albania Argentina Brazil 405 396 394 Brazil Jordan Tunisia 388 409 410 Peru Brazil Argentina 406 384 391 Mexico Costa Rica Mexico 429 413 424 Peru Colombia Kazakhstan 425 403 368 Costa Rica Tunisia Mexico 407 404 415 Jordan Colombia Jordan 399 386 399 Tunisia Colombia Malaysia 376 398 398 Peru Indonesia Indonesia 375 396 373 Argentina Indonesia Kazakhstan 388 382 393 10 10 20 30 40 10 30 40 20 30 40 20 % 0 % 0 % 0 15 5 15 35 5 25 25 5 15 35 35 25 Countries and economies are ranked in descending order of the percentage of top performers (Levels 5 and 6). OECD, PISA 2012 Database, Tables I.2.1a, I.2.3a, I.4.1a, I.4.3a, I.5.1a and I.5.3a. Source: 1 http://dx.doi.org/10.1787/888932935572 2 ... athemati i 65 e – Volume C ien C , eading and S r CS © m e in C o: Student Performan d and Can W OECD 2014 Kno S What Student

68 2 A f Student Perform A nce i n mA them o tic S A Profile All-rounders, or top performers in all three subjects, comprise between 6% and just over 8% of 15-year-old students in Korea (8.1%), New Zealand (8.0%), Australia (7.6%), Finland (7.4%), Canada (6.5%), Poland (6.1%), Belgium (6.1%), the Netherlands (6.0%) and the partner economy Chinese Taipei (6.1%), and even larger proportions are found in the countries and economies Shanghai-China (19.6%), Singapore (16.4%), Japan (11.3%) and Hong Kong-China (10.9%). Conversely, in two OECD countries and 17 partner countries and economies, fewer than 1% of students are top performers in all three subjects. Figure I.2.b shows the proportions of top performers in mathematics, reading and science for each country. Although on average across OECD countries, 9.3% and 3.3% of 15-year-olds reach Level 5 and Level 6 in mathematics, respectively, these proportions vary substantially across countries. For example, among OECD countries, Korea, Japan and Switzerland have at least 20% of top performers in mathematics, whereas Mexico and Chile have fewer than 1% and 2%, respectively. Among partner countries and economies, the overall proportion of these top performers also varies considerably from country to country; in some countries, no student achieves Level 6 in mathematics. At the same time, Shanghai-China, Singapore, Chinese Taipei and Hong Kong-China have the highest proportion of students performing at Level 5 or 6. Similar variations are shown in reading and science, with only slight differences in the patterns of these results among countries. Among countries with similar mean scores in PISA, there are remarkable differences in the percentage of top- performing students. For example, Denmark has a mean score of 500 points in mathematics in PISA 2012 and 10% of students perform at high proficiency levels in mathematics, which is less than the average of around 13%. New Zealand has a similar mean mathematics score of 500 points, but 15% of its students attain the highest levels of proficiency, which is above the average. Although only a small percentage of students in Denmark perform at the lowest levels (see Table I.2.1a), these results could signal the absence of a highly educated talent pool for the future. Having a large proportion of top performers in one subject is no guarantee of having a large proportion of top performers in the others. For example, Switzerland has one of the 10 largest shares of top performers in mathematics, but only a slightly-above-average share of top performers in reading and science. Across the three subjects and across all countries, girls are as likely to be top performers as boys. On average across OECD countries, 4.6% of girls and 4.3% of boys are top performers in all three subjects, and 15.6% of girls and 16.8% of boys are top performers in at least one subject (Table I.2.30). However, while the gender gap among students who are top performers only in science is small (0.9% of girls and 1.3% of boys), it is large among top performers in mathematics only (2.9% of girls and 5.9% of boys) and in reading only (3.2% of girls and 0.6% of boys). To increase the share of top-performing students, countries and economies need to look at the barriers posed by social background (examined in Volume II of this series), the relationship between performance and students’ attitudes towards learning (examined in Volume III), and schools’ organisation, resources and learning environment (examined in Volume IV). On average across OECD countries, 12.6% of students are top performers, meaning that they are proficient at Level 5 or 6. Among all participants in PISA 2012, the partner economy Shanghai-China (55.4%) has the largest proportion of students - China (33.7%). performing at Level 5 or 6, followed by Singapore (40.0%), Chinese Taipei (37.2%) and Hong Kong In Korea 30.9% of students are top performers in mathematics. Between 15% and 25% of students in Liechtenstein, Macao China, Japan, Switzerland, Belgium, the Netherlands, Germany, Poland, Canada, Finland and New Zealand - perform at Level 5 or above in mathematics. By contrast, in 36 countries, 10% of students or fewer perform at these levels. These include the OECD countries Denmark (10.0%), Italy (9.9%), Norway (9.4%), Israel (9.4%), Hungary (9.3%), the United States (8.8%), Sweden (8.0%), Spain (8.0%), Turkey (5.9%), Greece (3.9%) and Chile (1.6%). In Kazakhstan, Albania, Tunisia, Brazil, Mexico, Peru, Costa Rica, Jordan, Colombia, Indonesia and Argentina, fewer than 1% of students are top performers in mathematics (Figure I.2.22 and Table I.2.1a). Proficiency at Level 4 (scores higher than 545 but lower than or equal to 607 points) At Level 4, students can work effectively with explicit models on complex, concrete situations that may involve constraints or call for making assumptions. They can select and integrate different representations, including symbolic eading and S Kno © OECD 2014 What Student i e – Volume C ien C W r , CS athemati m e in C o: Student Performan d and Can S 66

69 2 o A nce i n mA f Student Perform A tic S A Profile them representations, linking them directly to aspects of real-world situations. Students at this level can use their limited range of skills and can reason with some insight, in straightforward contexts. They can construct and communicate explanations and arguments based on their interpretations, reasoning and actions. Question 3 in R (Figure I.2.57) involves rates and proportional reasoning, and it sits within Level 4 EVOLVING DOOR on the mathematics proficiency scale. In one minute, the door revolves 4 times bringing 4 × 3 = 12 sectors to the entrance, which enables 12 × 2 = 24 people to enter the building. In 30 minutes, 24 × 30 = 720 people can enter (hence, the correct answer is response option D). The high frequency of PISA items that involve proportional reasoning highlights its centrality to mathematical literacy, especially for students whose mathematics has reached a typical stage for 15-year-olds. Many real contexts involve direct proportion and rates, which as in this case are often used in chains of reasoning. Coordinating such a chain of reasoning requires to bring the information together in a devising a strategy logical sequence. This item also makes considerable demand on the fundamental mathematical capability, especially mathematisation in the formulating process. A student needs to understand the real situation, perhaps visualising how the doors rotate, presenting one sector at a time, making the only way for people to enter the building. This understanding of the real world problem enables the data given in the problem to be assembled in the right way. The questions in this unit have scientific been placed in the context category, even though they do not explicitly involve scientific or engineering concepts, as do many of the other items in this category. The scientific category includes items explaining why things are as they are in the real world. On average across OECD countries, 30.8% of students perform at proficiency Level 4, 5 or 6. More than three out of four students in Shanghai-China perform at one of these levels (75.6%), and more than one in two students in Singapore, Hong Kong - China, Chinese Taipei and Korea do. Countries and economies where more than one in three students are proficient at proficiency Level 4, 5 or 6 are Macao-China (48.8%), Liechtenstein (48.0%), Japan (47.4%), Switzerland (45.3%), the Netherlands (43.1%), Belgium (40.2%), Germany (39.1%), Canada (38.8%), Finland (38.4%), Poland (38.1%), Estonia (38.0%), Austria (35.3%), Viet Nam (34.6%) and Australia (33.8%). Yet in 17 participating countries and economies, fewer than 10% of students attain Level 4 or above. In Indonesia, Colombia, Argentina, Jordan, Peru, Tunisia, Costa Rica, Brazil, Mexico and Albania, fewer than 5% of students attain Level 4 or above (Figure I.2.22 and Table I.2.1a). Proficiency at Level 3 (scores higher than 482 but lower than or equal to 545 points) At Level 3, students can execute clearly described procedures, including those that require sequential decisions. Their interpretations are sufficiently sound to be the basis for building a simple model or for selecting and applying simple problem-solving strategies. Students at this level can interpret and use representations based on different information sources and reason directly from them. They typically show some ability to handle percentages, fractions and decimal numbers, and to work with proportional relationships. Their solutions reflect that they have engaged in basic interpretation and reasoning. EVOLVING (Figure I.2.57) requires Level 3 proficiency. This question may appear very simple: DOOR Question 1 in R finding the angle of 120 degrees between the two door wings, but the student responses indicate it is at Level 3. communication This is probably because of the demand arising from as well as and mathematisation representation , the specific knowledge of circle geometry that is needed. The context of three-dimensional revolving doors has to be understood from the written descriptions. It also needs to be understood that the three diagrams in the initial stimulus provide different two-dimensional information about just one revolving door (not three doors) – first the diameter, then the directions in which people enter and exit from the door, and thirdly connecting the wings mentioned within the text with the lines of the diagrams. The fundamental mathematical capability of representation is required at a high level to interpret these diagrams mathematically. They give the view from above, but students also need to visualise real revolving doors especially in answering Questions 2 and 3. On average across OECD countries, 54.5% of students are proficient at Level 3 or higher (that is, at Level 3, 4, 5 or 6). More than three out of four students in Shanghai-China (88.7%), Singapore (79.5%), Hong Kong-China (79.5%) and Korea (76.2%) attain Level 3 or above. More than two out of three students are proficient at these levels in Chinese Taipei (74.0%), Macao-China (72.8%), Japan (72.0%), Liechtenstein (70.7%), Switzerland (69.8%), Estonia (67.5%), the Netherlands (67.3%) and Finland (67.2%). By contrast, in 22 participating countries, fewer than one in three students attains these levels. In Peru, Colombia and Indonesia, fewer than 10% of students perform at those levels (Figure I.2.22 and Table I.2.1a). CS S Kno W and Can d o: Student Performan C e in m athemati What Student , r eading and S C ien C e – Volume i 67 OECD 2014 ©

70 2 S f Student Perform A nce i n mA them A tic o A Profile Proficiency at Level 2 (scores higher than 420 but lower than or equal to 482 points) At Level 2, students can interpret and recognise situations in contexts that require no more than direct inference. They can extract relevant information from a single source and make use of a single representational mode. Students at this level can employ basic algorithms, formulae, procedures or conventions to solve problems involving whole numbers. They are capable of making literal interpretations of the results. Results from longitudinal studies in Australia, Canada, Denmark and Switzerland show that students who perform below Level 2 often face severe disadvantages in their transition into higher education and the labour force in subsequent years. The proportion of students who perform below this baseline proficiency level thus indicates the degree of difficulty countries face in providing their populations with a minimum level of competencies (OECD, 2012). Question 1 in the unit HELEN THE CYCLIST (Figure I.2.55) is typical of Level 2 tasks. Question 1, a simple multiple choice item, requires comparison of speed when travelling 4 km in 10 minutes versus 2 km in 5 minutes. It is been classified within the employing process category because it requires the precise mathematical understanding that speed is a rate and that proportionality is the key. This question can be solved by recognising the doubles involved (2 km – 4 km; 5 km – 10 km), which is the very simplest notion of proportion. Consequently, with this Level 2 question, successful students demonstrate a very basic understanding of speed and of proportion calculations. If distance and time are in the same proportion, the speed is the same. Of course, students could correctly solve the problem in more complicated ways (e.g. calculating that both speeds are 24 km per hour) but this is not necessary. PISA results for this question do not incorporate information about the solution method used. The correct response option here is B (Helen’s average speed was the same in the first 10 minutes and in the next 5 minutes). Level 2 is considered the baseline level of mathematical proficiency that is required to participate fully in modern China, - society. More than 90% of students in the four top-performing countries and economies in PISA 2012, Shanghai Singapore, Hong K ong-China and Korea, meet this benchmark. Across OECD countries, an average of 77% of students attains Level 2 or higher: more than one in two students perform at these levels in all OECD countries except Chile (48.5%) and Mexico (45.3%). Only around one in four students in the partner countries Colombia, Peru and Indonesia attains this benchmark (Figure I.2.22 and Table I.2.1a). Proficiency at Level 1 (scores higher than 358 but lower than or equal to 420 points) or below At Level 1 students can answer questions involving familiar contexts where all relevant information is present and the questions are clearly defined. They are able to identify information and carry out routine procedures according to direct instructions in explicit situations. They can perform actions that are almost always obvious and follow immediately from the given stimuli. Students below Level 1 may be able to perform very direct and straightforward mathematical tasks, such as reading a single value from a well-labelled chart or table where the labels on the chart match the words in the stimulus and question, so that the selection criteria are clear and the relationship between the chart and the aspects of the context depicted are evident, and performing arithmetic calculations with whole numbers by following clear and well-defined instructions. Question 1 in GARAGE (Figure I.2.60) is a task that corresponds to the top of Level 1 in difficulty, very close to the Level 1/Level 2 boundary on the proficiency scale. It asks students to identify a picture of a building from the back, given the view from the front. The diagrams must be interpreted in relation to the real world positioning of “from the interpreting process. The correct response is C. Mental rotation tasks such as back”, so this question is classified in the this are solved by some people using intuitive spatial visualisation. Other people need explicit reasoning processes. They may analyse the relative positions of multiple features (door, window, nearest corner), discounting the multiple choice alternatives one by one. Others might draw a bird’s eye view, and then physically rotate it. This is just one example of how different students may use quite different methods to solve PISA questions: in this case explicit reasoning for some students is intuitive for others. Question 1 in CHARTS (Figure I.2.59), with a difficulty of 347.7, is a task below Level 1 on the mathematical proficiency scale, being one of the easiest tasks in the PISA 2012 item pool. It requires the student to find the bars for April, select the correct bar for the Metafolkies, and read the height of the bar to obtain the required response selection B (500). No scale reading or interpolation is required. All PISA participating countries and economies show students at Level 1 or below; but the largest proportions of students who attain only these levels are found in the lowest-performing countries. C d © OECD 2014 What Student S Kno W i e – Volume o: Student Performan ien C eading and S r , CS athemati m e in C and Can 68

71 2 tic f Student Perform A nce i n mA them A o S A Profile Across OECD countries, an average of 23.0% of students is proficient only at or below Level 1. In Shanghai-China, Singapore, Hong Kong-China and Korea, fewer than 10% of students perform at or below Level 1. Fewer than 15% do in Estonia, Macao-China, Japan, Finland, Switzerland, Chinese Taipei, Canada, Liechtenstein, Viet Nam, Poland and the Netherlands. By contrast, in 31 participating countries and economies more than one out of four students perform at these levels. In 15 countries the proportion of students who attain only Level 1 or below exceeds 50% (Figure I.2.22 and Table I.2.1a). Trends in the percentage of low- and top-performers in mathematics Changes in a country’s or economy’s average performance can result from changes at different levels of the performance distribution. For example, for some countries and economies, average improvement is driven by improvements among low-achieving students, where the share of students scoring below Level 2 is reduced. In other countries and economies, average improvement is driven mostly by changes among high-achieving students, where the share of students who perform at or above Level 5 increases. On average across OECD countries with comparable data, between 2003 and 2012 there was an increase of 0.7 percentage points in the share of students who do not meet the baseline proficiency level in mathematics and a reduction of 1.6 percentage points in the share of students at or above proficiency Level 5 (Figure I.2.23 and Table I.2.1b). However, these trends vary across countries. Some countries and economies saw a reduction in the proportion of low- performing students and a concurrent increase in the proportion of top-performing students. These are school systems that have seen improvements in performance both at the bottom and the top ends of the performance distribution. There are other countries where improvements are limited to reducing the share of low-performing students or increasing the share of top-performing students. Countries and economies can be grouped into categories based on whether they have: simultaneously reduced the share of low performers and increased the share of top performers between previous PISA assessments and PISA 2012; reduced the share of low performers but not increased the share of top performers between any previous PISA assessment and PISA 2012; increased the share of top performers but not reduced the share of low performers; and reduced the share of top performers or increased the share of low performers between PISA 2012 and any previous PISA assessment. The following section groups countries along these categories, first identifying those that have simultaneously reduced the share of low performers and increased the share of top performers between PISA 2003 and PISA 2012, between PISA 2006 and PISA 2012 or between PISA 2009 and PISA 2012. The remaining countries and economies are categorised as those that reduced the share of low performing students, increased the share of top performing students, or that saw an increase in the share of low performers or a reduction in the share of top performers. Moving everyone up: Reductions in the share of low performers and increases in that of top performers Countries and economies that have reduced the proportion of students scoring below Level 2 and increased the proportion of students scoring above Level 5 are ones that have been able to spread the improvements in their education systems across all levels of performance. Between 2003 and 2012 this was observed in Italy, Poland and Portugal. This reduction in the share of low-performers and increase in the share of high-performers was observed in Israel, Romania and Qatar between PISA 2006 and PISA 2012, and in Ireland, Malaysia and the Russian Federation between PISA 2009 and PISA 2012 (Figure I.2.23 and Table I.2.1b). Poland, for example, reduced the share of students scoring below Level 2 by eight percentage points while increasing the share of high achievers by seven percentage points between 2003 and 2012. A large part of this change is concentrated in the 2009 to 2012 period. In 2003, 2006 and 2009 about 20% of students were low-performers and around 10% were top-performers; by 2012 the share of students scoring below Level 2 dropped to 14% and the share of students scoring at or above Level 5 increased to 17%. Similarly, Portugal reduced the share of students scoring below Level 2 by five percentage points and increased the share of students scoring at or above Level 5 also by five percentage points during the period, with most of this change taking place between 2006 and 2009. Italy saw an overall reduction of seven percentage points in the share of students performing below Level 2 and an increase of three percentage points in the share of students scoring at or above Level 5, with most of this change taking place between 2006 and 2009 (Figure I.2.23 and Table I.2.1b). Annex B4 illustrates, for each country and economy, how mathematics performance at the 10th, 25th, 75th and 90th percentiles has evolved since 2003. Like the trends in the share of low- and top-performing students, it shows that average improvement in Poland and Italy, for example, is observed among low-, average and high-achieving students alike. i e in S Kno W and Can d o: Student Performan 69 OECD 2014 © What Student e – Volume C ien C eading and S r , CS athemati m C

72 2 o nce i n mA them A tic S A Profile f Student Perform A Reducing underperformance: Reductions in the share of low performers but no change in that of top performers Other countries and economies have concentrated change among those students who did not meet the baseline proficiency level. These countries and economies saw significant improvements in the performance of students who need it most and who now have basic skills and competencies to fully participate in society. Between 2003 and 2012, Brazil, Mexico, Tunisia and Turkey saw a reduction of more than five percentage points in the share of students scoring below proficiency Level 2 in mathematics. Germany also saw significant reductions in the proportion of students at proficiency Level 2, but no change in the proportion of those scoring at or above Level 5. Similarly, Bulgaria and Montenegro, both of which began participating in PISA after 2003, showed significant reductions in the proportion of students scoring at Level 2 between 2006 and 2012, as did Albania, Dubai (United Arab Emirates) and Kazakhstan between 2009 and 2012 (Figure I.2.23 and Table I.2.1b). Annex B4 shows the performance trajectories of these countries and economies, highlighting how the performance of their lowest achievers (those in the 10th percentile of performance) improved more than that of the highest-achieving students (those in the 90th percentile). By lifting the performance of their lowest-achieving students, these countries and economies have narrowed the gap between high- and low-achieving students and, in some cases, increased equity as well, as many low-achieving students are also from disadvantaged backgrounds (see Volume II, Chapter 2). Figure I.2.23 • • p ercentage of low-performing students and top performers in mathematics in 2003 and 2012 2003 2012 80 Students at or above prociency Level 5 70 60 50 40 Percentage of students 30 20 10 0 5.3 6.7 5.7 2.9 6.1 -5.4 -1.6 -5.0 -7.8 -8.1 -3.9 -2.0 -5.7 -6.3 -6.9 -5.9 -1.4 -4.3 Italy Spain Brazil Japan Latvia Korea France Turkey Tunisia Poland Austria Ireland Greece Finland Iceland Mexico Canada Norway Sweden Belgium Portugal Uruguay Thailand Hungary Australia Germany Denmark Indonesia Switzerland Netherlands Luxembourg New Zealand Liechtenstein United States Macao-China Czech Republic Slovak Republic Hong Kong-China Russian Federation OECD average 2003 3.9 2.5 7.6 5.3 5.5 0.7 3.7 5.1 5.7 4.4 7.5 6.5 2.6 9.8 7.7 -6.3 -7.7 -3.9 -5.2 -7.3 -8.1 0 -10.2 -10.2 -11.2 10 20 30 40 Percentage of students 50 60 70 Students below prociency Level 2 80 The chart shows only countries/economies that participated in both PISA 2003 and PISA 2012 assessments. Notes: The change between PISA 2003 and PISA 2012 in the share of students performing below Level 2 in mathematics is shown below the country/economy name. The change between PISA 2003 and PISA 2012 in the share of students performing at or above Level 5 in mathematics is shown above the country/economy name. Only statistically signicant changes are shown (see Annex A3). OECD average 2003 compares only OECD countries with comparable mathematics scores since 2003. Countries and economies are ranked in descending order of the percentage of students at or above prociency Level 5 in mathematics in 2012. OECD, PISA 2012 Database, Table I.2.1b. Source: 1 http://dx.doi.org/10.1787/888932935572 2 m © What Student S Kno W and Can d o: Student Performan C e in athemati CS , r eading and S i C ien C e – Volume OECD 2014 70

73 2 A f Student Perform A nce i n mA them o tic S A Profile Nurturing top performance: Increase in the share of top performers but no change in that of low performers Some countries and economies increased the proportion of students performing at or above Level 5. These are students who can handle complex mathematical content and processes. Higher proportions of these students signal a school system’s capacity to promote student performance at the highest level. Between 2003 and 2012, Korea and Macao-China saw around a six percentage-point increase in the share of students performing at this level. Other increases in the proportion of students scoring at or above Level 5 were observed in Chinese Taipei, Hong Kong-China, Japan, Serbia and Thailand (between 2006 and 2012) and in Estonia, Latvia, Shanghai-China and Singapore (between 2009 and 2012) (Figure I.2.23 and Table I.2.1b). As shown in Annex B4, the trajectories of these countries’ and economies’ low- and high-achieving students point to greater increases among the high achievers than among the low achievers. When comparing Korea’s mathematics scores in 2012 with those of 2003, for example, students in the 90th percentile improved by 20 scores points, and those at the 75th percentile improved by 18 points; however, there was no change in mathematics performance among those students in the 10th and 25th percentiles. That is, if those students at the bottom of the distribution performed at similar levels in 2003 and 2012, those at the top attained higher levels in 2012 than they did in 2003. Increase in the share of low performers or decrease in that of top performers There are 17 countries and economies, however, where the proportion of students who do not reach the baseline proficiency level increased or the proportion of students who reach the highest levels of proficiency decreased between a previous PISA assessment and PISA 2012. In these countries and economies there were fewer students performing at the top levels and more students who did not show the baseline level of mathematical literacy in 2012 than there were in a previous assessment (Figure I.2.23 and Table I.2.1b). Variation in student performance in mathematics The standard deviation in PISA scores, the difference between the top and bottom 5% of sampled students and the difference between the top and bottom 10%, or between the top and bottom quarters are all measures of the extent to which student performance varies among 15-year-olds. In fact, each of these measures gives more or less the same picture. Table I.2.3a shows the mean, standard deviation and percentiles of PISA mathematics scores for all participating countries and economies. As shown in Figure I.2.24, the ten PISA participants with the widest spread in scores (score-point difference between the top and bottom 10% of students) are Israel, Belgium, the Slovak Republic, New Zealand, France and Korea as well as the partner countries and economies Chinese Taipei, Singapore, Shanghai-China and Qatar. This group includes four of the highest-performing countries and economies (Chinese Taipei, Singapore, Shanghai-China and Korea), one of the lowest performers (Qatar) as well as two OECD countries that perform close to the OECD average (France, which is at the OECD average, and New Zealand, which is just above the OECD average) (Table I.2.3a). The ten participating countries/economies with the narrowest spread are Mexico and the partner countries Costa Rica, Indonesia, Kazakhstan, Colombia, Jordan, Argentina, Tunisia, Brazil and Thailand. All of these countries are among the 20 lowest-performing countries; seven of them are among the 10 lowest-performing countries. Less variation in performance is observed among the very lowest-performing countries, largely because there are fewer scores at the highest proficiency levels and, as a result, scores tend to be concentrated at the lower proficiency levels (Figure I.2.24 and Table I.2.3a). It is noteworthy that the relationship between average performance and the spread in student scores is weak, suggesting that high mean performance does not inevitably lead to large disparities in student performance. It is possible to combine a relatively narrow spread of scores and a relatively high average score, as does, for example, Estonia. Gender differences in mathematics performance Figure I.2.25 presents a summary of boys’ and girls’ performance in the PISA mathematics assessment (Table I.2.3a). On average across OECD countries, boys outperform girls in mathematics by 11 score points. Despite the stereotype that boys are better than girls at mathematics, boys show an advantage in only 38 out of the 65 countries and economies that participated in PISA 2012, and in only six countries is the gender gap larger than the equivalent of half a school year. As shown in Figure I.2.25, the largest difference in scores between boys and girls – in favour of boys – is seen in the partner country Colombia, and the OECD countries Luxembourg and Chile, a difference of around 25 points. In the partner countries Costa Rica, Liechtenstein and the OECD country Austria, this difference is between 22 and 24 points. e – Volume C S Kno W and Can d 71 OECD 2014 © i What Student C ien C eading and S r , CS athemati m e in o: Student Performan

74 2 o A nce i n mA f Student Perform A tic S A Profile them • Figure I.2.24 • r elationship between performance in mathematics and variation in performance 650 performance and performance and Above-average Above-average 1. United Kingdom below-average variation variation above-average 2. Czech Republic 3. Slovenia 4. Austria Shanghai-China 600 5. United States Lithuania 6. Japan Macao-China Singapore Chinese Taipei Performance in mathematics Hong Kong-China Canada 550 Korea Switzerland Poland Viet Nam Liechtenstein Finland Netherlands Estonia Belgium Germany 4 Ireland Australia 500 3 2 Denmark New Zealand OECD average: 494 points 1 Spain Latvia France 5 Russian Federation Slovak Republic Italy 6 Norway Portugal Israel Croatia Sweden Hungary Luxembourg 450 Greece Iceland Romania Turkey Kazakhstan Bulgaria Serbia Malaysia Thailand Mexico Chile Montenegro Costa Rica 400 Uruguay Argentina Brazil United Jordan Albania Arab Emirates Qatar Indonesia Tunisia Colombia Peru 350 performance and Below-average performance and Below-average variation below-average above-average variation OECD average: 239 points difference 300 More variation Less variation 190 230 170 310 250 210 290 150 270 Variation in mathematics performance (score-point difference between 90th and 10th percentiles) Source: OECD, PISA 2012 Database, Table I.2.3a. 2 1 http://dx.doi.org/10.1787/888932935572 In Korea, Japan and the partner economy Hong Kong-China, all of which are among the 10 top-performing countries, as well as in Italy, Spain, Ireland and New Zealand, and in the partner countries Peru, Brazil and Tunisia, this difference is between 15 and 20 points. In Luxembourg, a larger proportion of boys than girls attains the three highest proficiency levels, and far fewer boys than girls are found in the three lowest proficiency levels, leading to a marked overall gender difference in favour of boys (Tables I.2.2a and I.2.3a). In contrast, in only five countries do girls outperform boys in mathematics. The largest difference is seen in the partner country Jordan, where girls score around 21 points higher than boys. Girls also outperform boys in the partner countries Qatar, Thailand, Malaysia and in the OECD country Iceland (Figure I.2.25 and Table I.2.3a). In all of these countries more boys score at or below Level 1 than girls. The difference is particularly large in the partner country Jordan, where around 43% of boys score at or below Level 1, compared to around 30% of girls. In Iceland, while girls and boys are well-represented at all proficiency levels, far more boys than girls score below proficiency Level 1 (Table I.2.2a). Figure I.2.26 shows the average proportions of boys and girls in OECD countries within each of the defined mathematics proficiency levels. Larger proportions of boys than girls score at Level 5 or 6 (top performers) and at Level 4. Conversely, the proportion of girls is larger than the proportion of boys at all other proficiency levels, from Level 3. In almost all participating countries and economies, a larger proportion of boys than girls are top performers in mathematics (Level 5 or 6). In high-performing countries and economies, where a relatively large share of students performs at these levels, the difference in the proportion of boys and girls scoring at these levels is generally larger. and Can , CS athemati i e in C o: Student Performan d e – Volume W Kno S What Student OECD 2014 C © ien C eading and S r m 72

75 2 o A nce i n mA f Student Perform A tic S A Profile them Figure I.2.25 • • g ender differences in mathematics performance Girls Boys All students Gender differences Mean score on the mathematics scale – (boys girls) Jordan Qatar Thailand Malaysia Iceland United Arab Emirates Latvia Singapore Finland Sweden Girls perform Boys perform Bulgaria better better Russian Federation Albania Montenegro Lithuania Kazakhstan Norway Macao-China Slovenia Romania Poland Indonesia United States Estonia Chinese Taipei Shanghai-China Turkey Greece France Hungary Serbia OECD average Slovak Republic 11 score points Viet Nam Canada Netherlands OECD average Belgium Portugal Uruguay Croatia Israel Czech Republic Australia United Kingdom Switzerland Germany Argentina Denmark Mexico New Zealand Tunisia Ireland Hong Kong-China Spain Brazil Japan Korea Italy Peru Austria Liechtenstein Costa Rica Chile Luxembourg Colombia 450 -30 650 10 20 30 300 350 400 -10 500 550 600 -20 0 Mean score Score-point difference Statistically signicant gender differences are marked in a darker tone (see Annex A3). Note: girls). – Countries and economies are ranked in ascending order of the gender score-point difference (boys OECD, PISA 2012 Database, Table I.2.3a. Source: 2 1 http://dx.doi.org/10.1787/888932935572 o: Student Performan OECD 2014 m athemati CS , r eading and S C ien C C e in d and Can W Kno S What Student e – Volume i © 73

76 2 o A nce i n mA f Student Perform A tic S A Profile them Figure I.2.26 • • roficiency in mathematics among boys and girls p ECD average percentages of boys and girls at each level of mathematics proficiency o Boys Girls Level 6 2.4 4.2 Level 5 8.2 10.5 Level 4 17.6 18.7 24.3 Level 3 23.2 21.3 23.6 Level 2 15.8 14.2 Level 1 8.1 7.9 Below Level 1 % % 0 5 10 15 25 30 35 05101520253035 20 OECD, PISA 2012 Database, Table I.2.2a. Source: 2 1 http://dx.doi.org/10.1787/888932935572 For example, in the high-performing OECD countries Korea and Japan, and the partner economy Hong Kong-China, the share of boys who are top performers is around 9 percentage points larger than that of girls. In Israel, Austria, Italy, New Zealand and Luxembourg, which are situated in the middle of the performance distribution, the share of boys who attain at the highest proficiency levels is considerably larger than the share of girls who do, by a difference of 7.7 to 5.8 percentage points. This difference is also larger than 5 percentage points in Belgium, Chinese Taipei, the Slovak Republic, Spain, Canada, Liechtenstein, Switzerland and Germany (Table I.2.2a). While the proportion of girls is larger than the share of boys at the lower proficiency levels, there is considerable variation among countries and economies. In around a third of participating countries and economies, a higher proportion of boys than girls do not achieve the baseline level of proficiency. In Finland, Iceland and the partner countries Thailand, Jordan, Malaysia, the United Arab Emirates, Lithuania, Latvia and Singapore, a larger proportion of boys than girls perform below Level 2, the baseline proficiency level, and some of these countries, like Finland and the partner country Singapore, belong to the 15 top-performing countries and economies. Yet in many of the 15 lowest-performing countries and economies, including the OECD countries Chile and Mexico and the partner countries Costa Rica, Colombia, Brazil, Tunisia, Argentina and Peru, more girls than boys do not attain that level of proficiency. But in Luxembourg, which scores around the OECD average, and Liechtenstein, which scores well above the OECD average, the share of girls who score at or below Level 1 is considerably larger than that of boys by a difference of 8.6 and 6.1 percentage points, respectively (Table I.2.2a). Trends in gender differences in mathematics performance Among the countries and economies that showed a gender gap in mathematics performance in favour of boys in 2003, by 2012 the gender gap narrowed by nine score points or more in Finland, Greece, Macao-China, the Russian Federation and Sweden. Thus, in Greece, while boys outperformed girls in mathematics by 19 points in 2003, by 2012 this difference had shrunk to eight score points. In Finland, Macao-China, the Russian Federation, Sweden, Turkey and the United States, there was no longer a gender gap in mathematics performance favouring boys in 2012 compared to 2003. In Austria, Luxembourg and Spain, the gender gap favouring boys widened between 2003 and 2012. For example, in Austria in 2003, there was no observed gender gap in mathematics performance; but by 2012 there was a 22 score-point difference in performance in favour of boys. Iceland was one of the few countries where o: Student Performan e – Volume ien C eading and S r , CS athemati m e in C C d and Can W Kno S What Student OECD 2014 © i 74

77 2 A nce i n mA them A tic S f Student Perform A Profile o girls outperformed boys in mathematics in 2003; in 2012, girls still outperformed boys, but the gender gap had narrowed (Figure I.2.27 and Table I.2.3c). Countries seeking to reduce girls’ disadvantage in mathematics could examine the experiences of Korea, Latvia, China, the Russian Federation and Thailand. In Macao-China and the Russian Federation, for example, girls’ - Macao mathematics performance improved by around 20 score points while boys’ performance did not change, resulting in a narrowing of the gender gap in mathematics performance to the extent that the gender gap observed in 2003 lost statistical significance by 2012. In Thailand, boys’ performance did not change between PISA 2003 and PISA 2012, but girls’ performance improved by 14 score points. • Figure I.2.27 • c hange between 2003 and 2012 in gender differences in mathematics performance Gender differences in mathematics performance in 2012 Gender differences in mathematics performance in 2003 35 Boys perform better 30 25 20 Score-point difference 15 10 5 0 -5 -10 -15 Girls perform better -20 9 8 8 -9 15 -18 -10 -12 -11 Italy Brazil Spain Japan Latvia Korea France Turkey Tunisia Poland Austria Ireland Greece Finland Iceland Mexico Canada Norway Sweden Belgium Portugal Uruguay Thailand Hungary Australia Germany Denmark Indonesia Switzerland Netherlands Luxembourg New Zealand Liechtenstein United States Macao-China Czech Republic Slovak Republic Hong Kong-China Russian Federation OECD average 2003 Notes: Gender differences in PISA 2003 and PISA 2012 that are statistically signicant are marked in a darker tone (see Annex A3). Statistically signicant changes in the score-point difference between boys and girls in mathematics performance between PISA 2003 and PISA 2012 are shown next to the country/economy name. OECD average 2003 compares only OECD countries with comparable mathematics scores since 2003. Countries and economies are ranked in ascending order of gender differences (boys-girls) in 2012. OECD, PISA 2012 Database, Table I.2.3c. Source: 1 2 http://dx.doi.org/10.1787/888932935572 These trends are also reflected in the changes in the proportion of boys and girls who can be considered top performers in PISA (those who score at or above proficiency Level 5) or who are considered low performers in PISA (because they score below proficiency Level 2). Consistent with the fact that the gender gap in mathematics has narrowed or now favours girls in certain countries and economies, in Latvia, Portugal, the Russian Federation and Thailand the share of girls who perform below proficiency Level 2 shrunk between 2003 and 2012 with no concurrent change in the share of low-performing boys. In Macao-China and the Russian Federation during the period, the share of top-performing girls increased with no such increase among boys. In addition, Italy, Poland, Portugal and the Russian Federation show a reduction in the share of girls who perform below Level 2 and an increase in the share of girls who perform at Level 5 or 6 (Table I.2.2b). r m C o: Student Performan d and Can W Kno S What Student athemati , e in eading and S C ien C e – Volume i © OECD 2014 75 CS

78 2 A Profile f Student Perform A nce i n o them A tic S mA razil b : a S pi mproving in i Box I.2.4. With an economy that traditionally relied on the extraction of natural resources and suffered stagnating growth and spells of hyperinflation until the early 1990s, Brazil is today rapidly expanding its industrial and service sector. Its population of more than 190 million, which is spread across 27 states in geographic areas as vast and diverse as Rio de Janeiro and the Amazon River basin, recognises the critical role education plays in the country’s economic development. Like only a handful of other countries, Brazil’s performance in mathematics, reading and science has improved notably over the past decade. Its mean score in the PISA mathematics assessment has improved by an average of 4.1 point per year – from 356 points in 2003 to 391 points in 2012. Since 2000, reading scores have improved by an average of 1.2 score points per year; and, since 2006, science scores have risen by an average of 2.3 score points per year. Lowest-achieving students (defined as the 10% of students who score the lowest) have improved their performance by 65 score points – the equivalent of more than a year and a half of schooling. Despite these considerable improvements, around two out of three Brazilian students still perform below Level 2 in mathematics (in 2003, three in four students did). Not only have most Brazilian students remarkably improved their performance, Brazil has expanded enrolment in primary and secondary schools. While in 1995, 90% of students were enrolled in primary schools at age seven, only half of them continued to finish eighth grade. In 2003, 35% of 15-year-olds were not enrolled in school in grade 7 or above; by 2012 this percentage had shrunk to 22%. Enrolment rates for 15-year-olds thus increased, from 65% in 2003 to 78% in 2012. Many of the students who are now included in the school system come from rural communities or socio-economically disadvantaged families, so the population of students who participated in the PISA 2012 assessment is very different from that of 2003. PISA compares the performance of 15-year-old students who are enrolled in schools; but for those countries where this population has changed dramatically in a short period of time, trend data for students with similar background characteristics provide another way of examining how students’ performance is changing beyond changes in enrolment. Figure I.2.c compares the performance of students with similar socio-economic status across all years. The score attained by a socio-economically advantaged/average/disadvantaged student increased by 21/25/27 points, respectively, between 2003 and 2012. The figure also simulates alternate scenarios, assuming that the students who are now enrolled in schools – but probably weren’t in 2003 – score in the bottom half of the performance distribution, the bottom quarter of the performance distribution, or the bottom of the distribution and also come from the bottom half, bottom quarter, and bottom of the socio-economic distribution. Given that they assume that the newly enrolled students have lower scores than students who would have been enrolled in 2003, these simulations indicate the upper bounds of Brazil’s improvement in performance. For example, under the assumption that the newly enrolled students perform in the bottom quarter of mathematics performance, Brazil’s improvement in mathematics, had enrolment rates retained their 2003 levels, would have been 56 score points. Similarly, if the assumption is that newly enrolled students come from the bottom quarter of the socio-economic distribution, Brazil’s improvement in mathematics between 2003 and 2012 would have been 44 score points had enrolment rates not increased since 2003. Still, it is the observed enrolment rates and the observed performance in 2003 and 2012 that truly reflect the student population, its performance and the education challenges facing Brazil. Brazil’s increases in coverage are remarkable. However, although practically all students aged 7-14 start school at the beginning of the year, few continue until the end. They leave because the curriculum isn’t engaging, or because they want or need to work, or because of the prevalence of grade repetition. The pervasiveness of grade repetition in Brazil has been linked to high dropout rates, high levels of student disengagement, and the more than 12 years it takes students, on average, to complete eight grades of primary school. (PISA results suggest that repetition rates remain high in Brazil: in 2003, 33% of students reported having repeated at least one grade in primary or secondary education; in 2012, 36% of students reported so.) ... C d © OECD 2014 What Student S Kno W i e – Volume o: Student Performan ien C eading and S r , CS athemati m e in C and Can 76

79 2 o A nce i n mA f Student Perform A tic S A Profile them Figure I.2.c • • o bserved and expected trends in mathematics performance for b razil (2003-12) Change between 2003 and 2012 2003) – 2012 2003 (2012 -43 404 Total number of 15-year-olds 3 618 332 3 574 928 2 786 064 +426 210 2 359 854 Total 15-year-olds enrolled in grades 7 or higher Enrolment rates for 15-year-old students 65% 78% +19% S.E. Mean S.E. Mean S.E. Mean Mathematics performance 356 (4.8) 391 (2.1) +35.4 (5.6) c omparing the performance students with similar socio-economic backgrounds: Advantaged student in 2003 (6.0) 383 (5.2) 404 (2.3) +20.5 (4.7) +24.9 382 (4.0) 357 Average student in 2003 (1.6) +27.3 (1.7) 369 (3.9) 342 Disadvantaged student in 2003 (4.7) a v erage performance excluding newly enrolled students assuming that newly enrolled students are at: Bottom half of performance 356 (4.8) 406 (2.2) +49.7 (5.6) Bottom quarter of performance 356 (4.8) 412 (2.0) (5.6) +56.4 (1.8) (5.5) +58.6 (4.8) 356 415 Bottom of the distribution a erage performance excluding newly enrolled students assuming that newly enrolled students come from: v 356 (4.8) 397 (2.2) +40.5 (5.7) Bottom half of ESCS (4.8) Bottom quarter of ESCS 356 (5.7) 399 (2.3) +43.5 (5.7) Bottom of ESCS 356 (4.8) 400 (2.3) +44.1 Learning for Tomorrow’s World: First Results from PISA 2003 Notes: Enrolment rates are those reported as the coverage index 3 in Annex A3 in PISA index of economic, social and cultural (OECD, 2004) and in Annex A2 of this volume. An advantaged/disadvantaged student is one who has a (ESCS) that places him/her at the top/lower end of the fourth/first quartile of ESCS in 2003. Average students are those with an ESCS equal status to the average in 2003. Average performance in PISA 2012 that excludes newly enrolled students assuming that they come from the bottom half /quarter of performance and ESCS is calculated by randomly deleting 19% of the sample only among students scoring bottom half/quarter in the performance and ESCS distribution, respectively. Average performance in PISA 2012 that excludes the bottom of the performance or ESCS distribution excludes the bottom 19% of the sample in the performance and ESCS distribution, respectively. Despite the fact that primary and secondary education is managed and largely funded at the municipal and state levels, the central government has been a key actor in driving and shaping education reform. Over the past 15 years it has actively promoted reforms to increase funding, improve teacher quality, set national curriculum standards, improve high school completion rates, develop and put in place accountability measures, and set student achievement and learning targets for schools, municipalities and states. After Brazil’s economy stabilised, in the mid-1990s, the Cardoso administration increased federal spending on primary education through FUNDEF ( Fundo de Manutenção e Desenvolvimento do Ensino Fundamental ) and simultaneously distributed the funding more equitably, replacing a population-density formula that allocated the majority of funds to large cities and linking part of the funding to school enrolments. This was only possible after developing a student and school census to gather and consolidate information about schools and students. FUNDEF also raised teachers’ salaries, increased the number of teachers, increased the length of teacher-preparation programmes, and contributed to higher enrolments in rural areas. A conditional cash-transfer programme for families who send their 7-14 year-old children to school ( Bolsa Escola ) lifted many families out of subsistence-level poverty encouraging their interest that their children receive an education. In 2006, the Lula administration expanded FUNDEF to cover early childhood and after-school learning and increased overall funding for education, renaming the programme FUNDEB, as it now covered basic education more broadly. The administration also expanded the conditional cash transfers to cover students aged 15-17, thereby encouraging enrolment in upper secondary education, where enrolment is lowest. This expansion means that 6.1% of Brazil’s GDP is now spent on education and the country aims to devote 10% of its GDP to education by 2020. Funding for this important increase in education expenditure will come from the recently approved allocation of 75% of public revenues from oil to education. Improving the quality of teachers has also been at the centre of Brazil’s reform initiatives. A core element of FUNDEF was increasing teacher salaries, which rose 13% on average after FUNDEF, and more than 60% in the poorer, northeast region of the country. At the same time, the 1996 Law of Directive and Bases of National Education (LDB) ... CS C i What Student OECD 2014 77 ien C eading and S r , e – Volume athemati m e in C o: Student Performan d and Can W Kno S ©

80 2 o A nce i n mA f Student Perform A tic S A Profile them mandated that, by 2006, all new teachers have a university qualification, and that initial and in-service teacher training programmes be free of charge. These regulations came at a time when coverage was expanding significantly, leading to an increase in the number of teachers in the system. In 2000, for example, there were 430 467 secondary school teachers, and 88% of whom had a tertiary degree; in 2012 there were 497 797 teachers, 95% of whom had tertiary qualifications (INEP, 2000 and 2012). Subsequent reforms in the late 2000s sought to create standards for teachers’ career paths based on qualifications, not solely on tenure. The planned implementation of a new examination system for teacher certification, covering both content and pedagogy, has been delayed. Although universities are free to determine their curriculum for teacher-training programmes, the establishment of an examination system to certify teachers sends a strong signal of what content and pedagogical orientation should be developed. To encourage more students to enrol – and stay – in school, upper secondary education has become mandatory (this policy is being phased in so that enrolment will be obligatory for students aged 4 to 17 by 2016), and a new grade level has been added at the start of primary school. Giving students more opportunities to learn in school has also meant shifting to a full school day, as underscored in the 2011-2020 National Plan for Education. Most school days are just four hours long; and even though FUNDEB provided incentives for full-day schools, they were not sufficient to prompt the investments in infrastructure required for schools that accommodate two or three shifts in a day to become full-day schools. Although enrolment in full-day schools increased 24% between 2010 and 2012, overall coverage in full-day schools remains low: only 2 million out of a total of almost 30 million students attended such schools in 2012 (INEP, 2013). The reforms of the mid-1990s included provisions to improve the education information system and increase school accountability. It transformed the National Institute for Educational Studies and Research into an independent organisation responsible for the national assessment and evaluation of education. It turned a national ) for grades 4, 8 and 11 and Prova Brazil assessment system into the Evaluation System for Basic Education (SAEB/ the National Secondary Education Examination in Grade 11, which provides qualifications for further studies or entry into the labour market. SAEB changed over time to become a national census-based assessment for students in grades 4 and 8 and its results were combined with repetition and dropout rates in 2005 to create an index of schools quality, the Basic Education Development Index (IDEB). This gave schools, municipalities and states an incentive to reduce retention and dropout rates and a benchmark against which to which monitor their progress. The IDEB is set individually for each school and is scaled so that its levels are aligned with those of PISA. Results are widely published, and schools that show significant progress are granted more autonomy while schools that Fundescola remain low performers are given additional assistance. Support for schools is also offered through the programme. IDEB provides targets for each school; it is up to the schools, municipalities and states to develop strategic improvement plans. In line with Brazil’s progress in PISA, national performance as measured by the SAEB has also improved between 1999 and 2009 (Bruns, Evans and Luque, 2011). Perhaps a result of these reforms, not only are more Brazilian students attending school and performing at higher index of teacher shortage levels, they are also attending better-staffed schools (the dropped from 0.47 in 2003 to 0.19 in 2012, and the number of students per teacher in a school fell from 34 to 28 in the same period), and schools with better material resources (the index of quality of educational resources increased from -1.17 to -0.54). They are also attending schools with better learning environments, as shown by improved disciplinary climates and student-teacher relations. Students in 2012 also reported spending one-and-a-half hours less per week on homework than their counterparts in 2003 did. Sources: , The World Bank, Washington, D.C. Bruns, B., D. Evans and J. Luque (2011), Achieving World-Class Education in Brazil o Básica 2000 INEP (Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira) (2000), , Sinopse Estatística da Educaçã INEP, Brasilia. , INEP (Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira) (2012), Sinopse Estatística da Educaçao Básica 2012 INEP, Brasilia. INEP (Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira) (2013), Censo da Educação Básica: 2012, Resumo , INEP, Brasilia. Técnico Lessons from PISA for the OECD (2010b), u OECD Publishing. , Strong Performers, Successful Reformers in Education, nited States http://dx.doi.org/10.1787/9789264096660-en , OECD Publishing. ECD Economic Surveys: Brazil o OECD (2011), http://dx.doi.org/10.1787/eco_surveys-bra-2011-en o: Student Performan e – Volume © OECD 2014 What Student S Kno W and Can d C C e in m athemati CS , r eading and S C ien i 78

81 2 o nce i n mA them A tic S A Profile f Student Perform A S of mathematic Student performance in different area S This section focuses on student performance on the process subscales of formulating , employing and interpreting ; and on change and relationships the content subscales of , . uncertainty and data and quantity , space and shape In general, the correlation between scores on the subscales and overall mathematics scores is high: students tend to perform as well on the mathematics subscales as they do in mathematics overall. However, there is some variation at the country level in the relationship between subscale performance and overall mathematics performance, which perhaps reflects differences in emphasis in the curriculum. Process subscales The three process categories in the mathematics framework relate to three parts of the mathematical modelling cycle, a key feature of the way PISA assesses mathematics. As discussed earlier in this chapter, each item in the PISA 2012 mathematics survey was assigned to one of the process categories, even if solving an item often involves more than one of these processes. About a quarter of the items was designed primarily to elicit indicators of the process; about half of them required formulating situations mathematically mainly the employing mathematical concepts, facts, procedures, and reasoning process; and the remaining quarter process. interpreting, applying and evaluating mathematical outcomes emphasised the Student performance on the mathematics subscale formulating situations mathematically In order for individuals to use their mathematical knowledge and skills to solve a problem, they often first need to translate the problem into a form that is amenable to mathematical treatment. The framework refers to this process as one of formulating situations mathematically. In the PISA assessment, students may need to recognise or introduce simplifying assumptions that would help make the given mathematics item amenable to analysis. They have to identify which aspects of the problem are relevant to the solution and which might safely be ignored. They must recognise words, images, relationships or other features of the problem that can be given a mathematical form; and they need to express the relevant information in an appropriate way, for example in the form of a numeric calculation or as an algebraic expression. This process is sometimes referred to as the problem as expressed, usually in real-world terms, into a mathematical problem. For example, in translating a problem about some form of motion (such as travel on public transport, or riding a bicycle), the student may need to recognise a reference to “speed” and understand that this is referring to the relationship between the distance travelled distance as an essential step in giving the speed = over a given time period, and perhaps invoke the formula / time problem a clearly mathematical form. EVOLVING DOOR Question 2 and Question 3, Items listed in Figure I.2.9 that have been classified in this category are R F and C LIMBING Question 1 and Question 2. UJI M OUNT Across OECD countries, the average score attained on the formulating subscale is 492 points. A substantially lower score on the formulating subscale compared to average scores in the other processes or in mathematics overall might process more difficult. This would be expected when students indicate that some students might find the formulating have less experience with this process, for example, when most students in school work on mathematics problems that have already been “translated” into mathematical form. Top-performing countries and economies on this subscale are Shanghai-China, Singapore, Chinese Taipei, Hong Kong-China, Korea, Japan, Macao-China, Switzerland, Liechtenstein and the Netherlands (Figure I.2.28 and Table I.2.7). While across OECD countries, the average formulating score (492) is slightly lower than the average overall score for mathematics (494), this is not the case in the ten highest-performing countries on the overall mathematics scale. For nine of those countries and economies, the average national score on the formulating subscale is higher than the average overall score in mathematics. This is the case in Shanghai-China, Singapore, Hong Kong-China, Korea, Macao China, Switzerland and the Netherlands, w - here the mean score in formulating is between 4 and 12 points higher than the overall mathematics average, and is particularly evident in Chinese Taipei and Japan, where it is 19 and 18 points higher, respectively, than the overall mathematics average. This implies that in these countries, students find the formulation process to be a relatively easy aspect of mathematics. The only exception among this highest-performing formulating group is Liechtenstein, where the mean score is similar to the country’s mean overall mathematics score (Figure I.2.37). CS OECD 2014 S Kno W and Can d o: Student Performan C e in m athemati What Student , r eading and S C ien C e – Volume i © 79

82 2 o nce i n mA them A tic S A Profile f Student Perform A • • Figure I.2.28 formulating omparing countries’ and economies’ performance on the mathematics subscale c the OECD average above Statistically significantly Not statistically significantly different from the OECD average the OECD average below Statistically significantly m omparison c ean not statistically significantly different from that comparison country’s/economy’s score ountries/economies whose mean score is c country/economy score 624 Shanghai-China Singapore 582 Chinese Taipei Chinese Taipei Singapore, Hong Kong-China 578 Chinese Taipei, Korea 568 Hong Kong-China 562 Hong Kong-China, Japan Korea 554 Japan Korea 545 Macao-China Switzerland 538 Macao-China, Liechtenstein Switzerland 535 Liechtenstein Switzerland, Netherlands Liechtenstein, Finland Netherlands 527 Finland Netherlands, Estonia, Canada, Poland, Belgium 519 Finland, Canada, Poland, Belgium, Germany 517 Estonia 516 Finland, Estonia, Poland, Belgium, Germany Canada 516 Finland, Estonia, Canada, Belgium, Germany Poland 512 Finland, Estonia, Canada, Poland, Germany Belgium Estonia, Canada, Poland, Belgium, Denmark 511 Germany 502 Germany, Iceland, Austria, Australia, Viet Nam, New Zealand, Czech Republic Denmark 500 Iceland Denmark, Austria, Australia, Viet Nam, New Zealand, Czech Republic 499 Austria Denmark, Iceland, Australia, Viet Nam, New Zealand, Czech Republic, Ireland Denmark, Iceland, Austria, Viet Nam, New Zealand, Czech Republic, Ireland 498 Australia Denmark, Iceland, Austria, Australia, New Zealand, Czech Republic, Ireland, Slovenia, Norway, United Kingdom, Latvia Viet Nam 497 496 New Zealand Denmark, Iceland, Austria, Australia, Viet Nam, Czech Republic, Ireland, Slovenia, Norway, United Kingdom Czech Republic 495 Denmark, Iceland, Austria, Australia, Viet Nam, New Zealand, Ireland, Slovenia, Norway, United Kingdom, Latvia Austria, Australia, Viet Nam, New Zealand, Czech Republic, Slovenia, Norway, United Kingdom, Latvia 492 Ireland 492 Viet Nam, New Zealand, Czech Republic, Ireland, Norway, United Kingdom, Latvia Slovenia 489 Norway Viet Nam, New Zealand, Czech Republic, Ireland, Slovenia, United Kingdom, Latvia, France, Russian Federation, Slovak Republic 489 United Kingdom Viet Nam, New Zealand, Czech Republic, Ireland, Slovenia, Norway, Latvia, France, Luxembourg, Russian Federation, Slovak Republic, Portugal Latvia 488 Viet Nam, Czech Republic, Ireland, Slovenia, Norway, United Kingdom, France, Luxembourg, Russian Federation, Slovak Republic, Portugal 483 France Norway, United Kingdom, Latvia, Luxembourg, Russian Federation, Slovak Republic, Sweden, Portugal, Lithuania, Spain, United States United Kingdom, Latvia, France, Russian Federation, Slovak Republic, Sweden, Portugal, Lithuania, United States 482 Luxembourg Russian Federation 481 Norway, United Kingdom, Latvia, France, Luxembourg, Slovak Republic, Sweden, Portugal, Lithuania, Spain, United States, Italy Slovak Republic Norway, United Kingdom, Latvia, France, Luxembourg, Russian Federation, Sweden, Portugal, Lithuania, Spain, United States, Italy 480 479 Sweden France, Luxembourg, Russian Federation, Slovak Republic, Portugal, Lithuania, Spain, United States, Italy 479 Portugal United Kingdom, Latvia, France, Luxembourg, Russian Federation, Slovak Republic, Sweden, Lithuania, Spain, United States, Italy, Hungary 477 Lithuania France, Luxembourg, Russian Federation, Slovak Republic, Sweden, Portugal, Spain, United States, Italy, Hungary 477 Spain France, Russian Federation, Slovak Republic, Sweden, Portugal, Lithuania, United States, Italy, Hungary France, Luxembourg, Russian Federation, Slovak Republic, Sweden, Portugal, Lithuania, Spain, Italy, Hungary, Israel United States 475 Italy Russian Federation, Slovak Republic, Sweden, Portugal, Lithuania, Spain, United States, Hungary 475 469 Hungary Portugal, Lithuania, Spain, United States, Italy, Israel Israel 465 United States, Hungary, Croatia 453 Croatia Israel, Turkey, Greece, Serbia, Romania, Kazakhstan 449 Turkey Croatia, Greece, Serbia, Romania, Kazakhstan, Bulgaria 448 Greece Croatia, Turkey, Serbia, Romania, Kazakhstan 447 Serbia Croatia, Turkey, Greece, Romania, Kazakhstan, Bulgaria 445 Romania Croatia, Turkey, Greece, Serbia, Kazakhstan, Bulgaria 1, 2 Kazakhstan 442 Croatia, Turkey, Greece, Serbia, Romania, Bulgaria, Cyprus 1, 2 437 Bulgaria Turkey, Serbia, Romania, Kazakhstan, Cyprus 1, 2 437 Cyprus Kazakhstan, Bulgaria 426 United Arab Emirates Chile 420 United Arab Emirates, Thailand Chile Chile, Mexico, Uruguay, Malaysia 416 Thailand Thailand, Uruguay, Malaysia Mexico 409 406 Uruguay Thailand, Mexico, Malaysia, Montenegro, Costa Rica Thailand, Mexico, Uruguay, Montenegro, Costa Rica, Albania Malaysia 406 404 Montenegro Uruguay, Malaysia, Costa Rica 399 Uruguay, Malaysia, Montenegro, Albania, Jordan Costa Rica Albania Malaysia, Costa Rica 398 390 Jordan Costa Rica, Argentina 383 Argentina Jordan, Qatar, Brazil, Colombia, Tunisia Argentina, Brazil, Colombia, Tunisia Qatar 378 Brazil 376 Argentina, Qatar, Colombia, Tunisia, Peru, Indonesia 375 Colombia Argentina, Qatar, Brazil, Tunisia, Peru, Indonesia 373 Tunisia Argentina, Qatar, Brazil, Colombia, Peru, Indonesia 370 Brazil, Colombia, Tunisia, Indonesia Peru 368 Indonesia Brazil, Colombia, Tunisia, Peru 1. Note by Turkey: The information in this document with reference to ”Cyprus” relates to the southern part of the Island. There is no single authority representing both Turkish and Greek Cypriot people on the Island. Turkey recognises the Turkish Republic of Northern Cyprus (TRNC). Until a lasting and equitable solution is found within the context of the United Nations, Turkey shall preserve its position concerning the “Cyprus issue”. 2. Note by all the European Union Member States of the OECD and the European Union: The Republic of Cyprus is recognised by all members of the United Nations with the exception of Turkey. The information in this document relates to the area under the effective control of the Government of the Republic of Cyprus. OECD, PISA 2012 Database. Source: http://dx.doi.org/10.1787/888932935572 12 o: Student Performan e – Volume © OECD 2014 What Student S Kno W and Can d C C e in m athemati CS , r eading and S C ien i 80

83 2 o A nce i n mA them f Student Perform tic S A Profile A • Figure I.2.29 • formulating Summary descriptions of the six proficiency levels for the mathematical subscale P ercentage of students able to perform tasks at each level or above ( average) cd E o What students can do l evel 6 Students at or above Level 6 can apply a wide variety of mathematical content knowledge 5.0% to transform and represent contextual information or data, geometric patterns or objects into a mathematical form amenable to investigation. At this level, students can devise and follow a multi-step strategy involving significant modelling steps and extended calculation to formulate and solve complex real-world problems in a range of settings, for example involving material and cost calculations in a variety of contexts, or to find the area of an irregular region on a map; identify what information is relevant (and what is not) from contextual information about travel times, distances and speed to formulate appropriate relationships among them; apply reasoning across several linked variables to devise an appropriate way to present data in order to facilitate pertinent comparisons; and devise algebraic formulations that represent a given contextual situation. 5 At this level, students can use their understanding in a range of mathematical areas to 14.5% transform information or data from a problem context into mathematical form. They can transform information from different representations involving several variables, into a form suitable for mathematical treatment. They can formulate and modify algebraic expressions of relationships among variables; use proportional reasoning effectively to devise computations; gather information from different sources to formulate and solve problems involving geometric objects, features and properties, or analyse geometric patterns or relationships and express them in standard mathematical terms; transform a given model according to changed contextual circumstances; formulate a sequential calculation process based on text descriptions; and activate statistical concepts, such as randomness, or sample, and apply probability to formulate a model. 4 At Level 4, students can link information and data from related representations (for example, 31.1% a table and a map, or a spread sheet and a graphing tool) and apply a sequence of reasoning steps in order to formulate the mathematical expression needed to carry out a calculation or otherwise to solve a contextual problem. At this level, students can formulate a linear equation from a text description of a process, for example in a sales context, and formulate and apply cost comparisons to compare prices of sale items; identify which of given graphical representations corresponds to a given description of a physical process; specify a sequential calculation process in mathematical terms; identify geometrical features of a situation and use their geometric knowledge and reasoning to analyse a problem, for example to estimate areas or to link a contextual geometric situation involving similarity to the corresponding proportional reasoning; combine multiple decision rules needed to understand or implement a calculation where different constraints apply; and formulate algebraic expressions when the contextual information is reasonably straight-forward, for example to connect distance and speed information in time calculations. 3 At this level, students can identify and extract information and data from text, tables, graphs, 52.7% maps or other representations, and make use of them to express a relationship mathematically, including interpreting or adapting simple algebraic expressions related to an applied context. Students at this level can transform a textual description of a simple functional relationship into a mathematical form, for example with unit costs or payment rates; form a strategy involving two or more steps to link problem elements or to explore mathematical characteristics of the elements; apply reasoning with geometric concepts and skills to analyse patterns or identify properties of shapes or a specified map location, or to identify information needed to carry out some pertinent calculations, including calculations involving the use of simple proportional models and reasoning, where the relevant data and information is immediately accessible; and understand and link probabilistic statements to formulate probability calculations in contexts, such as in a manufacturing process or a medical test. 2 At this level, students can understand written instructions and information about simple 74.0% processes and tasks in order to express them in a mathematical form. They can use data presented in text or in a table (for example, giving information about the cost of some product or service) to formulate a computation required, such as to identify the length of a time period, or to present a cost comparison, or calculate an average; analyse a simple pattern, for example by formulating a counting rule or identifying and extending a numeric sequence; work effectively with different two- and three-dimensional standard representations of objects or situations, for example devising a strategy to match one representation with another compare different scenarios, or identify random experiment outcomes mathematically using standard conventions. 1 At this level students can recognise or modify and use an explicit simple model of a contextual 89.7% situation. Students can choose between several such models to match the situation. For example, they can choose between an additive and a multiplicative model in a shopping context; choose among given two-dimensional objects to represent a familiar three-dimensional object; and select one of several given graphs to represent growth of a population. CS OECD 2014 S Kno W and Can d o: Student Performan C e in m athemati What Student , r eading and S C ien C e – Volume i © 81

84 2 A Profile nce i n mA them A tic A f Student Perform o S Figure I.2.30 • • p roficiency in the mathematics subscale formulating P ercentage of students at each level of mathematics proficiency Level 3 Level 6 Level 5 Level 4 Below Level 1 Level 1 Level 2 Shanghai-China Shanghai-China Singapore Singapore Korea Korea Hong Kong-China Hong Kong-China Japan Japan Switzerland Switzerland Macao-China Macao-China Chinese Taipei Chinese Taipei Students at Level 1 Liechtenstein Liechtenstein or below Estonia Estonia Finland Finland Netherlands Netherlands Canada Canada Poland Poland Denmark Denmark Iceland Iceland Germany Germany Belgium Belgium Viet Nam Viet Nam Ireland Ireland Latvia Latvia Czech Republic Czech Republic Austria Austria Norway Norway Australia Australia Slovenia Slovenia New Zealand New Zealand OECD average OECD average United Kingdom United Kingdom Russian Federation Russian Federation France France Luxembourg Luxembourg Spain Spain Sweden Sweden Lithuania Lithuania Italy Italy Slovak Republic Slovak Republic United States United States Portugal Portugal Hungary Hungary Israel Israel Greece Greece Croatia Croatia Kazakhstan Kazakhstan Serbia Serbia Romania Romania Turkey Turkey Bulgaria Bulgaria United Arab Emirates United Arab Emirates Chile Chile Thailand Thailand Mexico Mexico Uruguay Uruguay Malaysia Malaysia Montenegro Montenegro Albania Albania Costa Rica Costa Rica Jordan Jordan Argentina Argentina Qatar Qatar Brazil Brazil Peru Peru Students at Level 2 Tunisia Tunisia or above Colombia Colombia Indonesia Indonesia % % 20 0 80 40 60 80 60 40 20 100 100 Countries and economies are ranked in descending order of the percentage of students at Levels 2, 3, 4, 5 and 6. Source: OECD, PISA 2012 Database, Table I.2.5. 1 http://dx.doi.org/10.1787/888932935572 2 m Kno What Student OECD 2014 W and Can d o: Student Performan C e in S athemati , r © i eading and S C ien C e – Volume CS 82

85 2 tic A nce i n mA them A f Student Perform S A Profile o In Croatia, Brazil, Tunisia, Malaysia, Viet Nam, Thailand and the OECD countries France and Italy, there is a difference formulating subscale and overall mathematics performance. of at least 10 points between student performance on the In all these countries, the scores in are lower than the overall mathematics scores. All these countries show formulating an average overall score in mathematics below the OECD average, except France, which is at the OECD average, and Viet Nam, which is above the OECD average. are given in Figure I.2.29 formulating situations mathematically Descriptions of the six levels of proficiency on the subscale and the distribution of students among these six proficiency levels is shown in Figure I.2.30. Student performance on the mathematics subscale employing mathematical concepts, facts, procedures, and reasoning To employ mathematical concepts, facts, procedures and reasoning for the PISA assessment, students need to recognise which elements of their “mathematics tool kit” are relevant to the problem as it has been presented, or as they have formulated it, and apply that knowledge in a systematic and organised way to work towards a solution. For example, in a problem about travel on public transport or riding a bicycle, once the basic relationships underlying the problem have been understood and expressed in a suitable mathematical form, the student may need to carry out a calculation, substitute values into a formula, solve an equation, or apply their knowledge of the conventions of graphing to extract data or present information mathematically. EVOLVING ? CAR HICH Question 1, W DOOR Items listed in Figure I.2.9 that have been classified in this category are R Question 2 and Question 3, C UJI HARTS ARAGE Question 2, C Question 5, G LIMBING Question 3, and M F OUNT Question 1, Question 2 and Question 3. H THE CYCLIST ELEN subscale is 493 points – 0.6 score point below employing Across OECD countries, the average score attained on the the average score in overall mathematics proficiency. This small difference reflects both the centrality of using mathematical concepts, facts, procedures and reasoning in school mathematics classes and the fact that about half of the items in the PISA 2012 mathematics assessment are categorised as predominantly requiring the use of employing processes. Top-performing countries and economies on this subscale are Shanghai-China, Singapore, Hong Kong-China, Korea, Chinese Taipei, Liechtenstein, Macao-China, Japan, Switzerland and Estonia (Figure I.2.31 and Table I.2.10). The great majority of participating countries and economies have an average employing score that is within about five score points of their average score on the overall mathematics proficiency scale. Only Chinese Taipei has an average subscale that is more than 10 points lower than its average score in mathematics (an 11-point score on the employing difference), indicating that more students have difficulty using this process. By contrast, Viet Nam’s average score on the employing subscale is 12 points higher than its average score on the mathematics proficiency scale, suggesting that students in that country find this aspect of problem solving relatively easy (Figure I.2.37). employing mathematical concepts, facts, procedures, and Descriptions of the six levels of proficiency on the subscale are given in Figure I.2.32 and the distribution of students among these six proficiency levels is shown in reasoning Figure I.2.33. Student performance on the mathematics subscale interpreting, applying and evaluating mathematical outcomes In interpreting mathematical outcomes, students need to make links between the outcomes and the situation from which they arose. For example, in a problem requiring a careful interpretation of some graphical data, students would have to make connections among the objects or relationships depicted in the graph, and the answer to the question might involve interpreting those objects or relationships. In a problem about travel on public transport or riding a bicycle, once the basic relationships underlying the problem have been understood and expressed in a suitable mathematical form, the required mathematical processing has been carried out, and results generated, the student may need to evaluate the results in relation to the original problem, or may need to show how the mathematical information obtained relates to the contextual elements of the problem. Question 1 and Question 2, HARTS Items listed in Figure I.2.9 that have been classified in this category are C CAR HICH W Question 1. ARAGE ? Question 1, and G CS OECD 2014 S Kno W and Can d o: Student Performan C e in m athemati What Student , r eading and S C ien C e – Volume i © 83

86 2 f Student Perform nce i n mA them A tic S A Profile o A Figure I.2.31 • • employing omparing countries’ and economies’ performance on the mathematics subscale c Statistically significantly the OECD average above Not statistically significantly different from the OECD average below Statistically significantly the OECD average m ean omparison c ountries/economies whose mean score is statistically significantly different from that comparison country’s/economy’s score not c country/economy score 613 Shanghai-China 574 Singapore Hong Kong-China Korea 558 553 Korea Hong Kong-China, Chinese Taipei Korea 549 Chinese Taipei Liechtenstein 536 Macao-China, Japan, Switzerland Macao-China 536 Liechtenstein, Japan 530 Japan Liechtenstein, Macao-China, Switzerland, Estonia, Viet Nam Switzerland 529 Liechtenstein, Japan, Estonia, Viet Nam 524 Estonia Japan, Switzerland, Viet Nam, Poland, Netherlands 523 Viet Nam Japan, Switzerland, Estonia, Poland, Netherlands, Canada, Germany, Belgium, Finland 519 Estonia, Viet Nam, Netherlands, Canada, Germany, Belgium, Finland Poland Estonia, Viet Nam, Poland, Canada, Germany, Belgium, Finland 518 Netherlands 517 Canada Viet Nam, Poland, Netherlands, Germany, Belgium, Finland Viet Nam, Poland, Netherlands, Canada, Belgium, Finland, Austria 516 Germany 516 Belgium Viet Nam, Poland, Netherlands, Canada, Germany, Finland, Austria Finland 516 Viet Nam, Poland, Netherlands, Canada, Germany, Belgium, Austria Germany, Belgium, Finland, Slovenia, Czech Republic 510 Austria Slovenia Austria, Czech Republic, Ireland 505 504 Czech Republic Austria, Slovenia, Ireland, Australia, France 502 Ireland Slovenia, Czech Republic, Australia, France, Latvia 500 Australia Czech Republic, Ireland, France, Latvia, New Zealand Czech Republic, Ireland Australia, Latvia, New Zealand, Denmark, Luxembourg, United Kingdom, Portugal France 496 495 Latvia Ireland, Australia, France, New Zealand, Denmark, Luxembourg, United Kingdom, Iceland, Portugal 495 New Zealand Australia, France, Latvia, Denmark, Luxembourg, United Kingdom, Iceland, Portugal Denmark 495 France, Latvia, New Zealand, Luxembourg, United Kingdom, Iceland, Portugal 493 France, Latvia, New Zealand, Denmark, United Kingdom, Iceland, Portugal, Russian Federation Luxembourg France, Latvia, New Zealand, Denmark, Luxembourg, Iceland, Portugal, Russian Federation, Norway, Italy, Slovak Republic United Kingdom 492 490 Latvia, New Zealand, Denmark, Luxembourg, United Kingdom, Portugal, Russian Federation, Norway, Italy, Slovak Republic Iceland France, Latvia, New Zealand, Denmark, Luxembourg, United Kingdom, Iceland, Russian Federation, Norway, Italy, Slovak Republic, Portugal 489 Lithuania, Spain Hungary, United States 487 Luxembourg, United Kingdom, Iceland, Portugal, Norway, Italy, Slovak Republic, Lithuania, Spain, Hungary, United States, Croatia Russian Federation United Kingdom, Iceland, Portugal, Russian Federation, Italy, Slovak Republic, Lithuania, Spain, Hungary, United States, Croatia Norway 486 Italy United Kingdom, Iceland, Portugal, Russian Federation, Norway, Slovak Republic, Lithuania, Spain, Hungary, United States, Croatia 485 United Kingdom, Iceland, Portugal, Russian Federation, Norway, Italy, Lithuania, Spain, Hungary, United States, Croatia Slovak Republic 485 482 Portugal, Russian Federation, Norway, Italy, Slovak Republic, Spain, Hungary, United States, Croatia Lithuania 481 Spain Portugal, Russian Federation, Norway, Italy, Slovak Republic, Lithuania, Hungary, United States, Croatia 481 Hungary Portugal, Russian Federation, Norway, Italy, Slovak Republic, Lithuania, Spain, United States, Croatia, Sweden 480 Portugal, Russian Federation, Norway, Italy, Slovak Republic, Lithuania, Spain, Hungary, Croatia, Sweden, Israel United States 478 Russian Federation, Norway, Italy, Slovak Republic, Lithuania, Spain, Hungary, United States, Sweden, Israel Croatia Hungary, United States, Croatia, Israel Sweden 474 Israel 469 United States, Croatia, Sweden Greece, Turkey, Romania Serbia 451 1, 2 , Bulgaria Greece Serbia, Turkey, Romania, Cyprus 449 1, 2 , United Arab Emirates, Bulgaria 448 Turkey Serbia, Greece, Romania, Cyprus 1, 2 , United Arab Emirates, Bulgaria 446 Serbia, Greece, Turkey, Cyprus Romania 1, 2 Greece, Turkey, Romania, United Arab Emirates, Bulgaria 443 Cyprus 1, 2 , Bulgaria, Kazakhstan Turkey, Romania, Cyprus 440 United Arab Emirates 1, 2 , United Arab Emirates, Kazakhstan 439 Greece, Turkey, Romania, Cyprus Bulgaria United Arab Emirates, Bulgaria, Thailand 433 Kazakhstan Kazakhstan, Malaysia Thailand 426 Malaysia 423 Thailand, Chile Malaysia, Mexico, Uruguay 416 Chile Mexico Chile, Uruguay 413 Uruguay Montenegro 409 Uruguay 408 Chile, Mexico, Montenegro, Costa Rica 401 Costa Rica Uruguay, Albania, Tunisia Costa Rica, Tunisia Albania 397 390 Tunisia Costa Rica, Albania, Brazil, Argentina, Jordan 388 Tunisia, Argentina, Jordan Brazil Tunisia, Brazil, Jordan Argentina 387 Jordan 383 Tunisia, Brazil, Argentina 373 Qatar Indonesia, Peru, Colombia 369 Indonesia Qatar, Peru, Colombia 368 Qatar, Indonesia, Colombia Peru 367 Colombia Qatar, Indonesia, Peru 1. Note by Turkey: The information in this document with reference to ”Cyprus” relates to the southern part of the Island. There is no single authority representing both Turkish and Greek Cypriot people on the Island. Turkey recognises the Turkish Republic of Northern Cyprus (TRNC). Until a lasting and equitable solution is found within the context of the United Nations, Turkey shall preserve its position concerning the “Cyprus issue”. 2. Note by all the European Union Member States of the OECD and the European Union: The Republic of Cyprus is recognised by all members of the United Nations with the exception of Turkey. The information in this document relates to the area under the effective control of the Government of the Republic of Cyprus. Source: OECD, PISA 2012 Database. http://dx.doi.org/10.1787/888932935572 12 o: Student Performan e – Volume © OECD 2014 What Student S Kno W and Can d C C e in m athemati CS , r eading and S C ien i 84

87 2 o A nce i n mA f Student Perform A tic S A Profile them • • Figure I.2.32 Summary descriptions of the six proficiency levels for employing the mathematical subscale Percentage of students able to perform tasks at each level or above average) o cd ( What students can do E l evel 6 Students at or above Level 6 can use a strong repertoire of knowledge and procedural skills 2.8% in a wide range of mathematical areas. They can form and follow a multi-step strategy to solve a problem involving several stages; apply reasoning in a connected way across several problem elements; set up and solve an algebraic equation with more than one variable; generate relevant data and information to explore problems, for example using a spread sheet to sort and analyse data; and justify their results mathematically and explain their conclusions and support them with well-formed mathematical arguments. At Level 6 students’ work is consistently precise and accurate. 5 Students at Level 5 can use a range of knowledge and skills to solve problems. They can 12.1% sensibly link information in graphical and diagrammatic form to textual information. They can apply spatial and numeric reasoning skills to express and work with simple models in reasonably well-defined situations and where the constraints are clear. They usually work systematically, for example to explore combinatorial outcomes, and can sustain accuracy in their reasoning across a small number of steps and processes. They are generally able to work competently with expressions, can work with formulae and use proportional reasoning, and are able to work with and transform data presented in a variety of forms. 4 At Level 4, students can identify relevant data and information from contextual material 30.7% and use it to perform such tasks as calculating distances, using proportional reasoning to apply a scale factor, converting different units to a common scale, or relating different graph scales to each other. They can work flexibly with distance-time-speed relationships, and can carry out a sequence of arithmetic calculations. They can use algebraic formulations, and follow a straightforward strategy and describe it. 3 Students at Level 3 frequently have sound spatial reasoning skills enabling them, for 54.8% example, to use the symmetry properties of a figure, recognise patterns presented in graphical form, or use angle facts to solve a geometric problem. Students at this level can connect two different mathematical representations, such as data in a table and in a graph, or an algebraic expression with its graphical representation, enabling them, for example, to understand the effect of changing data in one representation on the other. They can handle percentages, fractions and decimal numbers and work with proportional relationships. 2 Students at Level 2 can apply small reasoning steps to make direct use of given information 77.3% to solve a problem, for example, to implement a simple calculation model, identify a calculation error, analyse a distance-time relationship, or analyse a simple spatial pattern. At this level students show an understanding of place value in decimal numbers and can use that understanding to compare numbers presented in a familiar context; correctly substitute values into a simple formula; recognise which of a set of given graphs correctly represents a set of percentages and apply reasoning skills to understand and explore different kinds of graphical representations of data; and can understand simple probability concepts. 1 Students at Level 1 can identify simple data relating to a real-world context, such as that 91.9% presented in a structured table or in an advertisement where the text and data labels match directly; perform practical tasks, such as decomposing money amounts into lower denominations; use direct reasoning from textual information that points to an obvious strategy to solve a given problem, particularly where the mathematical procedural knowledge required would be limited to, for example, arithmetic operations with whole numbers, or ordering and comparing whole numbers; understand graphing techniques and conventions; and use symmetry properties to explore characteristics of a figure, such as comparin g side lengths and angles. CS S Kno W and Can d o: Student Performan C e in m athemati What Student , r eading and S C ien C e – Volume i 85 OECD 2014 ©

88 2 A Profile A nce i n mA them A tic S f Student Perform o • • Figure I.2.33 p employing roficiency in the mathematics subscale P ercentage of students at each level of mathematics proficiency Level 6 Level 1 Below Level 1 Level 2 Level 3 Level 4 Level 5 Shanghai-China Shanghai-China Singapore Singapore Hong Kong-China Hong Kong-China Korea Korea Estonia Estonia Macao-China Macao-China Japan Japan Switzerland Switzerland Students at Level 1 Finland Finland or below Viet Nam Viet Nam Liechtenstein Liechtenstein Poland Poland Chinese Taipei Chinese Taipei Canada Canada Netherlands Netherlands Austria Austria Ireland Ireland Germany Germany Latvia Latvia Belgium Belgium Denmark Denmark Slovenia Slovenia Czech Republic Czech Republic Australia Australia Iceland Iceland France France Russian Federation Russian Federation OECD average OECD average United Kingdom United Kingdom Norway Norway Luxembourg Luxembourg New Zealand New Zealand Lithuania Lithuania Spain Spain Portugal Portugal Italy Italy Slovak Republic Slovak Republic United States United States Hungary Hungary Croatia Croatia Sweden Sweden Israel Israel Greece Greece Serbia Serbia Romania Romania Turkey Turkey United Arab Emirates United Arab Emirates Bulgaria Bulgaria Kazakhstan Kazakhstan Thailand Thailand Malaysia Malaysia Chile Chile Mexico Mexico Uruguay Uruguay Montenegro Montenegro Albania Albania Costa Rica Costa Rica Tunisia Tunisia Argentina Argentina Brazil Brazil Jordan Jordan Qatar Qatar Students at Level 2 Peru Peru or above Colombia Colombia Indonesia Indonesia % % 20 0 20 40 60 100 80 100 80 60 40 Countries and economies are ranked in descending order of the percentage of students at Levels 2, 3, 4, 5 and 6. Source: OECD, PISA 2012 Database, Table I.2.8. 1 2 http://dx.doi.org/10.1787/888932935572 r S What Student OECD 2014 CS , m eading and S C C e – Volume i athemati e in C o: Student Performan d and Can W © Kno ien 86

89 2 f Student Perform nce i n mA them A tic S A Profile o A Figure I.2.34 • • interpreting omparing countries’ and economies’ performance on the mathematics subscale c Statistically significantly the OECD average above Not statistically significantly different from the OECD average below Statistically significantly the OECD average m ean omparison c ountries/economies whose mean score is statistically significantly different from that comparison country’s/economy’s score not c country/economy score 579 Shanghai-China 555 Singapore Hong Kong-China, Chinese Taipei 551 Singapore, Chinese Taipei Hong Kong-China 549 Chinese Taipei Singapore, Hong Kong-China, Liechtenstein, Korea Chinese Taipei, Korea, Japan 540 Liechtenstein 540 Korea Chinese Taipei, Liechtenstein, Japan 531 Japan Liechtenstein, Korea, Macao-China, Switzerland, Finland, Netherlands 530 Macao-China Japan, Switzerland, Finland, Netherlands 529 Japan, Macao-China, Finland, Netherlands, Canada Switzerland Japan, Macao-China, Switzerland, Netherlands 528 Finland Japan, Macao-China, Switzerland, Finland, Canada, Germany Netherlands 526 521 Switzerland, Netherlands, Germany, Poland Canada Netherlands, Canada, Poland, Australia, Belgium, Estonia, New Zealand, France, Austria Germany 517 515 Poland Canada, Germany, Australia, Belgium, Estonia, New Zealand, France, Austria, Denmark, Ireland Germany, Poland, Belgium, Estonia, New Zealand, France, Austria 514 Australia 513 Germany, Poland, Australia, Estonia, New Zealand, France, Austria, Denmark, Ireland Belgium Germany, Poland, Australia, Belgium, New Zealand, France, Austria, Denmark, Ireland 513 Estonia Germany, Poland, Australia, Belgium, Estonia, France, Austria, Denmark, Ireland New Zealand 511 511 Germany, Poland, Australia, Belgium, Estonia, New Zealand, Austria, Denmark, Ireland France Austria 509 Germany, Poland, Australia, Belgium, Estonia, New Zealand, France, Denmark, Ireland, United Kingdom 508 Denmark Poland, Belgium, Estonia, New Zealand, France, Austria, Ireland, United Kingdom 507 Ireland Poland, Belgium, Estonia, New Zealand, France, Austria, Denmark, United Kingdom, Viet Nam 501 Austria, Denmark, Ireland, Norway, Italy, Slovenia, Viet Nam, Spain, Luxembourg, Czech Republic United Kingdom 499 Norway United Kingdom, Italy, Slovenia, Viet Nam, Spain, Luxembourg, Czech Republic, Iceland, Portugal, United States United Kingdom, Norway, Slovenia, Viet Nam, Spain, Luxembourg, Czech Republic, Portugal 498 Italy United Kingdom, Norway, Italy, Viet Nam, Spain, Luxembourg, Czech Republic, Portugal Slovenia 498 497 Ireland, United Kingdom, Norway, Italy, Slovenia, Spain, Luxembourg, Czech Republic, Iceland, Portugal, United States, Latvia Viet Nam United Kingdom, Norway, Italy, Slovenia, Viet Nam, Luxembourg, Czech Republic, Iceland, Portugal, United States Spain 495 United Kingdom, Norway, Italy, Slovenia, Viet Nam, Spain, Czech Republic, Iceland, Portugal, United States Luxembourg 495 United Kingdom, Norway, Italy, Slovenia, Viet Nam, Spain, Luxembourg, Iceland, Portugal, United States, Latvia Czech Republic 494 492 Norway, Viet Nam, Spain, Luxembourg, Czech Republic, Portugal, United States, Latvia Iceland 490 Norway, Italy, Slovenia, Viet Nam, Spain, Luxembourg, Czech Republic, Iceland, United States, Latvia, Sweden Portugal 489 United States Norway, Viet Nam, Spain, Luxembourg, Czech Republic, Iceland, Portugal, Latvia, Sweden Latvia Viet Nam, Czech Republic, Iceland, Portugal, United States, Sweden 486 485 Portugal, United States, Latvia, Croatia Sweden 477 Croatia Sweden, Hungary, Slovak Republic, Russian Federation, Lithuania Croatia, Slovak Republic, Russian Federation, Lithuania 477 Hungary 473 Slovak Republic Croatia, Hungary, Russian Federation, Lithuania, Greece, Israel Croatia, Hungary, Slovak Republic, Lithuania, Greece, Israel Russian Federation 471 471 Lithuania Croatia, Hungary, Slovak Republic, Russian Federation, Greece, Israel 467 Greece Slovak Republic, Russian Federation, Lithuania, Israel 462 Israel Slovak Republic, Russian Federation, Lithuania, Greece 446 Turkey Serbia, Bulgaria, Romania 445 Serbia Turkey, Bulgaria, Romania 1, 2 , Chile, Thailand 441 Bulgaria Turkey, Serbia, Romania, Cyprus 1, 2 , Chile, Thailand Turkey, Serbia, Bulgaria, Cyprus Romania 438 1, 2 Bulgaria, Romania, Chile, Thailand 436 Cyprus 1, 2 , Thailand, United Arab Emirates Bulgaria, Romania, Cyprus Chile 433 1, 2 , Chile, United Arab Emirates 432 Thailand Bulgaria, Romania, Cyprus Chile, Thailand United Arab Emirates 428 Kazakhstan 420 Malaysia, Costa Rica Kazakhstan, Costa Rica, Montenegro, Mexico 418 Malaysia Kazakhstan, Malaysia, Montenegro, Mexico 418 Costa Rica Malaysia, Costa Rica, Mexico, Uruguay Montenegro 413 Mexico Malaysia, Costa Rica, Montenegro, Uruguay 413 409 Montenegro, Mexico Uruguay 401 Brazil 390 Argentina Colombia, Tunisia, Jordan, Indonesia Argentina, Tunisia, Jordan, Indonesia 387 Colombia Argentina, Colombia, Jordan, Indonesia, Albania Tunisia 385 Argentina, Colombia, Tunisia, Indonesia, Albania Jordan 383 379 Indonesia Argentina, Colombia, Tunisia, Jordan, Albania, Qatar, Peru 379 Albania Tunisia, Jordan, Indonesia, Qatar Indonesia, Albania, Peru Qatar 375 368 Peru Indonesia, Qatar 1. Note by Turkey: The information in this document with reference to ”Cyprus” relates to the southern part of the Island. There is no single authority representing both Turkish and Greek Cypriot people on the Island. Turkey recognises the Turkish Republic of Northern Cyprus (TRNC). Until a lasting and equitable solution is found within the context of the United Nations, Turkey shall preserve its position concerning the “Cyprus issue”. 2. Note by all the European Union Member States of the OECD and the European Union: The Republic of Cyprus is recognised by all members of the United Nations with the exception of Turkey. The information in this document relates to the area under the effective control of the Government of the Republic of Cyprus. Source: OECD, PISA 2012 Database. 12 http://dx.doi.org/10.1787/888932935572 CS OECD 2014 S Kno W and Can d o: Student Performan C e in m athemati What Student , r eading and S C ien C e – Volume i © 87

90 2 o A nce i n mA f Student Perform A tic S A Profile them Figure I.2.35 • • interpreting Summary descriptions of the six proficiency levels for the mathematical subscale Percentage of students able to perform tasks at each level or above average) ( What students can do o cd E l evel 6 At Level 6, students can link multiple complex mathematical representations in an analytic way 4.2% to identify and extract data and information that enables contextual questions to be answered, and can present their interpretations and conclusions in written form. For example, students may interpret two time-series graphs in relation to different contextual conditions; or link a relationship expressed both in a graph and in numeric form (such as in a price calculator) or in a spread sheet and graph, to present an argument or conclusion about contextual conditions. Students at this level can apply mathematical reasoning to data or information presented in order to generate a chain of linked steps to support a conclusion (for example, analysing a map using scale information; analysing a complex algebraic formula in relation to the variables represented; translating data into a new time-frame; performing a three-way currency conversion; or using a data-generation tool to find the information needed to answer a question). Students at this level can gather analysis, data and their interpretation across several different problem elements or across different questions about a context, showing a depth of insight and a capacity for sustained reasoning. 5 At Level 5, students can combine several processes in order to formulate conclusions based 14.5% on an interpretation of mathematical information with respect to context, such as formulating or modifying a model, solving an equation or carrying out computations, and using several reasoning steps to make the links to the identified context elements. At this level, students can make links between context and mathematics involving spatial or geometric concepts and complex statistical and algebraic concepts. They can easily interpret and evaluate a set of plausible mathematical representations, such as graphs, to identify which one highest reflects the contextual elements under analysis. Students at this level have begun to develop the ability to communicate conclusions and interpretations in written form. 4 At Level 4, students can apply appropriate reasoning steps, possibly multiple steps, to 33.0% extract information from a complex mathematical situation and interpret complicated mathematical objects, including algebraic expressions. They can interpret complex graphical representations to identify data or information that answers a question; perform a calculation or data manipulation (for example, in a spread sheet) to generate additional data needed to decide whether a constraint (such as a measurement condition or a size comparison) is met; interpret simple statistical or probabilistic statements in such contexts as public transport, or health and medical test interpretation, to link the meaning of the statements to the underlying contextual issues; conceptualise a change needed to a calculation procedure in response to a changed constraint; and analyse two data samples, for example relating to a manufacturing process, to make comparisons and draw and express conclusions. 3 Students at Level 3 begin to be able to use reasoning, including spatial reasoning, to support 55.9% their interpretations of mathematical information in order to make inferences about features of the context. They combine reasoning steps systematically to make various connections between mathematical and contextual material or when required to focus on different aspects of a context, for example where a graph shows two data series or a table contains data on two variables that must be actively related to each other to support a conclusion. They can test and explore alternative scenarios, using reasoning to interpret the possible effects of changing some of the variables under observation. They can use appropriate calculation steps to assist their analysis of data and support the formation of conclusions and interpretations, including calculations involving proportions and proportional reasoning, and in situations where systematic analysis across several related cases is needed. At this level, students can interpret and analyse relatively unfamiliar data presentations to support their conclusions. 2 At Level 2, students can link contextual elements of the problem to mathematics, for example 77.0% by performing appropriate calculations or reading tables. Students at this level can make comparisons repeatedly across several similar cases: for example, they can interpret a bar graph to identify and extract data to apply in a comparative condition where some insight is required. They can apply basic spatial skills to make connections between a situation presented visually and its mathematical elements; identify and carry out necessary calculations to support such comparisons as costs across several contexts; and can interpret a simple algebraic expression as it relates to a given context. 1 At Level 1, students can interpret data or information expressed in a direct way in order 91.2% to answer questions about the context described. They can interpret given data to answer questions about simple quantitative relational ideas (such as “larger”, “shorter time”, “in between”) in a familiar context, for example by evaluating measurements of an object against given criterion values, by comparing average journey times for two methods of transport, or by comparing specified characteristics of a small number of similar objects. Similarly, they can make simple interpretations of data in a timetable or schedule to identify times or events. Students at this level may show rudimentary understanding of such concepts as randomness and data interpretation, for example by identifying the plausibility of a statement about chance outcomes of a lottery, by understanding numeric and relational information in a well-labelled graph, and by understanding basic contextual implications of links between related graphs. o: Student Performan e – Volume © OECD 2014 What Student S Kno W and Can d C C e in m athemati CS , r eading and S C ien i 88

91 2 A Profile A nce i n mA them A tic S f Student Perform o subscale is 497 points, 3 score points above the interpreting Across OECD countries, the average score attained on the average score of 494 points on the overall mathematics proficiency scale. A substantially higher average score on the subscale might indicate that students find interpreting mathematical information a relatively less difficult interpreting aspect of the problem-solving process, perhaps because the task of evaluating mathematical results is commonly treated as part of that process in school mathematics classes. Top-performing countries and economies on this subscale are Shanghai-China, Singapore, Hong Kong-China, Chinese Taipei, Liechtenstein, Korea, Japan, Macao-China, Switzerland and Finland (Figure I.2.34 and Table I.2.13). While across OECD countries the average score on the interpreting subscale is slightly higher than the average score on the mathematics proficiency scale, this is not the case in eight of the ten highest-performing countries and economies is lower than the interpreting on the overall mathematics scale. In those countries and economies, the average score in average score in overall mathematics proficiency, with a difference ranging from less than 10 points in Switzerland, Japan, Macao-China and Hong Kong-China, to between 10 and 20 points in Chinese Taipei, Korea and Singapore, to 34 points in Shanghai-China. In the high-performing OECD country, the Netherlands, and the partner country Liechtenstein, the opposite pattern is observed (Figure I.2.37). In fact, performance on the interpreting subscale does not appear to be related to overall mathematics performance. In interpreting subscale than they do in mathematics overall, eight countries, students score at least ten points higher on the while in eight other countries the interpreting score is at least 10 points lower than the overall score. This latter group of countries includes the four highest-performing countries (Chinese Taipei, Korea, Singapore and Shanghai-China), one high-performing country (Viet Nam), and three countries that perform below the OECD average (Albania, Kazakhstan and the Russian Federation). Descriptions of the six levels of proficiency on the subscale interpreting, applying and evaluating mathematical outcomes are given in Figure I.2.35 and the distribution of students among these six proficiency levels is shown in Figure I.2.36. The relative strengths and weaknesses of countries in mathematics process subscales Figure I.2.37 shows the country mean for the overall mathematics scale and the difference between each process subscale and the overall mathematics scale. As the figure makes clear, the levels of performance on the process subscales are somewhat aligned with each other and with the overall mean mathematics performance. However, it is also clear that countries’ and economies’ strengths in the three processes vary considerably. Across all participating countries and economies, the average difference between the highest and lowest performance in mathematics processes is around 14 points. Within that variability, 16 countries/economies show the highest mean and 28 countries/economies have the highest score in formulating ; 21 countries/economies perform best in employing; interpreting. mean score in Shanghai-China shows the largest difference (46 points) between its highest ( formulating ) and lowest ( interpreting ) performance in processes, followed by Chinese Taipei, which has a difference 30 points between its highest ( formulating ) ) performance in processes. France shows a large difference (27 points) between its highest employing and lowest ( ) and lowest ( interpreting ( formulating ) performance in processes, the largest among OECD countries, and Singapore . shows the same difference as France but its strongest performance is in formulating while its weakest is in interpreting Viet Nam has a difference of 26 points between its strongest ( employing ) and weakest ( interpreting ) process subscales, and both Brazil and Croatia shows a difference of 25 points between their strongest and weakest process subscales. Peru, Turkey, Uruguay and Belgium show a negligible difference (2 to 3 score points) between their highest and lowest performance in processes (Figure I.2.37). The OECD average difference between the highest and lowest performance in processes is around 5 points. Switzerland, , Iceland, Japan, Korea, the Netherlands and Turkey have the highest mean score in formulating and four of these countries are the best-performing OECD countries. Austria, Belgium, the Czech Republic, Estonia, Hungary, Israel, Mexico, Poland, the Slovak Republic and Slovenia perform best in employing ; and the remaining 18 OECD countries . have the highest mean scores in interpreting - China, Ten partner countries and economies – Shanghai-China, Chinese Taipei, Singapore, Kazakhstan, Albania, Hong Kong ; ten other partner countries and Macao-China, Jordan, Qatar and Peru – have the highest mean scores in formulating economies – Brazil, Colombia, Costa Rica, Thailand, Indonesia, Montenegro, Argentina, Liechtenstein, Bulgaria and Uruguay – perform best in ; and the remaining eleven partner countries and economies have the highest mean interpreting . employing scores in CS OECD 2014 S Kno W and Can d o: Student Performan C e in m athemati What Student , r eading and S C ien C e – Volume i © 89

92 2 A Profile nce i n mA them A tic A f Student Perform o S Figure I.2.36 • • p roficiency in the mathematics subscale interpreting P ercentage of students at each level of mathematics proficiency Level 3 Level 6 Level 5 Level 4 Below Level 1 Level 1 Level 2 Shanghai-China Shanghai-China Hong Kong-China Hong Kong-China Finland Finland Singapore Singapore Korea Korea Japan Japan Macao-China Macao-China Chinese Taipei Chinese Taipei Students at Level 1 Canada Canada or below Estonia Estonia Switzerland Switzerland Liechtenstein Liechtenstein Poland Poland Netherlands Netherlands Denmark Denmark Ireland Ireland Viet Nam Viet Nam Australia Australia Germany Germany France France Belgium Belgium New Zealand New Zealand Norway Norway Austria Austria United Kingdom United Kingdom Slovenia Slovenia Spain Spain Czech Republic Czech Republic OECD average OECD average Latvia Latvia Portugal Portugal Italy Italy Iceland Iceland United States United States Luxembourg Luxembourg Sweden Sweden Croatia Croatia Russian Federation Russian Federation Hungary Hungary Lithuania Lithuania Slovak Republic Slovak Republic Greece Greece Israel Israel Serbia Serbia Romania Romania Turkey Turkey Bulgaria Bulgaria Thailand Thailand Chile Chile United Arab Emirates United Arab Emirates Kazakhstan Kazakhstan Malaysia Malaysia Costa Rica Costa Rica Mexico Mexico Montenegro Montenegro Uruguay Uruguay Brazil Brazil Argentina Argentina Albania Albania Colombia Colombia Tunisia Tunisia Qatar Qatar Students at Level 2 Jordan Jordan or above Indonesia Indonesia Peru Peru % % 20 0 80 40 60 80 60 40 20 100 100 Countries and economies are ranked in descending order of the percentage of students at Levels 2, 3, 4, 5 and 6. Source: OECD, PISA 2012 Database, Table I.2.11. 1 http://dx.doi.org/10.1787/888932935572 2 m Kno What Student OECD 2014 W and Can d o: Student Performan C e in S athemati , r © i eading and S C ien C e – Volume CS 90

93 2 S nce i n mA them A tic A A Profile o f Student Perform • Figure I.2.37 • omparing countries and economies on the different mathematics process subscales c Country’s/economy’s performance on the subscale is between 0 to 3 score points than on the overall mathematics scale higher than on the overall mathematics scale higher Country’s/economy’s performance on the subscale is between 3 to 10 score points than on the overall mathematics scale higher Country’s/economy’s performance on the subscale is 10 or more score points Country’s/economy’s performance on the subscale is between 0 to 3 score points than on the overall mathematics scale lower lower Country’s/economy’s performance on the subscale is between 3 to 10 score points than on the overall mathematics scale than on the overall mathematics scale Country’s/economy’s performance on the subscale is 10 or more score points lower Performance difference between the overall mathematics scale and each process subscale nterpreting Emplo f i ying athematics score m ormulating 0 12 -34 613 Shanghai-China 8 573 Singapore -18 1 Hong Kong-China -10 -3 7 561 -11 -11 19 560 Chinese Taipei 8 554 Korea -14 -1 538 -9 -2 7 Macao-China -5 Japan 536 18 -6 Liechtenstein 535 5 0 1 -2 7 531 Switzerland -2 Netherlands 523 4 -4 3 -8 Estonia -3 4 521 Finland 9 -3 0 519 Canada 518 -2 -2 3 Poland -3 1 -2 518 515 Belgium -2 1 -2 3 Germany 514 -3 2 -14 12 -15 Viet Nam 511 Austria 3 -6 4 506 Australia 10 -4 -6 504 501 Ireland -9 1 5 -3 4 501 Slovenia -9 -5 8 Denmark 500 2 11 500 New Zealand -5 -4 Czech Republic 499 -4 5 -5 -12 16 1 France 495 -1 c D average 494 -2 3 OE United Kingdom 7 -2 -5 494 Iceland 493 7 -3 0 Latvia 491 -3 5 -4 -8 5 Luxembourg 490 3 9 -3 489 Norway 0 3 Portugal 487 -8 2 Italy 0 13 -10 485 11 -3 -8 Spain 484 Russian Federation 5 -11 -1 482 -8 4 -1 Slovak Republic 482 -1 -6 8 United States 481 -1 479 Lithuania -8 3 Sweden -4 7 478 1 Hungary 477 -8 4 0 6 6 Croatia 471 -19 -5 2 Israel -2 466 453 -5 -4 14 Greece -2 449 Serbia 2 -3 Turkey 448 1 0 -2 Romania 445 -6 1 0 1, 2 Cyprus 3 440 -3 -4 2 0 Bulgaria 439 -2 -6 United Arab Emirates 434 -8 6 10 432 Kazakhstan -12 1 Thailand 427 -1 5 -11 10 Chile 423 -3 -6 -3 2 -15 421 Malaysia 0 0 -4 413 Mexico Montenegro 4 0 -6 410 -3 0 409 Uruguay -2 11 407 -8 -6 Costa Rica Albania 394 4 3 -16 Brazil 10 391 -4 -16 Argentina 388 -5 -1 1 2 -15 Tunisia -3 388 -2 Jordan -3 386 4 -9 -2 376 Colombia 11 1 376 -3 -1 Qatar 4 Indonesia 375 -7 -6 2 368 Peru 0 0 1. Note by Turkey: The information in this document with reference to ”Cyprus” relates to the southern part of the Island. There is no single authority representing both Turkish and Greek Cypriot people on the Island. Turkey recognises the Turkish Republic of Northern Cyprus (TRNC). Until a lasting and equitable solution is found within the context of the United Nations, Turkey shall preserve its position concerning the “Cyprus issue”. 2. Note by all the European Union Member States of the OECD and the European Union: The Republic of Cyprus is recognised by all members of the United Nations with the exception of Turkey. The information in this document relates to the area under the effective control of the Government of the Republic of Cyprus. OECD, PISA 2012 Database, Tables I.2.3a, I.2.7, I.2.10 and I.2.13. Source: 12 http://dx.doi.org/10.1787/888932935572 CS OECD 2014 S Kno W and Can d o: Student Performan C e in m athemati What Student , r eading and S C ien C e – Volume i © 91

94 2 A Profile nce i n mA them A tic S A o f Student Perform • ] Part 1/3 [ Figure I.2.38 • Where countries and economies rank on the different mathematics process subscales Statistically significantly the OECD average above Not statistically significantly different from the OECD average the OECD average below Statistically significantly Formulating subscale ange of ranks r o ll countries/economies a countries cd E m ean score pper rank ower rank pper rank ower rank l u u l 1 1 624 Shanghai-China 582 2 3 Singapore 578 3 2 Chinese Taipei 5 4 568 Hong Kong-China 4 6 2 1 562 Korea 2 1 554 Japan 6 5 Macao-China 545 7 8 538 3 9 8 3 Switzerland Liechtenstein 535 8 10 10 527 4 5 9 Netherlands 11 Finland 14 8 5 519 15 Estonia 517 5 9 11 15 11 9 5 516 Canada Poland 16 11 10 5 516 Belgium 16 13 10 7 512 13 17 511 7 11 Germany 502 11 Denmark 16 20 14 15 Iceland 17 21 11 500 17 Austria 499 11 23 16 16 Australia 498 12 23 18 497 Viet Nam 27 17 New Zealand 496 12 18 18 25 27 19 18 12 495 Czech Republic 27 492 15 20 21 Ireland 20 16 492 Slovenia 27 22 29 489 16 21 22 Norway United Kingdom 489 15 22 22 31 Latvia 488 23 30 483 34 France 27 25 20 Luxembourg 24 21 482 33 29 37 Russian Federation 481 27 20 Slovak Republic 28 38 28 480 21 37 Sweden 479 27 29 20 479 38 Portugal 28 28 30 477 38 Lithuania 32 38 477 28 23 Spain 30 29 22 475 39 United States 29 24 475 Italy 39 33 40 469 27 30 37 Hungary Israel 465 28 30 38 41 Croatia 453 41 45 32 31 Turkey 449 46 41 45 Greece 448 31 32 41 46 Serbia 447 41 47 445 Romania 41 442 43 48 Kazakhstan 45 Bulgaria 437 48 1, 2 Cyprus 437 46 48 United Arab Emirates 49 50 426 51 49 33 33 420 Chile 52 50 416 Thailand Mexico 409 34 53 34 51 406 52 56 Uruguay Malaysia 406 52 56 Montenegro 404 53 56 54 399 57 Costa Rica Albania 398 56 57 58 Jordan 59 390 Argentina 383 58 61 62 378 Qatar 59 Brazil 64 60 376 59 375 64 Colombia Tunisia 373 60 65 62 65 370 Peru 65 62 368 Indonesia 1. Note by Turkey: The information in this document with reference to ”Cyprus” relates to the southern part of the Island. There is no single authority representing both Turkish and Greek Cypriot people on the Island. Turkey recognises the Turkish Republic of Northern Cyprus (TRNC). Until a lasting and equitable solution is found within the context of the United Nations, Turkey shall preserve its position concerning the “Cyprus issue”. 2. Note by all the European Union Member States of the OECD and the European Union: The Republic of Cyprus is recognised by all members of the United Nations with the exception of Turkey. The information in this document relates to the area under the effective control of the Government of the Republic of Cyprus. OECD, PISA 2012 Database. Source: http://dx.doi.org/10.1787/888932935572 12 o: Student Performan e – Volume © OECD 2014 What Student S Kno W and Can d C C e in m athemati CS , r eading and S C ien i 92

95 2 nce i n mA them A tic S A A Profile o f Student Perform • ] Part 2/3 [ Figure I.2.38 • Where countries and economies rank on the different mathematics process subscales the OECD average above Statistically significantly Not statistically significantly different from the OECD average below the OECD average Statistically significantly Employing subscale ange of ranks r ll countries/economies a countries cd E o m ean score ower rank pper rank ower rank pper rank u l u l 1 613 Shanghai-China 1 2 574 Singapore 2 4 3 558 Hong Kong-China 1 5 3 Korea 1 553 5 Chinese Taipei 549 4 8 6 536 Liechtenstein Macao-China 536 6 7 4 10 2 6 Japan 530 2 Switzerland 529 4 7 10 12 9 5 3 Estonia 524 17 8 Viet Nam 523 17 519 4 10 10 Poland 17 10 10 4 Netherlands 518 12 17 Canada 517 5 10 Germany 5 18 12 11 516 10 12 17 Belgium 516 5 Finland 12 516 17 6 10 510 Austria 19 12 9 16 Slovenia 505 12 14 19 21 18 Czech Republic 504 11 15 22 12 23 502 Ireland 19 16 Australia 500 13 23 20 16 15 496 France 28 22 20 22 Latvia 495 29 28 22 20 15 495 New Zealand 29 495 16 21 23 Denmark Luxembourg 493 17 21 25 29 United Kingdom 23 23 16 492 32 32 Iceland 490 19 23 27 Portugal 26 36 24 17 489 37 28 Russian Federation 487 20 486 Norway 26 36 28 36 27 22 485 Italy 30 38 28 21 485 Slovak Republic 28 32 482 Lithuania 39 24 481 28 Spain 33 39 32 481 40 23 29 Hungary 24 480 40 33 United States 29 478 35 41 Croatia 38 Sweden 28 30 474 41 Israel 469 29 30 39 41 45 42 451 Serbia Greece 45 42 32 31 449 47 Turkey 42 32 31 448 42 446 48 Romania 1, 2 Cyprus 443 44 47 United Arab Emirates 440 45 48 Bulgaria 439 45 49 50 433 Kazakhstan 48 426 Thailand 49 51 52 Malaysia 423 50 53 Chile 416 33 34 51 52 34 413 Mexico 54 33 409 54 55 Montenegro Uruguay 408 53 56 Costa Rica 401 55 57 56 58 397 Albania Tunisia 390 57 61 58 Brazil 61 388 Argentina 387 58 61 61 383 Jordan 59 62 63 373 Qatar Indonesia 369 62 65 65 Peru 368 62 65 Colombia 367 63 1. Note by Turkey: The information in this document with reference to ”Cyprus” relates to the southern part of the Island. There is no single authority representing both Turkish and Greek Cypriot people on the Island. Turkey recognises the Turkish Republic of Northern Cyprus (TRNC). Until a lasting and equitable solution is found within the context of the United Nations, Turkey shall preserve its position concerning the “Cyprus issue”. 2. Note by all the European Union Member States of the OECD and the European Union: The Republic of Cyprus is recognised by all members of the United Nations with the exception of Turkey. The information in this document relates to the area under the effective control of the Government of the Republic of Cyprus. OECD, PISA 2012 Database. Source: http://dx.doi.org/10.1787/888932935572 12 CS OECD 2014 S Kno W and Can d o: Student Performan C e in m athemati What Student , r eading and S C ien C e – Volume i © 93

96 2 nce i n mA them A tic S A A Profile o f Student Perform • ] Part 3/3 [ Figure I.2.38 • Where countries and economies rank on the different mathematics process subscales the OECD average above Statistically significantly Not statistically significantly different from the OECD average the OECD average below Statistically significantly Interpreting subscale ange of ranks r ll countries/economies o E cd countries a m ean score u ower rank pper rank pper rank u l l ower rank 579 1 Shanghai-China 1 555 Singapore 3 2 2 4 551 Hong Kong-China 5 549 Chinese Taipei 3 Liechtenstein 7 540 4 7 4 2 1 540 Korea Japan 531 2 5 6 11 7 10 530 Macao-China 5 Switzerland 529 2 7 11 11 5 7 Finland 528 2 7 526 Netherlands 12 2 6 13 Canada 521 5 7 11 12 6 517 Germany 18 12 14 20 12 Poland 515 6 Australia 514 18 13 12 7 7 20 14 13 Belgium 513 Estonia 14 13 20 513 8 New Zealand 22 8 511 14 16 22 France 511 9 16 14 15 Austria 509 9 17 23 17 11 508 Denmark 23 17 Ireland 507 12 17 18 23 29 22 21 15 501 United Kingdom 30 499 16 23 22 Norway 498 Italy 29 23 22 17 28 498 17 21 23 Slovenia Viet Nam 497 22 33 Spain 495 18 25 25 32 495 31 Luxembourg 26 24 20 Czech Republic 26 18 494 24 33 Iceland 492 21 26 28 33 Portugal 26 27 20 35 490 35 United States 489 21 27 28 35 Latvia 486 31 485 Sweden 33 27 25 36 477 39 35 Croatia 35 29 28 477 39 Hungary Slovak Republic 28 41 36 30 473 471 37 41 Russian Federation 41 471 37 Lithuania Greece 467 29 31 39 42 42 40 31 462 Israel 30 Turkey 32 446 32 43 46 43 Serbia 45 445 47 Bulgaria 441 43 438 Romania 48 44 1, 2 Cyprus 436 45 48 50 433 33 33 46 Chile 50 46 432 Thailand 50 48 428 United Arab Emirates Kazakhstan 53 51 420 Malaysia 418 51 55 51 418 Costa Rica 54 Montenegro 413 53 56 56 Mexico 413 34 34 53 56 409 54 Uruguay Brazil 401 57 57 58 Argentina 61 390 Colombia 387 58 61 385 62 Tunisia 58 Jordan 63 59 383 60 379 65 Indonesia Albania 379 61 64 63 64 375 Qatar 65 64 368 Peru 1. Note by Turkey: The information in this document with reference to ”Cyprus” relates to the southern part of the Island. There is no single authority representing both Turkish and Greek Cypriot people on the Island. Turkey recognises the Turkish Republic of Northern Cyprus (TRNC). Until a lasting and equitable solution is found within the context of the United Nations, Turkey shall preserve its position concerning the “Cyprus issue”. 2. Note by all the European Union Member States of the OECD and the European Union: The Republic of Cyprus is recognised by all members of the United Nations with the exception of Turkey. The information in this document relates to the area under the effective control of the Government of the Republic of Cyprus. OECD, PISA 2012 Database. Source: http://dx.doi.org/10.1787/888932935572 12 o: Student Performan e – Volume © OECD 2014 What Student S Kno W and Can d C C e in m athemati CS , r eading and S C ien i 94

97 2 o A nce i n mA them A f Student Perform S A Profile tic Gender differences in performance on the process subscales Figures I.2.39a, b and c show the extent of gender-related differences in performance on the three mathematical processes. In most countries, boys and girls show similar performance on the processes subscales as on the mathematics proficiency scale. Boys also outnumber girls in the top three proficiency levels of the subscales, while girls outnumber boys in the lower levels of the subscales (Tables I.2.6, I.2.9 and I.2.12). formulating subscale by around 16 points. The On average across OECD countries, boys outperform girls on the largest differences in favour of boys are observed in Luxembourg (33 points), Austria (32 points), Chile (29 points), Italy (24 points), New Zealand (23 points) and Korea (22 points). Ireland, Switzerland and Mexico show a gender difference of 20 points. The difference was less than 10 points in the United States (8 points). Among partner countries and economies, boys outperform girls by 33 points in Costa Rica, and by between 20 and 30 points in Colombia, Liechtenstein, Brazil, Tunisia, Peru, Hong Kong-China, and Uruguay. Several partner countries and economies show gender differences of less than 10 points, including Macao-China (9 points), Shanghai-China (8 points), Kazakhstan (7 points) and Montenegro (6 points). Only one country shows performance differences in favour of girls – Qatar (9 points). employing On average among OECD countries, boys outperform girls on the subscale by 9 points. In only one OECD country, Iceland, do girls outperform boys – by 7 points. Among partner countries and economies, girls outperform employing subscale in 6 countries and economies, notably in Jordan (25 points), Thailand (17 points), Qatar boys on the (15 points), Malaysia (9 points), Latvia (6 points) and Singapore (6 points). Boys outperform girls by more than 20 points in the partner countries Colombia (28 points) and Costa Rica (23 points). On average across OECD countries, boys outperform girls on the interpreting subscale by 9 points. The largest differences in favour of boys are recorded in Chile (22 points), Spain (21 points) and Luxembourg (20 points). Among partner countries and economies, large differences in favour of boys are recorded in Liechtenstein (27 points), Costa Rica (21 points) and Colombia (21 points). In Iceland and Finland, girls outperform boys by 11 points, and four partner countries show differences in favour of girls, with measurable differences in Jordan (25 points), Qatar (23 points), Thailand (15 points) and Malaysia (11 points). Content subscales The four content categories in the PISA 2012 assessment – change and relationships , space and shape , quantity uncertainty and data and – aim to capture broad groups of mathematical phenomena that involve different kinds of mathematical thinking and expertise, and that relate to broad parts of the mathematics curriculum found in all countries and economies. PISA outcomes presented according to this categorisation may reflect differences in curriculum priorities and in course content available to 15-year-olds. For example, in previous PISA assessment, a different profile of outcomes related to the uncertainty and data category compared to the other areas was observed and could be attributed to the fact that the teaching of probability and statistics is not uniform among countries/economies or even within them. Similarly, it might be expected that students who have studied predominantly basic computation and quantitative skills (related most quantity strongly to the category) might have different outcomes from those whose courses emphasised algebra and the study of mathematical functions and relations (which link most strongly to the change and relationships category); and that students in school systems that emphasise geometry can be expected to perform better on the items related to the space and shape category. change and relationships Student performance on the mathematics subscale PISA items in this category emphasise the relationships among objects, and the mathematical processes associated with changes in those relationships. Items listed in Figure I.2.9 that have been classified in this category are H ELEN THE UJI OUNT ELEN M LIMBING F Question 2. The questions in H Question 1, Question 2 and Question 3, and C CYCLIST CYCLIST relate to the relationships among the variables speed, distance and time in relation to travel by bicycle. THE also involves thinking about the relationships among the variables distance, speed and time UJI F C M LIMBING OUNT in relation to a walking trip. subscale is 493 points. The ten top-performing countries, with change and relationships The OECD average score on the a mean score of at least 530 points on this subscale, are Shanghai-China, Singapore, Hong Kong-China, Chinese Taipei, Korea, Macao-China, Japan, Liechtenstein, Estonia and Switzerland (Figure I.2.40 and Table I.2.16). The average score among OECD countries on this subscale is one point lower than the average score on the overall mathematics proficiency scale (Figure I.2.52). CS OECD 2014 S Kno W and Can d o: Student Performan C e in m athemati What Student , r eading and S C ien C e – Volume i © 95

98 2 A Profile A nce i n mA f Student Perform A tic S o them • Figure I.2.39a • subscale formulating ender differences in performance on the g All students Girls Boys Gender differences Mean score on the formulating subscale – (boys girls) Qatar Thailand Jordan Malaysia Latvia Albania Iceland Singapore United Arab Emirates Sweden Girls perform Boys perform Finland better better Norway Lithuania Bulgaria Russian Federation Montenegro Indonesia Kazakhstan Romania United States Slovenia Shanghai-China Macao-China Turkey Chinese Taipei Estonia United Kingdom Serbia Greece Canada Poland OECD average Belgium 16 score points Israel France Netherlands OECD average Slovak Republic Croatia Portugal Hungary Denmark Australia Czech Republic Argentina Viet Nam Germany Spain Japan Mexico Switzerland Ireland Uruguay Hong Kong-China Korea New Zealand Italy Peru Tunisia Brazil Liechtenstein Colombia Chile Austria Costa Rica Luxembourg 20 30 650 600 550 500 450 400 300 40 350 -20 0 -10 10 Score-point difference Mean score Note: Statistically signicant gender differences are marked in a darker tone (see Annex A3). girls). – Countries and economies are ranked in ascending order of the gender score-point difference (boys OECD, PISA 2012 Database, Table I.2.7. Source: http://dx.doi.org/10.1787/888932935572 2 1 m athemati CS i r e – Volume C and Can Kno S What Student W OECD 2014 © eading and S d o: Student Performan C e in ien C , 96

99 2 A Profile A nce i n mA f Student Perform A tic S o them • Figure I.2.39b • subscale employing ender differences in performance on the g All students Girls Boys Gender differences Mean score on the employing subscale – (boys girls) Jordan Thailand Qatar Malaysia Iceland Latvia United Arab Emirates Singapore Sweden Russian Federation Bulgaria Finland Albania Montenegro Poland Lithuania Girls perform Boys perform Kazakhstan better better Macao-China Norway Romania United States Slovenia Shanghai-China Chinese Taipei Indonesia Estonia Greece Turkey Croatia Slovak Republic Netherlands Viet Nam France Hungary OECD average Switzerland 9 score points Israel Portugal Serbia OECD average Canada Australia Uruguay Hong Kong-China Germany Belgium Czech Republic Denmark United Kingdom Ireland Argentina Mexico Spain New Zealand Japan Korea Italy Brazil Liechtenstein Peru Tunisia Austria Costa Rica Luxembourg Chile Colombia -20 10 650 600 550 500 450 400 350 300 0 40 30 20 -10 -30 Mean score Score-point difference Note: Statistically signicant gender differences are marked in a darker tone (see Annex A3). Countries and economies are ranked in ascending order of the gender score-point difference (boys – girls). OECD, PISA 2012 Database, Table I.2.10. Source: 1 http://dx.doi.org/10.1787/888932935572 2 S CS athemati m e in What Student 97 Kno W and Can d o: Student Performan C OECD 2014 ien e – Volume C eading and S r , © i C

100 2 A Profile A nce i n mA f Student Perform A tic S o them • Figure I.2.39c • subscale interpreting ender differences in performance on the g All students Girls Boys Gender differences Mean score on the interpreting subscale – (boys girls) Jordan Qatar Thailand Malaysia Iceland Finland Bulgaria United Arab Emirates Kazakhstan Singapore Russian Federation Montenegro Sweden Latvia Lithuania Slovenia Boys perform Girls perform Macao-China better better Indonesia Norway Chinese Taipei Poland Tunisia France Hungary Albania Estonia Romania Viet Nam Serbia Shanghai-China United States Uruguay OECD average Greece 9 score points Czech Republic Turkey Canada OECD average Slovak Republic Australia Argentina Netherlands Belgium Korea Mexico New Zealand Hong Kong-China Germany Portugal Brazil Switzerland United Kingdom Denmark Croatia Peru Austria Japan Ireland Israel Italy Luxembourg Spain Colombia Costa Rica Chile Liechtenstein 450 400 350 300 20 30 -30 0 10 -20 650 -10 600 550 500 Score-point difference Mean score Note: Statistically signicant gender differences are marked in a darker tone (see Annex A3). Countries and economies are ranked in ascending order of the gender score-point difference (boys – girls). Source: OECD, PISA 2012 Database, Table I.2.13. 1 http://dx.doi.org/10.1787/888932935572 2 © i athemati m e in C o: Student Performan d and Can W r eading and S CS OECD 2014 What Student S Kno e – Volume C ien C , 98

101 2 f Student Perform nce i n mA them A tic A A Profile o S • Figure I.2.40 • omparing countries’ and economies’ performance on the mathematics subscale c change and r elationships above the OECD average Statistically significantly Not statistically significantly different from the OECD average the OECD average below Statistically significantly omparison ean m c ountries/economies whose mean score is not statistically significantly different from that comparison country’s/economy’s score country/economy c score Shanghai-China 624 Singapore 580 564 Hong Kong-China Chinese Taipei, Korea Hong Kong-China, Korea 561 Chinese Taipei Korea Hong Kong-China, Chinese Taipei 559 542 Macao-China Japan, Liechtenstein Macao-China, Liechtenstein 542 Japan Liechtenstein 542 Macao-China, Japan 530 Estonia Switzerland, Canada 530 Switzerland Estonia, Canada Estonia, Switzerland, Finland, Netherlands Canada 525 520 Finland Canada, Netherlands, Germany, Belgium, Viet Nam Netherlands 518 Canada, Finland, Germany, Belgium, Viet Nam, Poland 516 Finland, Netherlands Belgium, Viet Nam, Poland, Australia, Austria Germany 513 Belgium Finland, Netherlands, Germany, Viet Nam, Poland, Australia, Austria Finland, Netherlands, Germany, Belgium, Poland, Australia, Austria, Ireland, New Zealand, Czech Republic, Slovenia 509 Viet Nam 509 Poland Netherlands, Germany, Belgium, Viet Nam, Australia, Austria, Ireland, New Zealand, Czech Republic Australia 509 Germany, Belgium, Viet Nam, Poland, Austria Austria Germany, Belgium, Viet Nam, Poland, Australia, Ireland, New Zealand, Czech Republic 506 501 Viet Nam, Poland, Austria, New Zealand, Czech Republic, Slovenia, France, Latvia, United Kingdom, Denmark Ireland New Zealand 501 Viet Nam, Poland, Austria, Ireland, Czech Republic, Slovenia, France, Latvia, United Kingdom, Denmark 499 Czech Republic Viet Nam, Poland, Austria, Ireland, New Zealand, Slovenia, France, Latvia, United Kingdom, Denmark, Russian Federation 499 Slovenia Viet Nam, Ireland, New Zealand, Czech Republic, France, Latvia, United Kingdom, Denmark 497 Ireland, New Zealand, Czech Republic, Slovenia, Latvia, United Kingdom, Denmark, Russian Federation, United States France 496 Latvia Ireland, New Zealand, Czech Republic, Slovenia, France, United Kingdom, Denmark, Russian Federation, United States, Portugal Ireland, New Zealand, Czech Republic, Slovenia, France, Latvia, Denmark, Russian Federation, United States, Portugal 496 United Kingdom Ireland, New Zealand, Czech Republic, Slovenia, France, Latvia, United Kingdom, Russian Federation, United States, Portugal Denmark 494 Czech Republic, France, Latvia, United Kingdom, Denmark, United States, Luxembourg, Iceland, Portugal Russian Federation 491 France, Latvia, United Kingdom, Denmark, Russian Federation, Luxembourg, Iceland, Portugal, Spain, Hungary, Lithuania United States 488 Russian Federation, United States, Iceland, Portugal, Hungary Luxembourg 488 Russian Federation, United States, Luxembourg, Portugal, Spain, Hungary Iceland 487 486 Latvia, United Kingdom, Denmark, Russian Federation, United States, Luxembourg, Iceland, Spain, Hungary, Lithuania, Norway Portugal 482 Spain United States, Iceland, Portugal, Hungary, Lithuania, Norway, Italy, Slovak Republic 481 United States, Luxembourg, Iceland, Portugal, Spain, Lithuania, Norway, Italy, Slovak Republic Hungary 479 United States, Portugal, Spain, Hungary, Norway, Italy, Slovak Republic Lithuania Portugal, Spain, Hungary, Lithuania, Italy, Slovak Republic, Croatia 478 Norway Spain, Hungary, Lithuania, Norway, Slovak Republic, Croatia Italy 477 474 Slovak Republic Spain, Hungary, Lithuania, Norway, Italy, Sweden, Croatia, Israel Slovak Republic, Croatia, Israel 469 Sweden 468 Croatia Norway, Italy, Slovak Republic, Sweden, Israel Slovak Republic, Sweden, Croatia, Turkey Israel 462 1, 2 Israel, Greece, Romania, United Arab Emirates, Serbia, Cyprus 448 Turkey 1, 2 Turkey, Romania, United Arab Emirates, Serbia, Cyprus Greece 446 1, 2 , Bulgaria 446 Romania Turkey, Greece, United Arab Emirates, Serbia, Cyprus 1, 2 , Bulgaria 442 United Arab Emirates Turkey, Greece, Romania, Serbia, Cyprus 1, 2 , Bulgaria, Kazakhstan Turkey, Greece, Romania, United Arab Emirates, Cyprus 442 Serbia 1, 2 Turkey, Greece, Romania, United Arab Emirates, Serbia, Bulgaria Cyprus 440 1, 2 , Kazakhstan Bulgaria 434 Romania, United Arab Emirates, Serbia, Cyprus Serbia, Bulgaria 433 Kazakhstan Chile Thailand 414 Chile 411 Thailand, Mexico, Costa Rica, Malaysia Chile, Costa Rica, Uruguay, Malaysia 405 Mexico Chile, Mexico, Uruguay, Malaysia, Montenegro 402 Costa Rica Mexico, Costa Rica, Malaysia, Montenegro Uruguay 401 Malaysia 401 Chile, Mexico, Costa Rica, Uruguay, Montenegro 399 Costa Rica, Uruguay, Malaysia Montenegro Jordan, Tunisia, Argentina 388 Albania 387 Jordan Albania, Tunisia, Argentina Tunisia 379 Albania, Jordan, Argentina, Brazil, Indonesia Argentina 379 Albania, Jordan, Tunisia, Brazil, Indonesia Tunisia, Argentina, Indonesia Brazil 372 364 Indonesia Brazil, Qatar, Colombia 363 Qatar Colombia Colombia Qatar, Peru 357 349 Peru Colombia 1. Note by Turkey: The information in this document with reference to ”Cyprus” relates to the southern part of the Island. There is no single authority representing both Turkish and Greek Cypriot people on the Island. Turkey recognises the Turkish Republic of Northern Cyprus (TRNC). Until a lasting and equitable solution is found within the context of the United Nations, Turkey shall preserve its position concerning the “Cyprus issue”. 2. Note by all the European Union Member States of the OECD and the European Union: The Republic of Cyprus is recognised by all members of the United Nations with the exception of Turkey. The information in this document relates to the area under the effective control of the Government of the Republic of Cyprus. Source: OECD, PISA 2012 Database. 12 http://dx.doi.org/10.1787/888932935572 CS OECD 2014 S Kno W and Can d o: Student Performan C e in m athemati What Student , r eading and S C ien C e – Volume i © 99

102 2 o A nce i n mA f Student Perform A tic S A Profile them Fourteen countries and economies score more than three points higher on this subscale than on the overall mathematics scale. Eleven of these countries and economies score more than five points above the overall mathematics scale. They include Shanghai-China, which scores 11 points higher (the largest difference) on the change and relationships subscale than on the overall mathematics scale, followed by Estonia, the Russian Federation, the United Arab Emirates, Liechtenstein, Canada, Singapore, the United States, Japan, Latvia and Korea. Seven of these countries and economies score well above the OECD average on the overall mathematics proficiency scale. subscale that At the other end of the spectrum, 28 countries show average scores on the change and relationships are more than three points lower than the average score on the overall mathematics proficiency scale. Among these countries, Brazil, Colombia, Malaysia and Peru score between 19 and 20 points lower on the subscale than on the overall mathematics proficiency scale; Qatar, Thailand, Norway, Chile, Montenegro and Indonesia score between 10 and 14 points lower; and 14 other countries and economies also score lower on the subscale than on the overall proficiency scale, by a difference of at least 5 points (Figure I.2.52). and the change and relationships Figure I.2.41 describes the six levels of proficiency on the mathematics subscale distribution of students among these six proficiency levels is shown in Figure I.2.42. • • Figure I.2.41 Summary descriptions of the six proficiency levels for the mathematical subscale change and relationships Percentage of students able to perform tasks at each level or above ( l evel What students can do average) o E cd 6 At Level 6, students use significant insight, abstract reasoning and argumentation skills, and 4.5% technical knowledge and conventions to solve problems involving relationships among variables and to generalise mathematical solutions to complex real-world problems. They can create and use an algebraic model of a functional relationship incorporating multiple quantities. They apply deep geometrical insight to work with complex patterns; and they can use complex proportional reasoning, and complex calculations with percentages to explore quantitative relationships and change. 5 At Level 5, students can solve problems by using algebraic and other formal mathematical 14.5% models, including in scientific contexts. They can use complex and multi-step problem- solving skills, and can reflect on and communicate reasoning and arguments, for example in evaluating and using a formula to predict the quantitative effect of change in one variable on another. They can use complex proportional reasoning, for example to work with rates, and they can work competently with formulae and with expressions including inequalities. 4 Students at Level 4 can understand and work with multiple representations, including algebraic 31.9% models of real-world situations. They can reason about simple functional relationships between variables, going beyond individual data points to identifying simple underlying patterns. They can use some flexibility in interpretation and reasoning about functional relationships (for example, in exploring distance-time-speed relationships) and can modify a functional model or graph to fit a specified change to the situation; and they can communicate the resulting explanations and arguments. 3 At Level 3, students can solve problems that involve working with information from two related 54.2% representations (text, graph, table, formulae), requiring some interpretation, and use reasoning in familiar contexts. They show some ability to communicate their arguments. Students at this level can make a straightforward modification to a given functional model to fit a new situation; and they use a range of calculation procedures to solve problems, including ordering data, time difference calculations, substitution of values into a formula, or linear interpolation. 2 Students at Level 2 can locate relevant information about a relationship from data provided 75.1% in a table or graph and make direct comparisons, for example, to match given graphs to a specified change process. They can reason about the basic meaning of simple relationships expressed in text or numeric form by linking text with a single representation of a relationship (graph, table, simple formula), and can correctly substitute numbers into simple formulae, sometimes expressed in words. At this level, student can use interpretation and reasoning skills in a straightforward context involving linked quantities. 1 Students at Level 1 can evaluate single given statements about a relationship expressed clearly 89.6% and directly in a formula, or in a graph. Their ability to reason about relationships, and to change in those relationships, is limited to simple expressions and to those located in familiar situations. They may apply simple calculations needed to solve problems related to clearly expressed relationships. o: Student Performan e – Volume © OECD 2014 What Student S Kno W and Can d C C e in m athemati CS , r eading and S C ien i 100

103 2 A Profile A nce i n mA them A tic S f Student Perform o • • Figure I.2.42 p change and relationships roficiency in the mathematics subscale P ercentage of students at each level of mathematics proficiency Level 6 Level 1 Below Level 1 Level 2 Level 3 Level 4 Level 5 Shanghai-China Shanghai-China Singapore Singapore Hong Kong-China Hong Kong-China Estonia Estonia Korea Korea Macao-China Macao-China Japan Japan Canada Canada Students at Level 1 Chinese Taipei Chinese Taipei or below Liechtenstein Liechtenstein Finland Finland Switzerland Switzerland Netherlands Netherlands Viet Nam Viet Nam Ireland Ireland Poland Poland Germany Germany Belgium Belgium Australia Australia Latvia Latvia Denmark Denmark Austria Austria Czech Republic Czech Republic Slovenia Slovenia United Kingdom United Kingdom Russian Federation Russian Federation France France New Zealand New Zealand United States United States OECD average OECD average Iceland Iceland Spain Spain Portugal Portugal Lithuania Lithuania Luxembourg Luxembourg Hungary Hungary Italy Italy Norway Norway Slovak Republic Slovak Republic Sweden Sweden Croatia Croatia Israel Israel Greece Greece Turkey Turkey Romania Romania Serbia Serbia United Arab Emirates United Arab Emirates Kazakhstan Kazakhstan Bulgaria Bulgaria Thailand Thailand Chile Chile Uruguay Uruguay Mexico Mexico Malaysia Malaysia Montenegro Montenegro Costa Rica Costa Rica Albania Albania Jordan Jordan Argentina Argentina Tunisia Tunisia Brazil Brazil Qatar Qatar Students at Level 2 Peru Peru or above Colombia Colombia Indonesia Indonesia % % 20 0 20 40 80 60 80 100 100 60 40 Countries and economies are ranked in descending order of the percentage of students at Levels 2, 3, 4, 5 and 6. Source: OECD, PISA 2012 Database, Table I.2.14. 1 2 http://dx.doi.org/10.1787/888932935572 e – Volume d and Can W Kno C ien C r i © OECD 2014 101 eading and S , CS athemati m e in C S What Student o: Student Performan

104 2 f Student Perform nce i n mA them A tic S A Profile o A • • Figure I.2.43 c omparing countries’ and economies’ performance on the mathematics subscale space and shape Statistically significantly above the OECD average Not statistically significantly different from the OECD average the OECD average Statistically significantly below ean m omparison c c country/economy not ountries/economies whose mean score is score statistically significantly different from that comparison country’s/economy’s score 649 Shanghai-China 592 Chinese Taipei Singapore 580 Korea 573 Korea Singapore, Hong Kong-China Korea, Japan 567 Hong Kong-China 558 Japan Macao-China 558 Japan Hong Kong-China, Macao-China 544 Switzerland Liechtenstein 539 Switzerland Liechtenstein 524 Poland Canada, Belgium, Netherlands, Germany, Viet Nam, Finland 513 Estonia Estonia, Belgium, Netherlands, Germany, Viet Nam, Finland Canada 510 Estonia, Canada, Netherlands, Germany, Viet Nam, Finland 509 Belgium 507 Netherlands Estonia, Canada, Belgium, Germany, Viet Nam, Finland, Slovenia, Austria, Czech Republic Estonia, Canada, Belgium, Netherlands, Viet Nam, Finland, Slovenia, Austria, Czech Republic 507 Germany 507 Viet Nam Estonia, Canada, Belgium, Netherlands, Germany, Finland, Slovenia, Austria, Czech Republic, Latvia, Denmark, Australia, Russian Federation Finland 507 Estonia, Canada, Belgium, Netherlands, Germany, Viet Nam, Slovenia, Austria Netherlands, Germany, Viet Nam, Finland, Austria, Czech Republic, Latvia, Russian Federation 503 Slovenia 501 Netherlands, Germany, Viet Nam, Finland, Slovenia, Czech Republic, Latvia, Denmark, Australia, Russian Federation, Portugal Austria 499 Czech Republic Netherlands, Germany, Viet Nam, Slovenia, Austria, Latvia, Denmark, Australia, Russian Federation, Portugal, New Zealand, Slovak Republic 497 Latvia Viet Nam, Slovenia, Austria, Czech Republic, Denmark, Australia, Russian Federation, Portugal, New Zealand, Slovak Republic, France Viet Nam, Austria, Czech Republic, Latvia, Australia, Russian Federation, Portugal, New Zealand, Slovak Republic 497 Denmark Viet Nam, Austria, Czech Republic, Latvia, Denmark, Russian Federation, Portugal, New Zealand, Slovak Republic Australia 497 496 Russian Federation Viet Nam, Slovenia, Austria, Czech Republic, Latvia, Denmark, Australia, Portugal, New Zealand, Slovak Republic, France, Iceland, Italy Portugal 491 Austria, Czech Republic, Latvia, Denmark, Australia, Russian Federation, New Zealand, Slovak Republic, France, Iceland, Italy, Luxembourg Czech Republic, Latvia, Denmark, Australia, Russian Federation, Portugal, Slovak Republic, France, Iceland, Italy, Luxembourg 491 New Zealand Czech Republic, Latvia, Denmark, Australia, Russian Federation, Portugal, New Zealand, France, Iceland, Italy, Luxembourg, Norway Slovak Republic 490 Latvia, Russian Federation, Portugal, New Zealand, Slovak Republic, Iceland, Italy, Luxembourg France 489 Russian Federation, Portugal, New Zealand, Slovak Republic, France, Italy, Luxembourg Iceland 489 Russian Federation, Portugal, New Zealand, Slovak Republic, France, Iceland Luxembourg, Norway Italy 487 486 Luxembourg Portugal, New Zealand, Slovak Republic, France, Iceland, Italy, Norway 480 Norway Slovak Republic, Italy, Luxembourg, Ireland, Spain, United Kingdom, Hungary, Lithuania Norway, Spain, United Kingdom, Hungary, Lithuania 478 Ireland 477 Spain Norway, Ireland, United Kingdom, Hungary, Lithuania Norway, Ireland, Spain, Hungary, Lithuania, Sweden United Kingdom 475 Hungary Norway, Ireland, Spain, United Kingdom, Lithuania, Sweden, United States 474 Norway, Ireland, Spain, United Kingdom, Hungary, Sweden, United States 472 Lithuania United Kingdom, Hungary, Lithuania, United States, Croatia Sweden 469 United States 463 Hungary, Lithuania, Sweden, Croatia Croatia 460 Sweden, United States, Kazakhstan, Israel Croatia, Israel, Romania, Serbia, Turkey, Bulgaria Kazakhstan 450 Croatia, Kazakhstan, Romania, Serbia, Turkey, Bulgaria Israel 449 Romania 447 Kazakhstan, Israel, Serbia, Turkey, Bulgaria Serbia Kazakhstan, Israel, Romania, Turkey, Bulgaria 446 1, 2 , Malaysia, Thailand 443 Turkey Kazakhstan, Israel, Romania, Serbia, Bulgaria, Greece, Cyprus 1, 2 , Malaysia, Thailand Bulgaria 442 Kazakhstan, Israel, Romania, Serbia, Turkey, Greece, Cyprus 1, 2 , Malaysia, Thailand Turkey, Bulgaria, Cyprus 436 Greece 1, 2 Turkey, Bulgaria, Greece, Malaysia, Thailand Cyprus 436 1, 2 , Thailand 434 Turkey, Bulgaria, Greece, Cyprus Malaysia 1, 2 , Malaysia, United Arab Emirates Turkey, Bulgaria, Greece, Cyprus Thailand 432 United Arab Emirates 425 Thailand, Chile United Arab Emirates, Albania, Uruguay, Mexico 419 Chile Albania Chile, Uruguay, Mexico, Montenegro 418 Chile, Albania, Mexico, Montenegro Uruguay 413 Mexico 413 Chile, Albania, Uruguay, Montenegro 412 Montenegro Albania, Uruguay, Mexico Costa Rica 397 385 Jordan Argentina, Indonesia, Tunisia, Brazil, Qatar 385 Jordan, Indonesia, Tunisia, Brazil, Qatar Argentina Jordan, Argentina, Tunisia, Brazil, Qatar Indonesia 383 Tunisia 382 Jordan, Argentina, Indonesia, Brazil, Qatar 381 Brazil Jordan, Argentina, Indonesia, Tunisia, Qatar 380 Qatar Jordan, Argentina, Indonesia, Tunisia, Brazil 370 Colombia Peru 369 Colombia Peru 1. Note by Turkey: The information in this document with reference to ”Cyprus” relates to the southern part of the Island. There is no single authority representing both Turkish and Greek Cypriot people on the Island. Turkey recognises the Turkish Republic of Northern Cyprus (TRNC). Until a lasting and equitable solution is found within the context of the United Nations, Turkey shall preserve its position concerning the “Cyprus issue”. 2. Note by all the European Union Member States of the OECD and the European Union: The Republic of Cyprus is recognised by all members of the United Nations with the exception of Turkey. The information in this document relates to the area under the effective control of the Government of the Republic of Cyprus. Source: OECD, PISA 2012 Database. 12 http://dx.doi.org/10.1787/888932935572 o: Student Performan e – Volume © OECD 2014 What Student S Kno W and Can d C C e in m athemati CS , r eading and S C ien i 102

105 2 A Profile f Student Perform A nce i n mA them A tic S o space and shape Student performance on the mathematics subscale PISA items in this category emphasise spatial relationships among objects, and measurement and other geometric aspects Question 1 and of the spatial world. Items listed in Figure I.2.9 that have been classified in this category are G ARAGE Question 2, and R Question 1 and Question 2. The questions in G involve spatial reasoning ARAGE EVOLVING DOOR EVOLVING (Question 1), and working with measurements and area calculations with a model of a real-world object. R DOOR involves knowledge of angle relationships, spatial reasoning and some calculations with circle geometry. Across OECD countries, the average score attained on the space and shape subscale is 490 points. Top-performing countries and economies on this subscale are Shanghai-China, Chinese Taipei, Singapore, Korea, Hong Kong-China, Macao-China, Japan, Switzerland, Liechtenstein and Poland (Figure I.2.43 and Table I.2.19). The average score among OECD countries on this subscale is four points lower than the average score on the overall mathematics proficiency scale (Figure I.2.52). However, this difference varies widely among countries. Figure I.2.44 • • Summary descriptions of the six proficiency levels for the mathematical subscale space and shape Percentage of students able to perform tasks at each level or above l o E cd average) What students can do ( evel 6 At Level 6, students can solve complex problems involving multiple representations or 4.5% calculations; identify, extract, and link relevant information, for example by extracting relevant dimensions from a diagram or map and using scale to calculate an area or distance; use spatial reasoning, significant insight and reflection, for example, by interpreting text and related contextual material to formulate a useful geometric model and applying it while taking into account contextual constraints; recall and apply relevant procedural knowledge from their base of mathematical knowledge, such as in circle geometry, trigonometry, Pythagoras’s rule, or area and volume formulae to solve problems; and can generalise results and findings, communicate solutions and provide justifications and argumentation. 5 At Level 5, students can solve problems that require appropriate assumptions to be made, or 13.4% that involve reasoning from assumptions provided while taking into account explicitly stated constraints, for example, in exploring and analysing the layout of a room and the furniture it contains. They solve problems using theorems or procedural knowledge, such as symmetry properties, or similar triangle properties or formulae including those for calculating area, perimeter or volume of familiar shapes. They use well-developed spatial reasoning, argument and insight to infer relevant conclusions and to interpret and link different representations, for example to identify a direction or location on a map from textual information. 4 Students at Level 4 can solve problems by using basic mathematical knowledge, such as angle 29.7% and side-length relationships in triangles, and by doing so in a way that involves multistep, visual and spatial reasoning, and argumentation in unfamiliar contexts. They can link and integrate different representations, for example to analyse the structure of a three-dimensional object based on two different perspectives of it; and can compare objects using geometric properties. 3 At Level 3, students can solve problems that involve elementary visual and spatial reasoning 51.9% in familiar contexts, such as calculating a distance or a direction from a map or a GPS device; link different representations of familiar objects or appreciate properties of objects under some simple specified transformation; and devise simple strategies and apply basic properties of triangles and circles. They can use appropriate supporting calculation techniques, such as scale conversions needed to analyse distances on a map. 2 At Level 2, students can solve problems involving a single familiar geometric representation 74.2% (for example, a diagram or other graphic) by comprehending and drawing conclusions in relation to clearly presented basic geometric properties and associated constraints. They can also evaluate and compare spatial characteristics of familiar objects in a situation where given constraints apply, such as comparing the height or circumference of two cylinders having the same surface area, or deciding whether a given shape can be dissected to produce another specified shape. 1 Students at Level 1 can recognise and solve simple problems in a familiar context using 90.0% pictures or drawings of familiar geometric objects and applying basic spatial skills, such as recognising elementary symmetry properties, comparing lengths or angle sizes, or using procedures, such as dissection of shapes. OECD 2014 © i e – Volume C ien C eading and S 103 , CS athemati m e in C o: Student Performan d and Can W Kno S What Student r

106 2 A Profile nce i n mA them A tic A f Student Perform o S Figure I.2.45 • • p roficiency in the mathematics subscale space and shape P ercentage of students at each level of mathematics proficiency Level 3 Level 6 Level 5 Level 4 Below Level 1 Level 1 Level 2 Shanghai-China Shanghai-China Japan Japan Korea Korea Singapore Singapore Hong Kong-China Hong Kong-China Macao-China Macao-China Switzerland Switzerland Liechtenstein Liechtenstein Students at Level 1 Chinese Taipei Chinese Taipei or below Poland Poland Estonia Estonia Finland Finland Canada Canada Denmark Denmark Netherlands Netherlands Latvia Latvia Germany Germany Viet Nam Viet Nam Slovenia Slovenia Austria Austria Belgium Belgium Iceland Iceland Russian Federation Russian Federation Czech Republic Czech Republic Australia Australia New Zealand New Zealand France France Luxembourg Luxembourg OECD average OECD average Slovak Republic Slovak Republic Italy Italy Ireland Ireland Portugal Portugal Norway Norway Spain Spain United Kingdom United Kingdom Hungary Hungary Sweden Sweden Lithuania Lithuania Croatia Croatia United States United States Kazakhstan Kazakhstan Israel Israel Romania Romania Serbia Serbia Bulgaria Bulgaria Greece Greece Turkey Turkey Malaysia Malaysia Thailand Thailand United Arab Emirates United Arab Emirates Albania Albania Chile Chile Uruguay Uruguay Mexico Mexico Montenegro Montenegro Costa Rica Costa Rica Argentina Argentina Qatar Qatar Jordan Jordan Indonesia Indonesia Tunisia Tunisia Students at Level 2 Brazil Brazil or above Peru Peru Colombia Colombia % % 20 0 80 40 60 80 60 40 20 100 100 Countries and economies are ranked in descending order of the percentage of students at Levels 2, 3, 4, 5 and 6. Source: OECD, PISA 2012 Database, Table I.2.17. 1 http://dx.doi.org/10.1787/888932935572 2 m Kno What Student OECD 2014 W and Can d o: Student Performan C e in S athemati , r © i eading and S C ien C e – Volume CS 104

107 2 nce i n mA them A tic S A A Profile o f Student Perform subscale than on their overall space and shape Ten countries and economies score more than 10 points higher on the proficiency scale. These differences are quiet large in some countries, with Shanghai-China showing the largest difference (36 points), followed by Chinese Taipei (32 points), Albania (23 points), Japan (21 points), Macao-China (20 points), Korea (19 points), Kazakhstan (18 points), Malaysia (14 points), the Russian Federation (14 points) and Switzerland (13 points). Five of the best-performing countries and economies on the mathematics scale, Shanghai-China, Chinese Taipei, Korea, Macao-China and Japan, are included in this group. Conversely, nine countries score at least 10 points lower on the space and shape subscale than on the overall proficiency scale. Ireland shows the largest difference (24 points), while in the eight other countries, differences range from 10 to 20 points: the United Kingdom (19 points), the United States (18 points), Israel (17 points), Greece (17 points), the Netherlands (16 points), Finland (12 points), Croatia (11 points) and Brazil (11 points) (Figure I.2.52). Figure I.2.44 describes the six levels of proficiency on the mathematics subscale and the distribution space and shape of students among these six proficiency levels is shown in Figure I.2.45. Student performance on the mathematics subscale quantity PISA items in this category emphasise comparisons and calculations based on quantitative relationships and numeric properties of objects and phenomena. Items listed in Figure I.2.9 that have been classified in this category are W HICH R CA DOOR EVOLVING Question 1 and Question 3, and R UJI F OUNT M LIMBING Question 2 and Question 3, C ? ? CAR HICH Question 3. The questions in W involve reasoning about quantities of given properties of different objects, and computation with percentages. C LIMBING M OUNT EVOLVING F UJI also involves calculations with given quantities. R Question 3 involves reasoning and calculations using given quantitative information. DOOR The average score on the quantity subscale is 495 points. The ten top-performing countries and economies on this subscale are Shanghai-China, Singapore, Hong Kong-China, Chinese Taipei, Liechtenstein, Korea, the Netherlands, Switzerland, Macao-China and Finland (Figure I.2.46 and Table I.2.22). The average score among OECD countries on the quantity subscale is one point higher than the average score on the quantity overall mathematics proficiency scale (Figure I.2.52). Twenty-two countries and economies have an average score that is within about three score points of their average score on the overall mathematics proficiency scale. subscale than on the overall mathematics scale, and seven other countries also quantity Israel scores 13 points higher on the score higher on this subscale than on the main scale by at least five points: Croatia (9 points), the Netherlands (9 points), Finland (8 points), Serbia (7 points), Spain (7 points), the Czech Republic (6 points) and Italy (5 points). subscale than on the main proficiency scale, and Jordan scores quantity Shanghai-China scores 22 points lower on the 19 points lower. Japan (18 points), Chinese Taipei (16 points), Korea (16 points), Indonesia (13 points) and Malaysia (11 points) score at least 10 points lower on the subscale than on the main scale. F igure I.2.47 describes the six levels of proficiency on the mathematics subscale quantity and the distribution of students among these six proficiency levels is shown in Figure I.2.48. Student performance on the mathematics subscale uncertainty and data PISA items in this category emphasise interpreting and working with data and with different data presentation forms, and problems involving probabilistic reasoning. Items listed in Figure I.2.9 that have been classified in this category are CAR ? Question 1, and C HARTS Question 1, Question 2 and Question 3. The question in W HICH CAR ? involves HICH W interpreting data in a two-way table to identify an object that satisfies various criteria. The questions in C HARTS involve interpreting a bar chart and understanding the relationships depicted in the chart. subscale is 493 points. Top-performing uncertainty and data Across OECD countries, the average score on the countries and economies on this subscale are Shanghai-China, Singapore, Hong Kong-China, Chinese Taipei, Korea, the Netherlands, Japan, Liechtenstein, Macao-China and Switzerland (Figure I.2.49 and Table I.2.25). The average uncertainty and data score among OECD countries on the subscale is one point lower than the average score on the overall mathematics scale, but the difference between the two sets of scores varies widely among countries (Figure I.2.52). CS OECD 2014 S Kno W and Can d o: Student Performan C e in m athemati What Student , r eading and S C ien C e – Volume i © 105

108 2 f Student Perform nce i n mA them A tic S A Profile o A • • Figure I.2.46 c omparing countries’ and economies’ performance on the mathematics subscale quantity the OECD average Statistically significantly above Not statistically significantly different from the OECD average the OECD average below Statistically significantly m c omparison ean not score ountries/economies whose mean score is c country/economy statistically significantly different from that comparison country’s/economy’s score 591 Shanghai-China Hong Kong-China 569 Singapore Hong Kong-China 566 Singapore 543 Chinese Taipei Liechtenstein, Korea Chinese Taipei, Korea, Netherlands, Switzerland, Macao-China 538 Liechtenstein Korea 537 Chinese Taipei, Liechtenstein, Netherlands, Switzerland, Macao-China 532 Netherlands Liechtenstein, Korea, Switzerland, Macao-China, Finland, Estonia Liechtenstein, Korea, Netherlands, Macao-China, Finland, Estonia 531 Switzerland 531 Macao-China Liechtenstein, Korea, Netherlands, Switzerland, Finland Finland 527 Netherlands, Switzerland, Macao-China, Estonia 525 Netherlands, Switzerland, Finland, Belgium, Poland, Japan Estonia Estonia, Poland, Japan, Germany, Canada, Viet Nam 519 Belgium 519 Poland Estonia, Belgium, Japan, Germany, Canada, Austria, Viet Nam Estonia, Belgium, Poland, Germany, Canada, Austria, Viet Nam Japan 518 Belgium, Poland, Japan, Canada, Austria, Viet Nam Germany 517 515 Belgium, Poland, Japan, Germany, Austria, Viet Nam Canada Poland, Japan, Germany, Canada, Viet Nam, Ireland, Czech Republic Austria 510 Belgium, Poland, Japan, Germany, Canada, Austria, Ireland, Czech Republic, Slovenia, Denmark, Australia, New Zealand 509 Viet Nam 505 Ireland Austria, Viet Nam, Czech Republic, Slovenia, Denmark, Australia, New Zealand 505 Czech Republic Austria, Viet Nam, Ireland, Slovenia, Denmark, Australia, New Zealand Viet Nam, Ireland, Czech Republic, Denmark, Australia 504 Slovenia Viet Nam, Ireland, Czech Republic, Slovenia, Australia, New Zealand, Iceland, France, United Kingdom Denmark 502 500 Australia Viet Nam, Ireland, Czech Republic, Slovenia, Denmark, New Zealand, Iceland, France, United Kingdom New Zealand 499 Viet Nam, Ireland, Czech Republic, Denmark, Australia, Iceland, France, Luxembourg, United Kingdom, Norway Denmark, Australia, New Zealand, France, Luxembourg, United Kingdom, Norway, Spain 496 Iceland Denmark, Australia, New Zealand, Iceland, Luxembourg, United Kingdom, Norway, Spain, Italy France 496 New Zealand, Iceland, France, United Kingdom, Norway, Spain, Italy Luxembourg 495 United Kingdom Denmark, Australia, New Zealand, Iceland, France, Luxembourg, Norway, Spain, Italy, Latvia, Slovak Republic 494 New Zealand, Iceland, France, Luxembourg, United Kingdom, Spain, Italy, Latvia, Slovak Republic Norway 492 491 Iceland, France, Luxembourg, United Kingdom, Norway, Italy, Latvia, Slovak Republic Spain France, Luxembourg, United Kingdom, Norway, Spain, Latvia, Slovak Republic Italy 491 487 Latvia United Kingdom, Norway, Spain, Italy, Slovak Republic, Lithuania, Sweden, Portugal, Croatia, Israel, United States 486 Slovak Republic United Kingdom, Norway, Spain, Italy, Latvia, Lithuania, Sweden, Portugal, Croatia, Israel, Russian Federation, United States Lithuania Latvia, Slovak Republic, Sweden, Portugal, Croatia, Israel, Russian Federation, United States, Hungary 483 482 Sweden Latvia, Slovak Republic, Lithuania, Portugal, Croatia, Israel, Russian Federation, United States, Hungary Latvia, Slovak Republic, Lithuania, Sweden, Croatia, Israel, Russian Federation, United States, Hungary 481 Portugal Croatia 480 Latvia, Slovak Republic, Lithuania, Sweden, Portugal, Israel, Russian Federation, United States, Hungary Israel Latvia, Slovak Republic, Lithuania, Sweden, Portugal, Croatia, Russian Federation, United States, Hungary 480 478 Russian Federation Slovak Republic, Lithuania, Sweden, Portugal, Croatia, Israel, United States, Hungary 478 United States Latvia, Slovak Republic, Lithuania, Sweden, Portugal, Croatia, Israel, Russian Federation, Hungary 476 Lithuania, Sweden, Portugal, Croatia, Israel, Russian Federation, United States Hungary 456 Serbia Greece 455 Greece Serbia 1, 2 443 Romania Bulgaria, Turkey, Cyprus 1, 2 Romania, Turkey, Cyprus Bulgaria 443 1, 2 , United Arab Emirates Turkey 442 Romania, Bulgaria, Cyprus 1, 2 Romania, Bulgaria, Turkey Cyprus 439 431 United Arab Emirates Turkey, Kazakhstan United Arab Emirates, Chile, Thailand 428 Kazakhstan 421 Kazakhstan, Thailand Chile Kazakhstan, Chile, Mexico, Uruguay, Malaysia 419 Thailand Thailand, Uruguay, Malaysia, Costa Rica Mexico 414 411 Thailand, Mexico, Malaysia, Montenegro, Costa Rica Uruguay Malaysia 409 Thailand, Mexico, Uruguay, Montenegro, Costa Rica 409 Montenegro Uruguay, Malaysia, Costa Rica 406 Mexico, Uruguay, Malaysia, Montenegro Costa Rica Argentina, Albania Brazil 393 391 Argentina Brazil, Albania Brazil, Argentina, Tunisia 386 Albania Tunisia 378 Albania, Colombia, Qatar, Jordan Colombia 375 Tunisia, Qatar, Jordan, Peru 371 Qatar Tunisia, Colombia, Jordan, Peru, Indonesia 367 Jordan Tunisia, Colombia, Qatar, Peru, Indonesia Peru Colombia, Qatar, Jordan, Indonesia 365 362 Indonesia Qatar, Jordan, Peru 1. Note by Turkey: The information in this document with reference to ”Cyprus” relates to the southern part of the Island. There is no single authority representing both Turkish and Greek Cypriot people on the Island. Turkey recognises the Turkish Republic of Northern Cyprus (TRNC). Until a lasting and equitable solution is found within the context of the United Nations, Turkey shall preserve its position concerning the “Cyprus issue”. 2. Note by all the European Union Member States of the OECD and the European Union: The Republic of Cyprus is recognised by all members of the United Nations with the exception of Turkey. The information in this document relates to the area under the effective control of the Government of the Republic of Cyprus. Source: OECD, PISA 2012 Database. http://dx.doi.org/10.1787/888932935572 12 o: Student Performan e – Volume © OECD 2014 What Student S Kno W and Can d C C e in m athemati CS , r eading and S C ien i 106

109 2 o A nce i n mA f Student Perform A tic S A Profile them Figure I.2.47 • • quantity Summary descriptions of the six proficiency levels on the mathematical subscale Percentage of students able to perform tasks at each level or above E cd ( o l What students can do average) evel 6 At Level 6 and above, students conceptualise and work with models of complex quantitative 3.9% processes and relationships; devise strategies for solving problems; formulate conclusions, arguments and precise explanations; interpret and understand complex information, and link multiple complex information sources; interpret graphical information and apply reasoning to identify, model and apply a numeric pattern. They can analyse and evaluate interpretive statements based on data provided; work with formal and symbolic expressions; plan and implement sequential calculations in complex and unfamiliar contexts, including working with large numbers, for example to perform a sequence of currency conversions, entering values correctly and rounding results. Students at this level work accurately with decimal fractions; they use advanced reasoning concerning proportions, geometric representations of quantities, combinatorics and integer number relationships; and they interpret and understand formal expressions of relationships among numbers, including in a scientific context. 5 At Level 5, students can formulate comparison models and compare outcomes to determine 14.0% highest price, and interpret complex information about real-world situations (including graphs, drawings and complex tables, for example two graphs using different scales). They can generate data for two variables and evaluate propositions about the relationship between them. Students can communicate reasoning and argument; recognise the significance of numbers to draw inferences; and provide a written argument evaluating a proposition based on data provided. They can make an estimation using knowledge about daily life; calculate relative and/or absolute change; calculate an average; calculate relative and/or absolute difference, including percentage difference, given raw difference data; and can convert units (for example calculations involving areas in different units). 4 At Level 4, students can interpret complex instructions and situations; relate text-based 32.5% numerical information to a graphic representation; identify and use quantitative information from multiple sources; deduce system rules from unfamiliar representations; formulate a simple numeric model; set up comparison models; and explain their results. They can carry out accurate and more complex or repeated calculations, such as adding 13 given times in hour/minute format; carry out time calculations using given data on distance and speed of a journey; perform simple division of large multiples in context; carry out calculations involving a sequence of steps; and accurately apply a given numeric algorithm involving a number of steps. Students at this level can perform calculations involving proportional reasoning, divisibility or percentages in simple models of complex situations. 3 At Level 3, students can use basic problem-solving processes, including devising a simple 55.4% strategy to test scenarios, understand and work with given constraints, use trial and error, and use simple reasoning in familiar contexts. At this level students can interpret a text description of a sequential calculation process, and correctly implement the process; identify and extract data presented directly in textual explanations of unfamiliar data; interpret text and diagrams describing a simple pattern; and perform calculations, including working with large numbers, calculations with speed and time, conversion of units (for example from an annual rate to a daily rate). They understand place value involving mixed 2- and 3-decimal values and including working with prices; can order a small series of (4) decimal values; calculate percentages of up to 3-digit numbers; and apply calculation rules given in natural language. 2 At Level 2, students can interpret simple tables to identify and extract relevant quantitative 76.5% information, and can interpret a simple quantitative model (such as a proportional relationship) and apply it using basic arithmetic calculations. They can identify the links between relevant textual information and tabular data to solve word problems; interpret and apply simple models involving quantitative relationships; identify the simple calculation required to solve a straight-forward problem; carry out simple calculations involving basic arithmetic operations; order 2- and 3-digit whole numbers and decimal numbers with one or two decimal places; and calculate percentages. 1 At Level 1, students can solve basic problems in which relevant information is explicitly 90.8% presented, and the situation is straightforward and very limited in scope. Students at this level can handle situations where the required computational activity is obvious and the mathematical task is basic, such as a one-step simple arithmetic operation, or to total the columns of a simple table and compare the results. They can read and interpret a simple table of numbers; extract data and perform simple calculations; use a calculator to generate relevant data; and extrapolate from the data generated, using reasoning and calculation with a simple linear model. CS OECD 2014 S Kno W and Can d o: Student Performan C e in m athemati What Student , r eading and S C ien C e – Volume i © 107

110 2 A Profile A nce i n mA them A tic S f Student Perform o • • Figure I.2.48 p quantity roficiency in the mathematics subscale P ercentage of students at each level of mathematics proficiency Level 6 Level 1 Below Level 1 Level 2 Level 3 Level 4 Level 5 Shanghai-China Shanghai-China Hong Kong-China Hong Kong-China Singapore Singapore Korea Korea Finland Finland Estonia Estonia Macao-China Macao-China Liechtenstein Liechtenstein Students at Level 1 Switzerland Switzerland or below Poland Poland Chinese Taipei Chinese Taipei Netherlands Netherlands Japan Japan Viet Nam Viet Nam Canada Canada Austria Austria Germany Germany Ireland Ireland Belgium Belgium Denmark Denmark Slovenia Slovenia Czech Republic Czech Republic Latvia Latvia Norway Norway Australia Australia Iceland Iceland New Zealand New Zealand France France OECD average OECD average Luxembourg Luxembourg United Kingdom United Kingdom Spain Spain Italy Italy Lithuania Lithuania Sweden Sweden Russian Federation Russian Federation Portugal Portugal Slovak Republic Slovak Republic Croatia Croatia United States United States Hungary Hungary Israel Israel Greece Greece Serbia Serbia Bulgaria Bulgaria Romania Romania Turkey Turkey United Arab Emirates United Arab Emirates Kazakhstan Kazakhstan Chile Chile Thailand Thailand Mexico Mexico Uruguay Uruguay Montenegro Montenegro Malaysia Malaysia Costa Rica Costa Rica Albania Albania Argentina Argentina Brazil Brazil Tunisia Tunisia Qatar Qatar Colombia Colombia Students at Level 2 Peru Peru or above Jordan Jordan Indonesia Indonesia % % 20 0 20 40 60 100 80 100 80 60 40 Countries and economies are ranked in descending order of the percentage of students at Levels 2, 3, 4, 5 and 6. Source: OECD, PISA 2012 Database, Table I.2.20. 1 2 http://dx.doi.org/10.1787/888932935572 r S What Student OECD 2014 CS , m eading and S C C e – Volume i athemati e in C o: Student Performan d and Can W © Kno ien 108

111 2 f Student Perform nce i n mA them A tic S A Profile o A Figure I.2.49 • • c omparing countries’ and economies’ performance on the mathematics subscale uncertainty and data the OECD average Statistically significantly above Not statistically significantly different from the OECD average the OECD average below Statistically significantly m c omparison ean statistically significantly different from that comparison country’s/economy’s score not ountries/economies whose mean score is c country/economy score 592 Shanghai-China 559 Singapore Hong Kong-China Hong Kong-China Singapore, Chinese Taipei 553 549 Chinese Taipei Hong Kong-China Netherlands, Japan 538 Korea Netherlands 532 Korea, Japan, Liechtenstein, Macao-China 528 Japan Korea, Netherlands, Liechtenstein, Macao-China, Switzerland, Viet Nam 526 Liechtenstein Netherlands, Japan, Macao-China, Switzerland, Viet Nam, Finland, Poland Netherlands, Japan, Liechtenstein, Switzerland, Viet Nam Macao-China 525 522 Switzerland Japan, Liechtenstein, Macao-China, Viet Nam, Finland, Poland, Canada Viet Nam 519 Japan, Liechtenstein, Macao-China, Switzerland, Finland, Poland, Canada, Estonia 519 Liechtenstein, Switzerland, Viet Nam, Poland, Canada Finland 517 Poland Liechtenstein, Switzerland, Viet Nam, Finland, Canada, Estonia, Germany, Ireland Switzerland, Viet Nam, Finland, Poland 516 Canada 510 Estonia Viet Nam, Poland, Germany, Ireland, Belgium, Australia, New Zealand, Denmark Germany 509 Poland, Estonia, Ireland, Belgium, Australia, New Zealand, Denmark, United Kingdom Ireland Poland, Estonia, Germany, Belgium, Australia, New Zealand, Denmark, United Kingdom 509 508 Estonia, Germany, Ireland, Australia, New Zealand, Denmark, United Kingdom Belgium Australia 508 Estonia, Germany, Ireland, Belgium, New Zealand, Denmark, United Kingdom 506 New Zealand Estonia, Germany, Ireland, Belgium, Australia, Denmark, United Kingdom, Austria 505 Denmark Estonia, Germany, Ireland, Belgium, Australia, New Zealand, United Kingdom, Austria 502 Germany, Ireland, Belgium, Australia, New Zealand, Denmark, Austria, Norway, Iceland United Kingdom 499 Austria New Zealand, Denmark, United Kingdom, Norway, Slovenia, Iceland, France United Kingdom, Austria, Slovenia, Iceland, France, United States 497 Norway Austria, Norway, Iceland, France Slovenia 496 United Kingdom, Austria, Norway, Slovenia, France, United States Iceland 496 Austria, Norway, Slovenia, Iceland, Czech Republic, United States, Spain, Portugal France 492 France, United States, Spain, Portugal, Luxembourg, Sweden, Italy Czech Republic 488 Norway, Iceland, France, Czech Republic, Spain, Portugal, Luxembourg, Sweden, Italy United States 488 Spain 487 France, Czech Republic, United States, Portugal, Luxembourg, Sweden, Italy 486 Portugal France, Czech Republic, United States, Spain, Luxembourg, Sweden, Italy, Latvia 483 Czech Republic, United States, Spain, Portugal, Sweden, Italy, Latvia Luxembourg 483 Czech Republic, United States, Spain, Portugal, Luxembourg, Italy, Latvia, Hungary Sweden Italy 482 Czech Republic, United States, Spain, Portugal, Luxembourg, Sweden, Latvia, Hungary 478 Latvia Portugal, Luxembourg, Sweden, Italy, Hungary, Lithuania, Slovak Republic Sweden, Italy, Latvia, Lithuania, Slovak Republic, Croatia, Israel Hungary 476 Lithuania 474 Latvia, Hungary, Slovak Republic, Croatia, Israel Slovak Republic 472 Latvia, Hungary, Lithuania, Croatia, Israel, Russian Federation Hungary, Lithuania, Slovak Republic, Israel, Russian Federation, Greece 468 Croatia Hungary, Lithuania, Slovak Republic, Croatia, Russian Federation, Greece Israel 465 463 Russian Federation Slovak Republic, Croatia, Israel, Greece Greece Croatia, Israel, Russian Federation 460 1, 2 448 Serbia Turkey, Cyprus 1, 2 , Romania 447 Serbia, Cyprus Turkey 1, 2 Serbia, Turkey, Romania 442 Cyprus 1, 2 , Thailand, United Arab Emirates, Bulgaria, Chile 437 Romania Turkey, Cyprus Romania, United Arab Emirates, Bulgaria, Chile 433 Thailand Romania, Thailand, Bulgaria, Chile 432 United Arab Emirates 432 Bulgaria Romania, Thailand, United Arab Emirates, Chile, Malaysia Romania, Thailand, United Arab Emirates, Bulgaria 430 Chile Malaysia Bulgaria, Costa Rica 422 Costa Rica, Kazakhstan, Mexico 415 Montenegro Costa Rica 414 Malaysia, Montenegro, Kazakhstan, Mexico, Uruguay Kazakhstan 414 Montenegro, Costa Rica, Mexico, Uruguay 413 Mexico Montenegro, Costa Rica, Kazakhstan Costa Rica, Kazakhstan, Brazil, Tunisia 407 Uruguay 402 Brazil Uruguay, Tunisia Tunisia 399 Uruguay, Brazil, Jordan Tunisia, Argentina, Colombia, Albania, Indonesia Jordan 394 Argentina 389 Jordan, Colombia, Albania, Indonesia, Qatar Jordan, Argentina, Albania, Indonesia Colombia 388 386 Albania Jordan, Argentina, Colombia, Indonesia, Qatar 384 Indonesia Jordan, Argentina, Colombia, Albania, Qatar Qatar Argentina, Albania, Indonesia 382 373 Peru 1. Note by Turkey: The information in this document with reference to ”Cyprus” relates to the southern part of the Island. There is no single authority representing both Turkish and Greek Cypriot people on the Island. Turkey recognises the Turkish Republic of Northern Cyprus (TRNC). Until a lasting and equitable solution is found within the context of the United Nations, Turkey shall preserve its position concerning the “Cyprus issue”. 2. Note by all the European Union Member States of the OECD and the European Union: The Republic of Cyprus is recognised by all members of the United Nations with the exception of Turkey. The information in this document relates to the area under the effective control of the Government of the Republic of Cyprus. Source: OECD, PISA 2012 Database. http://dx.doi.org/10.1787/888932935572 12 CS OECD 2014 S Kno W and Can d o: Student Performan C e in m athemati What Student , r eading and S C ien C e – Volume i © 109

112 2 o A nce i n mA them f Student Perform tic S A Profile A Colombia (12 points), Tunisia (12 points) and Brazil (11 points) score more than 10 points higher on the subscale than on the mathematics proficiency scale. Twenty other countries scores between three and ten points lower on this subscale than on the overall proficiency scale. Eleven countries and economies score 10 points or more lower on the uncertainty and data subscale than they do on the mathematics proficiency scale. Shanghai-China (21 points lower), the Russian Federation (19 points lower) and Kazakhstan (18 points lower) show the largest differences. Korea (16 points), Singapore (14 points), Macao-China (13 points), Latvia (12 points), Chinese Taipei (11 points), the Czech Republic (11 points), Estonia (10 points) and the Slovak Republic (10 points) complete this group. and the distribution uncertainty and data Figure I.2.50 describes the six levels of proficiency in the mathematics subscale of students among these six proficiency levels is shown in Figure I.2.51. • Figure I.2.50 • Summary descriptions of the six proficiency levels on the mathematical subscale uncertainty and data Percentage of students able to perform tasks at each level or above evel What students can do average) cd l E o ( 6 At Level 6, students can interpret, evaluate and critically reflect on a range of complex 3.2% statistical or probabilistic data, information and situations to analyse problems. Students at this level bring insight and sustained reasoning across several problem elements; they understand the connections between data and the situations they represent and are able to make use of those connections to explore problem situations fully. They bring appropriate calculation techniques to bear to explore data or to solve probability problems; and they can produce and communicate conclusions, reasoning and explanations. 5 At Level 5, students can interpret and analyse a range of statistical or probabilistic data, 12.5% information and situations to solve problems in complex contexts that require linking of different problem components. They can use proportional reasoning effectively to link sample data to the population they represent, can appropriately interpret data series over time, and are systematic in their use and exploration of data. Students at this level can use statistical and probabilistic concepts and knowledge to reflect, draw inferences and produce and communicate results. 4 Students at Level 4 can activate and employ a range of data representations and statistical 30.6% or probabilistic processes to interpret data, information and situations to solve problems. They can work effectively with constraints, such as statistical conditions that might apply in a sampling experiment, and they can interpret and actively translate between two related data representations (such as a graph and a data table). Students at this level can perform statistical and probabilistic reasoning to make contextual conclusions. 3 At Level 3, students can interpret and work with data and statistical information from a 54.4% single representation that may include multiple data sources, such as a graph representing several variables, or from two related data representations ,such as a simple data table and graph. They can work with and interpret descriptive statistical, probabilistic concepts and conventions in contexts such as coin tossing or lotteries, and draw conclusions from data, such as calculating or using simple measures of centre and spread. Students at this level can perform basic statistical and probabilistic reasoning in simple contexts. 2 Students at Level 2 can identify, extract and comprehend statistical data presented in a simple 76.9% and familiar form such as a simple table, a bar graph or pie chart. They can identify, understand and use basic descriptive statistical and probabilistic concepts in familiar contexts, such as tossing coins or rolling dice. At this level students can interpret data in simple representations, and apply suitable calculation procedures that connect given data to the problem context represented. 1 At Level 1, students can identify and read information presented in a small table or simple 91.7% well-labelled graph to locate and extract specific data values while ignoring distracting information, and recognise how these relate to the context. Students at this level can recognise and use basic concepts of randomness to identify misconceptions in familiar experimental contexts, such as lottery outcomes. o: Student Performan e – Volume © OECD 2014 What Student S Kno W and Can d C C e in m athemati CS , r eading and S C ien i 110

113 2 A Profile nce i n mA them A tic S A o f Student Perform • Figure I.2.51 • p roficiency in the mathematics subscale uncertainty and data ercentage of students at each level of mathematics proficiency P Level 4 Level 1 Level 6 Level 5 Below Level 1 Level 3 Level 2 Shanghai-China Shanghai-China Hong Kong-China Hong Kong-China Singapore Singapore Viet Nam Viet Nam Korea Korea Japan Japan Macao-China Macao-China Estonia Estonia Students at Level 1 Chinese Taipei Chinese Taipei or below Finland Finland Poland Poland Canada Canada Netherlands Netherlands Liechtenstein Liechtenstein Switzerland Switzerland Ireland Ireland Denmark Denmark Australia Australia Norway Norway Germany Germany United Kingdom United Kingdom Belgium Belgium Austria Austria New Zealand New Zealand Slovenia Slovenia Iceland Iceland Czech Republic Czech Republic OECD average OECD average United States United States Spain Spain Latvia Latvia Portugal Portugal France France Sweden Sweden Italy Italy Hungary Hungary Luxembourg Luxembourg Lithuania Lithuania Slovak Republic Slovak Republic Croatia Croatia Russian Federation Russian Federation Greece Greece Israel Israel Serbia Serbia Turkey Turkey Romania Romania Thailand Thailand Bulgaria Bulgaria Chile Chile United Arab Emirates United Arab Emirates Malaysia Malaysia Montenegro Montenegro Kazakhstan Kazakhstan Mexico Mexico Costa Rica Costa Rica Uruguay Uruguay Brazil Brazil Tunisia Tunisia Albania Albania Jordan Jordan Argentina Argentina Qatar Qatar Students at Level 2 Colombia Colombia or above Indonesia Indonesia Peru Peru % % 20 0 80 40 80 60 60 40 20 100 100 Countries and economies are ranked in descending order of the percentage of students at Levels 2, 3, 4, 5 and 6. Source: OECD, PISA 2012 Database, Table I.2.23. 1 http://dx.doi.org/10.1787/888932935572 2 eading and S o: Student Performan and Can W Kno C e in m athemati CS , r d C C S What Student e – Volume i © OECD 2014 111 ien

114 2 o A nce i n mA them f Student Perform tic S A Profile A The relative strengths and weaknesses of countries in different mathematics content areas Figure I.2.52 shows the country means for the overall mathematics scale and the difference in performance between each content subscale and the overall mathematics scale. As the figure makes clear, the levels of performance on the content subscales are relatively well aligned with each other and with overall mean mathematics performance, as is the case with the process subscales. However, it is also clear that the relative strength of countries in relation to the four content categories varies considerably; in fact, there is even more variability than is the case with the process subscales. It is also evident that while space and shape is frequently the strongest area among some of the higher-performing is the weakest of the four countries, this is certainly not always the case; and similarly, while change and relationships areas in several of the lower-performing countries, this is by no means true for all countries and economies. Among OECD countries, where the average score on the easiest subscale ( quantity ) and the most difficult subscale ( space ), relative to overall mathematical performance, is about 6 points, Japan shows the largest difference between and shape space and shape ) content areas of 39 points; Turkey has the smallest difference quantity ) and weakest ( its strongest ( between its strongest and weakest content areas, as it did between its strongest and weakest process areas, this time of about 7 points. Between these extremes there is a great spread, with an average difference between the strongest and change and weakest performance of about 17 points. Within that variation, six countries had the highest mean score for relationships (Estonia, Canada, Australia, Hungary, France and Turkey); six countries performed strongest in space and shape (Japan, Korea, Switzerland, the Slovak Republic, Poland and Portugal); 13 performed strongest in quantity (Israel, the Netherlands, Finland, Spain, the Czech Republic, Italy, Luxembourg, Austria, Belgium, Iceland, Germany, Slovenia (the United Kingdom, Chile, uncertainty and data and Mexico) ; and the remaining nine had the highest mean scores in Norway, Greece, Ireland, the United States, New Zealand, Denmark, and Sweden). Among partner countries and economies, Shanghai-China shows the largest difference (about 58 points) between its ); while the smallest difference between the strongest content category ( space and shape ) and its weakest ( quantity best and worst performance in the content subscales is around 11 points, seen in Uruguay, Bulgaria, Lithuania and Romania. Once again, between these extremes there is a great spread, with an average difference between the best and worst performance of about 22 points. Within that variation, three countries had the highest mean score for change and relationships ; 11 countries performed best in space and shape ; five had the highest mean score in quantity; and uncertainty and data . 12 performed best in Figure I.2.53 shows the mean score on each of the four content scales for all countries, and indicates the range of ranks (highest and lowest) that might apply to each country, taking into account the statistical uncertainty in the estimates of ranks. Gender differences in performance on the content subscales Figures I.2.54a, b, c and d, show the performance differences between boys and girls on the content subscales. On average, a larger proportion of boys than girls attains the top two proficiency levels on all four of the content subscales (Tables I.2.15, I.2.18, I.2.21 and I.2.25). change and relationships subscale, boys outperform girls by 11 points, on average across OECD countries. On the Differences of more than 20 points, in favour of boys, are seen in Chile (32 points), Colombia (29 points), Luxembourg (25 points), Austria (23 points), Japan (22 points), Korea, Liechtenstein and Costa Rica (21 points each). Twenty-four other countries and economies show significant differences in favour of boys. Six partner countries and economies show girls outperforming boys on the change and relationships subscale: Jordan (29 points), Thailand (20 points), Qatar (18 points), Malaysia (15 points), Latvia (9 points), and Kazakhstan (8 points). By contrast, in no OECD country did girls outperform boys on the subscale. On the subscale, boys outperform girls by 15 points, on average across OECD countries. Differences of more space and shape than 20 points, in favour of boys, are seen in 18 countries and economies, with the largest differences in Austria (37 points), Luxembourg (34 points), Colombia (34 points) and Chile (31 points). Twenty-seven other countries and economies show differences in favour of boys. In Iceland, girls outperform boys by a statistically significant 8 points. Statistically significant differences in favour of girls are observed in Albania (10 points), Qatar (15 points) and Jordan (15 points). Boys outperform girls on the quantity subscale by an average of 11 points across OECD countries. Differences of more than 20 points in favour of boys are seen in Colombia (31 points), Costa Rica (29 points), Luxembourg (23 points), Chile (22 points), Peru (22 points) and Liechtenstein (22 points). Meanwhile, only in four countries do girls outperform boys: Qatar (19 points), Thailand (16 points), Sweden (7 points) and Singapore (6 points). o: Student Performan e – Volume © OECD 2014 What Student S Kno W and Can d C C e in m athemati CS , r eading and S C ien i 112

115 2 i n mA them A tic S nce A Profile o f Student Perform A • Figure I.2.52 • omparing countries and economies on the different mathematics content subscales c Country’s/economy’s performance on the subscale is between 0 to 3 score points higher than on the overall mathematics scale Country’s/economy’s performance on the subscale is between 3 to 10 score points higher than on the overall mathematics scale Country’s/economy’s performance on the subscale is 10 or more score points higher than on the overall mathematics scale Country’s/economy’s performance on the subscale is between 0 to 3 score points than on the overall mathematics scale lower Country’s/economy’s performance on the subscale is between 3 to 10 score points than on the overall mathematics scale lower lower than on the overall mathematics scale Country’s/economy’s performance on the subscale is 10 or more score points Performance difference between the overall mathematics scale and each content subscale m athematics score ncertainty and data c hange and relationships Space and shape Quantity u Shanghai-China -22 11 613 36 -21 6 Singapore 573 7 -5 -14 -8 4 6 Hong Kong-China 3 561 560 -16 32 Chinese Taipei 1 -11 Korea -16 -16 19 5 554 538 Macao-China 4 20 -8 -13 -8 536 6 21 -18 Japan 4 7 535 Liechtenstein -9 3 Switzerland 13 0 -9 -1 531 523 -5 -16 9 9 Netherlands Estonia 521 9 -8 4 -10 Finland 519 2 -12 8 0 -3 Canada 518 7 -8 -2 -1 1 7 -8 518 Poland -7 Belgium 515 -6 4 -1 Germany 514 -5 4 -6 2 -4 Viet Nam 511 -2 -2 8 -7 5 -5 Austria 506 1 504 -4 5 Australia -8 4 4 7 Ireland 501 0 -24 -5 3 Slovenia 501 -2 2 -3 -6 500 Denmark 5 2 New Zealand -1 -9 1 6 500 0 499 6 -11 0 Czech Republic -3 1 -6 2 495 France OE 494 D average -1 -4 1 -1 c 8 United Kingdom 494 2 -19 0 4 Iceland 493 -6 -4 3 491 Latvia -12 -3 6 6 Luxembourg -2 -3 5 -7 490 -10 489 -12 3 7 Norway Portugal 487 -1 4 -6 -1 Italy 485 -9 2 5 -3 7 -7 -3 2 484 Spain 482 -19 -4 14 9 Russian Federation -10 482 -7 8 5 Slovak Republic 7 United States 481 7 -18 -4 Lithuania 479 -5 4 -7 0 -10 Sweden 478 -9 3 4 -1 -2 -3 Hungary 4 477 Croatia 9 -11 -3 471 -3 -1 466 -4 -17 13 Israel 2 -17 -7 7 453 Greece Serbia -7 -1 7 -3 449 -6 448 0 -5 -1 Turkey Romania 445 1 3 -1 -8 1, 2 -3 0 440 3 -1 Cyprus Bulgaria 439 -4 3 4 -7 8 434 United Arab Emirates -9 -3 -2 Kazakhstan 432 1 18 -4 -18 -13 -8 5 6 Thailand 427 8 -1 -4 -12 423 Chile 2 421 -19 14 -11 Malaysia -9 0 0 -1 413 Mexico 5 -1 2 -11 410 Montenegro Uruguay 2 -2 3 -8 409 Costa Rica 7 -1 -10 -5 407 Albania -8 -8 23 -6 394 391 Brazil -20 -11 1 11 3 Argentina 388 -10 -3 0 -9 -5 -10 12 388 Tunisia -19 Jordan 386 2 8 -1 -8 Colombia 376 12 -20 -1 5 -6 -14 376 Qatar 4 Indonesia 375 -11 7 -13 9 P 5 2 -3 368 eru -19 1. Note by Turkey: The information in this document with reference to ”Cyprus” relates to the southern part of the Island. There is no single authority representing both Turkish and Greek Cypriot people on the Island. Turkey recognises the Turkish Republic of Northern Cyprus (TRNC). Until a lasting and equitable solution is found within the context of the United Nations, Turkey shall preserve its position concerning the “Cyprus issue”. 2. Note by all the European Union Member States of the OECD and the European Union: The Republic of Cyprus is recognised by all members of the United Nations with the exception of Turkey. The information in this document relates to the area under the effective control of the Government of the Republic of Cyprus. Source: OECD, PISA 2012 Database, Tables I.2.3a, I.2.16, I.2.19, I.2.22 and I.2.25. 12 http://dx.doi.org/10.1787/888932935572 CS OECD 2014 S Kno W and Can d o: Student Performan C e in m athemati What Student , r eading and S C ien C e – Volume i © 113

116 2 f Student Perform nce i n mA them A tic S A Profile o A • ] Part 1/4 [ Figure I.2.53 • Where countries and economies rank on the different mathematics content subscales above the OECD average Statistically significantly Not statistically significantly different from the OECD average the OECD average Statistically significantly below Change and relationships subscale ange of ranks r countries o cd a E ll countries/economies m ean score pper rank ower rank ower rank pper rank l u l u 624 Shanghai-China 1 1 2 2 580 Singapore Hong Kong-China 5 564 3 5 Chinese Taipei 561 3 1 1 559 Korea 5 3 Macao-China 542 6 8 Japan 2 542 8 6 2 Liechtenstein 542 6 8 10 Estonia 530 3 4 9 530 11 9 5 3 Switzerland 12 Canada 525 4 6 10 14 11 8 5 Finland 520 5 16 11 9 518 Netherlands Germany 516 17 12 10 6 17 13 11 7 513 Belgium Viet Nam 21 13 509 Poland 509 13 13 20 7 12 19 Australia 9 15 509 15 Austria 506 9 21 14 17 Ireland 501 12 25 19 19 25 501 New Zealand 17 12 Czech Republic 499 12 19 19 27 20 17 13 499 Slovenia 25 21 France 497 13 19 28 28 20 496 Latvia 496 13 20 20 28 United Kingdom 29 Denmark 494 15 23 20 Russian Federation 491 24 32 United States 488 18 24 26 33 20 488 32 28 23 Luxembourg 24 20 487 Iceland 33 28 36 Portugal 486 19 26 27 Spain 36 32 26 23 482 38 Hungary 481 22 28 31 38 Lithuania 479 32 Norway 24 28 33 38 478 34 28 25 477 Italy 38 474 Slovak Republic 40 34 29 25 38 469 28 30 41 Sweden Croatia 468 38 41 Israel 462 28 30 39 42 448 Turkey 47 42 32 31 46 Greece 446 31 32 42 47 Romania 446 42 48 442 United Arab Emirates 43 442 42 48 Serbia 1, 2 Cyprus 440 45 48 Bulgaria 434 46 49 Kazakhstan 48 49 433 51 50 414 Thailand 411 52 50 34 33 Chile Mexico 405 33 54 34 51 402 52 56 Costa Rica Uruguay 401 52 56 Malaysia 401 52 56 54 399 56 Montenegro Albania 388 57 58 57 Jordan 59 387 Tunisia 379 58 61 61 379 Argentina 58 Brazil 62 60 372 61 364 64 Indonesia Qatar 363 62 63 63 65 357 Colombia 65 64 349 Peru 1. Note by Turkey: The information in this document with reference to ”Cyprus” relates to the southern part of the Island. There is no single authority representing both Turkish and Greek Cypriot people on the Island. Turkey recognises the Turkish Republic of Northern Cyprus (TRNC). Until a lasting and equitable solution is found within the context of the United Nations, Turkey shall preserve its position concerning the “Cyprus issue”. 2. Note by all the European Union Member States of the OECD and the European Union: The Republic of Cyprus is recognised by all members of the United Nations with the exception of Turkey. The information in this document relates to the area under the effective control of the Government of the Republic of Cyprus. Source: OECD, PISA 2012 Database. http://dx.doi.org/10.1787/888932935572 12 o: Student Performan e – Volume © OECD 2014 What Student S Kno W and Can d C C e in m athemati CS , r eading and S C ien i 114

117 2 nce i n mA them A tic S A A Profile o f Student Perform • ] Part 2/4 [ Figure I.2.53 • Where countries and economies rank on the different mathematics content subscales the OECD average above Statistically significantly Not statistically significantly different from the OECD average below the OECD average Statistically significantly Space and shape subscale ange of ranks r ll countries/economies a countries cd E o m ean score ower rank pper rank ower rank pper rank u l u l 1 649 Shanghai-China 1 2 592 Chinese Taipei 2 4 3 580 Singapore 1 5 3 Korea 1 573 6 Hong Kong-China 567 4 7 6 558 Macao-China 558 2 2 5 7 Japan 544 Switzerland 3 3 8 9 Liechtenstein 539 8 9 4 10 Poland 524 4 10 Estonia 8 14 11 5 513 16 510 5 9 11 Canada 10 5 Belgium 509 17 11 19 11 12 Netherlands 507 5 Germany 12 19 11 5 507 Viet Nam 21 11 507 Finland 12 6 507 18 11 16 12 9 503 Slovenia 20 501 24 Austria 15 16 9 16 Czech Republic 25 17 499 10 497 Latvia 26 18 19 12 497 Denmark 25 16 Australia 497 12 16 20 25 18 496 Russian Federation 28 21 Portugal 491 13 22 31 491 New Zealand 30 23 21 15 490 14 22 22 32 Slovak Republic 22 France 489 16 24 31 Iceland 489 16 21 25 30 25 22 31 16 Italy 487 Luxembourg 19 31 28 22 486 36 Norway 480 22 27 31 32 478 36 27 Ireland 23 23 36 27 32 Spain 477 United Kingdom 37 32 28 23 475 24 Hungary 38 32 28 474 33 472 Lithuania 38 27 469 Sweden 29 39 36 463 28 29 37 40 United States Croatia 460 39 41 Kazakhstan 450 41 45 Israel 40 46 31 30 449 447 Romania 41 46 41 Serbia 46 446 41 Turkey 443 30 32 49 442 Bulgaria 49 42 50 436 31 32 46 Greece 1, 2 Cyprus 436 46 49 50 434 Malaysia 46 Thailand 432 46 51 52 United Arab Emirates 425 50 54 Chile 419 33 33 51 Albania 55 418 52 413 53 56 Uruguay Mexico 413 34 34 53 56 Montenegro 412 54 56 57 57 397 Costa Rica Jordan 385 58 62 58 Argentina 62 385 Indonesia 383 58 63 63 382 Tunisia 58 59 63 381 Brazil Qatar 380 60 63 65 Peru 370 64 65 Colombia 64 369 1. Note by Turkey: The information in this document with reference to ”Cyprus” relates to the southern part of the Island. There is no single authority representing both Turkish and Greek Cypriot people on the Island. Turkey recognises the Turkish Republic of Northern Cyprus (TRNC). Until a lasting and equitable solution is found within the context of the United Nations, Turkey shall preserve its position concerning the “Cyprus issue”. 2. Note by all the European Union Member States of the OECD and the European Union: The Republic of Cyprus is recognised by all members of the United Nations with the exception of Turkey. The information in this document relates to the area under the effective control of the Government of the Republic of Cyprus. OECD, PISA 2012 Database. Source: http://dx.doi.org/10.1787/888932935572 12 CS OECD 2014 S Kno W and Can d o: Student Performan C e in m athemati What Student , r eading and S C ien C e – Volume i © 115

118 2 nce i n mA them A tic S A A Profile o f Student Perform • ] Part 3/4 [ Figure I.2.53 • Where countries and economies rank on the different mathematics content subscales the OECD average above Statistically significantly Not statistically significantly different from the OECD average the OECD average below Statistically significantly Quantity subscale ange of ranks r a ll countries/economies o E cd countries m ean score pper rank ower rank ower rank pper rank u l u l 591 1 Shanghai-China 1 569 Singapore 3 2 2 3 566 Hong Kong-China 5 543 Chinese Taipei 4 Liechtenstein 7 538 4 8 4 3 1 537 Korea 10 532 1 4 5 Netherlands Switzerland 531 1 4 6 10 9 531 Macao-China 7 Finland 5 11 8 3 527 12 Estonia 525 3 6 9 6 519 Belgium 16 12 10 11 17 10 5 519 Poland Japan 11 17 11 5 518 17 12 11 6 517 Germany 7 17 515 11 13 Canada 15 9 510 Austria 19 13 Viet Nam 13 24 509 Ireland 505 11 15 17 22 17 23 Czech Republic 505 11 16 Slovenia 22 15 12 504 18 Denmark 502 12 17 18 24 26 19 21 14 500 Australia 27 499 14 20 21 New Zealand 22 16 496 Iceland 29 23 29 496 16 23 22 France 25 Luxembourg 495 18 22 29 32 22 25 16 494 United Kingdom 33 Norway 492 25 25 18 Spain 491 33 27 25 20 491 33 28 25 21 Italy 36 29 Latvia 487 29 486 22 28 37 Slovak Republic 483 Lithuania 39 32 33 29 482 40 Sweden 25 30 Portugal 41 32 481 25 480 41 Croatia 33 30 25 480 Israel 41 32 478 35 41 Russian Federation 34 United States 26 30 478 41 Hungary 476 27 30 36 41 Serbia 456 43 42 Greece 42 31 31 455 43 44 Romania 47 443 47 Bulgaria 443 44 442 Turkey 48 44 32 32 1, 2 Cyprus 439 45 47 49 United Arab Emirates 431 47 50 48 Kazakhstan 428 51 49 33 33 421 Chile 53 50 419 Thailand Mexico 414 34 54 34 51 411 52 56 Uruguay Malaysia 409 52 56 Montenegro 409 53 56 406 53 56 Costa Rica Brazil 393 57 58 57 Argentina 59 391 Albania 386 58 60 378 62 Tunisia 59 60 375 Colombia 62 61 371 63 Qatar Jordan 367 62 65 62 65 365 Peru 65 63 362 Indonesia 1. Note by Turkey: The information in this document with reference to ”Cyprus” relates to the southern part of the Island. There is no single authority representing both Turkish and Greek Cypriot people on the Island. Turkey recognises the Turkish Republic of Northern Cyprus (TRNC). Until a lasting and equitable solution is found within the context of the United Nations, Turkey shall preserve its position concerning the “Cyprus issue”. 2. Note by all the European Union Member States of the OECD and the European Union: The Republic of Cyprus is recognised by all members of the United Nations with the exception of Turkey. The information in this document relates to the area under the effective control of the Government of the Republic of Cyprus. OECD, PISA 2012 Database. Source: http://dx.doi.org/10.1787/888932935572 12 o: Student Performan e – Volume © OECD 2014 What Student S Kno W and Can d C C e in m athemati CS , r eading and S C ien i 116

119 2 A Profile nce i n mA them A tic S A o f Student Perform • ] Part 4/4 [ Figure I.2.53 • Where countries and economies rank on the different mathematics content subscales Statistically significantly the OECD average above Not statistically significantly different from the OECD average the OECD average below Statistically significantly Uncertainty and data subscale ange of ranks r ll countries/economies o cd countries a E ean score m pper rank ower rank ower rank pper rank l u l u Shanghai-China 1 1 592 Singapore 559 2 2 4 Hong Kong-China 3 553 4 3 549 Chinese Taipei 5 7 2 1 538 Korea 3 1 532 Netherlands 8 5 Japan 528 2 4 6 10 6 11 Liechtenstein 526 Macao-China 525 7 10 13 7 Switzerland 522 3 6 Viet Nam 15 8 519 14 Finland 519 4 10 7 Poland 8 16 10 4 517 Canada 516 4 7 11 14 Estonia 19 14 7 510 12 Germany 21 14 14 7 509 509 15 14 Ireland 8 21 Belgium 21 15 14 8 508 9 508 14 Australia 16 21 16 506 9 22 15 New Zealand Denmark 23 17 16 10 505 502 United Kingdom 11 17 18 24 Austria 19 21 26 14 499 27 497 15 20 22 Norway 20 16 496 Slovenia 27 23 496 16 20 23 27 Iceland 23 France 18 492 24 30 Czech Republic 488 20 25 27 32 19 488 26 26 34 United States Spain 487 28 33 25 20 Portugal 486 20 27 27 35 Luxembourg 34 31 24 483 27 Sweden 35 29 28 23 483 482 30 27 Italy 23 35 Latvia 37 32 478 27 Hungary 39 34 29 476 35 474 Lithuania 39 40 35 28 472 Slovak Republic 30 468 37 41 Croatia 31 465 29 Israel 38 42 Russian Federation 463 39 42 31 30 Greece 460 42 40 Serbia 44 43 448 447 45 43 32 32 Turkey 1, 2 Cyprus 44 442 46 437 45 49 Romania 433 Thailand 46 50 United Arab Emirates 432 46 50 Bulgaria 50 46 432 Chile 430 33 33 47 50 50 422 52 Malaysia 55 Montenegro 415 52 Costa Rica 55 414 52 414 52 55 Kazakhstan Mexico 413 34 34 52 55 Uruguay 407 55 57 56 58 402 Brazil Tunisia 399 56 59 61 394 Jordan 58 63 Argentina 389 59 59 388 Colombia 63 386 60 63 Albania Indonesia 384 60 64 64 Qatar 382 63 Peru 373 65 65 1. Note by Turkey: The information in this document with reference to ”Cyprus” relates to the southern part of the Island. There is no single authority representing both Turkish and Greek Cypriot people on the Island. Turkey recognises the Turkish Republic of Northern Cyprus (TRNC). Until a lasting and equitable solution is found within the context of the United Nations, Turkey shall preserve its position concerning the “Cyprus issue”. 2. Note by all the European Union Member States of the OECD and the European Union: The Republic of Cyprus is recognised by all members of the United Nations with the exception of Turkey. The information in this document relates to the area under the effective control of the Government of the Republic of Cyprus. Source: OECD, PISA 2012 Database. http://dx.doi.org/10.1787/888932935572 12 CS OECD 2014 S Kno W and Can d o: Student Performan C e in m athemati What Student , r eading and S C ien C e – Volume i © 117

120 2 f Student Perform A nce i n mA them A tic S o A Profile • • Figure I.2.54a g subscale change and relationships ender differences in performance on the Girls Boys All students Gender differences Mean score on (boys – girls) the change and relationships subscale Jordan Thailand Qatar Malaysia Latvia Kazakhstan Sweden Russian Federation United Arab Emirates Montenegro Iceland Albania Bulgaria Indonesia Turkey Macao-China Girls perform Boys perform Singapore better better Poland Romania Lithuania Finland Norway Slovak Republic Greece Chinese Taipei Slovenia United States Croatia Serbia Estonia Hungary Czech Republic Viet Nam Belgium OECD average Netherlands 11 score points Portugal Shanghai-China Uruguay OECD average Mexico Germany France Switzerland Australia Israel Ireland Canada Peru United Kingdom Argentina Denmark Hong Kong-China Spain New Zealand Tunisia Italy Brazil Costa Rica Liechtenstein Korea Japan Austria Luxembourg Colombia Chile -10 4030 20 10 0 300 -20 -30 -40 650 600 550 500 450 400 350 Mean score Score-point difference Statistically signicant gender differences are marked in a darker tone (see Annex A3). Note: girls). – Countries and economies are ranked in ascending order of the gender score-point difference (boys OECD, PISA 2012 Database, Table I.2.16. Source: 2 1 http://dx.doi.org/10.1787/888932935572 o: Student Performan e in © OECD 2014 What Student S Kno W and Can d m i e – Volume C ien C eading and S r , CS athemati C 118

121 2 A Profile A nce i n mA f Student Perform A tic S o them • Figure I.2.54b • subscale space and shape ender differences in performance on the g Girls Boys All students Gender differences Mean score on the space and shape subscale (boys – girls) Jordan Qatar Albania Iceland Singapore Thailand Lithuania United Arab Emirates Finland Latvia Shanghai-China Bulgaria Malaysia Sweden Boys perform Girls perform Norway better better Estonia Russian Federation Montenegro Slovenia Macao-China Chinese Taipei United States Poland Kazakhstan Romania Canada Serbia Greece Turkey Israel United Kingdom Denmark OECD average Slovak Republic 15 score points Argentina Portugal Croatia OECD average Netherlands Germany France Uruguay Hungary Spain Belgium Japan Switzerland Australia Korea Hong Kong-China Mexico Czech Republic Indonesia Italy Liechtenstein Viet Nam Brazil Ireland New Zealand Tunisia Costa Rica Peru Chile Colombia Luxembourg Austria 20 0 4030 -10 -20 650 600 550 500 450 400 350 300 10 Score-point difference Mean score Statistically signicant gender differences are marked in a darker tone (see Annex A3). Note: girls). – Countries and economies are ranked in ascending order of the gender score-point difference (boys OECD, PISA 2012 Database, Table I.2.19. Source: http://dx.doi.org/10.1787/888932935572 2 1 o: Student Performan eading and S C ien d and Can W Kno S i What Student e – Volume e in C m athemati CS , C 119 OECD 2014 © r

122 2 S f Student Perform A nce i n mA them A tic o A Profile • Figure I.2.54c • subscale quantity ender differences in performance on the g Boys Girls All students Gender differences Mean score on the quantity subscale (boys – girls) Qatar Thailand Jordan Malaysia Sweden United Arab Emirates Singapore Iceland Finland Bulgaria Boys perform Girls perform Latvia better better Russian Federation Montenegro Kazakhstan Romania Lithuania Indonesia Norway Poland Macao-China Albania United States Viet Nam Slovenia Estonia Hungary Chinese Taipei Serbia France Hong Kong-China Canada Shanghai-China Uruguay Czech Republic Switzerland OECD average Australia 11 score points Greece Netherlands Belgium OECD average Slovak Republic Portugal Korea Argentina Israel United Kingdom Turkey Ireland New Zealand Germany Tunisia Denmark Croatia Mexico Austria Italy Brazil Japan Spain Liechtenstein Peru Chile Luxembourg Costa Rica Colombia 4030 20 10 0 -10 -20 -30 650 600 300 500 450 400 350 550 Score-point difference Mean score Statistically signicant gender differences are marked in a darker tone (see Annex A3). Note: girls). – Countries and economies are ranked in ascending order of the gender score-point difference (boys OECD, PISA 2012 Database, Table I.2.22. Source: http://dx.doi.org/10.1787/888932935572 2 1 i C © OECD 2014 What Student S Kno W and Can d e in e – Volume C ien C eading and S r , CS athemati m o: Student Performan 120

123 2 S f Student Perform A nce i n mA them A tic o A Profile • Figure I.2.54d • subscale uncertainty and data ender differences in performance on the g All students Boys Girls Mean score on Gender differences the uncertainty and data subscale (boys – girls) Jordan Thailand Malaysia Qatar Iceland United Arab Emirates Finland Russian Federation Singapore Bulgaria Boys perform Girls perform Albania better better Latvia Indonesia Slovenia Lithuania Montenegro Sweden Kazakhstan Norway Viet Nam France Romania Poland Macao-China United States Chinese Taipei Tunisia Shanghai-China Greece Estonia Australia Belgium OECD average Hungary 9 score points New Zealand Mexico Canada OECD average Turkey Netherlands Uruguay Croatia Peru Czech Republic Slovak Republic Israel Serbia Argentina Brazil Portugal Hong Kong-China Japan Colombia United Kingdom Denmark Germany Ireland Switzerland Italy Spain Korea Austria Chile Costa Rica Liechtenstein Luxembourg 30 20 10 0 -10 -20 -30 -40 650 600 550 500 450 400 350 300 Mean score Score-point difference Statistically signicant gender differences are marked in a darker tone (see Annex A3). Note: girls). – Countries and economies are ranked in ascending order of the gender score-point difference (boys OECD, PISA 2012 Database, Table I.2.25. Source: 1 http://dx.doi.org/10.1787/888932935572 2 OECD 2014 CS S Kno W and Can d o: Student Performan C e in m 121 What Student © i e – Volume C ien C eading and S r , athemati

124 2 S f Student Perform A nce i n mA them A tic o A Profile subscale by an average of 9 points – the uncertainty and data Across OECD countries, boys outperform girls on the smallest average difference of the four content subscales. The largest performance difference in favour of boys (23 points) is seen in Luxembourg. In Liechtenstein this difference is about 22 points, and in 31 other countries and economies boys outperform girls on this subscale by less than 20 points. Iceland and Finland are the only OECD countries where girls outperform boys on this subscale (11 and 5 points in favour for girls, respectively), but among partner countries and economies, four show substantial differences in favour of girls: Jordan (30 points), Thailand (16 points), Malaysia (15 points) and Qatar (13 points). urkey t : a S pi mproving in i Box I.2.5. When it first participated in PISA, in 2003, Turkey was among the lowest-performing OECD countries in mathematics, reading and science. Yet Turkey’s performance in all three domains has improved markedly since then, at an average yearly rate of 3.2, 4.1 and 6.4 points per year. In 2003, for example, the average 15-year-old student in Turkey scored 423 points in mathematics. With an average annual increase of 3.2 points, the average score in mathematics in 2012 was 448 points – an improvement over 2003 scores that is the equivalent of more than half a year of schooling. Much of this improvement was concentrated among students with the greatest educational needs. The mathematics th scores of Turkey’s lowest-achieving students (the 10 percentile) improved from 300 to 338 points between 2003 and 2012, with no significant change among the highest-achieving students during the period. Consistent with this trend, the share of students who perform below proficiency Level 2 in mathematics shrank from 52% in 2003 to 42% in 2012. Between-school differences in average mathematics performance did not change between 2003 and 2012, but differences in performance among students within schools narrowed during that time, meaning that much of the improvement in mathematics performance observed between 2003 and 2012 is the result of low-performing students across all schools improving their performance (Table II.2.1b). The observed improvement in mathematics was concentrated among socio-economically disadvantaged and low- achieving students. Between 2003 and 2012, both the average difference in performance between advantaged and disadvantaged students and the degree to which students’ socio-economic status predicts their performance shrank. In 2003, advantaged students outperformed disadvantaged students by almost 100 score points; in 2012, the difference was around 60 score points. In 2003, 28% of the variation in students’ scores (around the OECD average) was explained by students’ socio-economic status; by 2012, 15% of the variation (below the OECD average) was explained by students’ socio-economic status. While all students, on average, improved their scores no matter where their schools were located, students attending schools in towns (population of 3 000 to 100 000) improved their mathematics scores by 59 points between 2003 and 2012 – more than the increase observed among students in cities or large cities (population greater than 100 000; no change in performance detected). Turkey has a highly centralised school system: education policy is set centrally at the Ministry of National Education and schools have comparatively little autonomy. Education policy is guided by a two-year Strategic Plan and a four-year Development Plan. The Basic Education Programme (BEP), launched in 1998, sought to expand primary education, improve the quality of education and overall student outcomes, narrow the gender gap in performance, align performance indicators with those of the European Union, develop school libraries, ensure that qualified teachers were employed, integrate information and communication technologies into the education system, and create local learning centres, based in schools, that are open to everyone (OECD, 2007). The Master Implementation - 05), designed in collaboration with UNICEF, and the Secondary Project (2006-11), in collaboration with Plan (2001 the World Bank, included multiple projects to improve both equity and quality in the education system. The Standards for Primary Education, piloted in 2010 and recently expanded to all primary institutions, defines quality standards for assessments, and - primary education, guides schools in achieving these standards, develops a system of school self guides local and central authorities in addressing inequalities among schools. One of the major changes introduced with the BEP programme involved the compulsory education law. This change was first implemented in the 1997/98 school year, and in 2003 the first students graduated from the eight-year compulsory education system. Since the launch of this programme, the attendance rate among primary students increased from around 85% to nearly 100%, while the attendance rate in pre-primary programmes increased from 10% to 25%. In addition, the system was expanded to include 3.5 million more pupils, average class size was reduced to roughly 30 students, all students learn at least one foreign language, computer laboratories were established in every primary school, and overall physical conditions were improved in all 35 000 rural schools. ... o: Student Performan eading and S © OECD 2014 What Student S Kno W and Can d C C e in m athemati CS , i e – Volume C ien r 122

125 2 f Student Perform A nce i n mA them A tic S o A Profile Resources devoted to the programme exceeded USD 11 billion. This programme did not directly affect school participation for most of the 15-year-olds assessed by PISA, who are mainly in secondary schools where enrolment rates are close to 60%. In 2012, compulsory education was increased from 8 to 12 years of schooling, and the school system was redefined into three levels (primary, lower secondary and upper secondary) of four years each. Fifteen-year-old students in Turkey are the least likely among students in all OECD countries to have attended pre - primary education. Several initiatives are in place to change this, but none has yet had a direct impact on the students who participated in PISA 2012. Early childhood education and care is featured in the current Development Plan (2014-18) and other on-going programmes include the Mobile Classroom (for children aged 36 - 66 months from low-income families), the Summer Preschool (for children aged 60-66 months), the Turkey Country Programme, and the Pre-School Education Project. New curricula were introduced in the 2006/07 school year, starting from the 6th grade. The secondary school mathematics and language curricula were also revised and a new science curriculum was applied in the 9th grade for the 2008/09 school year. In PISA 2012 students had already been taught the new curriculum for four years, although their primary school education was part of the former system. The standards of the new curricula were intended to meet PISA goals: “Increased importance has been placed on students’ doing mathematics which means exploring mathematical ideas, solving problems, making connections among mathematical ideas, and (Talim ve Terbiye Kurulu [TTKB] [Board of Education], 2008). applying them in real life situations” The curricular reform was designed not only to change the content of school education and encourage the introduction of innovative teaching methods, but above all to change the teaching philosophy and culture within schools. The new curricula and teaching materials emphasise “student-centred learning”, giving students a more active role than before, when memorising information had been the predominant approach. They also reflect the assumption, on which PISA is based, that schools should equip students with the skills needed to ensure success at school and in life, in general. In 2003, more than one in four students reported having arrived late for school at least once in the two weeks prior to the PISA test; by 2012, more than four in ten students reported having arrived late. By contrast, students’ sense of belonging at school seems to have improved during the same period. Students in 2012 also spent one half an hour less per week in mathematics instruction than students in 2003 did, and almost an hour and a half less per week in after-school study. Students in 2012 attended schools with better physical infrastructure and better educational resources than their counterparts in 2003 did. Throughout 2004 and 2005, private-sector investments funded 14 000 additional classrooms in the country. Taxes were reduced for private businesses that invested in education. This was particularly helpful in provinces where there was large internal migration (OECD, 2006). Several policies had sought to change the culture and management of schools. Schools were obliged to propose a plan of work, including development targets and strategic plans for reaching them. More democratic governance, parental involvement and teamwork were suggested. In 2004, a project aimed at teaching students democratic skills was started in all primary and secondary schools, with many responsibilities assigned to student assemblies. In addition, more transparent and performance-oriented inspection tools were introduced. Teachers were also the target of policy changes. New arrangements were implemented in 2008 to train teachers for upper secondary education through five-year graduate programmes. The arrangements also stipulated that graduates in other fields, such as science or literature, who wanted to teach would also have to attend a year- and-a-half of graduate training in education. The Teacher Formation Programmes of Education Faculties (2008) links pre-service training courses to the Ministry’s curriculum and teacher-practice standards while giving more autonomy to faculties on the courses that should be taught. The New Teacher Programme, introduced in 2011, established stricter requirements for certain subjects. Several projects implemented over the past decade have addressed equity issues. The Girls to Schools Now campaign, in collaboration with UNICEF, that started in 2003 aimed to ensure that all girls aged 6 to 14 attend primary school. Efforts to increase enrolment in school continue through programmes like the Address-Based Population Registry System, the Education with Transport , which creates a registry to identify non-schooled children programme, which benefits students who have no access to school, and the Complementary Transitional Training ... CS eading and S S Kno W and Can d o: Student Performan C e in m athemati What Student , 123 OECD 2014 © i e – Volume C ien C r

126 2 tic f Student Perform A nce i n mA them A o S A Profile tries to ensure that 10-14 year-olds acquire a basic education even if they have never been Programme, which The Project for Increasing Enrolment Rates Especially . enrolled in a school or if they had dropped out of school in a pilot phase in the 16 provinces with the lowest enrolment rates among girls, addresses families’ for Girls, awareness about the links between education and the labour market. Since 2003, textbooks for all primary students have been supplied free of charge by the Ministry of National Education. The International Inspiration and the Strengthening Special Education Project, begun in 2010, are designed to promote Project, begun in 2011, disadvantaged students’ performance. Sources: utlook: Turkey, o Education Policy OECD (2013d), OECD Publishing. LICY%20 oo n RKEY_E u TL ou .pdf o %20P on CATI u K%20T http://www.oecd.org/edu/ED OECD (2007), ational Policies for Education: Basic Education in Turkey n , OECD Publishing. Reviews of http://dx.doi.org/10.1787/9789264030206-en , OECD Publishing. Economic Survey of Turkey: 2006 OECD (2006), http://dx.doi.org/10.1787/eco_surveys-tur-2006-en ğ Ilkögretim Matematik Dersi 6-8 Sınıflar Ö Talim ve Terbiye Kurulu (TTKB) (2008), retim Programı ve Kılavuzu (Teaching Syllabus , Milli E and Curriculum Guidebook for Elementary School Mathematics Course: Grades 6 to 8) ı, Ankara. ğ itim Bakanlı ğ C d © OECD 2014 What Student S Kno W i e – Volume o: Student Performan ien C eading and S r , CS athemati m e in C and Can 124

127 2 o A nce i n mA them A tic S A Profile f Student Perform of S a S pi mathematic S xample e S unit • Figure I.2.55 • helen t S cycli the Helen has just got a new bike. It has a speedometer which sits on the handlebar. The speedometer can tell Helen the distance she travels and her average speed for a trip. This unit is concerned with journeys by bicycle. Its storyline about an individual person places it into the personal context category. Slight changes in the context of the unit could place these questions into the or scientific occupational categories. These categories are designed to ensure breadth of appeal to students in the contexts used in the assessment and are a checklist to promote inclusion of all aspects of life. They are not reporting categories. The concern with change and relationships relationships between distance, time and speed puts these questions in the content category. HELEN THE CYCLIST ESTI on 1 – Qu On one trip, Helen rode 4 km in the first 10 minutes and then 2 km in the next 5 minutes. Which one of the following statements is correct? ’s average speed was greater in the first 10 minutes than in the next 5 minutes. Helen A. Helen ’s average speed was the same in the first 10 minutes and in the next 5 minutes. B. ’s average speed was less in the first 10 minutes than in the next 5 minutes. Helen C. It is not possible to tell anything about Helen ’s average speed from the information given. D. Scoring Level 6 d escription: Compare average speeds given distances travelled and times taken 669 Level 5 athematical content area: m Change and relationships 607 Level 4 Personal ontext: c 545 Employ Process: Level 3 482 Question format: Simple multiple choice Level 2 420 d 440.5 ifficulty: Level 1 358 Below Level 1 f ull c redit . Helen’s average speed was the same in the first 10 minutes and in the next 5 minutes. B n redit c o Other responses. Missing. and Can m e in C o: Student Performan d W Kno S What Student OECD 2014 125 i e – Volume C ien C eading and S r , CS athemati ©

128 2 o A nce i n mA them A f Student Perform S A Profile tic Comment Question 1, a simple multiple choice item, requires comparison of speed when travelling 4 km in 10 minutes versus 2 km process category because it requires the precise mathematical employing in 5 minutes. It is been classified within the understanding that speed is a rate and that proportionality is the key. This question can be solved by recognising the doubles involved (2 km – 4 km; 5 km – 10 km), which is the very simplest notion of proportion. Consequently, with this Level 2 question, successful students demonstrate a very basic understanding of speed and of proportion calculations. If distance and time are in the same proportion, the speed is the same. f course, students could correctly solve the o problem in more complicated ways (e.g. calculating that both speeds are 24 km per hour) but this is not necessary. PISA results for this question do not incorporate information about the solution method used. The correct response option here is B (Helen’s average speed was the same in the first 10 minutes and in the next 5 minutes). – Qu HELEN THE CYCLIST ESTI on 2 Helen rode 6 km to her aunt’s house. Her speedometer showed that she had averaged 18 km/h for the whole trip. Which one of the following statements is correct? It took Helen 20 minutes to get to her aunt’ A. s house. It took Helen 30 minutes to get to her aunt’ s house. B. It took Helen 3 hours to get to her aunt’ C. s house. It is not possible to tell how long it took Helen to get to her aunt’ D. s house. Scoring Level 6 escription: Calculate time travelled given average speed and distance travelled d 669 Level 5 Change and relationships m athematical content area: 607 Level 4 ontext: Personal c 545 Process: Employ Level 3 482 Simple multiple choice Question format: Level 2 420 d ifficulty: 510.6 Level 1 358 Below Level 1 f c redit ull A. It took Helen 20 minutes to get to her aunt’ s house. n o c redit Other responses. Missing. Comment Question 2 is at Level 3. Again, it is classified in the employing process category and can be solved by simple proportional reasoning, from the understanding of the meaning of the speed: 18 kilometres travelled in one hour. For one third of the distance, the time is one third of an hour, which is 20 minutes (hence the correct answer A: It took Helen 20 minutes to get to her aunt’s house). Information about the percentage of students choosing each multiple choice is available for future analysis through the public databases. on 3 ESTI – Qu HELEN THE CYCLIST Helen rode her bike from home to the river, which is 4 km away. It took her 9 minutes. She rode home using a shorter route of 3 km. This only took her 6 minutes. What was Helen’s average speed, in km/h, for the trip to the river and back? Average speed for the trip: ...km/h What Student e – Volume S Kno W and Can d i OECD 2014 © o: Student Performan C e in m athemati CS , r eading and S C ien C 126

129 2 o A nce i n mA them f Student Perform tic S A Profile A Level 6 Scoring 669 Level 5 607 Calculate average speed over two trips given two distances travelled and the times taken escription: d Level 4 545 athematical content area: Change and relationships m Level 3 482 Personal ontext: c Level 2 Employ Process: 420 Level 1 Constructed response manual Question format: 358 Below Level 1 ifficulty: 696.6 d ull f redit c 28 redit c o n Other responses. [Incorrect method: average of speeds for 2 trips (26.67 and 30)]. 28.3 Missing. Comment Question 3 requires a deeper understanding of the meaning of average speed, appreciating the importance of linking total time with total distance. Average speed cannot be obtained just by averaging the speeds, even though in this specific case the incorrect answer (28.3 km/hr) obtained by averaging the speeds (26.67 km/hr and 30 km/hr) is not much different from the correct answer of 28 km/hr. There are both mathematical and real world understandings of this reasoning and phenomenon, leading to high demands on the fundamental mathematical capabilities of mathematisation and also using symbolic, formal and technical language and operations. and argumentation For students who know to work from total time (9 + 6 = 15 minutes) and total distance (4 + 3 = 7 km), the answer can be obtained simply by proportional reasoning (7 km in ¼ hour is 28 km in 1 hour), or by more complicated formula approaches (e.g. distance / time = 7 / (15/60) = 420 / 15 = 28). This question has been classified as an employing process because the greatest part of the demand was judged to arise from the mathematical definition of average speed and possibly also the unit conversion, especially for students using speed–distance–time formulas. It is one of the more difficult tasks of the item pool, and sits in Level 6 on the proficiency scale. General comment on this unit Some indication of the increasing difficulty of the three questions of this unit can be appreciated by looking at the overall strategies for the three questions. In Question 1, two rates are to be compared. In Question 2, the solution strategy goes from speed and distance, to time with a unit conversion. In Question 3, the four quantities have to be combined in a way that students often find counter-intuitive. Instead of combining the distance-time information for each trip, the two distances and the two times are combined, giving new distance and time, and so average speed. In the most elegant solutions, all the arithmetic is simple, but in practice students’ methods may often involve more complicated calculation. © r , eading and S C ien C e – Volume i OECD 2014 CS athemati m e in C o: Student Performan d and Can W Kno S What Student 127

130 2 A Profile A nce i n mA them A tic S f Student Perform o • Figure I.2.56 • fuji mount climbing L NG MOUNT FUJ i i c MB i olcano in Japan. Mount Fuji is a famous dormant v CLIMBING MOUNT FUJI – Qu ESTI on 1 Mount Fuji is only open to the public for climbing from 1 July to 27 August each year. About 200 000 people climb Mount Fuji during this time. On average, about how many people climb Mount Fuji each day? A. 340 710 B. 3 400 C. 7 100 D. 7 400 E. Level 6 Scoring 669 Level 5 607 d escription: Identify an average daily rate given a total number and a specific time period (dates provided) Level 4 545 m athematical content area: Quantity Level 3 482 Societal c ontext: Level 2 Process: Formulate 420 Level 1 Simple multiple choice Question format: 358 Below Level 1 d ifficulty: 464 redit c f ull C. 3 400 n o c redit Other responses. Missing. Comment Question 1 goes beyond personal concerns of a walker to wider community issues – in this case possibly concerns of societal use of the public trail. Items classified as involve such things as voting systems, public transport, government, public policies, demographics, advertising, national statistics and economics. Although individuals are involved in these things in a personal way, in the societal context category the focus of problems is more on the community perspective. Allocation to the context category is only carried out in order to ensure a balance across the assessment and is not used for reporting. With minor rewording, presenting the challenges from the point of view of the decisions made by park rangers, this unit could have belonged to the occupational category. eading and S r , CS i m e in C o: Student Performan d and Can W Kno S What Student OECD 2014 © e – Volume C ien C athemati 128

131 2 o A nce i n mA them f Student Perform tic S A Profile A Question 1 is presented in the simple multiple choice format (choose one out of four). Question 2 requires the answer 11 a.m. and so is a constructed response item with expert scoring needed to ensure that all equivalent ways of writing the time are picked up. Question 3, requiring the number 40 for full score, or the number 0.4 (answering in metres) for partial credit, also had expert scoring. Question 1 requires calculation of the number of days the trail is open using the given dates, and then calculation of quantity content category because it involves quantification of time and of an an average. It has been allocated to the average. The formula for average is required and this is indeed a relationship, but in this question the focus is on its use in finding the number of people per day, rather than inherently about the relationship. For this reason, the question is change and relationships not in the category. Question 3 has similar characteristics, involving units of length. The correct response to Question 1 is C: 3400. CLIMBING MOUNT FUJI – Qu on 2 ESTI The Gotemba walking trail up Mount Fuji is about 9 kilometres (km) long. Walkers need to return from the 18 km walk by 8 p.m. Toshi estimates that he can walk up the mountain at 1.5 kilometres per hour on average, and down at twice that speed. These speeds take into account meal breaks and rest times. Using Toshi’s estimated speeds, what is the latest time he can begin his walk so that he can return by 8 p.m.? ... Scoring escription: d Calculate the start time for a trip given two different speeds, a total distance to travel and a finish time Level 6 669 Change and relationships athematical content area: m Level 5 Societal ontext: c 607 Level 4 Formulate Process: 545 Level 3 Question format: Constructed response expert 482 Level 2 ifficulty: d 641.6 420 Level 1 358 Below Level 1 ull f c redit [with or without a.m., or an equivalent way of writing time, for example, 11:00] 11 (a.m.) o redit n c Other responses. Missing. Comment Question 2 is allocated to the change and relationships category, because here the relationship between distance and time, encapsulated as speed, is paramount. From information about distances and speed, the time to go up and the time to go down have to be quantified, and then used in combination with the finishing time to get the starting time. Had the times to go up and down been given directly, rather than indirectly through distance and speed, then the question could have also belonged in the quantity category. Because PISA questions are set in real contexts, they usually involve multiple mathematical topics and underlying mathematical phenomena, so it is necessary to make judgements about the major source of demand in order to categorise them. Allocating the process category similarly requires judgement about the major demand of the item. Question 1 has been allocated to the formulating category, because of the judgement that the major demand in this relatively easy item is to take the two pieces of real world information (open season and total number of climbers), and to set up the mathematical problem to be solved: find the length of the open season from the dates and use it with the information about the total to find the average. Expert judgement is that the major cognitive demand for 15-year-olds lies in this movement from the real world problem to the mathematical relationships, rather than in the ensuing whole number calculations. Question 2 has also been allocated to the formulating process category, because again the major demand is judged to arise from the e in OECD 2014 m What Student Kno W and Can d S o: Student Performan C athemati CS , r eading and S C ien C e – Volume i © 129

132 2 o A nce i n mA them A tic f Student Perform A Profile S transformation from the real world data to the mathematical problem, identifying all the relationships involved, rather than in carrying out the calculations or in interpreting the answer as a starting time of 11 a.m. In this difficult item, the mathematical structure involves multiple relationships: starting time = finishing time – duration, duration = time up + time down, time up (down) = distance / speed (or equivalent proportional reasoning), time down = half time up, and appreciating the simplifying assumptions that average speeds already include consideration of variable speed during the day and that no further allowance is required for breaks. CLIMBING MOUNT FUJI – Qu ESTI on 3 Toshi wore a pedometer to count his steps on his walk along the Gotemba trail. His pedometer showed that he walked 22 500 steps on the way up. Estimate Toshi’s average step length for his walk up the 9 km Gotemba trail. Give your answer in centimetres (cm). Answer: ... cm Level 6 Scoring 669 Level 5 607 escription: d Divide a length given in km by a specific number and express the quotient in cm Level 4 545 athematical content area: m Quantity Level 3 482 ontext: c Societal Level 2 Employ Process: 420 Level 1 Constructed response manual Question format: 358 Below Level 1 d ifficulty: 610 f c redit ull 40 P artial c redit Responses with the digit 4 based on incorrect conversion to centimetres. • 0.4 [answer given in metres]. • 4 000 [incorrect conversion]. redit o c n Other responses. Missing. Comment employing Question 3 has been allocated to the category. There is one main relationship involved: the distance walked = number of steps × average step length. To use this relationship to solve the problem, there are two obstacles: rearranging the formula (which is probably done by students informally rather than formally using the written relationship) so that the average step length can be found from distance and number of steps, and making appropriate unit conversions. For this question, it was judged that the major cognitive demand comes from carrying out these steps; hence it has been categorised in the employing process, rather than identifying the relationships and assumptions to be made (the formulating process) or interpreting the answer in real world terms. W and Can d o: Student Performan C e in m athemati CS , r e – Volume Kno S What Student OECD 2014 © C eading and S C ien i 130

133 2 A Profile A nce i n mA them A tic S f Student Perform o • Figure I.2.57 • revolving door r EVOLV i NG DOO r A rev olving door includes three wings which rotate within a circular-shaped space. The inside diameter of this space is 2 metres (200 centimetres). The three door wings divide the space into three equal sectors. The plan below shows the door wings in three different positions viewed from the top. Entrance Wings 200 cm Exit The stimulus for these three questions concerns a revolving door, which is common in cold and hot countries to prevent heat moving into or out of buildings. ESTI on 1 REVOLVING DOOR – Qu What is the size in degrees of the angle formed by two door wings? Size of the angle: ...º Level 6 Scoring 669 Level 5 607 Compute the central angle of a sector of a circle d escription: Level 4 545 m Space and shape athematical content area: Level 3 482 Scientific c ontext: Level 2 Employ Process: 420 Level 1 Constructed response manual Question format: 358 Below Level 1 d ifficulty: 512.3 redit c f ull [accept the equivalent reflex angle: 240]. 120 redit c o n Other responses. Missing. Comment The first question may appear very simple: finding the angle of 120 degrees between the two door wings, but the communication, student responses indicate it is at Level 3. This is probably because of the demand arising from as well as the specific knowledge of circle geometry that is needed. The context of mathematisation and representation three-dimensional revolving doors has to be understood from the written descriptions. It also needs to be understood that the three diagrams in the initial stimulus provide different two-dimensional information about just one revolving door (not three doors) – first the diameter, then the directions in which people enter and exit from the door, and thirdly connecting the wings mentioned within the text with the lines of the diagrams. The fundamental mathematical capability OECD 2014 © i e – Volume C ien C 131 r , CS athemati m e in C o: Student Performan d and Can W Kno S What Student eading and S

134 2 o nce i n mA them A tic S A Profile f Student Perform A is required at a high level to interpret these diagrams mathematically. This question is allocated to the representation of content category because it requires knowledge that there are 360 degrees in a complete revolution, space and shape and because of the requirement for spatial understanding of the diagrams. These diagrams give the view from above, but students also need to visualise real revolving doors especially in answering Questions 2 and 3. REVOLVING DOOR – Qu on 2 ESTI Possible air ow in this position The two door openings (the dotted arcs in the diagram) are the same size. If these openings are too wide the revolving wings cannot provide a sealed space and air could then flow freely between the entrance and the exit, causing unwanted heat loss or gain. This is shown in the diagram opposite. What is the maximum arc length in centimetres (cm) that each door opening can have, so that air never flows freely between the entrance and the exit? Maximum arc length: ... cm Level 6 Scoring 669 Level 5 607 escription: Interpret a geometrical model of a real life situation to calculate the length of an arc d Level 4 545 Space and shape athematical content area: m Level 3 482 Scientific c ontext: Level 2 Formulate Process: 420 Level 1 Question format: Constructed response expert 358 Below Level 1 840.3 ifficulty: d redit c ull f 100π th calculated as 1/6 Ans wers in the range from 103 to 105. [Accept answers . Also accept an of the circumference ) ( 3 answer of 100 only if it is clear that this response resulted from using ote: Answer of 100 without supporting = 3. π n working could be obtained by a simple guess that it is the same as the radius (length of a single wing).] redit c o n Other responses. . [states the total size of the openings rather than the size of “each” opening] • 209 Missing. Comment Question 2 was one of the most challenging questions in the survey, lying towards the upper end of Level 6. It addresses the main purpose of revolving doors, which is to provide an airlock between inside and outside the building and it requires substantial geometric reasoning, which places it in the space and shape content category. The complexity of coding such a multi-step response in so many countries led to this item being assessed only as full credit or no credit. For full credit, the complex geometrical reasoning showing that the maximum door opening is one sixth of the circumference needed to be followed by an accurate calculation in centimetres. The item is classified in the formulating process, and it draws very heavily on the mathematisation fundamental mathematical capability, because the real situation has to be carefully analysed and this analysis needs to be translated into geometric terms and back again at multiple points to the contextual situation of the door. As the diagram supplied in the question shows, air will pass from the outside to the inside, or vice versa, if the wall between the front and back openings is shorter than the circumference subtended by one sector. Since the sectors each subtend one third of the circumference, and there are two walls, together the walls must close at least two thirds of the circumference, leaving no more than one third for the two openings. Arguing from symmetry of front and back, each opening cannot be more than one sixth of the circumference. There is further geometric reasoning required to check that the airlock is indeed maintained if this opening length is used. The question therefore draws very heavily on the reasoning and argument fundamental mathematical capability. m e – Volume C ien e in athemati CS , r eading and S C o: Student Performan d and Can W Kno S What Student OECD 2014 © C i 132

135 2 A Profile nce i n mA A A tic S f Student Perform o them on 3 REVOLVING DOOR – Qu ESTI The door makes 4 complete rotations in a minute. There is room for a maximum of two people in each of the three door sectors. What is the maximum number of people that can enter the building through the door in 30 minutes? A. 60 B. 180 C. 240 D. 0 72 Level 6 Scoring 669 Level 5 607 d escription: Identify information and construct an (implicit) quantitative model to solve the problem Level 4 545 athematical content area: m Quantity Level 3 482 ontext: c Scientific Level 2 Process: Formulate 420 Level 1 Simple multiple choice Question format: 358 Below Level 1 d ifficulty: 561.3 redit c ull f D . 720 c redit n o Other responses. Missing. Comment Question 3 addresses a different type of challenge, involving rates and proportional reasoning, and it sits within Level 4 on the mathematics proficiency scale. In one minute, the door revolves 4 times bringing 4 × 3 = 12 sectors to the entrance, which enables 12 × 2 = 24 people to enter the building. In 30 minutes, 12 × 30 = 720 people can enter (hence, quantity the correct answer is response option D). The question is allocated to the content category because of the way in which the multiple relevant quantities (number of people per sector [2], number of sectors per revolution [3], number of revolutions per minute [4], number of minutes [30]) have to be combined by number operations to produce the required number of persons to enter in 30 minutes. The high frequency of PISA items that involve proportional reasoning highlights its centrality to mathematical literacy, especially for students whose mathematics has reached a typical stage for 15-year-olds. Many real contexts involve direct proportion and rates, which as in this case are often used in chains of reasoning. Coordinating such a chain of reasoning requires devising a strategy to bring the information together in a logical sequence. This item also makes considerable demand on the mathematisation fundamental mathematical capability, especially in the formulating process. A student needs to understand the real situation, perhaps visualising how the doors rotate, presenting one sector at a time, making the only way for people to enter the building. This understanding of the real world problem enables the data given in the problem to be assembled in the right way. General comment on this unit The questions in this unit have been allocated to the scientific context category, even though they do not explicitly involve scientific or engineering concepts, as do many of the other items in this category. The scientific category includes items that explain why things are as they are in the real world. Question 2 is a good example of such an essentially scientific endeavour. Formal geometric proof is not required by the question, but in answering this item correctly, the highest students will have almost constructed such a proof. C ien C OECD 2014 e in athemati CS , r m eading and S C o: Student Performan d and Can W Kno S What Student e – Volume i © 133

136 2 tic A nce i n mA them A f Student Perform S A Profile o • Figure I.2.58 • Which car? which car? Chris has just received her car driving licence and wants to buy her first car. This table below shows the details of four cars she finds at a local car dealer. Dezal Castel Bolte Alpha m odel: 2000 2001 1999 y ear 2003 4 450 a dvertised price (zeds) 4 800 3 990 4 250 d istance travelled 105 000 115 000 128 000 109 000 (kilometres) 1.783 1.82 Engine capacity (litres) 1.79 1.796 WHICH CAR? on 1 ESTI – Qu of these conditions: Chris wants a car that meets all The distance travelled is not higher than 120 000 kilometres. • It was made in the year 2000 or a later year. • higher than 4 500 zeds. not The advertised price is • • Which car meets Chris’s conditions? Alpha A. Bolte B. Castel C. Dezal D. Level 6 669 Scoring Level 5 607 Level 4 d Select a value that meets four numerical conditions/statements set within a financial context escription: 545 Level 3 u ncertainty and data m athematical content area: 482 Level 2 Personal ontext: c 420 Process: Interpret Level 1 358 Question format: Simple multiple choice Below Level 1 ifficulty: 327.8 d f ull c redit B . Bolte. redit n c o Other responses. Missing. Kno , CS athemati m e in C o: Student Performan d and Can W eading and S S What Student OECD 2014 © C C e – Volume i r ien 134

137 2 A Profile A nce i n f Student Perform them A tic S o mA ESTI – Qu WHICH CAR? on 2 Which car’s engine capacity is the smallest? Alpha A. B. Bolte C. Castel D. Dezal Level 6 Scoring 669 Level 5 607 escription: Choose the smallest decimal number in a set of four, in context d Level 4 545 Quantity m athematical content area: Level 3 482 Personal ontext: c Level 2 Process: Employ 420 Level 1 Question format: Simple multiple choice 358 Below Level 1 ifficulty: 490.9 d redit c ull f D . Dezal. redit c o n Other responses. Missing. on 3 ESTI – Qu WHICH CAR? Chris will have to pay an extra 2.5% of the advertised cost of the car as taxes. How much are the extra taxes for the Alpha? Extra taxes in zeds: ... Level 6 Scoring 669 Level 5 607 d Calculate 2.5% of a value in the thousands within a financial context escription: Level 4 545 Quantity athematical content area: m Level 3 482 c ontext: Personal Level 2 Employ Process: 420 Level 1 Constructed response manual Question format: 358 Below Level 1 552.6 ifficulty: d f c redit ull 120 c redit n o Other responses. [ • 2.5% of 4 800 zeds eeds to be evaluated]. n Missing. General comment on this unit Because buying a car is a situation which many people face in their everyday life, all three questions have been allocated to the personal context category. Question 1 and Question 2 are simple multiple choice responses, and Question 3, which asks for a single number, is a constructed response item that does not require expert scoring. Question 1 has been allocated to . The item requires knowledge of the basic row-column conventions of a table, as well uncertainty and data as co-ordinated data-handling ability to identify where the three conditions are simultaneously satisfied. The solution also requires basic knowledge of large whole numbers, but the expert judgement is that this knowledge is unlikely to be the main source of difficulty in the item for 15-year-old students. The correct response is B: Bolte. athemati m e in 135 o: Student Performan d and Can W Kno S What Student , CS OECD 2014 r eading and S C ien C e – Volume i © C

138 2 o A nce i n f Student Perform them A tic S A Profile mA content category because it is well known that even at age 15, quantity In contrast, Question 2 has been allocated to the many students have misconceptions about the base ten and place value ideas required to order “ragged” decimal numbers. Credit is given here for response option D: Dezal. quantity Question 3 is also allocated to the content category because the calculation of 2.5% of the advertised cost, 120 zeds, is expected to be a much larger source of cognitive demand than identifying the correct data from the table. The difficulty for this age group in dealing with decimal numbers and percentages is reflected in the empirical results, with Question 1 being an easy item, Question 2 close to the international average and Question 3 above it. To allocate the items to process categories, it is necessary to consider how the real world situation is involved. Items in formulating category have their major demand in the transition from the real world problem to the mathematical the problem. Items in the employing category have their major demand within the mathematical world. Items in the interpreting category have their major demand in using mathematical information to give a real world solution. Questions 2 and 3 are allocated to the category. This is because in both of these items, the major source of cognitive employing demand has been identified as being within mathematics: the concept of decimal notation and the calculation of a percentage. In Question 1, a table of data is presented, and its construction (with the identification of key variables etc.) represents a mathematisation of the real situation. The question then requires these mathematical entities as presented to be interpreted in relation to the real world constraints and situation they represent. o: Student Performan i C ien C r , CS athemati m e in C e – Volume d and Can W Kno S What Student OECD 2014 © eading and S 136

139 2 o A nce i n mA them A tic S A Profile f Student Perform • • Figure I.2.59 S hart c In January, the new CDs of the bands 4U2Rock and The Kicking Kangaroos were released. In February, the CDs of the bands No One’s Darling and The Metalfolkies followed. The following graph shows the sales of the bands’ CDs from January to June. Sales of CDs per month 2 250 4U2Rock 2 000 The Kicking Kangaroos 1 750 No One’s Darling 1 500 The Metalfolkies 1 250 1 000 750 500 Number of CDs sold per month 250 0 JanFeb Mar Apr MayJun Month The three questions making up the unit C are all of below average difficulty in the main survey. All three items HARTS hoice, so the demand for communication is only receptive. The unit presents a bar chart showing are simple multiple c 6 months of sales data for music. The complication of the bar chart is that it displays four separate data series (four different music bands). Students have to read values from the graphical representation of data and draw conclusions. . All three items have all been classified in the This is a common task type in the content category uncertainty and data context category because it provides information about community behaviour, in this case, aggregated music societal choices. C – Qu on 1 ESTI HARTS The Metalfolkies How many CDs did the band sell in April? A. 250 B. 500 1 000 C. 1 270 D. Level 6 669 Scoring Level 5 607 Level 4 Read a bar chart escription: d 545 Level 3 u ncertainty and data athematical content area: m 482 Level 2 Societal c ontext: 420 Process: Interpret Level 1 358 Simple multiple choice Question format: Below Level 1 ifficulty: 347.7 d redit f ull c B . 500 redit n o c Other responses. Missing. athemati OECD 2014 © i 137 C ien C eading and S r , CS m e in C o: Student Performan d and Can W Kno S What Student e – Volume

140 2 A Profile nce i n mA them A tic S f Student Perform o A Comment Question 1, with a difficulty of 347.7, is below Level 1 on the mathematical proficiency scale, being one of the easiest tasks in the PISA 2012 item pool. It requires the student to find the bars for April, select the correct bar for the Metafolkies, and read the height of the bar to obtain the required response selection B (500). n o scale reading or interpolation is process category. required. This question is classified in the interpreting C on 2 HARTS – Qu ESTI sell more CDs than the band No One’s Darling The Kicking Kangaroos for the first In which month did the band time? No month A. B. Mar ch A pril C. D. May Scoring Level 6 669 Level 5 Read a bar chart and compare the height of two bars escription: d 607 Level 4 m ncertainty and data u athematical content area: 545 ontext: c Societal Level 3 482 Process: Interpret Level 2 420 Simple multiple choice Question format: Level 1 358 415 ifficulty: d Below Level 1 ull c redit f C. April. n o c redit Other responses. Missing. Comment Question 2 is a little more difficult, and lies near the bottom of Level 3 on the scale. The bars representing two bands need to be identified and the heights compared, starting from January and working through the year. n o reading of the vertical scale is required. It is only necessary to make visual comparisons of adjacent bars against a very simple characteristic (which is bigger), –and to identify the correct response option C (April). In comparison with Question 1, Question 2 is a little more demanding of (receptive component), representation , communication and devising strategies, and similar on the other fundamental mathematical capabilities. It is also classified in the interpreting process category. HARTS – Qu ESTI on 5 C The Kicking Kangaroos The manager of is worried because the number of their CDs that sold decreased from February to June. What is the estimate of their sales volume for July if the same negative trend continues? A. 70 CDs B. 370 CDs C. 670 CDs D. 1 340 CDs athemati e – Volume What Student Kno W and Can d o: Student Performan S C e in m OECD 2014 © C CS , r eading and S C ien i 138

141 2 o A nce i n f Student Perform them A tic S A Profile mA Scoring Level 6 669 Level 5 d escription: Interpret a bar chart and estimate the number of CDs sold in the future assuming that the linear trend continues 607 Level 4 athematical content area: ncertainty and data u m 545 c Societal ontext: Level 3 482 Process: Employ Level 2 420 Question format: Simple multiple choice Level 1 358 428.2 ifficulty: d Below Level 1 f c redit ull B . 370 CDs. o c redit n Other responses. Missing. Comment Question 5 requires identifying the data series for the Kangaroos band and observing the negative trend noted in the lead-in to the item stimulus. It involves some work with numbers and also an appreciation that the correct answer to choose may be an approximation to a calculated answer. There are several ways to continue the trend by one more month. A student might work out each monthly decrease and average them, which involves a lot of calculation. A student might take one fifth of the total decrease from February to June. Another student might place a ruler along the tops of the bars for the Kangaroos and find that the July bar would show something between 250 and 500. The correct response option is B (370 CDs), and the task lies in Level 2 on the mathematics scale. The question has been allocated to the Employing process because it was judged that most students at this level are likely to take the calculation routes, and that carrying these out accurately is likely to present the greatest difficulty for the item. r eading and S C C e – Volume o: Student Performan e in m athemati CS , d and Can W Kno S What Student ien 139 OECD 2014 © i C

142 2 S f Student Perform A nce i n mA them A tic o A Profile • Figure I.2.60 • garage A garage manufacturer’s “basic” range includes models with just one window and one door. George chooses the following model from the “basic” range. The position of the window and the door are shown here. The unit G content category because they deal with space and shape consists of two questions, both in the ARAGE spatial visualisation and reading building plans, and both in the occupational context category, because these questions may arise in the construction, painting or other completion of a building project. Because of the need to derive mathematical information from the diagrams, both questions require activation of the representation fundamental mathematical capability. ESTI on 1 – Qu GARAGE The illustrations below show different “basic” models as viewed from the back. Only one of these illustrations matches the model above chosen by George. Which model did George choose? Circle A, B, C or D. a B c D Level 6 Scoring 669 Level 5 607 d u se space ability to identify a 3D view corresponding to another given 3D view escription: Level 4 545 m Space and shape athematical content area: Level 3 c ontext: o ccupational 482 Level 2 Interpret Process: 420 Level 1 Simple multiple choice Question format: 358 Below Level 1 419.6 d ifficulty: OECD 2014 What Student S Kno W and Can d athemati © CS o: Student Performan C e in i e – Volume C ien C eading and S r , m 140

143 2 o A nce i n mA them A f Student Perform S A Profile tic redit c ull f [Graphic C]. C. redit c o n Other responses. Missing. Comment Question 1 lies very close to the Level 1/Level 2 boundary on the proficiency scale. It asks students to identify a picture of a building from the back, given the view from the front. The diagrams must be interpreted in relation to the real world positioning of “from the back”, so this question is classified in the process. The correct response is C. interpreting Mental rotation tasks such as this are solved by some people using intuitive spatial visualisation. ther people need o explicit reasoning processes. They may analyse the relative positions of multiple features (door, window, nearest corner), o thers might draw a bird’s eye view, and then physically discounting the multiple choice alternatives one by one. rotate it. This is just one example of how different students may use quite different methods to solve PISA questions: in this case explicit reasoning for some students is intuitive for others. ESTI on 2 GARAGE – Qu The two plans below show the dimensions, in metres, of the garage George chose. Front view Side view 2.50 1.00 1.00 2.40 2.40 0.50 1.00 6.00 2.00 0.50 1.00 Drawing not to scale. Note: The roof is made up of two identical rectangular sections. Calculate the total area of the roof. Show your work. ... Scoring d escription: Interpret a plan and calculate the area of a rectangle using the Pythagorean theorem or measurement Level 6 669 athematical content area: m Space and shape Level 5 607 ccupational c ontext: o Level 4 Employ Process: 545 Level 3 Constructed response expert Question format: 482 Level 2 687.3 ifficulty: d 420 Level 1 358 Below Level 1 f redit c ull An y value from 31 to 33, either showing no working at all or supported by working that shows the use of the Pythagorean 2 [ theorem (or including elements indicating that this method was used) nits (m u ) not required] . 2 12√7.25 m • 2 • 12 × 2.69 = 32.28 m 2 • 32.4 m W and Can d o: Student Performan C e in m What Student Kno OECD 2014 S athemati CS , r eading and S C ien C e – Volume i © 141

144 2 o A nce i n f Student Perform them A tic S A Profile mA c redit Partial W orking shows correct use of the Pythagorean theorem but makes a calculation error or uses incorrect length or does not double roof area. 2 2 • 2.5 [correct use of Pythagoras theorem with calculation error]. + 1 = 6, 12 × √6 = 29.39 2 2 2 • 2 + 1 [incorrect length used]. = 5, 2 x 6 x √5 = 26.8 m • 6 × 2.6 = 15.6 [Did not double roof area]. Working does not show use of Pythagorean theorem but uses reasonable value for width of roof (for example, any value from 2.6 to 3) and completes rest of calculation correctly. • 2.75 × 12 = 33 • 3 × 6 × 2 = 36 • 12 × 2.6 = 31.2 c o n redit Other responses. • 2.5 × 12 = 30 [Estimate of width of roof lies outside the acceptable range which is from 2.6 to 3]. • 3.5 × 6 × 2 = 42 [Estimate of width of roof lies outside the acceptable range which is from 2.6 to 3]. Missing. Plan for answering Garage, Question 2 Vertical projection of roof Slant Front height view of roof Horizontal projection Area of Area of of roof one side whole roof (2.5m) Length of Side one side view (6m) Comment Question 2 requires complicated calculation, with multiple calls upon the mathematical diagrams, and knowing to use Pythagoras’s theorem. For this reason, it has been classified in the employing process. There are multiple reasons why this item is at Level 5 for partial credit answers and at Level 6 for full credit answers. Question 2 requires a constructed response, although in this case the explanation of reasoning is only used to award partial credit for incorrect answers, rather than being scored for quality of explanation. There is high level demand for the capability, in representation understanding and deriving exact information from the front and side views presented. Mathematisation is also called upon, especially in reconciling the apparent 1.0 m height of the roof from the side view with the real situation and with the front view. The devising strategies capability is called up at a high level to make a plan to get the area from the information presented. The plan above shows the basic structure of the solution. To carry out such a plan also requires careful monitoring. Future analysis of the data beyond the scope of this first report may show interesting differences between the students who score partial credit. m athemati CS , r eading and S e – Volume C ien C OECD 2014 What Student S Kno W © and Can d o: Student Performan C e in i 142

145 2 f Student Perform nce i n mA them A tic S A Profile o A Notes 1. The GDP values represent per capita GDP in 2012 at current prices, adjusted for differences in purchasing power among OECD countries. 2. It should be borne in mind, however, that the number of countries involved in this comparison is small, and that the trend line is therefore strongly affected by the particular characteristics of the countries included in the comparison. 3. Spending per student is approximated by multiplying public and private expenditure on educational institutions per student in 2012 at each level of education by the theoretical duration of education at the respective level, up to the age of 15. Cumulative expenditure be the typical number of years spent by a student from the for a given country is approximated as follows: let n(0), n(1) and n(2) be the E(2) age of 6 up to the age of 15 years in primary, lower secondary and upper secondary education. Let E(0), E(1) and annual expenditure per student in USD converted using purchasing power parities in primary, lower secondary and upper secondary E education, respectively. The cumulative expenditure is then calculated by multiplying current annual expenditure by the typical duration of study n for each level of education i using the following formula: 2 i ( n CE ) i ( E * ) = i = 0 4. For this purpose, the respective data were standardised across countries and then averaged over the different aspects. 5. For more details, see Butler and Adams (2007). 6. For trend purposes, Dubai (UAE) and the rest of the United Arab Emirates are counted as separate economies. Dubai (UAE) implemented PISA 2009 in 2009 and the rest of the United Arab Emirates implemented PISA 2009 in 2010, as part of PISA 2009+. 7. As described in more detail in Annex A5, the annualised change takes into account the specific year in which the assessment was conducted. In the case of mathematics, this is especially relevant for the PISA 2009 assessment as Costa Rica, Malaysia and the United Arab Emirates (excluding Dubai) implemented the assessment in 2010 as part of PISA 2009+. 8. Normally, when comparing two concurrent means, the significance is indicated by calculating the ratio of the difference of the means to the standard error of the difference of the means. If the absolute value of this ratio is greater than 1.96, then a true difference is indicated with 95% confidence. When comparing two means taken at different times, with instruments that have a subset of common items, as in different PISA surveys, an extra error term, known as the link error, is introduced, and the resulting statement of significant difference is more conservative. For more details, see Annex A5. 9. By accounting for students’ gender, age, socio-economic status, immigrant background and language spoken at home, the adjusted trends allow for a comparison of trends in performance assuming no change in the underlying population or the effective samples’ average socio-economic status, age and percentage of girls, students with an immigrant background or students that speak a language at home that is different than the language of assessment. is unavailable for Albania in PISA 2012. Albania improved throughout its PISA index of social, economic and cultural status 10. The participation in PISA, but it is impossible to calculate adjusted trends for the country. References Bruns, B., D. Evans, and J. Luque (2011), Achieving World-Class Education in Brazil , The World Bank, Washington, D.C. R.J. Adams (2007), “The impact of differential investment of student effort on the outcomes of international studies”, Butler, J. and Journal of Applied Measurement, Vol. 3, No. 8, pp. 279-304. Journal of Applied Gebhardt, E. and R.J. Adams (2007), “The influence of equating methodology on reported trends in PISA”, Vol. 8, No. 3, pp. 305-322. Measurement, INEP , INEP, Sinopse Estatística da Educação Básica 2000 (Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira) (2000), Brasilia. INEP (Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira) (2012), Sinopse Estatística da Educaçao Básica 2012 , INEP, Brasilia. Censo da Educação Básica: 2012, Resumo (Instituto Nacional de Estudos e Pesquisas Educacionais Anísio Teixeira) (2013), INEP Técnico , INEP, Brasilia. , OECD Publishing. PISA 2012 Technical Report (forthcoming), OECD (2013a), OECD utlook 2013: First Results from the Survey of Adult Skills o ECD Skills o , OECD Publishing. http://dx.doi.org/10.1787/9789264204256-en CS OECD 2014 S Kno W and Can d o: Student Performan C e in m athemati What Student , r eading and S C ien C e – Volume i © 143

146 2 tic A nce i n mA them A f Student Perform S A Profile o PISA, OECD Publishing. The Survey of Adult Skills: Reader’s Companion, (2013b), OECD http://dx.doi.org/10.1787/9789264204027-en OECD (2013c), PISA 2012 Assessment and Analytical Framework: Mathematics, Reading, Science, Problem Solving and Financial Literacy, PISA, OECD Publishing. http://dx.doi.org/10.1787/9789264190511-en o OECD Publishing. utlook: Turkey, OECD (2013d), Education Policy u K%20T TL ou LICY%20 o %20P on CATI u http://www.oecd.org/edu/ED .pdf n RKEY_E oo (2012), OECD Learning beyond Fifteen: Ten Years after PISA , PISA, OECD Publishing. http://dx.doi.org/10.1787/9789264172104-en ECD Economic Surveys: Brazil OECD (2011), , OECD Publishing. o http://dx.doi.org/10.1787/eco_surveys-bra-2011-en OECD (2010a), Pathways to Success: How Knowledge and Skills at Age 15 Shape Future Lives in Canada , PISA, OECD Publishing. http://dx.doi.org/10.1787/9789264081925-en , Strong Performers, Successful Reformers in Education, nited States u Lessons from PISA for the OECD (2010b), OECD Publishing. http://dx.doi.org/10.1787/9789264096660-en , OECD Publishing. n ational Policies for Education: Basic Education in Turkey Reviews of (2007), OECD http://dx.doi.org/10.1787/9789264030206-en (2006), OECD Economic Survey of Turkey: 2006 , OECD Publishing. http://dx.doi.org/10.1787/eco_surveys-tur-2006-en Learning for Tomorrow’s World: First Results from PISA 2003, OECD PISA, OECD Publishing. (2004), http://dx.doi.org/10.1787/9789264006416-en s Talim ve Terbiye Kurulu (TTKB) (2008), retim Programı ve Kılavuzu (Teaching Syllabus and ğ lkögretim Matematik Dersi 6–8 Sınıflar Ö ğ Curriculum Guidebook for Elementary School Mathematics Course: Grades 6 to 8) , Milli E ğ ı, Ankara. itim Bakanlı and Turner, R., J. Dossey, W. Blum (2013), “Using mathematical competencies to predict item difficulty in PISA”, in M. Prenzel, M. Niss utcomes of the PISA Research Conference 2009, o Research on PISA: Research M. Kobarg, K. Schöps and S. Rönnebeck (eds.), Dordrecht, Springer, pp. 23-37. o: Student Performan e – Volume © OECD 2014 What Student S Kno W and Can d C C e in m athemati CS , r eading and S C ien i 144

147 3 Measuring Opportunities to Learn Mathematics This chapter examines whether and how exposure to mathematics content, known as “opportunity to learn”, is associated with student performance. The analysis is based on students’ responses to questions that appeared in the PISA Student Questionnaire on the degree to which they encountered various types of mathematics problems during their schooling, how familiar they were with certain formal mathematics content, and how frequently they had been taught to solve specific mathematics tasks involving formal or applied mathematics. OECD 2014 athemati , r eading and S C What Student C e – Volume i © CS 145 m e in C o: Student Performan d and Can W Kno S ien

148 3 Measuring Opp rtunities t O Learn Mathe M atics O Previous research has shown a relationship between students’ exposure to subject content in school, what is known as “opportunity to learn”, and student performance (e.g. Schmidt et al., 2001). Building on previous measures of opportunity to learn (Carroll, 1963; Wiley and Harnischfeger, 1974; Sykes, Schneider and Planck, 2009; Schmidt et al., 2001), the PISA 2012 assessment included questions to students on the mathematics theories, concepts and content to which they have been exposed to in school, and the amount of class time they spent studying this content. What the data tell us Asian countries and economies – Shanghai-China, Singapore, Students in the high-performing East • Hong Kong - China, Chinese Taipei, Korea, Macao-China and Japan – are more frequently exposed to formal mathematics than students in most of the other PISA-participating countries and economies. • Exposure to more ad vanced mathematics content, such as algebra and geometry, appears to be related to high performance on the PISA mathematics assessment, even if the causal nature of this relationship cannot be established. • Strong mathematics performance in PISA is not only related to opportunities to learn formal mathematics, suc h as solving a quadratic equation, using complex numbers, or calculating the volume of a box, but also to opportunities to learn applied mathematics (using mathematics in a real-world context). Six questions were created in the Student Questionnaire to cover both the content and time aspects of students’ opportunity to learn. Four of the questions focused on the degree to which students encountered various types of mathematics problems or tasks during their schooling, which all form part of the PISA mathematics framework and assessment. Some of the tasks included in those questions involved formal mathematics content, such as solving an equation or calculating the volume of a box (see Question 4 at the end of this chapter). Others involved using mathematics in a real-world applied context (see Question 6 at the end of this chapter). Another type of task required using mathematics in its own context, such as using geometric theorems to determine the height of a pyramid (see Question 5 at the end of this chapter). The last type of tasks involved formal mathematics, but situated in a word problem like those typically found in textbooks (see Question 3 at the end of this chapter) where it is obvious to students what mathematics knowledge and skills are needed to solve them. Students were asked to indicate how frequently they encountered similar tasks in their mathematics lessons using a four-point scale: never, rarely, sometimes, or frequently. In another question, students were asked how familiar they were with certain formal mathematics content, including such topics as quadratic functions, radicals and the cosine of an angle (see Question 2 at the end of this chapter). Responses to these tasks were recorded on a five-point scale indicating the degree to which students had heard of the topic. Having heard of a topic more often was assumed to reflect a greater degree of opportunity to learn. In addition, a question asked students to indicate, on a four-point scale, how frequently they had been taught to solve eight specific mathematics tasks (see Question 1 at the end of this chapter). These tasks included both formal and applied mathematics. All but the last question were used to create three indices: “formal mathematics”, “word problems”, and “applied mathematics”. Values of these indices range from 0 to 3, indicating the degree of exposure to opportunity to learn, with 0 corresponding to no exposure and 3 to frequent exposure. (For more details on how these indices are constructed, see the section in blue at the end of this chapter.). When interpreting these data, it needs to be borne in mind that the 15-year-olds assessed by PISA are, in some countries, dispersed over a range of grades and mathematical programmes and will therefore be exposed to a range of mathematical content. On average, 15-year-olds in OECD countries indicated that they encounter applied mathematics tasks and word problems “sometimes” and formal mathematics tasks somewhat less frequently (Figures I.3.1a, b, c and Table I.3.1). C d © OECD 2014 What Student S Kno W i e – Volume o: Student Performan ien C eading and S r , CS athemati m e in C and Can 146

149 3 rtunities t O Learn Mathe M atics Measuring Opp O • • Figure I.3.1a Students’ exposure to word problems Exposure to word problems Iceland Spain Liechtenstein Jordan Switzerland France Slovenia Austria Finland Poland Chile Montenegro Luxembourg Croatia Germany Hungary Slovak Republic Canada Russian Federation Denmark Thailand Peru Belgium Sweden Indonesia Colombia Albania OECD average Romania United Kingdom Kazakhstan Malaysia Norway United Arab Emirates Mexico Australia Ireland Estonia United States Italy Qatar Latvia Korea Israel Czech Republic New Zealand Tunisia Lithuania Costa Rica Argentina Japan Netherlands Singapore Bulgaria Serbia Brazil Portugal Chinese Taipei Hong Kong-China Greece Uruguay Turkey Shanghai-China Macao-China Viet Nam Index of exposure to word problems 2.0 0.5 1.0 1.5 2.5 Countries and economies are ranked in descending order of the index of exposure to word problems. Source: OECD, PISA 2012 Database, Table I.3.1. 2 1 http://dx.doi.org/10.1787/888932935591 , OECD 2014 Kno W and Can d o: Student Performan C e in m athemati CS S r eading and S C ien C What Student e – Volume i © 147

150 3 Measuring Opp rtunities t O Learn Mathe M atics O • Figure I.3.1b • Students’ exposure to formal mathematics Exposure to formal mathematics Shanghai-China Singapore Macao-China Jordan United Arab Emirates Russian Federation Albania Croatia Korea Japan Serbia Latvia Romania Estonia United States Canada Chinese Taipei Kazakhstan Bulgaria Hungary Viet Nam Slovenia Turkey Greece Montenegro Spain France Poland Hong Kong-China Belgium Italy Israel Czech Republic Peru Mexico Colombia Portugal Finland Qatar Thailand Slovak Republic OECD average Chile Australia Germany Lithuania Uruguay United Kingdom Denmark Indonesia Malaysia Liechtenstein Austria Costa Rica New Zealand Netherlands Ireland Luxembourg Brazil Switzerland Argentina Tunisia Iceland Sweden 2.0 1.5 1.0 2.5 0.5 Index of exposure to formal mathematics Countries and economies are ranked in descending order of the index of exposure to formal mathematics. OECD, PISA 2012 Database, Table I.3.1. Source: 1 http://dx.doi.org/10.1787/888932935591 2 r S and Can d o: Student Performan C e in m What Student athemati CS , W eading and S C ien C e – Volume i © OECD 2014 Kno 148

151 3 Measuring Opp rtunities t O Learn Mathe M atics O • Figure I.3.1c • Students’ exposure to applied mathematics Exposure to applied mathematics Thailand Indonesia Mexico Jordan Kazakhstan Albania Colombia Portugal Netherlands Tunisia Romania United Arab Emirates Chile Canada Peru Malaysia Brazil France Australia Poland Liechtenstein Qatar Spain United States Iceland Singapore Russian Federation Denmark Turkey Germany New Zealand Switzerland Hungary Lithuania OECD average Bulgaria Slovenia Argentina Montenegro Belgium Luxembourg Ireland United Kingdom Greece Slovak Republic Latvia Croatia Korea Serbia Austria Hong Kong-China Estonia Norway Israel Italy Chinese Taipei Japan Sweden Costa Rica Finland Uruguay Viet Nam Shanghai-China Macao-China Czech Republic Index of exposure to applied mathematics 1.5 0.5 1.0 2.5 2.0 Countries and economies are ranked in descending order of the index of exposure to applied mathematics. Source: OECD, PISA 2012 Database, Table I.3.1. http://dx.doi.org/10.1787/888932935591 2 1 o: Student Performan 149 OECD 2014 © i e – Volume and Can C ien C What Student eading and S r , CS athemati m e in W Kno S d C

152 3 O rtunities t O Learn Mathe M atics Measuring Opp tudent achievement S pportunity to learn and o To examine the overall relationship between opportunity to learn and achievement, a three-level model was fitted to the data showing that at all three levels – country, school and student – there was a statistically significant relationship between opportunity to learn and student performance. Therefore, examinations of the relationship between opportunity to learn and achievement can be made at student, school and country levels simultaneously. For applied mathematics, the relationship at all three levels is curvilinear (e.g. quadratic): on average, the more frequently students are exposed to problems involving applied mathematics, the better their mathematics performance, but only up to a point; after this point, performance declines. Figure I.3.2 graphically portrays the nature of the relationship averaged over the 65 countries, as well as over the OECD countries. • • Figure I.3.2 elationship between mathematics performance and students’ exposure to applied mathematics r OECD countries 64 partner countries and economies 510 490 470 Mean score in mathematics 450 430 Frequently Sometimes Rarely Never How frequently the applied mathematics 1.0 0.5 0.0 2.5 2.0 3.0 1.5 tasks occurred in student schooling experiences Index of exposure to applied mathematics Source: OECD, PISA 2012 Database. http://dx.doi.org/10.1787/888932935591 2 1 Among OECD countries, student performance is higher by about 40 points as the frequency of the encounters increased from “never” to “rarely”; but at a point between “rarely” and “sometimes” student performance reached a peak after which more frequent encounters with such problems had a negative relationship to performance. Fifteen-year-olds who frequently encounter applied problems scored about ten PISA score points below students who sometimes encounter such problems. For both of the other opportunity-to-learn variables, i.e. word problems and formal mathematics – the relationship is linear. Exposure to word problems is positively related to performance at both the school and student levels, but not at the country level; the relationship between exposure to formal mathematics and performance is significant at all three levels. Within each country the relationship between opportunity to learn and performance can be observed at both the school and student levels. These relationships were analysed using a two-level model. Of the 64 countries and economies that participated in PISA 2012 with available data for the index of opportunity to learn formal mathematics, all but Albania and Liechtenstein show a positive and statistically significant relationship between exposure to formal mathematics and performance at both the student and school levels (Figure I.3.3). Among the OECD countries, the average impact of the degree of exposure to algebra and geometry topics on performance is around 50 points at the student level (i.e. increase in PISA mathematics score associated with one unit increase in the index of exposure to formal mathematics). The student level impact of the degree of exposure to word problems on performance is more limited, involving 49 countries with an OECD average estimated impact of 4 points (Table I.3.2). What Student © OECD 2014 S i e – Volume C ien C eading and S r , CS athemati m e in C o: Student Performan d and Can W Kno 150

153 3 O M atics Measuring Opp Learn Mathe rtunities t O • Figure I.3.3 • c ountry-level regressions between opportunity to learn variables and mathematics performance at the student and school levels Student School pplied mathematics f Word problems a ormal mathematics f ormal mathematics W ord problems a pplied mathematics L alia ustr a L L L L ustria L L Q a OECD b elgium L L anada L Q L L c L Q Q c hile L L L Q L epublic L L L c zech r Q L Q L enmark d L L Q L L L Estonia L L Q L inland f L L f r ance Q L L L Germany L L L L L Greece L ungary h L L L L L L Q i celand Q i L Q L reland L L i srael L Q L i L L taly Q L Q L L L L Q L Japan k L L L ea or l L L L Q L uxembourg L Q m L L Q L exico L Q L etherlands n L n L L ew Zealand L Q n orway L Q m L m Poland L L L L L L Portugal Slovak epublic r L Q L L Q L L Slovenia L L L Spain L Q L L L Sweden L L L Q L Q L L Q L Switzerland L t urke L y L u nited k ingdom L Q L Q L L L nited States L L u lbania a rgentina L L L L a L razil Q L L b Partners b ulgaria Q L Q L c L Q L olombia L Q L L c osta r ica Q L L Q L L Q L roatia c ong- L hina h c L ong k Q L ndonesia L i Q L Jordan L L Q azakhstan L L k L atvia L L l iechtenstein L l ithuania L L L Q L l L acao- c hina L Q L m L alaysia L Q L m L Q ontenegro L L m Peru L L Q L Q L L Qatar L L L Q Q Q L L L omania r Q L L L L ation ussian f eder r L L Serbia Q L Shanghai- L L L L hina c L L Singapore L L Q L L hinese t L L c Q aipei L hailand L Q L L Q t L t unisia L L L L nited Q L rab Emirates a L L u ruguay L L L L Q u iet n am L L v Note: “L” and “Q” show a statistically significant relationship between the opportunity to learn variables and mathematics performance. “L” when the relationship is linear and “Q” when it is quadratic. Source: OECD, PISA 2012 Database, Table I.3.2. 2 http://dx.doi.org/10.1787/888932935591 1 , Kno W and Can d o: Student Performan C e in m athemati CS What Student r eading and S C ien C e – Volume i OECD 2014 S © 151

154 3 Learn Mathe M atics Measuring Opp O rtunities t O • • Figure I.3.4a r elationship between the index of exposure to word problems and students’ mathematics performance Iceland Iceland Sweden Sweden Poland Poland Thailand Thailand Luxembourg Luxembourg Finland Finland Lithuania Lithuania Jordan Jordan Mexico Mexico Chile Chile Malaysia Malaysia New Zealand New Zealand Latvia Latvia Japan Japan Estonia Estonia Australia Australia Slovak Republic Slovak Republic Korea Korea Canada Canada Argentina Argentina Russian Federation Russian Federation OECD average OECD average United Kingdom United Kingdom Montenegro Montenegro Switzerland Switzerland Spain Spain United States United States Ireland Ireland Denmark Denmark Romania Romania Colombia Colombia Peru Peru Albania Albania Tunisia Tunisia Chinese Taipei Chinese Taipei Costa Rica Costa Rica Qatar Qatar Portugal Portugal Italy Italy Germany Germany Slovenia Slovenia Austria Austria Czech Republic Czech Republic Kazakhstan Kazakhstan Croatia Croatia France France Hungary Hungary Israel Israel Bulgaria Bulgaria United Arab Emirates United Arab Emirates Belgium Belgium Viet Nam Viet Nam Brazil Brazil Indonesia Indonesia Turkey Turkey Liechtenstein Liechtenstein Serbia Serbia Hong Kong-China Hong Kong-China Uruguay Uruguay Greece Greece Netherlands Netherlands Singapore Singapore Macao-China Macao-China Shanghai-China Shanghai-China -4 20 16 0 12 -8 8 4 Increase in PISA mathematics score associated with a one-unit increase in the index of exposure to word problems Note: For the index of exposure to word problems the estimates come from a linear regression, positive values thus signal that greater exposure is more strongly associated with students’ mathematics performance. Countries and economies are ranked in descending order of the strength of the relationship between the index of exposure to word problems and mathematics performance. Source: OECD, PISA 2012 Database, Table I.3.2. 1 http://dx.doi.org/10.1787/888932935591 2 e in OECD 2014 What Student S Kno W and Can d o: Student Performan C © m athemati CS , r eading and S C ien C e – Volume i 152

155 3 O Learn Mathe M atics O rtunities t Measuring Opp • • Figure I.3.4b r elationship between the index of exposure to formal mathematics and students’ mathematics performance Singapore Singapore United Kingdom United Kingdom Korea Korea New Zealand New Zealand Australia Australia Finland Finland Chinese Taipei Chinese Taipei Macao-China Macao-China United States United States Spain Spain Ireland Ireland Israel Israel Latvia Latvia Canada Canada Estonia Estonia Portugal Portugal Slovak Republic Slovak Republic Shanghai-China Shanghai-China Denmark Denmark Poland Poland Japan Japan Netherlands Netherlands Switzerland Switzerland Czech Republic Czech Republic OECD average OECD average Malaysia Malaysia Russian Federation Russian Federation United Arab Emirates United Arab Emirates Hong Kong-China Hong Kong-China Lithuania Lithuania France France Uruguay Uruguay Germany Germany Croatia Croatia Chile Chile Viet Nam Viet Nam Serbia Serbia Colombia Colombia Luxembourg Luxembourg Peru Peru Hungary Hungary Austria Austria Italy Italy Thailand Thailand Greece Greece Qatar Qatar Bulgaria Bulgaria Montenegro Montenegro Belgium Belgium Mexico Mexico Brazil Brazil Jordan Jordan Turkey Turkey Costa Rica Costa Rica Liechtenstein Liechtenstein Romania Romania Argentina Argentina Iceland Iceland Kazakhstan Kazakhstan Slovenia Slovenia Indonesia Indonesia Sweden Sweden Tunisia Tunisia Albania Albania -10 60 30 50 40 90 8070 20 10 0 Increase in PISA mathematics score associated with a one-unit increase in the index of exposure to formal mathematics Note: For the index of exposure to formal mathematics the estimates come from a linear regression, positive values thus signal that greater exposure is more strongly associated with students’ mathematics performance. Countries and economies are ranked in descending order of the strength of the relationship between the index of exposure to formal mathematics and mathematics performance. OECD, PISA 2012 Database, Table I.3.2. Source: 1 http://dx.doi.org/10.1787/888932935591 2 , What Student Kno W and Can d o: Student Performan C e in m athemati CS S r eading and S C ien C i © OECD 2014 153 e – Volume

156 3 O Learn Mathe M atics O rtunities t Measuring Opp • • Figure I.3.4c r elationship between the index of exposure to applied mathematics and students’ mathematics performance Turkey Turkey Belgium Belgium Kazakhstan Kazakhstan Uruguay Uruguay Austria Austria Greece Greece Shanghai-China Shanghai-China Russian Federation Russian Federation Hungary Hungary Albania Albania Poland Poland United Arab Emirates United Arab Emirates Indonesia Indonesia Tunisia Tunisia Argentina Argentina Australia Australia Germany Germany Portugal Portugal Viet Nam Viet Nam Israel Israel Korea Korea Slovenia Slovenia Brazil Brazil Qatar Qatar Japan Japan Latvia Latvia Canada Canada United States United States Czech Republic Czech Republic Bulgaria Bulgaria United Kingdom United Kingdom Hong Kong-China Hong Kong-China Finland Finland Denmark Denmark Estonia Estonia Ireland Ireland Mexico Mexico Serbia Serbia Italy Italy Netherlands Netherlands OECD average OECD average Costa Rica Costa Rica Chile Chile Slovak Republic Slovak Republic Croatia Croatia Colombia Colombia Singapore Singapore Peru Peru Macao-China Macao-China Lithuania Lithuania Malaysia Malaysia Jordan Jordan New Zealand New Zealand Spain Spain France France Thailand Thailand Luxembourg Luxembourg Montenegro Montenegro Liechtenstein Liechtenstein Romania Romania Chinese Taipei Chinese Taipei Switzerland Switzerland Iceland Iceland Sweden Sweden -12 -8 -14 -16 2 0-2 -4-6 -10 Steepness of the u-shaped relationship between the index of exposure to applied mathematics and students’ mathematics performance in PISA For the index of exposure to applied mathematics the estimates are from a regression with a quadratic term, meaning that negative values indicate an inverted-u Note: shape relationship between the index and students’ mathematics performance. Lower negative numbers point to steeper inverted u-shaped relationships. and economies are ranked in descending order of the strength of the relationship between the index of exposure to applied mathematics and Countries mathematics performance. OECD, PISA 2012 Database, Table I.3.2. Source: http://dx.doi.org/10.1787/888932935591 2 1 C CS , r eading and S Kno ien C i S What Student OECD 2014 © W and Can d o: Student Performan C e in m athemati e – Volume 154

157 3 rtunities t O Learn Mathe M atics O Measuring Opp It is noteworthy that in the high-performing East Asian countries and economies on the PISA assessment – Shanghai- China, Singapore, Hong Kong-China, Chinese Taipei, Korea, Macao-China and Japan – the exposure to formal mathematics is significantly stronger than in the remaining PISA participating countries and economies (2.1 versus 1.7). The exposure to word problems shows the opposite pattern. In this case the exposure to word problems is less strong in the high-performing East Asian countries and economies than in the other countries (1.4 versus 1.8). For the index of exposure to applied mathematics, the difference between high-performing East Asian participants and other countries and economies is about 0.2 points (1.8 versus 2.0) (Table I.3.1). The results suggest that opportunities to learn formal mathematics are related to PISA performance. Furthermore, exposure to more advanced mathematics content, such as algebra and geometry, appears to be related to high performance on the PISA mathematics assessment, even if the causal nature of this relationship cannot be established. At the student level, the estimated effect of a greater degree of familiarity with such content on performance is almost 50 points (Figure I.3.4b and Table I.3.2). The results could indicate that students exposed to advanced mathematics content are also good at applying that content to PISA tasks. Alternatively, the results could indicate that high-performing students attend mathematics classes that offer more advanced mathematics content. Exposure to word problems, which are usually designed by textbook writers as applications of mathematics, are also related to performance, but not as strongly (Figure I.3.4a and Table I.3.2). In 47 of the 65 participating countries and economies, the opportunity-to-learn variable measuring the frequency of student encounters with applied mathematics tasks was related to PISA performance at either the student or school level 1 or both (Figures I.3.3 and I.3.5). Again, the causal nature of the relationship cannot be established. In some countries the relationship is likely to be the result of low-performing students attending programmes and tracks that offer more applied mathematics content. • Figure I.3.5 • Significance of exposure to applied mathematics Where exposure is related to performance, at the school and student levels School Significant Not significant Brazil, Croatia, France, Japan, Jordan, Lithuania, Bulgaria, Canada, Chile, Colombia, Costa Rica, Significant Luxembourg, Macao-China, Malaysia, Denmark, Finland, Iceland, Ireland, Italy, Mexico, Peru, Qatar, Romania, Shanghai - China, Montenegro, New Zealand, Netherlands, Slovak Republic, Switzerland, Thailand, Norway, Serbia, Singapore, Spain, Sweden, United Kingdom, Uruguay Chinese Taipei, Turkey Student Austria, Estonia, Indonesia, Israel, Kazakhstan, Not significant Albania, Argentina, Australia, Belgium, Czech Republic, Germany, Greece, Tunisia, United Arab Emirates - China, Hungary, Korea, Hong Kong Latvia, Liechtenstein, Poland, Portugal, Russian Federation, Slovenia, United States, Viet Nam OECD, PISA 2012 Database, Table I.3.2. Source: In all 40 countries and economies showing a relationship between applied mathematics and performance at the student level, except Uruguay, Turkey and Shanghai-China, the relationship is curvilinear. This means that the positive relationship between applied mathematics and performance at the student level holds until a certain point, and then it becomes negative. The average of the top-achieving East Asian countries on the applied mathematics index (1.76) falls between “rarely” and “sometimes” on the index. As shown in Figure I.3.2, the average is just at the inflection point as the curve begins its downward slope. The other 58 countries’/economies’ mean places them further down the curve where the decline in performance is greater (Table I.3.1). In 20 of them, namely Uruguay, the United Kingdom, Finland, the Slovak Republic, Thailand, Canada, Ireland, Bulgaria, Chile, Denmark, Peru, Costa Rica, Switzerland, Iceland, Qatar, Colombia, Mexico, Romania, Italy and Shanghai-China there is a relationship between applied mathematics and performance at both the school and student levels (Figure I.3.5). CS S Kno W and Can d o: Student Performan C e in m athemati What Student , r eading and S C ien C e – Volume i 155 OECD 2014 ©

158 3 rtunities t O Learn Mathe M atics Measuring Opp O Educators and education policy makers tend to agree that the capacity of students to apply mathematical content is central to their success later in life, because modern economies tend to pay people not for what they know but for what they can do with what they know. They often debate the extent to which mathematics that is related to real-world problems should be incorporated into school curricula. Some argue that students learn advanced mathematics content best when studying it in an applied context; others contend that contextual material could detract from the content and therefore exposure to advanced mathematics content with as little contextual material as possible will be most effective in helping students learn and apply the content. PISA results on the opportunity-to-learn measure do not answer the question directly, but they suggest that it is a matter of balance. It appears that strong mathematics performance in PISA is not only related to opportunities to learn formal mathematics, but also to opportunities to learn applied mathematics. Learning formal mathematics is necessary, but not sufficient by itself. Even with a higher level of opportunities related to formal mathematics, a degree of exposure to applied mathematics problems is, up to some point, positively related to performance. to learn d ifference S in opportunitie S Decisions on curriculum content, whether taken at the national, regional, local or school level, have direct consequences on students’ academic achievement (Schmidt et al., 2001 and Sykes, Schneider and Plank, 2009). As an integral feature of curricula, opportunities to learn thus fall under the purview of education policy. Given the significant relationship between opportunities to learn and performance, as described above, policy makers can learn through PISA how their decisions about curricula are ultimately reflected in student performance. Students were asked about the frequency with which they had encountered six types of fairly common real-world mathematics problems during their time at school (see Question 1 at the end of this chapter). The average proportion of students across OECD countries who answered “frequently” ranged from 11.2% (calculating the power consumption of an electric appliance per week, Figure I.3.6 and Table I.3.10) to 25.4% (calculating how many square metres of tiles were needed to cover a floor, Figure I.3.7 and Table I.3.5). The average proportion of 15-year-olds who rarely or never were taught to do these kinds of tasks ranged from 35.9% to 57.2%. Countries varied widely on these measures, though some of this variation may be due to differences in what students in different countries and contexts consider to be frequent. For example, in some countries and economies, namely Hong Kong-China, the Czech Republic, Macao-China and Viet Nam, fewer than 10% of students say they frequently encounter an applied problem like one that requires them to calculate the taxes imposed when purchasing a computer. In Viet Nam, only 3.6% of 15-year-olds say they are frequently exposed to such a problem. By contrast, 60% to 61% of students in OECD and partner countries and economies say they frequently encounter formal mathematics tasks like the two items that involved solving quadratic equations (Tables I.3.7 and I.3.9); and there was much less variation between countries. PISA also categorised mathematics problems into four types – formal mathematics (Figure I.3.8), word problems (Figure I.3.9), applied problems in mathematics (Figure I.3.10), and real-world problems (Figure I.3.11) – in order to more finely distinguish between formal and applied mathematics. PISA found that an average of 68.4% of students in OECD countries said they frequently encounter formal mathematics tasks (e.g. 2x + 3 = 7, and finding the volume of a box) in their mathematics lessons. This proportion varies from a high of 85.4% in Iceland to a low of 49.0% in Portugal (Figure I.3.8 and Table I.3.11). Among partner countries and economies, the proportion of students who are frequently exposed to these types of tasks ranges from 78.4% in Croatia to 43.2% in Brazil. By contrast, only around 6.5% of students in OECD countries rarely or never encounter this type of problem. A second category of mathematics problem includes formal mathematics concepts placed in a word problem of the kind often found in textbooks. These types of word problems do have an “applied” component, but they are often perceived by students as contrived real-world problems. Students can often recognise such word problems as requiring the same computations that they are being asked to perform in the lesson, but with verbiage surrounding the computation. The examples given included purchasing furniture with a discount, and finding the age of someone, given his/her relationship to the age of others. e – Volume o: Student Performan © OECD 2014 What Student S Kno W and Can i C C ien C eading and S r , CS athemati m e in d 156

159 3 rtunities t O Learn Mathe M atics Measuring Opp O • Figure I.3.6 • p ercentage of students who reported having seen applied mathematics problems like “calculating the power consumption of an electric appliance per week” fr equently or sometimes Sometimes Frequently Jordan Kazakhstan Albania Qatar Singapore Peru Thailand United Arab Emirates Romania Mexico Russian Federation Indonesia Brazil Colombia Bulgaria Chile Turkey Greece Shanghai-China Tunisia Iceland Slovak Republic Portugal Korea Hungary Argentina Poland Spain Lithuania Netherlands Israel Viet Nam Denmark Malaysia Chinese Taipei Latvia Montenegro Costa Rica Slovenia United Kingdom Canada Croatia Sweden OECD average Finland Ireland Macao-China Uruguay France Japan United States Austria Estonia Germany Serbia New Zealand Luxembourg Norway Czech Republic Switzerland Australia Belgium Hong Kong-China Italy Liechtenstein 10 0 60 9080 70 50 30 20 40 100 Percentage of students who reported having seen the content frequently or sometimes Countries and economies are ranked in descending order of the percentage of students who reported having seen applied mathematics problems, for instance calculating the power consumption of an electronic appliance per week frequently (see Question 1 at the end of this chapter). OECD, PISA 2012 Database, Table I.3.10. Source: 2 http://dx.doi.org/10.1787/888932936427 1 Kno What Student r S W and Can d o: Student Performan C e in m athemati CS 157 OECD 2014 © i e – Volume C ien C eading and S ,

160 3 rtunities t O Learn Mathe M atics Measuring Opp O • Figure I.3.7 • p ercentage of students who reported having seen applied mathematics problems like “calculating how many squar e metres of tiles you need to cover a floor” frequently or sometimes Sometimes Frequently Poland Germany Netherlands Iceland Austria Liechtenstein Kazakhstan Korea Switzerland Slovenia Slovak Republic Albania Lithuania Romania Spain Hungary Ireland Jordan Estonia Mexico Montenegro OECD average Sweden France Denmark Thailand Russian Federation United Arab Emirates Finland United States Indonesia Belgium Peru Canada Qatar United Kingdom Croatia Chile Norway Argentina Luxembourg Bulgaria Colombia New Zealand Turkey Malaysia Latvia Australia Shanghai-China Italy Serbia Brazil Tunisia Czech Republic Greece Chinese Taipei Singapore Israel Japan Macao-China Uruguay Costa Rica Portugal Viet Nam Hong Kong-China 10 0 60 9080 70 50 30 20 40 100 Percentage of students who reported having seen the content frequently or sometimes Countries and economies are ranked in descending order of the percentage of students who reported having seen applied mathematics problems, for instance calculating how many square metres of tiles you need to cover a oor, frequently (see Question 1 at the end of this chapter). OECD, PISA 2012 Database, Table I.3.5. Source: http://dx.doi.org/10.1787/888932936427 2 1 e in m © OECD 2014 What Student S Kno W and Can d o: Student Performan i e – Volume C ien C eading and S r , CS athemati C 158

161 3 rtunities t O Learn Mathe M atics Measuring Opp O Figure I.3.8 • • p ercentage of students who reported having seen formal mathematics problems equently or sometimes in their mathematics lessons fr Sometimes Frequently Iceland Japan Denmark Croatia Finland Liechtenstein Germany Ireland Switzerland Slovak Republic Canada Russian Federation Jordan United Kingdom Spain Hungary United States France Austria Estonia Slovenia Chile United Arab Emirates Luxembourg OECD average Czech Republic Romania Montenegro Netherlands Australia Latvia Thailand Poland Indonesia Lithuania Kazakhstan Greece Albania Malaysia Belgium Italy Bulgaria Sweden Turkey Mexico Korea Norway Colombia Singapore Peru Costa Rica New Zealand Israel Serbia Tunisia Viet Nam Qatar Hong Kong-China Portugal Uruguay Argentina Macao-China Chinese Taipei Shanghai-China Brazil 9080 10 20 100 40 60 0 70 50 30 Percentage of students who reported having seen the content frequently or sometimes Countries and economies are ranked in descending order of the percentage of students who reported having seen formal mathematics problems, for instance solving an equation or nding the volume of a box, frequently (see Question 4 at the end of this chapter). OECD, PISA 2012 Database, Table I.3.11. Source: 1 http://dx.doi.org/10.1787/888932936427 2 CS e – Volume S Kno W and Can d o: Student Performan C e in m athemati What Student , r eading and S C ien 159 OECD 2014 © i C

162 3 rtunities t O Learn Mathe M atics Measuring Opp O • Figure I.3.9 • p ercentage of students who reported having seen word problems in their mathematics lessons fr equently or sometimes Sometimes Frequently Iceland Jordan Spain Liechtenstein Switzerland France Slovenia Austria Montenegro Finland Chile Poland Luxembourg Croatia Hungary Germany Slovak Republic Canada Russian Federation Peru Belgium Thailand Denmark Sweden Albania Indonesia Colombia Romania OECD average Kazakhstan United Kingdom Malaysia United Arab Emirates Mexico Norway Australia Ireland Estonia Italy Qatar United States Latvia Korea Israel Tunisia Czech Republic New Zealand Costa Rica Argentina Lithuania Japan Netherlands Serbia Singapore Bulgaria Brazil Portugal Chinese Taipei Turkey Greece Uruguay Shanghai-China Hong Kong-China Viet Nam Macao-China 10 0 60 9080 70 50 30 20 40 100 Percentage of students who reported having seen the content frequently or sometimes Countries and economies are ranked in descending order of the percentage of students who reported having seen word problems in their mathematics lessons frequently (see Question 3 at the end of this chapter). OECD, PISA 2012 Database, Table I.3.12. Source: http://dx.doi.org/10.1787/888932936427 2 1 e in m © OECD 2014 What Student S Kno W and Can d o: Student Performan i e – Volume C ien C eading and S r , CS athemati C 160

163 3 rtunities t O Learn Mathe M atics Measuring Opp O Some 44.5% of 15-year-olds in OECD countries say they frequently encounter this type of word problem in their mathematics lessons (Figure I.3.9 and Table I.3.12), while an average of 12.7% of students rarely or never encounter such word problems. In France, Spain, Switzerland, Iceland and Slovenia, and in the partner countries Jordan and Liechtenstein, around 60% of students are exposed to these types of word problems frequently. In the PISA categorisation of mathematics, two types of applied contexts were studied: mathematics as a context in itself (applied problems in mathematics), and real-world contexts. Across OECD countries, the proportion of students who frequently encounter these two types of problems in their lessons is significantly smaller than the proportion of those who frequently encounter formal mathematics problems and word problems. Applied problems in mathematics require the use of mathematics theorems, such as finding the height of a pyramid or determining prime numbers – tasks with a primarily mathematical context but that also have more practical applications. Some 34% of 15-year-old students in OECD countries say they encounter these problems during their mathematics lessons, but nearly one in four students say they rarely or never encounter these types of problems (Figure I.3.10 and Table I.3.13). Among OECD countries, only Turkey shows that just over half of its students frequently encounter these types of problems during their lessons. By contrast, in Israel, nearly one in five students never encounters these types of problems in mathematics class. An average of 21.2% of students in OECD countries say they frequently encounter mathematics problems that are set in a real-world context; and about 33.6% of students encounters such problems rarely or never in class (Figure I.3.11 and Table I.3.14). In Mexico, Portugal, Iceland, Chile, Canada, the Netherlands, and in the partner countries and economies Thailand, Jordan, Indonesia, Tunisia, the United Arab Emirates and Colombia, at least 30% of students frequently encounter these kinds of problems in class. When looking across the four types of problems in the typology, two observations can be made. First, the typology represents a rough continuum in the percentage of students who are frequently exposed to each type of problem, declining steadily from formal mathematics (68%) to mathematically-oriented word problems (45%) to applied problems in mathematics (34%) to real-world applied problems (21%) (Figure I.3.12). At the other end of this distribution, the percentage of students who indicated that they never or rarely have such lessons increased over the same continuum from 7% to 13% to 24% to 34%. Second, the opportunities to learn the different types of mathematics problems varied greatly among countries – and even more so within countries. To measure students’ familiarity with mathematics content, PISA 2012 asked students how often they had heard of 13 mathematics topics. Tables I.3.15 to I.3.27 show the proportion of students in a country who indicated they had never heard of a particular topic, heard of it once or twice, heard of it a few times, heard of it often, or knew it well. The variation in responses, both across the mathematics topics and across countries, is striking. Considered along with other PISA opportunity-to-learn measures, such as encounters with particular types of problems, these results suggest a wide variation in opportunity to learn – one that is similar to that found in other international mathematics studies, such as the Trends in International Mathematics and Science Study (Mullis et al., 2012). Assuming familiarity with mathematics topics is related to exposure and by extension to opportunity, the average country results for the 13 topics can be divided into three categories reflecting varying degrees of exposure: the topics with low, medium and high exposure. Fewer than 40% of students say they “heard often” or “know well” the mathematics topics in the category “low exposure” and more than 60% in the category “high exposure” do (Table I.3.28). There were clear differences in opportunity to learn different mathematics content. On average, students identified topics such as linear equations (Figure I.3.13), radicals and polygons as those that they had heard of often and knew well; other topics, such as complex numbers (Figure I.3.14) and exponential functions, which are typically taught in later grades, were much less well known among 15-year-olds (Figure I.3.15). Only 42% of students in OECD countries reported that they know linear equations well, but when the category “heard of it often” was included, almost two out of three (64.4%) 15-year-olds say they have heard of them. However, this varies considerably across countries. In Iceland, only 17.8% of 15-year-old students say they either know linear equations well or have often heard about them. By contrast, at least 90% of students in Japan, Korea and Estonia, and in the partner countries and economies Croatia, Macao-China and the Russian Federation have frequent opportunities to learn about linear equations. In the partner country Viet Nam, less than 10% of students have a similar exposure to linear equations – the core topic of an elementary algebra course. OECD 2014 CS S Kno W and Can d o: Student Performan C e in m 161 What Student © i e – Volume C ien C eading and S r , athemati

164 3 rtunities t O Learn Mathe M atics Measuring Opp O • Figure I.3.10 • p ercentage of students who reported having seen applied problems in mathematics in their mathematics lessons fr equently or sometimes Sometimes Frequently Kazakhstan Indonesia Romania Albania Liechtenstein Thailand Turkey Tunisia Japan Poland Jordan Bulgaria Russian Federation Malaysia Switzerland Singapore Montenegro Greece France Germany Mexico Colombia United Arab Emirates Portugal Peru Netherlands Lithuania Austria Croatia Qatar Belgium Estonia Korea Spain Hungary Serbia Slovak Republic Latvia Chile Canada OECD average Luxembourg Brazil Slovenia Hong Kong-China Italy United States Australia Viet Nam Israel Macao-China Chinese Taipei Iceland Shanghai-China Ireland Argentina Uruguay Denmark United Kingdom New Zealand Czech Republic Costa Rica Finland Norway Sweden 10 0 60 9080 70 50 30 20 40 100 Percentage of students who reported having seen the content frequently or sometimes Countries and economies are ranked in descending order of the percentage of students who reported having seen applied problems in mathematics, for instance geometrical theorems or prime numbers, frequently (see Question 5 at the end of this chapter). OECD, PISA 2012 Database, Table I.3.13. Source: http://dx.doi.org/10.1787/888932936427 2 1 e in m © OECD 2014 What Student S Kno W and Can d o: Student Performan i e – Volume C ien C eading and S r , CS athemati C 162

165 3 rtunities t O Learn Mathe M atics Measuring Opp O • Figure I.3.11 • ercentage of students who reported having seen real-world problems p in their mathematics lessons fr equently or sometimes Frequently Sometimes Indonesia Thailand Jordan Colombia Mexico Iceland Netherlands Tunisia Canada Portugal Chile United Arab Emirates Brazil Kazakhstan Albania Israel United States Peru Australia Qatar Ireland Argentina Denmark Russian Federation New Zealand France Spain Malaysia Romania OECD average United Kingdom Belgium Hungary Shanghai-China Turkey Sweden Luxembourg Costa Rica Norway Singapore Germany Montenegro Greece Bulgaria Lithuania Poland Slovenia Italy Uruguay Switzerland Slovak Republic Latvia Serbia Chinese Taipei Liechtenstein Austria Croatia Finland Korea Estonia Viet Nam Hong Kong-China Macao-China Czech Republic Japan 10 60 9080 70 50 30 20 40 0 100 Percentage of students who reported having seen the content frequently or sometimes Countries and economies are ranked in descending order of the percentage of students who reported having seen real-world problems frequently (see Question 6 at the end of this chapter). Source: OECD, PISA 2012 Database, Table I.3.14. 1 http://dx.doi.org/10.1787/888932936427 2 d What Student o: Student Performan C e in m athemati CS r S Kno W and Can 163 OECD 2014 © i e – Volume C ien C eading and S ,

166 3 rtunities t O Learn Mathe M atics Measuring Opp O • Figure I.3.12 • Student exposure to mathematics problems Percentage of students who reported having seen the four types of mathematics problems frequently or sometimes, o ECD average % 80 70 60 50 40 30 20 10 0 Word problems Real-world applied problems Applied problems Formal mathematics in mathematics in mathematics OECD, PISA 2012 Database, Tables I.3.11, I.3.12, I.3.13 and I.3.14. Source: http://dx.doi.org/10.1787/888932936427 1 2 There is also a substantial variation of the familiarity with mathematics topics within some countries, suggesting considerable variability in the implemented curriculum. The point can be illustrated with the algebra topic of quadratic function. For example, in the United Kingdom the distribution of how often students had heard of the topic was almost even across the five response categories (never heard of it, heard of it once or twice, heard of it a few times, heard of it often, or knew it well), with around one in five students self-reporting to fall into each of these categories. A similar type of distribution can be found in Poland, Greece, Colombia and Mexico. For other countries, there is a higher degree of consistency in student reports about their familiarity with mathematics topics. In Shanghai-China, 81% knew the topic well while fewer than 2% had never heard of it. Conversely, in Sweden, 63% of 15-year-old students had never heard of it while fewer than 5% knew it well (Figure I.3.16). OECD countries also show considerable variation on the opportunity-to-learn indices (Figures I.3.1a, b, c and Table I.3.1). The OECD countries Portugal and Mexico had a mean of 2.2 on the applied mathematics index, which implied that, on average, 15-year-old students are sometimes to frequently exposed to these types of problems, while the mean for the Czech Republic was 1.6, between “sometimes” and “rarely”. This is a relatively large difference between these countries, given the limited range of the scale. Even larger differences are observed among partner countries and economies: Thailand had a mean of 2.4, indicating that the country’s 15-year-olds are between “sometimes” and “frequently” exposed to these types of mathematics problems, while Macao-China shows a mean similar to that of the Czech Republic. Variations on the formal mathematics index are even larger, with Shanghai-China having a mean of 2.3 (students in these countries encounter such tasks in mathematics class “sometimes” to “frequently”) while Sweden shows a mean of 0.8 (meaning students there almost never encounter such problems in their mathematics class). Using the formal and applied mathematics scales, countries can be categorised into four different groups (Figure I.3.17). The horizontal axis represents the OECD average frequency with which the country’s 15-year-olds have the opportunity to learn formal mathematics, while the vertical axis represents the OECD average frequency of the opportunity to learn applied mathematics. The upper right quadrant shows the countries whose students indicated that, on average, they have more opportunities to learn both applied and formal mathematics. Of the 19 countries in this group, eight of them are OECD countries. Six OECD countries (the United Kingdom, Ireland, Luxembourg, Norway, Sweden and Austria) and three partner countries (Uruguay, Costa Rica and Argentina) are included in the group shown in the lower left quadrant, which includes countries whose students have fewer opportunities to learn both formal and applied mathematics. In partner countries and economies such as Shanghai-China and Macao-China, students reported more opportunities to learn formal mathematics, on average, but fewer opportunities to learn applied mathematics. e – Volume o: Student Performan © OECD 2014 What Student S Kno W and Can i C C ien C eading and S r , CS athemati m e in d 164

167 3 rtunities t O Learn Mathe M atics Measuring Opp O • Figure I.3.13 • p ercentage of students who reported having seen linear equations often or knowing the concept well and understanding it Know it well, understand the concept Heard of it often Macao-China Croatia Russian Federation Japan Korea Slovenia Serbia Estonia Germany Singapore Jordan Montenegro Czech Republic Bulgaria Slovak Republic United States Canada United Arab Emirates Israel Hungary Romania Austria Liechtenstein Chile Latvia Kazakhstan Australia France Qatar Albania Netherlands OECD average Spain Denmark Ireland Italy New Zealand United Kingdom Malaysia Peru Lithuania Thailand Finland Switzerland Mexico Hong Kong-China Colombia Luxembourg Uruguay Turkey Portugal Chinese Taipei Argentina Costa Rica Greece Belgium Indonesia Poland Brazil Tunisia Sweden Iceland Viet Nam 10 0 60 9080 70 50 30 20 40 100 Percentage of students who reported having heard the concept often or knowing well and understanding it Countries and economies are ranked in descending order of the percentage of students who reported knowing the linear equations concept well and understanding it (see Question 2 at the end of this chapter). OECD, PISA 2012 Database, Table I.3.18. Source: 2 http://dx.doi.org/10.1787/888932936427 1 Kno What Student , S W and Can d o: Student Performan C e in m athemati 165 OECD 2014 © i e – Volume C ien C eading and S r CS

168 3 rtunities t O Learn Mathe M atics Measuring Opp O • Figure I.3.14 • p ercentage of students who reported having seen complex numbers often or knowing the concept well and understanding it Know it well, understand the concept Heard of it often Korea Jordan Shanghai-China United Arab Emirates Tunisia Kazakhstan Chinese Taipei Qatar Macao-China Hong Kong-China Montenegro United States Croatia Singapore Peru Israel Argentina Romania Spain Serbia Thailand Canada Italy Chile Austria Luxembourg Hungary Colombia United Kingdom Turkey Costa Rica Portugal Australia Viet Nam Liechtenstein France OECD average Germany Albania Malaysia Bulgaria Mexico Ireland Switzerland Uruguay Russian Federation Belgium Greece Brazil Slovenia New Zealand Latvia Estonia Denmark Indonesia Slovak Republic Poland Sweden Japan Czech Republic Netherlands Lithuania Iceland Finland 10 0 60 9080 70 50 30 20 40 100 Percentage of students who reported having heard the concept often or knowing well and understanding it Countries and economies are ranked in descending order of the percentage of students who reported knowing the complex numbers concept well and understanding it (see Question 2 at the end of this chapter). OECD, PISA 2012 Database, Table I.3.20. Source: http://dx.doi.org/10.1787/888932936427 2 1 C e in © OECD 2014 What Student S Kno W and Can d i e – Volume C ien C eading and S r , CS athemati m o: Student Performan 166

169 3 rtunities t O Learn Mathe M atics Measuring Opp O • Figure I.3.15 • p ercentage of students who reported having seen exponential functions often or knowing the concept well and understanding it Know it well, understand the concept Heard of it often Shanghai-China United Arab Emirates Jordan Macao-China Hong Kong-China Singapore Albania Canada United States Qatar Spain Germany Chinese Taipei Viet Nam Netherlands Liechtenstein Poland Montenegro Croatia Iceland Uruguay Colombia Peru Brazil Italy Australia Austria Slovenia New Zealand Bulgaria OECD average Costa Rica Mexico Israel Chile Switzerland Portugal Argentina Romania Greece Luxembourg Finland Belgium Japan Korea Denmark Kazakhstan Malaysia Latvia Tunisia Serbia Slovak Republic Turkey France Ireland Lithuania Indonesia Sweden Czech Republic United Kingdom Thailand Hungary Estonia Russian Federation 10 0 60 9080 70 50 30 20 40 100 Percentage of students who reported having heard the concept often or knowing well and understanding it Countries and economies are ranked in descending order of the percentage of students who reported knowing the exponential functions concept well and understanding it (see Question 2 at the end of this chapter). OECD, PISA 2012 Database, Table I.3.15. Source: 2 http://dx.doi.org/10.1787/888932936427 1 Kno What Student , S W and Can d o: Student Performan C e in m athemati 167 OECD 2014 © i e – Volume C ien C eading and S r CS

170 3 rtunities t O Learn Mathe M atics Measuring Opp O Figure I.3.16 • • ercentage of students who reported having seen quadratic functions often p or knowing the concept well and understanding it Know it well, understand the concept Heard of it often Shanghai-China Russian Federation Israel Japan Jordan Estonia Singapore United Arab Emirates Viet Nam Macao-China Korea Hungary Latvia Chinese Taipei Croatia Germany Romania Lithuania Belgium Montenegro Spain Slovenia France Serbia Netherlands Austria Qatar Denmark Bulgaria United States Turkey Malaysia Portugal Ireland OECD average Liechtenstein Kazakhstan Canada Switzerland Hong Kong-China Czech Republic Italy Luxembourg Poland Australia Finland Uruguay Indonesia United Kingdom Slovak Republic Colombia Peru Greece Thailand Argentina Costa Rica New Zealand Mexico Brazil Tunisia Albania Iceland Chile Sweden 50 100 40 60 9080 70 0 30 20 10 Percentage of students who reported having heard the concept often or knowing well and understanding it Countries and economies are ranked in descending order of the percentage of students who reported knowing the quadratic functions concept well and understanding it (see Question 2 at the end of this chapter). OECD, PISA 2012 Database, Table I.3.17. Source: 1 http://dx.doi.org/10.1787/888932936427 2 o: Student Performan , © OECD 2014 What Student S Kno W and Can d r C e in m athemati i e – Volume C ien C eading and S CS 168

171 3 O O Learn Mathe M atics Measuring Opp rtunities t • Figure I.3.17 • e xposure to applied mathematics vs. exposure to formal mathematics 2.5 2.4 Thailand Indonesia 2.3 Mexico Jordan Kazakhstan 2.2 Albania Colombia Tunisia Index of exposure to applied mathematics Romania Portugal Netherlands Australia 2.1 United Germany Arab Emirates Chile Peru Canada France Malaysia Brazil Iceland United States 2.0 Spain Poland Singapore Liechtenstein Denmark Turkey Qatar Russian Federation Switzerland Hungary Lithuania New Zealand OECD average 1.9 Slovenia Bulgaria Montenegro Argentina Ireland Latvia Belgium United Kingdom Greece Croatia Luxembourg Hong Kong-China 1.8 Korea Austria Serbia Israel Slovak Republic Italy Sweden Estonia Chinese Costa Rica Japan 1.7 Taipei Finland Uruguay Shanghai- China Viet Nam 1.6 Macao-China Czech Republic 1.5 OECD average 2.3 2.4 1.5 1.7 1.6 1.8 1.4 1.3 1.2 1.1 1.0 0.9 0.8 1.9 2.0 2.1 2.2 0.7 Index of exposure to formal mathematics OECD, PISA 2012 Database, Tables I.3.1. Source: 1 2 http://dx.doi.org/10.1787/888932936427 169 CS athemati m e in C o: Student Performan OECD 2014 e – Volume d i C and Can ien W © Kno S What Student C eading and S r ,

172 3 Learn Mathe M atics O Measuring Opp O rtunities t u S tion S ed for the con Que S truction of the three opportunity to learn indice S S Six questions were used from the Student Questionnaire to cover both the content and the time aspects of the opportunity to learn. These questions are shown below. Question 1 h ow often have you encountered the following types of mathematics tasks during your time at school? (Please tick only one bo x on each row.) Never Sometimes Frequently Rarely a) Working out from a how long n 1 n 2 3 n 4 n it would take to get from one place to another. b) Calculating how much more expensive n 3 n 4 2 n 1 n a computer would be after adding tax. c) Calculating how many square metres of tiles n n 2 3 1 n n 4 you need to cover a floor. d) Understanding scientific tables presented n n 2 n 3 n 4 1 in an article. 2 e) + 5 = 29 Solving an equation like: 6x 4 n 3 1 n n 2 n f) Finding the actual distance between two places n 1 n 4 n 2 n 3 on a map with a 1:10,000 scale. g) Solving an equation like 2(x+3) = (x + 3)(x - 3) n 2 4 3 1 n n n h) Calculating the power consumption n 1 n 4 n 3 n 2 of an electronic appliance per week. i) Solving an equation like: 3x+5=17 4 n 1 n 2 n 3 n Question 2 t hinking about mathematical concepts: ho w familiar are you with the following terms? (Please tick only one box in each row.) Know it well, Heard of it Never Heard of it Heard of it understand heard of it often once or twice a few times the concept a) Exponential Function n 1 n 2 n 3 n 4 n 5 b) Divisor n 5 n 1 4 n n 2 n 3 c) Quadratic Function 1 n 5 4 n 3 n 2 n n d) Linear Equation 2 n 1 n 4 n 3 5 n n e) Vectors 3 n 1 n n 2 n 4 n 5 f) Complex Number 2 n 1 n n 5 n 4 n 3 g) Rational Number 1 5 n 4 n 3 n 2 n n h) Radicals 5 n 4 n 3 n n 1 n 2 i) Polygon 1 n 5 4 n 3 n 2 n n j) Congruent Figure n 5 1 n 2 4 n 3 n n k) Cosine 3 n 2 n 1 n 5 n 4 n l) Arithmetic Mean 1 n 2 n 3 n n 4 n 5 m) Probability 3 1 n 2 n 5 n n 4 n The next four questions are about students’ experience with different kinds of mathematics problems at school. They include some descriptions of problems and dark blue-coloured boxes, each containing a mathematics problem. The students had to read each problem but did not have to solve it. o: Student Performan e – Volume © OECD 2014 What Student S Kno W and Can d C C e in m athemati CS , r eading and S C ien i 170

173 3 O Learn Mathe M atics Measuring Opp O rtunities t Question 3 In the box is a series of problems. Each requires you to understand a problem written in text and perform the appropriate u sually the problem talks about practical situations, but the numbers and people and places mentioned calculations. are made up. All the information you need is given. Here are two examples: 1. is two years older than and is four times as old as . When is 30, how old is ? 2. Mr bought a television and a bed. The television cost <$625> but he got a 10% discount. The bed cost <$200>. He paid <$20> for delivery. How much money did Mr spend? We want to know about your experience with these types of word problems at school. o not solve them! d (Please tick only one box in each row.) Frequently Sometimes Rarely Never a) How often have you encountered these types n 2 n 1 4 n 3 n ? mathematics lessons of problems in your b) How often have you encountered these types of n 4 n 3 n n 1 2 problems in the tests you have taken at school? Question 4 Below are examples of another set of mathematical skills. 1) Solve 2x + 3 = 7. 2) Find the volume of a box with sides 3m, 4m and 5m. We want to know about your experience with these types of problems at school. o not solve them! d (Please tick only one box in each row.) Frequently Sometimes Rarely Never a) How often have you encountered these types 2 n 1 n 4 n 3 n of problems in your mathematics lessons ? How often have you encountered these types of b) n n 2 n 1 n 3 4 problems in the tests you have taken at school? Question 5 In the next type of problem, you have to use mathematical knowledge and draw conclusions. There is no practical . application provided. Here are two examples 1) Here you need to use geometrical theorems: S 12 cm D C 12 cm A B 12 cm Determine the height of the pyramid. 2) Here you have to know what a prime number is: If n is any number: can (n+1)² be a prime number? We want to know about your experience with these types of problems at school. d o not solve them! (Please tick only one box in each row.) Never Sometimes Rarely Frequently a) How often have you encountered these types 3 n 2 n 1 n 4 n ? mathematics lessons of problems in your b) How often have you encountered these types of n 3 n 2 n 1 n 4 problems in the tests you have taken at school? , Kno W and Can d o: Student Performan C e in m athemati CS What Student r eading and S OECD 2014 S C ien C e – Volume i © 171

174 3 rtunities t O Learn Mathe M atics Measuring Opp O Question 6 In this type of problem, you have to apply suitable mathematical knowledge to find a useful answer to a problem that arises in everyday life or work. The data and information are about real situations. Here are two examples. Example 1 A TV reporter says “This graph shows that there is a huge increase in the number of robberies from 1998 to 1999.” Number of robberies per year 520 Year 1999 515 510 Year 1998 505 Example 2 For years the relationship between a person’s recommended maximum heart rate and the person’s age was described by the following formula: Recommended maximum heart rate = 220 – age Recent research showed that this formula should be modified slightly. The new formula is as follows: Recommended maximum heart rate = 208 – (0.7 × age) From which age onwards does the recommended maximum heart rate increase as a result of the introduction of the new formula? Show your work. We want to know about your experience with these types of problems at school. o not solve them! d (Please check only one box in each row.) Rarely Never Sometimes Frequently a) How often have you encountered these types n 1 n n 4 3 n 2 ? mathematics lessons of problems in your b) How often have you encountered these types of n n n 1 4 n 3 2 tests you have taken at school? problems in the he three opportunity to learn indice t S From these questions, three indices were constructed: • The inde x of exposure to word problems This index was coded using the frequency choices for the word-problem type of task (Question 3) as follows: frequently = 3, sometimes and rarely = 1, and never = 0. x of exposure to applied mathematics The inde • This index was constructed as the mean of the applied tasks involving both the mathematics contexts (Question 5) and the real-world contexts (Question 6). Each was separately scaled as: frequently = 3, sometimes = 2, rarely =1, and never = 0. The inde x of exposure to formal mathematics • This index was created as the average of three scales. wo separate scales were constructed using the item asking for the degree of the student’s familiarity with 7 of the T – 13 mathematics content areas (Question 2). The five response categories reflecting the degree to which they had heard of the topic were scaled 0 to 4 with 0 representing “never heard of it” 4 representing they “knew it well”. o: Student Performan e – Volume © OECD 2014 What Student S Kno W and Can d C C e in m athemati CS , r eading and S C ien i 172

175 3 Measuring Opp rtunities t O Learn Mathe M atics O The frequency codes for the three topics – exponential functions, quadratic functions, and linear equations – were averaged to define familiarity with algebra. Similarly, the average of four topics defined a geometry scale, including vectors, polygons, congruent figures, and cosines. – he third scale was derived from the item where students indicated how often they had been confronted with T problems defined as formal mathematics (Question 4). The frequency categories were coded as “frequently”, “sometimes”, and “rarely” equalling 1 and “never” equal to 0, resulting in a dichotomous variable. The algebra, geometry and formal mathematics tasks were averaged to form the index “formal mathematics”, which ranged in values from 0 to 3, similar to the other three indices. 173 CS athemati eading and S C ien C e – Volume i © OECD 2014 , m e in C o: Student Performan d and Can W Kno S What Student r

176 3 atics rtunities t O Learn Mathe M O Measuring Opp Note 1. The 18 countries/economies that show no relationship between the frequency of student encounters with applied mathematics problems and the performance of 15-year-olds on PISA are the United States, Poland, Hong Kong-China, Greece, Albania, Latvia, Germany, the Czech Republic, Hungary, Australia, Belgium, Argentina, Slovenia, Portugal, Liechtenstein, Korea, the Russian Federation and Viet Nam. References (8), pp. 723-733. Teachers College Record, 64 (1963), “A model of school learning”, Carroll, J.B. OECD PISA 2012 Assessment and Analytical Framework: Mathematics, Reading, Science, Problem Solving and Financial (2013), , OECD Publishing. Literacy http://dx.doi.org/10.1787/9789264190511-en Mullis, I.V.S., Chestnut Hill, Boston College, Massachusetts. TIMSS 2011 International Results in Mathematics, et al. (2012), San Francisco. Jossey-Bass, national Comparison of Curriculum and Learning, - et al. (2001), Schmidt, W.H., Why Schools Matter: A Cross and D.N. Plank (2009), Handbook of Education Policy Research , Routledge, New York. Sykes, G., B. Schneider (1974), “Explosion of a myth: Quantity of schooling and exposure to instruction, major educational Wiley, D.E. and A. Harnischfeger vehicles”, , 3(4), pp. 7-12. Educational Researcher r S © OECD 2014 i e – Volume C ien C eading and S Kno , CS athemati m e in C o: Student Performan d and Can W What Student 174

177 4 A Profile of Student Performance in Reading This chapter examines student performance in reading in PISA 2012. It provides examples of assessment questions, relating them to each PISA proficiency level, discusses gender differences in student performance, compares countries’ and economies’, performance in reading, and highlights trends in reading performance up to 2012. © e in athemati CS , r eading and S What Student ien C e – Volume i m OECD 2014 175 C o: Student Performan d and Can W Kno S C

178 4 A Profile f Student Perform A nce i n r e A ding o What can 15-y ear-old students do in reading? This chapter compares countries’ and economies’ performance, shows some regions’ performance, and analyses the changes over the various PISA assessments. It highlights the differences between girls’ and boys’ performance and provides examples of assessment questions at each PISA proficiency level. Reading literacy focuses on the ability of students to use written information in real-life situations. PISA defines reading literacy as understanding, using, reflecting on and engaging with written texts, in order to achieve one’s goals, to develop one’s knowledge and potential, and to participate in society (OECD, 2009). This definition goes beyond the traditional notion of decoding information and literal interpretation of what is written towards more applied tasks. PISA’s conception of reading literacy encompasses the range of situations in which people read, the different ways written texts are presented through different media, and the variety of ways that readers approach and use texts, from the functional and finite, such as finding a particular piece of practical information, to the deep and far-reaching, such as understanding other ways of doing, thinking and being. Reading literacy was the major domain assessed in 2000, the first PISA assessment, and in 2009, the fourth PISA assessment. In this fifth PISA assessment, mathematics was the major domain, thus less time was devoted to assessing students’ reading skills. As a result, only an update on overall performance is possible, rather than the kind of in-depth analysis of knowledge and skills shown in the PISA 2009 report (OECD, 2009). This chapter presents the results of the paper-based assessment in PISA 2012. Thirty-two of the 65 participating countries and economies participated in the computer-based (digital reading assessment). Annex B3 presents results on both the computer-based scale and a combined paper-and-computer scale. What the data tell us • Of the 64 countries and economies that ha ve comparable data in reading performance since 2000, 32 show an improvement in mean reading performance, 22 show no change, and 10 show a deterioration in performance. • Among OECD countries, Chile, Estonia, German y, Hungary, Israel, Japan, Korea, Luxembourg, Mexico, Poland, Portugal, Switzerland and Turkey all improved their reading performance across successive PISA assessments. • Between 2000 and 2012, Albania, Israel and Poland increased the share of top-performing students and simultaneously reduced the share of students who do not meet the baseline level of proficiency in reading. T he gender gap in reading performance – favouring girls – widened in 11 countries and economies between 2000 • and 2012. Student performance in reading The metric for the overall reading scale is based on a mean for participating OECD countries set at 500, with a standard deviation of 100. These were set when reporting the results of the first PISA reading assessment, administered in 2000 (OECD, 2001). To help interpret what students’ scores mean in substantive terms, the scale is divided into levels of proficiency that indicate the kinds of tasks that students at those levels are capable of completing successfully (OECD, 2009). Average performance in reading One way to summarise student performance and to compare the relative standing of countries in reading is through countries’ and economies’ mean performance, both relative to each other and to the OECD mean. For PISA 2012, the OECD mean is 496, with a standard deviation of 94. This establishes the benchmark against which each country’s and each economy’s reading performance in PISA 2012 is compared. When interpreting mean performance, only those differences among countries and economies that are statistically significant should be taken into account. Figure I.4.1 shows each country/economy’s mean score and also for which pairs of countries/economies the differences between the means are statistically significant. For each country/economy shown in the middle column, the countries/economies whose mean scores are not statistically significantly different are listed in the right column. In all other cases, country/economy A scores higher than country/economy B if country/ economy A is situated above country/economy B in the middle column, and scores lower if country/economy A is situated below country/economy B. For example: Shanghai-China ranks first and Hong Kong-China ranks second, but the performance of Singapore, which appears third on the list, cannot be distinguished with confidence from that of Hong Kong-China. o: Student Performan e in © OECD 2014 What Student S Kno W and Can d m i e – Volume C ien C eading and S r , CS athemati C 176

179 4 f Student Perform nce i n r e A ding A Profile o A • Figure I.4.1 • omparing countries’ and economies’ performance in reading c Statistically significantly the OECD average above Not statistically significantly different from the OECD average the OECD average below Statistically significantly m c ean omparison c ountries/economies whose mean score is not statistically significantly different from that comparison country’s/economy’s score score country/economy Shanghai-China 570 545 Hong Kong-China Singapore, Japan, Korea 542 Singapore Hong Kong-China, Japan, Korea 538 Japan Hong Kong-China, Singapore, Korea 536 Korea Hong Kong-China, Singapore, Japan Ireland, Chinese Taipei, Canada, Poland, Liechtenstein 524 Finland 523 Ireland Finland, Chinese Taipei, Canada, Poland, Liechtenstein 523 Chinese Taipei Finland, Ireland, Canada, Poland, Estonia, Liechtenstein 523 Canada Finland, Ireland, Chinese Taipei, Poland, Liechtenstein Finland, Ireland, Chinese Taipei, Canada, Estonia, Liechtenstein, New Zealand, Australia, Netherlands, Viet Nam Poland 518 516 Estonia Chinese Taipei, Poland, Liechtenstein, New Zealand, Australia, Netherlands, Viet Nam Finland, Ireland, Chinese Taipei, Canada, Poland, Estonia, New Zealand, Australia, Netherlands, Switzerland, Macao-China, Belgium, Liechtenstein 516 Viet Nam, Germany 512 New Zealand Poland, Estonia, Liechtenstein, Australia, Netherlands, Switzerland, Macao-China, Belgium, Viet Nam, Germany, France Australia Poland, Estonia, Liechtenstein, New Zealand, Netherlands, Switzerland, Macao-China, Belgium, Viet Nam, Germany, France 512 511 Netherlands Poland, Estonia, Liechtenstein, New Zealand, Australia, Switzerland, Macao-China, Belgium, Viet Nam, Germany, France, Norway Switzerland Liechtenstein, New Zealand, Australia, Netherlands, Macao-China, Belgium, Viet Nam, Germany, France, Norway 509 Liechtenstein, New Zealand, Australia, Netherlands, Switzerland, Belgium, Viet Nam, Germany, France, Norway 509 Macao-China Liechtenstein, New Zealand, Australia, Netherlands, Switzerland, Macao-China, Viet Nam, Germany, France, Norway Belgium 509 Poland, Estonia, Liechtenstein, New Zealand, Australia, Netherlands, Switzerland, Macao-China, Belgium, Germany, France, Norway, Viet Nam 508 United Kingdom, United States 508 Germany Liechtenstein, New Zealand, Australia, Netherlands, Switzerland, Macao-China, Belgium, Viet Nam, France, Norway, United Kingdom 505 France New Zealand, Australia, Netherlands, Switzerland, Macao-China, Belgium, Viet Nam, Germany, Norway, United Kingdom, United States Norway Netherlands, Switzerland, Macao-China, Belgium, Viet Nam, Germany, France, United Kingdom, United States, Denmark 504 499 United Kingdom Viet Nam, Germany, France, Norway, United States, Denmark, Czech Republic 498 United States Viet Nam, France, Norway, United Kingdom, Denmark, Czech Republic, Italy, Austria, Hungary, Portugal, Israel Denmark 496 Norway, United Kingdom, United States, Czech Republic, Italy, Austria, Hungary, Portugal, Israel United Kingdom, United States, Denmark, Italy, Austria, Latvia, Hungary, Spain, Luxembourg, Portugal, Israel, Croatia Czech Republic 493 490 Italy United States, Denmark, Czech Republic, Austria, Latvia, Hungary, Spain, Luxembourg, Portugal, Israel, Croatia, Sweden United States, Denmark, Czech Republic, Italy, Latvia, Hungary, Spain, Luxembourg, Portugal, Israel, Croatia, Sweden 490 Austria Latvia Czech Republic, Italy, Austria, Hungary, Spain, Luxembourg, Portugal, Israel, Croatia, Sweden 489 United States, Denmark, Czech Republic, Italy, Austria, Latvia, Spain, Luxembourg, Portugal, Israel, Croatia, Sweden, Iceland 488 Hungary 488 Spain Czech Republic, Italy, Austria, Latvia, Hungary, Luxembourg, Portugal, Israel, Croatia, Sweden Luxembourg Czech Republic, Italy, Austria, Latvia, Hungary, Spain, Portugal, Israel, Croatia, Sweden 488 488 Portugal United States, Denmark, Czech Republic, Italy, Austria, Latvia, Hungary, Spain, Luxembourg, Israel, Croatia, Sweden, Iceland, Slovenia United States, Denmark, Czech Republic, Italy, Austria, Latvia, Hungary, Spain, Luxembourg, Portugal, Croatia, Sweden, Iceland, Slovenia, Israel 486 Lithuania, Greece, Turkey, Russian Federation 485 Croatia Czech Republic, Italy, Austria, Latvia, Hungary, Spain, Luxembourg, Portugal, Israel, Sweden, Iceland, Slovenia, Lithuania, Greece, Turkey 483 Sweden Italy, Austria, Latvia, Hungary, Spain, Luxembourg, Portugal, Israel, Croatia, Iceland, Slovenia, Lithuania, Greece, Turkey, Russian Federation Iceland 483 Hungary, Portugal, Israel, Croatia, Sweden, Slovenia, Lithuania, Greece, Turkey Slovenia Portugal, Israel, Croatia, Sweden, Iceland, Lithuania, Greece, Turkey, Russian Federation 481 477 Lithuania Israel, Croatia, Sweden, Iceland, Slovenia, Greece, Turkey, Russian Federation Israel, Croatia, Sweden, Iceland, Slovenia, Lithuania, Turkey, Russian Federation 477 Greece 475 Turkey Israel, Croatia, Sweden, Iceland, Slovenia, Lithuania, Greece, Russian Federation 475 Russian Federation Israel, Sweden, Slovenia, Lithuania, Greece, Turkey 463 Slovak Republic 1, 2 Serbia 449 Cyprus 1, 2 , United Arab Emirates, Chile, Thailand, Costa Rica, Romania, Bulgaria Cyprus Serbia 446 United Arab Emirates 442 Serbia, Chile, Thailand, Costa Rica, Romania, Bulgaria Serbia, United Arab Emirates, Thailand, Costa Rica, Romania, Bulgaria 441 Chile 441 Thailand Serbia, United Arab Emirates, Chile, Costa Rica, Romania, Bulgaria 441 Costa Rica Serbia, United Arab Emirates, Chile, Thailand, Romania, Bulgaria Serbia, United Arab Emirates, Chile, Thailand, Costa Rica, Bulgaria 438 Romania Bulgaria Serbia, United Arab Emirates, Chile, Thailand, Costa Rica, Romania 436 Montenegro 424 Mexico Montenegro 422 Mexico Brazil, Tunisia, Colombia Uruguay 411 410 Brazil Uruguay, Tunisia, Colombia Uruguay, Brazil, Colombia, Jordan, Malaysia, Indonesia, Argentina, Albania 404 Tunisia 403 Colombia Uruguay, Brazil, Tunisia, Jordan, Malaysia, Indonesia, Argentina Jordan 399 Tunisia, Colombia, Malaysia, Indonesia, Argentina, Albania, Kazakhstan Malaysia Tunisia, Colombia, Jordan, Indonesia, Argentina, Albania, Kazakhstan 398 396 Tunisia, Colombia, Jordan, Malaysia, Argentina, Albania, Kazakhstan Indonesia Tunisia, Colombia, Jordan, Malaysia, Indonesia, Albania, Kazakhstan Argentina 396 Albania Tunisia, Jordan, Malaysia, Indonesia, Argentina, Kazakhstan, Qatar, Peru 394 393 Kazakhstan Jordan, Malaysia, Indonesia, Argentina, Albania, Qatar, Peru 388 Albania, Kazakhstan, Peru Qatar 384 Peru Albania, Kazakhstan, Qatar 1. Note by Turkey: The information in this document with reference to ”Cyprus” relates to the southern part of the Island. There is no single authority representing both Turkish and Greek Cypriot people on the Island. Turkey recognises the Turkish Republic of Northern Cyprus (TRNC). Until a lasting and equitable solution is found within the context of the United Nations, Turkey shall preserve its position concerning the “Cyprus issue”. 2. Note by all the European Union Member States of the OECD and the European Union: The Republic of Cyprus is recognised by all members of the United Nations with the exception of Turkey. The information in this document relates to the area under the effective control of the Government of the Republic of Cyprus. Source: OECD, PISA 2012 Database. 12 http://dx.doi.org/10.1787/888932935610 CS OECD 2014 S Kno W and Can d o: Student Performan C e in m athemati What Student , r eading and S C ien C e – Volume i © 177

180 4 A i n r e nce ding A Profile o f Student Perform A [ Figure I.4.2 • Part 1/3 ] • r eading performance among pi S a 2012 participants, at national and regional levels Reading scale r ange of ranks o a E cd countries ll countries/economies m ean score ower rank u u l l ower rank pper rank pper rank 1 1 Shanghai-China 570 4 2 Hong Kong-China 545 542 Singapore 4 2 Japan 2 2 1 5 538 3 2 5 Korea 1 536 527 nited States) u Massachusetts ( 525 Australian Capital Territory (Australia) Finland 3 5 6 10 524 10 523 3 6 6 Ireland 523 Chinese Taipei 6 10 Canada 523 3 6 6 10 nited States) 521 u Connecticut ( Veneto (Italy) 521 521 Trento (Italy) Lombardia (Italy) 521 Western Australia (Australia) 519 518 Friuli Venezia Giulia (Italy) 7 14 Poland 518 4 9 Flemish community (Belgium) 518 517 Victoria (Australia) Estonia 10 9 6 14 516 516 7 18 Liechtenstein n 513 ew South Wales (Australia) 12 7 512 New Zealand 19 11 512 Australia 8 12 12 18 Netherlands 14 11 21 6 511 511 Madrid (Spain) 509 avarre (Spain) n Switzerland 8 14 13 21 509 13 509 22 Macao-China Belgium 509 9 15 15 20 Viet Nam 508 12 23 508 Queensland (Australia) Germany 15 22 13 9 508 u nited Kingdom) 506 Scotland ( 506 Piemonte (Italy) 16 France 505 10 16 23 Castile and Leon (Spain) 505 504 Asturias (Spain) 17 24 11 504 17 Norway 502 Valle d’Aosta (Italy) 501 Catalonia (Spain) 500 South Australia (Australia) u nited Kingdom) 500 England ( 499 German-speaking community (Belgium) United Kingdom 499 14 19 20 26 499 Galicia (Spain) 498 Emilia Romagna (Italy) 498 Basque Country (Spain) 498 nited Kingdom) u n orthern Ireland ( 28 United States 498 14 20 21 French community (Belgium) 497 497 Bolzano (Italy) Marche (Italy) 497 27 23 20 16 496 Denmark Aragon (Spain) 493 Puglia (Italy) 493 23 23 16 493 Czech Republic 31 u mbria (Italy) 492 u nited States) 492 Florida ( Liguria (Italy) 490 490 La Rioja (Spain) Alentejo (Portugal) 490 OECD countries are shown in bold black. Partner countries are shown in bold blue. Participating economies and subnational entities that are not included in national results Notes: are shown in bold blue italics. Regions are shown in black italics (OECD countries) or blue italics (partner countries). 1. Note by Turkey: The information in this document with reference to ”Cyprus” relates to the southern part of the Island. There is no single authority representing both Turkish and Greek Cypriot people on the Island. Turkey recognises the Turkish Republic of Northern Cyprus (TRNC). Until a lasting and equitable solution is found within the context of the United Nations, Turkey shall preserve its position concerning the “Cyprus issue”. 2. Note b y all the European Union Member States of the OECD and the European Union: The Republic of Cyprus is recognised by all members of the United Nations with the exception of Turkey. The information in this document relates to the area under the effective control of the Government of the Republic of Cyprus. Countries, economies and subnational entities are ranked in descending order of mean reading performance. OECD, PISA 2012 Database. Source: 12 http://dx.doi.org/10.1787/888932935610 o: Student Performan e – Volume © OECD 2014 What Student S Kno W and Can d C C e in m athemati CS , r eading and S C ien i 178

181 4 o i n r e A ding A Profile nce f Student Perform A • • Figure I.4.2 [ Part 2/3 ] a eading performance among pi r S 2012 participants, at national and regional levels Reading scale r ange of ranks o a cd countries ll countries/economies E ean score m pper rank ower rank u l l ower rank u pper rank 19 490 Italy 34 25 26 34 490 18 26 25 Austria Latvia 489 26 35 Hungary 25 27 18 488 36 488 35 27 27 20 Spain 20 28 35 488 26 Luxembourg 37 25 28 18 488 Portugal 488 Toscana (Italy) 486 19 31 25 40 Israel Cantabria (Spain) 485 Croatia 485 28 39 Tasmania (Australia) 485 30 40 30 23 483 Sweden 39 Iceland 483 25 30 33 Perm Territory region (Russian Federation) 482 39 481 27 30 35 Slovenia Lazio (Italy) 480 480 Abruzzo (Italy) u nited Kingdom) 480 Wales ( Lithuania 477 42 37 42 36 Greece 31 28 477 477 Andalusia (Spain) 476 Molise (Italy) 476 Balearic Islands (Spain) 27 31 36 42 475 Turkey 42 Russian Federation 475 38 Basilicata (Italy) 474 468 nited Arab Emirates) u Dubai ( orthern Territory (Australia) 466 n 464 Campania (Italy) Sardegna (Italy) 464 43 463 32 32 43 Slovak Republic 462 Murcia (Spain) Extremadura (Spain) 457 Sicilia (Italy) 455 451 nited Arab Emirates) u Sharjah ( 451 Querétaro (Mexico) 1, 2 Cyprus 449 44 45 448 Distrito Federal (Mexico) Aguascalientes (Mexico) 447 44 446 48 Serbia Chihuahua (Mexico) 444 442 45 50 United Arab Emirates uevo León (Mexico) 442 n 441 50 45 33 33 Chile Thailand 441 51 45 Costa Rica 45 441 51 Colima (Mexico) 440 Romania 51 46 438 437 Mexico (Mexico) Durango (Mexico) 436 Jalisco (Mexico) 436 51 45 436 Bulgaria Calabria (Italy) 434 433 Rio Grande do Sul (Brazil) 431 Manizales (Colombia) Coahuila (Mexico) 431 431 nited Arab Emirates) u Abu Dhabi ( 430 Quintana Roo (Mexico) Ciudad Autónoma de Buenos Aires (Argentina) 429 Baja California (Mexico) 428 Federal District (Brazil) 428 Mato Grosso do Sul (Brazil) 428 OECD countries are shown in bold black. Partner countries are shown in bold blue. Participating economies and subnational entities that are not included in national results Notes: are shown in bold blue italics. Regions are shown in black italics (OECD countries) or blue italics (partner countries). 1. Note by Turkey: The information in this document with reference to ”Cyprus” relates to the southern part of the Island. There is no single authority representing both Turkish and Greek Cypriot people on the Island. Turkey recognises the Turkish Republic of Northern Cyprus (TRNC). Until a lasting and equitable solution is found within the context of the Turkey shall preserve its position concerning the “Cyprus issue”. United Nations, 2. Note by all the European Union Member States of the OECD and the European Union: The Republic of Cyprus is recognised by all members of the United Nations with the exception of Turkey. The information in this document relates to the area under the effective control of the Government of the Republic of Cyprus. Countries, economies and subnational entities are ranked in descending order of mean reading performance. OECD, PISA 2012 Database. Source: http://dx.doi.org/10.1787/888932935610 12 CS OECD 2014 S Kno W and Can d o: Student Performan C e in m athemati What Student , r eading and S C ien C e – Volume i © 179

182 4 o i n r e A ding A Profile nce f Student Perform A ] Part 3/3 • [ Figure I.4.2 • eading performance among r a S 2012 participants, at national and regional levels pi Reading scale r ange of ranks o a E cd countries ll countries/economies ean score m pper rank ower rank u pper rank l ower rank u l Espírito Santo (Brazil) 427 Minas Gerais (Brazil) 427 Yucatán (Mexico) 426 425 Morelos (Mexico) San Luis Potosí (Mexico) 425 53 424 34 34 52 Mexico 423 Baja California Sur (Mexico) 423 Puebla (Mexico) Medellin (Colombia) 423 Santa Catarina (Brazil) 423 Bogota (Colombia) 422 53 52 422 Montenegro Paraná (Brazil) 422 São Paulo (Brazil) 422 421 Tamaulipas (Mexico) Tlaxcala (Mexico) 418 418 ayarit (Mexico) n 417 Sinaloa (Mexico) nited Arab Emirates) 415 Fujairah ( u 415 nited Arab Emirates) u Ras Al Khaimah ( nited Arab Emirates) 414 Ajman ( u Guanajuato (Mexico) 414 414 Hidalgo (Mexico) Campeche (Mexico) 413 412 Zacatecas (Mexico) 411 Paraíba (Brazil) 411 Uruguay 56 54 Veracruz (Mexico) 410 54 410 56 Brazil Cali (Colombia) 408 Rio de Janeiro (Brazil) 408 54 404 60 Tunisia Colombia 60 55 403 Piauí (Brazil) 403 nited Arab Emirates) 400 u u mm Al Quwain ( Rondônia (Brazil) 400 62 56 Jordan 399 63 57 398 Malaysia 397 Sergipe (Brazil) 397 Ceará (Brazil) 396 Amapá (Brazil) 396 56 63 Indonesia 63 396 57 Argentina 395 Tabasco (Mexico) 64 394 58 Albania 393 Goiás (Brazil) orte (Brazil) n 393 Rio Grande do 393 Kazakhstan 64 59 Bahia (Brazil) 388 63 388 Qatar 65 387 Pará (Brazil) Peru 63 65 384 Acre (Brazil) 383 Amazonas (Brazil) 382 382 Mato Grosso (Brazil) Tocantins (Brazil) 381 377 Roraima (Brazil) Pernambuco (Brazil) 376 Chiapas (Mexico) 371 369 Maranhão (Brazil) 368 Guerrero (Mexico) Alagoas (Brazil) 355 OECD countries are shown in bold black. Partner countries are shown in bold blue. Participating economies and subnational entities that are not included in national results Notes: are shown in bold blue italics. Regions are shown in black italics (OECD countries) or blue italics (partner countries). 1. Note by Turkey: The information in this document with reference to ”Cyprus” relates to the southern part of the Island. There is no single authority representing both Turkish and Greek Cypriot people on the Island. Turkey recognises the Turkish Republic of Northern Cyprus (TRNC). Until a lasting and equitable solution is found within the context of the United Nations, Turkey shall preserve its position concerning the “Cyprus issue”. 2. Note by all the European Union Member States of the OECD and the European Union: The Republic of Cyprus is recognised by all members of the United Nations with the exception of Turkey. The information in this document relates to the area under the effective control of the Government of the Republic of Cyprus. Countries, economies and subnational entities are ranked in descending order of mean reading performance. Source: OECD, PISA 2012 Database. 12 http://dx.doi.org/10.1787/888932935610 o: Student Performan e – Volume © OECD 2014 What Student S Kno W and Can d C C e in m athemati CS , r eading and S C ien i 180

183 4 ding f Student Perform A nce i n r e A o A Profile Moreover, countries and economies are divided into three broad groups: those whose mean scores are statistically around the OECD mean (highlighted in dark blue), those whose mean scores are above the OECD mean (highlighted in pale blue), and those whose mean scores are below the OECD mean (highlighted in medium blue). As shown in Figure I.4.1, Shanghai-China, Hong Kong-China, Singapore, Japan and Korea are the five highest-performing countries and economies in reading. Shanghai-China has a mean score of 570 points in reading – the equivalent of more than a year-and-a-half of schooling above the OECD average of 496 score points, and 25 score points above the second best-performing participant, Hong Kong-China. Finland, Ireland, Chinese Taipei, Canada, Poland, Estonia and Liechtenstein perform at least 20 score points above the OECD average. Ten other countries and economies – New Zealand, Australia, the Netherlands, Switzerland, Macao-China, Belgium, Viet Nam, Germany, France and Norway – also score above the OECD average. Meanwhile, the United Kingdom, the United States, Denmark and the Czech Republic perform around the OECD average; and 39 countries and economies perform below the OECD average. Among OECD countries, performance differences are large: 114 score points separate the mean scores of the highest- and lowest-performing OECD countries; when the partner countries and economies are considered along with OECD countries, this difference amounts to 185 score points. Because the figures are derived from samples, it is not possible to determine a country’s or economy’s precise ranking among all countries and economies. However, it is possible to determine, with confidence, a range of rankings in which the country’s/economy’s performance level lies (Figure I.4.2). For entities other than those for which full samples were drawn (i.e. Shanghai-China, Hong Kong-China, Chinese Taipei and Macao-China), it is not possible to calculate a rank order but the mean score provides a possibility to position subnational entities against the performance of the countries and economies. For example, Massachusetts shows a score between the performance of top-performer Korea and Finland. Trends in average reading performance The change in a school system’s average performance over time indicates how and to what extent the system is progressing towards achieving the goal of providing all students with the knowledge and skills needed to become full participants in a knowledge-based society. Trends in reading performance up to 2012 are available for 64 countries 1 and economies. PISA 2012 results for 30 countries and economies can be compared with data from all the previous cycles (PISA 2000, 2003, 2006 and 2009); for the other countries and economies, annualised trends can be calculated even if these countries/economies did not begin their participation in PISA assessments in PISA 2000, missed some assesments between PISA 2000 and 2012, or have results from previous assessments that are not comparable over time. The following analyses calculate the average trend using all the available information. Results are presented as the annualised change – the average yearly change in performance observed throughout a country’s or economy’s 2 participation in PISA. (For further details on the estimation of the annualised change, see Annex A5). Of the 64 countries and economies with comparable data in reading performance, 32 show a positive annualised trend in mean reading performance across all PISA assessments, 22 show no change, and the remaining 10 countries and economies show a deteriorating annualised trend in average student performance. Among OECD countries, average yearly improvements (i.e. positive annualised change) in reading performance across successive PISA assessments are observed in Chile, Estonia, Germany, Hungary, Israel, Japan, Korea, Luxembourg, Mexico, Poland, Portugal, Switzerland and Turkey. Figure I.4.3 shows that Montenegro, Peru, Qatar, Serbia and Singapore saw an average yearly improvement of more than five score points in reading throughout their participation in subsequent PISA assessments. Albania, Chinese Taipei, Turkey and Shanghai-China saw an average yearly improvement of more than four score points, and Chile, Israel and Tunisia saw an average yearly improvement of more than three score points. These are significant improvements. Most of these countries and economies, except Shanghai-China and Singapore, have participated in at least three PISA assessments. Six other countries and economies show a yearly improvement of at least two score points in reading; 11 countries and economies saw a yearly improvement of at least one score point; and three countries and economies saw an annual improvement in performance, albeit of less than one score point. In 2000, the average 15-year-old in Peru scored 327 points on the PISA reading assessment, 370 score points in 2009 and 384 points in 2012. Improvements over time were also consistent in Turkey, where the average reading performance athemati S Kno W and Can d o: Student Performan C e in 181 OECD 2014 What Student © i e – Volume C ien C eading and S r , CS m

184 4 f Student Perform nce i n r A A ding A Profile o e improved relatively steadily from 441 points to 475 points between 2003 and 2012. Poland also saw consistent progress across the five PISA assessments, moving from a below-OECD-average score of 479 score points in reading in 2000 to an above-OECD-average score of 518 points in 2012. Korea’s improvement in PISA and recent education policies and programmes are outlined in Box I.4.1. Figure I.4.3 • • nnualised change in reading performance throughout participation in a pi S a R eading score-point difference associated with one calendar year 15 10 5 0 Annualised change in reading performance -5 -10 5 5 2 5 5 5 5 5 5 5 5 4 5 4 3 5 2 3 3 4 4 4 4 5 3 4 2 5 5 3 4 2 4 5 5 3 5 5 3 5 5 5 3 4 3 2 5 2 5 5 5 5 5 5 5 3 3 5 3 2 4 3 4 4 4 Italy Peru Chile Israel Brazil Spain Japan Qatar Latvia Korea Serbia Jordan France Turkey Tunisia Poland Austria Ireland Estonia Greece Finland Iceland Mexico Croatia Albania Canada Norway Sweden Bulgaria Belgium Portugal Slovenia Uruguay Malaysia Thailand Hungary Australia Romania Lithuania Germany Denmark Argentina Indonesia Colombia Singapore Costa Rica Kazakhstan Switzerland Dubai (UAE) Netherlands Montenegro Luxembourg New Zealand Liechtenstein United States Macao-China Chinese Taipei Czech Republic Slovak Republic Shanghai-China United Kingdom Hong Kong-China Russian Federation OECD average 2000 United Arab Emirates* * United Arab Emirates excluding Dubai. Statistically signicant score point changes are marked in a darker tone (see Annex A3). Notes: The number of comparable reading scores used to calculate the annualised change is shown next to the country/economy name. The annualised change is the average annual change in PISA score points from a country’s/economy’s earliest participation in PISA to PISA 2012. It is calculated taking into account all of a country’s/economy‘s participation in PISA. For more details on the calculation of the annualised change, see Annex A5. OECD average 2000 compares only OECD countries with comparable reading scores since 2000. Countries and economies are ranked in descending order of the annualised change in reading performance. Source: OECD, PISA 2012 Database, Table I.4.3b. http://dx.doi.org/10.1787/888932935610 2 1 The average change experienced over successive PISA assessments doesn’t capture the extent to which this change is steady, or whether it is decelerating or accelerating. Of the 32 countries and economies that show a statistically an annualised improvement in reading performance, 29 participated in at least two PISA assessments in addition to PISA 2012, so it is possible to determine whether their improvement is steady, accelerating or decelerating. The average reading performance in Chinese Taipei, Israel, Japan, Luxembourg, Macao-China, the Russian Federation and Thailand shows an improvement, the rate of which is higher in the later PISA assessments than in the earlier assessments. Improvements in reading have remained relatively steady in Albania, Brazil, Estonia, Germany, Hong Kong-China, Hungary, Indonesia, Mexico, Montenegro, Peru, Portugal, Poland, Switzerland, Tunisia and Turkey, and was slower in the later PISA assessments than the earlier assessments in Chile, Colombia, Korea, Latvia, Liechtenstein, Qatar and Serbia (Figure I.4.4). Other countries and economies show no annualised improvement, but this is because of a deterioration between their first two PISA assessments followed by improvements in later assessments. This was observed in Argentina, Bulgaria, France, Italy, Norway, Romania and Spain. Spain, for example, saw a decline in performance between PISA 2000 and PISA 2003 which continued through PISA 2006. But this initially negative trend reversed itself between 2006 and 2009 to the extent that Spain’s performance in PISA 2012 was similar to that recorded in PISA 2000. athemati CS , r eading and S e in ien C e – Volume i C o: Student Performan d and Can W Kno S What Student OECD 2014 © m C 182

185 4 A A nce i n r e f Student Perform ding A Profile o • • Figure I.4.4 urvilinear trajectories of average reading performance across c assessments a S pi Rate of acceleration or deceleration in per formance (quadratic term) Decelerating Accelerating Steadily changing PISA reading score PISA reading score PISA reading score 2003 2000 2006 2009 2012 2000 2003 2012 2006 2009 2003 2000 2006 2009 2012 Countries/economies Albania Israel Hungary Poland Thailand Chile Qatar Brazil Japan Indonesia Chinese Taipei Portugal Colombia Serbia with positive annualised change Estonia Mexico Luxembourg Switzerland Korea Germany Macao-China Montenegro Tunisia Latvia Hong Kong-China Peru Russian Federation Turkey Liechtenstein PISA reading score PISA reading score PISA reading score 2003 2009 2003 2000 2000 2006 2009 2012 2003 2000 2006 2009 2012 2006 2012 Countries/economies Croatia Argentina Romania Jordan Austria Bulgaria Spain Lithuania Belgium France Netherlands Czech Republic Italy Slovak Republic with no signicant annualised change Denmark Norway United Kingdom Greece United States PISA reading score PISA reading score PISA reading score 2000 2006 2009 2009 2006 2012 2009 2012 2012 2003 2003 2003 2000 2006 2000 Countries/economies Australia Slovenia Ireland Canada Sweden with negative annualised change Finland Uruguay Iceland New Zealand Figures are for illustrative purposes only. Countries and economies are grouped according to the direction and signicance of their annualised Notes: change and their rate of acceleration. Countries and economies with data from only one PISA assessments other than 2012 are excluded. Source: OECD, PISA 2012 Database, Table I.4.3b. http://dx.doi.org/10.1787/888932935610 2 1 o: Student Performan r C ien C e – Volume , CS athemati m e in C eading and S d and Can W Kno S What Student i © OECD 2014 183

186 4 A i n r e A ding A Profile o f Student Perform nce ] Figure I.4.5 [ Part 1/2 • • m ultiple comparisons of reading performance between 2000 and 2012 countries/economies with countries/economies with countries/economies with countries/economies with reading reading c ountries/economies with similar higher performance in 2000 ountries/economies with similar c higher performance in 2000 c ountries/economies with similar lower performance in 2000 eading r lower performance in 2000 r eading performance performance performance in 2000 but lower performance in performance in 2000 but with similar performance performance in 2000 but higher performance in performance but similar performance performance in 2000 in 2012 but higher performance in 2012 2012 and similar performance in 2012 in 2012 but lower performance in 2012 2012 in 2012 in 2012 in 2000 Hong Kong-China 525 545 New Zealand, Sweden, Australia, Ireland Japan, Korea Finland, Canada 545 525 Hong Kong-China Japan 522 538 Hong Kong-China, Korea Japan 522 538 United States, New Zealand, Sweden, Finland Australia, Canada, Ireland, Belgium Hong Kong-China, Japan, Ireland 525 536 New Zealand, Sweden, Australia Korea Finland Canada 536 525 Korea 546 524 Hong Kong-China, Japan Finland Finland 546 524 Poland, Canada, Ireland, Liechtenstein, Korea Hong Kong-China, Japan Canada, Korea New Zealand, Sweden, Australia 523 527 Ireland Ireland 527 523 Finland Poland, Liechtenstein Finland 534 Ireland Japan Poland, Liechtenstein Hong Kong-China, Korea New Zealand, Australia 523 523 Canada 534 Canada 479 518 Greece, Austria, Czech Republic, Germany, Switzerland, Liechtenstein New Zealand, Finland, Poland United States, France, Poland 518 479 Hungary, Spain, Portugal, Italy Australia, Canada, Ireland, Sweden, Denmark, Iceland, Belgium Norway New Zealand, Finland, Liechtenstein 516 United States, Greece, Austria, 483 Poland, Germany, Switzerland Sweden, Denmark, Iceland 516 483 Liechtenstein Czech Republic, Hungary, Spain, France, Australia, Canada, Portugal, Italy Ireland, Belgium, Norway Hong Kong-China, Japan, Canada, Poland, Germany, France, New Zealand 529 512 Australia 529 512 New Zealand Ireland, Korea Belgium, Switzerland, Norway, Liechtenstein Poland, Germany, France, Australia 512 New Zealand Hong Kong-China, Japan, Canada, 528 512 528 Australia Ireland, Korea Belgium, Switzerland, Norway, Liechtenstein New Zealand, Australia Belgium 507 509 Sweden, Denmark, Iceland United States, France, Switzerland, Norway Japan Poland, Germany, Liechtenstein 509 507 Belgium 509 Switzerland 509 Austria, Czech Republic, Hungary, Spain, Switzerland United States, Poland, Germany, France, 494 New Zealand, Australia Sweden, Iceland 494 Belgium, Norway, Liechtenstein Denmark, Italy Germany 484 508 Greece, Austria, Czech Republic, United States, New Zealand, Poland, Switzerland, Liechtenstein 484 Sweden, Denmark, Iceland 508 Germany Hungary, Spain, Italy France, Australia, Belgium, Norway 505 Sweden New Zealand, Australia Poland Germany, Liechtenstein United States, Belgium, Denmark, Iceland 505 505 France France 505 Switzerland, Norway 505 504 Sweden New Zealand, Australia Poland 505 Norway Norway 504 United States, France, Belgium, Iceland Germany, Czech Republic, Denmark, Switzerland Liechtenstein 498 Poland Latvia, Germany, Hungary, Austria, Czech Republic, France, Sweden, Iceland 498 504 United States United States 504 Japan, Liechtenstein Israel, Portugal Belgium, Spain, Denmark, Switzerland, Norway, Italy Belgium, Switzerland 497 496 United States, Austria, Czech Republic, 497 Sweden, Iceland Latvia, Hungary, Israel, 496 Poland, Germany, Denmark Denmark Liechtenstein Portugal France, Spain, Norway, Italy Czech Republic Czech Republic 492 493 United States, Austria, Hungary, Spain, 493 Poland, Germany, Switzerland, Iceland Latvia, Israel, Portugal Sweden, Norway 492 Liechtenstein Denmark, Italy 490 Sweden, Iceland Latvia, Israel, Portugal United States, Austria, Czech Republic, Italy 487 Poland, Germany, Switzerland, 490 Greece 487 Italy Liechtenstein Hungary, Spain, Denmark 492 Austria 492 490 United States, Czech Republic, Hungary, Austria Poland, Germany, Switzerland, Latvia, Israel, Portugal 490 Sweden, Iceland Spain, Denmark, Italy Liechtenstein Israel, Portugal Latvia 458 489 Greece, Russian Federation United States, Austria, Latvia 489 458 Czech Republic, Sweden, Hungary, Spain, Denmark, Iceland, Italy United States, Sweden, Hungary 488 480 Greece, Austria, Czech Republic, Spain, Hungary 480 Poland, Germany, Switzerland, Latvia, Israel 488 Portugal, Italy Denmark, Iceland Liechtenstein United States, Austria, Czech Republic, Spain 493 488 493 Spain 488 Sweden, Iceland Latvia, Israel, Portugal Poland, Germany, Switzerland, Liechtenstein Hungary, Denmark, Italy 488 Greece, Latvia, Hungary, Israel, Poland, Liechtenstein United States, Austria, 470 Portugal Portugal 488 470 Russian Federation Czech Republic, Sweden, Spain, Denmark, Iceland, Italy United States, Austria, 486 Israel Thailand, Bulgaria, Argentina Greece, Latvia, Portugal, 452 486 Israel 452 Czech Republic, Sweden, Russian Federation Hungary, Spain, Denmark, Iceland, Italy United States, Hong Kong-China, Japan, Sweden 516 483 Greece, Latvia, Austria, Poland, Germany, France, 516 Sweden 483 Czech Republic, Hungary, Ireland, Belgium, Korea Denmark, Switzerland, Israel, Spain, Iceland, Portugal, Norway, Liechtenstein Russian Federation, Italy Iceland 507 483 United States, France, Belgium, Norway Greece, Latvia, Austria, Poland, Germany, Iceland 483 507 Sweden Hungary, Israel, Spain, Czech Republic, Denmark, Portugal, Russian Federation, Switzerland, Liechtenstein Italy Hungary, Israel, Portugal, 477 Poland, Latvia, Germany, Liechtenstein, 474 Greece 477 Sweden, Iceland 474 Greece Russian Federation Italy Russian Federation Latvia Russian Federation 462 475 Sweden, Iceland Greece, Israel, Portugal 475 462 Chile 410 441 Argentina, Mexico Thailand, Bulgaria, Romania 441 410 Chile 441 Thailand 431 441 Israel Bulgaria, Romania Argentina, Mexico Chile Thailand 431 428 Romania 438 Argentina, Mexico Thailand, Bulgaria Chile 438 428 Romania Argentina Thailand, Mexico, Romania Israel 436 Chile 436 430 Bulgaria Bulgaria 430 Argentina 424 422 Mexico Mexico 424 Thailand, Chile, Romania Bulgaria 422 Brazil 396 410 Argentina 410 396 Brazil Indonesia 371 371 396 Albania, Peru Argentina Indonesia 396 418 Argentina 396 418 Brazil, Thailand, Israel, Bulgaria, Chile, 396 Argentina Albania, Indonesia, Peru Mexico, Romania Albania 349 394 Albania 349 394 Argentina, Indonesia Peru 327 Peru 327 Peru 384 Albania, Argentina, Indonesia 384 Note: Only countries and economies that participated in the PISA 2000 and PISA 2012 assessments are shown. Countries and economies are ranked in descending order of their mean reading performance in PISA 2012. OECD, PISA 2012 Database, Table I.4.3b. Source: 12 http://dx.doi.org/10.1787/888932935610 o: Student Performan e – Volume © OECD 2014 What Student S Kno W and Can d C C e in m athemati CS , r eading and S C ien i 184

187 4 f Student Perform nce i n r e A ding A Profile o A ] Figure I.4.5 [ Part 2/2 • • m ultiple comparisons of reading performance between 2000 and 2012 ountries/economies with c c ountries/economies with ountries/economies with c ountries/economies with c eading r r eading countries/economies with similar higher performance in 2000 countries/economies with similar higher performance in 2000 countries/economies with similar lower performance in 2000 reading lo wer performance in 2000 reading performance performance performance in 2000 but lower performance in performance in 2000 but with similar performance performance in 2000 but higher performance in performance but similar performance performance in 2000 in 2012 but higher performance in 2012 2012 and similar performance in 2012 in 2012 but lower performance in 2012 2012 in 2012 in 2012 in 2000 Hong Kong-China 525 Hong Kong-China New Zealand, Sweden, Australia, Ireland Japan, Korea Finland, Canada 545 525 545 Japan 522 522 Japan 538 538 United States, New Zealand, Sweden, Finland Hong Kong-China, Korea Australia, Canada, Ireland, Belgium Canada Korea 525 Korea 525 536 New Zealand, Sweden, Australia Hong Kong-China, Japan, Ireland Finland 536 546 Finland 524 Finland 546 524 Poland, Canada, Ireland, Hong Kong-China, Japan Liechtenstein, Korea Canada, Korea New Zealand, Sweden, Australia 523 527 Ireland Ireland 527 523 Finland Poland, Liechtenstein Hong Kong-China, Japan 523 Canada Japan Poland, Liechtenstein Hong Kong-China, Korea Finland Ireland 534 New Zealand, Australia 523 Canada 534 518 Greece, Austria, Czech Republic, Germany, Switzerland, Liechtenstein New Zealand, Finland, Poland 479 United States, France, 479 518 Poland Australia, Canada, Ireland, Hungary, Spain, Portugal, Italy Sweden, Denmark, Iceland, Belgium Norway Liechtenstein 516 United States, Greece, Austria, New Zealand, Finland, 483 483 Poland, Germany, Switzerland Sweden, Denmark, Iceland 516 Liechtenstein France, Australia, Canada, Czech Republic, Hungary, Spain, Ireland, Belgium, Norway Portugal, Italy New Zealand 529 512 Australia Hong Kong-China, Japan, Canada, Poland, Germany, France, 512 New Zealand 529 Ireland, Korea Belgium, Switzerland, Norway, Liechtenstein 512 New Zealand Hong Kong-China, Japan, Canada, Poland, Germany, France, 528 Australia 512 528 Australia Belgium, Switzerland, Ireland, Korea Norway, Liechtenstein 509 507 509 Sweden, Denmark, Iceland United States, France, Switzerland, Norway Japan Poland, Germany, Liechtenstein New Zealand, Australia 507 Belgium Belgium 509 Switzerland 509 Austria, Czech Republic, Hungary, Spain, Switzerland United States, Poland, Germany, France, 494 New Zealand, Australia Sweden, Iceland 494 Belgium, Norway, Liechtenstein Denmark, Italy Germany 484 508 Greece, Austria, Czech Republic, Poland, Switzerland, Liechtenstein United States, New Zealand, Sweden, Denmark, Iceland Germany 484 508 France, Australia, Belgium, Hungary, Spain, Italy Norway New Zealand, Australia Poland Germany, Liechtenstein United States, Belgium, Denmark, Iceland 505 505 France France 505 505 Sweden Switzerland, Norway 505 505 504 Sweden New Zealand, Australia Poland Norway 504 Germany, Czech Republic, Iceland United States, France, Belgium, Norway Denmark, Switzerland Liechtenstein Poland Latvia, Germany, Hungary, Austria, Czech Republic, France, Sweden, Iceland 498 504 United States United States 504 498 Japan, Liechtenstein Israel, Portugal Belgium, Spain, Denmark, Switzerland, Norway, Italy United States, Austria, Czech Republic, Denmark Denmark 497 496 496 497 Belgium, Switzerland Latvia, Hungary, Israel, Sweden, Iceland Poland, Germany, Liechtenstein Portugal France, Spain, Norway, Italy 493 493 Iceland Sweden, Norway 492 Latvia, Israel, Portugal Czech Republic Poland, Germany, Switzerland, Czech Republic 492 United States, Austria, Hungary, Spain, Denmark, Italy Liechtenstein 487 490 Greece United States, Austria, Czech Republic, Italy Poland, Germany, Switzerland, Italy Latvia, Israel, Portugal Sweden, Iceland 490 487 Liechtenstein Hungary, Spain, Denmark Poland, Germany, Switzerland, Austria 492 490 United States, Czech Republic, Hungary, 492 Austria 490 Latvia, Israel, Portugal Sweden, Iceland Liechtenstein Spain, Denmark, Italy Latvia Israel, Portugal Greece, Russian Federation 458 489 United States, Austria, 489 458 Latvia Czech Republic, Sweden, Hungary, Spain, Denmark, Iceland, Italy Hungary Latvia, Israel Poland, Germany, Switzerland, United States, Sweden, 488 480 Hungary 480 488 Greece, Austria, Czech Republic, Spain, Portugal, Italy Denmark, Iceland Liechtenstein Latvia, Israel, Portugal United States, Austria, Czech Republic, 488 Sweden, Iceland 488 493 493 Spain Spain Poland, Germany, Switzerland, Liechtenstein Hungary, Denmark, Italy Greece, Latvia, Hungary, Israel, 488 470 Poland, Liechtenstein United States, Austria, Portugal 470 Portugal 488 Russian Federation Czech Republic, Sweden, Spain, Denmark, Iceland, Italy United States, Austria, Israel 452 Thailand, Bulgaria, Argentina Greece, Latvia, Portugal, 486 486 452 Israel Russian Federation Czech Republic, Sweden, Hungary, Spain, Denmark, Iceland, Italy United States, Hong Kong-China, Japan, 483 516 Sweden Greece, Latvia, Austria, Poland, Germany, France, 516 Sweden 483 Czech Republic, Hungary, Ireland, Belgium, Korea Denmark, Switzerland, Israel, Spain, Iceland, Portugal, Norway, Liechtenstein Russian Federation, Italy 507 483 United States, France, Belgium, Norway Greece, Latvia, Austria, Iceland Poland, Germany, 507 Iceland 483 Sweden Hungary, Israel, Spain, Czech Republic, Denmark, Portugal, Russian Federation, Switzerland, Liechtenstein Italy Greece 474 477 Greece 474 Poland, Latvia, Germany, Liechtenstein, 477 Sweden, Iceland Hungary, Israel, Portugal, Italy Russian Federation Russian Federation Sweden, Iceland Latvia 475 Greece, Israel, Portugal 462 475 Russian Federation 462 Thailand, Bulgaria, Romania Argentina, Mexico 441 410 Chile Chile 410 441 431 Thailand 431 441 Argentina, Mexico Bulgaria, Romania Israel Chile 441 Thailand Chile 428 Romania 428 Thailand, Bulgaria 438 Romania Argentina, Mexico 438 Thailand, Mexico, Romania Bulgaria Israel Chile 436 430 Bulgaria 430 436 Argentina 422 424 Thailand, Chile, Romania Bulgaria Argentina 424 422 Mexico Mexico 396 Brazil Brazil 396 410 Argentina 410 371 Indonesia 396 Argentina Albania, Peru 396 371 Indonesia Brazil, Thailand, Israel, Bulgaria, Chile, 396 418 418 Argentina Argentina Albania, Indonesia, Peru 396 Mexico, Romania 349 Albania 394 Peru Argentina, Indonesia 394 349 Albania 327 327 384 Albania, Argentina, Indonesia 384 Peru Peru Note: Only countries and economies that participated in the PISA 2000 and PISA 2012 assessments are shown. Countries and economies are ranked in descending order of their mean reading performance in PISA 2012. Source: OECD, PISA 2012 Database, Table I.4.3b. 12 http://dx.doi.org/10.1787/888932935610 CS OECD 2014 S Kno W and Can d o: Student Performan C e in m athemati What Student , r eading and S C ien C e – Volume i © 185

188 4 f Student Perform nce i n r A A ding A Profile o e At any point in time, countries and economies share similar levels of performance with other countries and economies. But since the pace of change varies over time and across school systems, the relative standing of countries and economies evolves. Figure I.4.5 shows, for each country and economy with comparable results in 2000 and PISA 2012, those other countries and economies that had similar reading performance in 2000 but whose performance improved or deteriorated in 2012. In 2000, for example, Germany was similar in reading performance to Austria, the Czech Republic, Greece, Hungary, Italy, Liechtenstein, Poland, Spain and Switzerland; but after improvements in performance, it scored higher than Austria, the Czech Republic, Greece, Hungary, Italy and Spain in 2012. In 2000, Germany’s score in PISA was lower than those of Australia and New Zealand; but by 2012, the country had reached the same performance level as these two countries. Along the same lines, Chile had similar levels of performance as Argentina and Mexico in 2000. By 2012, Chile showed better performance than these two and attained the same level of performance as Bulgaria, Romania and Thailand– all of which had higher average reading scores than Chile in PISA 2000. Figure I.4.6 shows the relationship between each country’s and economy’s average reading performance in PISA 2000 and 3 their annualised change between 2000 and 2012. Countries and economies that show the strongest improvement in this period are more likely to have had comparatively low performance in PISA 2000 or their earliest comparable PISA score. • Figure I.4.6 • a S pi elationship between annualised change in performance and average r 2000 reading scores PISA 2000 performance PISA 2000 performance OECD average above below OECD average 6 Peru 5 Performance improved Albania 4 Israel Chile 3 Poland Annualised change in reading performance Indonesia Hong Kong-China Switzerland 2 Germany Latvia Portugal Romania Japan Thailand Liechtenstein Brazil Mexico 1 Korea Hungary Russian Federation Denmark Italy Greece Bulgaria Norway OECD average 2000 Belgium 0 Performance deteriorated Austria Spain France Czech Republic Ireland United Canada -1 States New Zealand Iceland Australia Finland Argentina -2 Sweden -3 375 425 475 525 575 350 400 450 500 550 325 Mean score in reading in PISA 2000 Annualised score point change in reading that are statistically signicant are indicated in a darker tone (see Annex A3). Notes: The annualised change is the average annual change in PISA score points from a country’s/economy’s earliest participation in PISA to PISA 2012. It is calculated taking into account all of a country’s/economy’s participation in PISA. For more details on the calculation of the annualised change, see Annex A5. OECD average 2000 considers only those countries with comparable reading scores since PISA 2000. The correlation between a country’s/economy’s mean score in 2000 and its annualised performance is -0.67. Source: OECD, PISA 2012 Database, Table I.4.3b. 2 1 http://dx.doi.org/10.1787/888932935610 Kno eading and S , CS athemati C e in C o: Student Performan d and Can W r S What Student OECD 2014 © i ien C e – Volume m 186

189 4 ding f Student Perform A nce i n r e A o A Profile In fact, the correlation between a country’s/economy’s PISA 2000 reading score and their annualised change in reading is -0.67. Among other things, this means that 45% of the variation in the annualised change since 2000 can be explained by a country’s/economy’s PISA 2000 reading score. Of the 20 countries and economies that showed an annualised improvement in reading performance and participated in PISA in the 2000 assessment, eleven had an average reading performance of 470 points in PISA 2000, well below the OECD average. It is by no means the case that all low-performing countries improve at a faster pace. Greece, Hungary, Poland and Portugal, for example, had relatively similar levels of performance in PISA 2000 (between 470 and 480 score points in reading), yet by 2012, the degree of improvement, if any, varied among them. Poland improved by 2.8 score points per year, Portugal by 1.6 points and Hungary by 1.0 per year, while no improvement was observed in Greece. Similarly, while Mexico, Argentina and Chile had similar levels of performance in PISA 2000 (between 410 and 422 score points), by 2012 improvements were observed in Chile and Mexico, but no improvement was observed in Argentina. Indeed, even some of those countries and economies that scored at or above the OECD average in the earlier assessments of PISA showed annualised improvements across their participation in PISA. These include Chinese Taipei, Estonia, Hong Kong-China, Japan, Korea, Macao-China, Shanghai-China, Singapore and Switzerland (Figure I.4.6 and Table I.4.3b). Trends in reading performance adjusted for sampling and demographic changes Improvements in a country’s or economy’s overall reading performance may be the result of specific education policies; they may also be due to demographic or socio-economic changes that shift the country’s/economy’s population profile. For example, because of trends in migration, the characteristics of the PISA reference population – 15-year-olds enrolled in school – may have shifted; or, as a result of development, the socio-economic status of students who were assessed in PISA 2012 is higher than that of students assessed in 2000. Adjusted trends shed light on changes in reading performance that are not due to alterations in the demographic characteristics of the student population or the sample. Figure I.4.7 presents the adjusted annualised changes in reading performance. These adjusted trends assume that the socio-economic status of students and their age, as well as the proportion of girls, students with an immigrant background and students speaking a language at home different from the language of instruction remain intact across PISA cycles, using the PISA 2012 sample as the reference. In short, it assumes that the population and sample characteristics observed in 2012 along these student-level attributes did 4 If countries and economies see a difference between the adjusted trends and not change between 2000 and 2012. the observed trends, particularly when the observed trend tends lower (or negative) in relation to the adjusted trend (non-negative), that means that changes in the student population are having adverse effects on performance. It is the observed, not the adjusted, trends that measure the quality of education in a school system. Annex A5 provides details on how adjusted trends are calculated. After accounting for these differences in population and samples, 21 countries and economies experience an average yearly improvement in reading performance. Colombia, Croatia, Dubai (United Arab Emirates), Indonesia, Jordan, Mexico, New Zealand, Costa Rica, the Slovak Republic and Sweden have similar adjusted and un-adjusted trends, meaning that either the PISA samples or the reference population have not changed much during their participation in PISA; that even if the students’ characteristics have changed, these have not affected their performance in school; or that improved education services have offset any negative effect on average reading performance related to changes in the population. After accounting for changes in students’ background characteristics, the observed improvements in Japan, Luxembourg, Malaysia, Romania and Turkey are greater. In these countries, improvements in reading performance were unrelated to changes in the student population; had students in the previous assessment shared the same characteristics as students who took the PISA 2012 test, the observed improvements would have been even greater. In Brazil, Estonia, Germany, Hungary, Liechtenstein, Macao-China, Portugal, the Russian Federation, Switzerland and Thailand the overall observed improvement loses statistical significance. In Korea, the observed improvement in reading performance becomes negative after accounting for students’ background characteristics. In these countries and economies, a large part of the observed improvement can be attributed to the changes in the student population. Observed improvements in the remaining countries and economies remain, indicating that they are not fully explained by changes in the background characteristics of students. In these cases, changes in other student characteristics, such as students’ attitudes towards learning, or the resources, policies and practices implemented in the school system may account for the improvements. Observed improvements remain, but are smaller in magnitude in Chile, Hong Kong-China, Israel, Latvia, Mexico, Montenegro and Poland. In these countries and economies, at least a third of the improvement is the result of a change in the student population – or the sample – towards students whose background characteristics are typically associated with better reading outcomes. athemati S Kno W and Can d o: Student Performan C e in 187 OECD 2014 What Student © i e – Volume C ien C eading and S r , CS m

190 4 f Student Perform nce i n r e A ding o A Profile A Informative as they may be, adjusted trends are merely hypothetical scenarios that help to determine the source of changes in students’ performance over time. Observed trends depicted in Figure I.4.7 and throughout this chapter summarise the overall evolution of a school system, highlighting the challenges that countries and economies face in improving students’ and schools’ performance in reading. • Figure I.4.7 • pi a djusted and observed annualised performance change in average a reading scores S After accounting for social and demographic changes Before accounting for social and demographic changes 14 12 10 8 6 4 2 0 -2 -4 Annualised change in reading performance -6 -8 -10 Italy Peru Chile Israel Brazil Spain Japan Qatar Latvia Korea Serbia Jordan France Turkey Tunisia Poland Austria Ireland Estonia Greece Iceland Finland Mexico Croatia Canada Norway Sweden Bulgaria Belgium Portugal Slovenia Uruguay Malaysia Thailand Hungary Australia Romania Lithuania Germany Denmark Argentina Indonesia Colombia Singapore Costa Rica Kazakhstan Switzerland Dubai (UAE) Netherlands Montenegro Luxembourg New Zealand Liechtenstein United States Macao-China Chinese Taipei Czech Republic Slovak Republic Shanghai-China United Kingdom Hong Kong-China Russian Federation OECD average 2000 United Arab Emirates* * United Arab Emirates excluding Dubai. Statistically signicant values are marked in a darker tone (see Annex A3). Notes: The annualised change is the average annual change in PISA score points. It is calculated taking into account all of a country’s/economy’s participation in PISA. For more details on the calculation of the annualised change, see Annex A5. The annualised change adjusted for demographic changes assumes that the average age and as well as PISA index of social, cultural and economic status, the percentage of female students, those with an immigrant background and those who speak a language other than the assessment at home is the same in previous assessments as those observed in 2012. For more details on the calculation of the adjusted annualised change, see Annex A5. OECD average 2000 considers only those countries with comparable reading scores since PISA 2000. Countries and economies are ranked in descending order of the annualised change after accounting for demographic changes. Source: OECD, PISA 2012 Database, Tables I.4.3b and I.4.4. 1 2 http://dx.doi.org/10.1787/888932935610 : Korea a S pi mproving in i Box I.4.1. Korea has consistently performed at the top level in PISA, and has still improved over time. In PISA 2000, Korea performed on a par with New Zealand, Sweden, Australia, Hong Kong-China, Japan and Ireland; by 2012 Korea outperformed the first three. Performance in reading, for example, has improved by an average of almost one score point per year since 2000. As a result, Korea’s average score in reading increased from 525 points in 2003 to 536 points in 2012. This improvement was concentrated at the top of the performance distribution: the percentage of students scoring at or above proficiency Level 5 in mathematics increased by more than eight percentage points since 2000 to 14% in 2012. While the mathematics scores among the top 10% of students have improved by more than 30 points during the period, no change was observed among low-achieving students. Korea’s performance in science also improved consistently throughout its participation in PISA: science performance increased by an average of 2.6 points per year since 2006 so that average scores in science rose from 522 points in PISA 2006 to 538 points in PISA 2012. ... OECD 2014 athemati e – Volume C ien C eading and S r , m © i What Student S Kno W and Can d o: Student Performan C e in CS 188

191 4 ding f Student Perform A nce i n r e A o A Profile Korea’s improvements in reading were concentrated among high-achieving students. the average improvement of high-achieving students outpaced that of lower-achieving students. Higher standards in language literacy were put in place in the mid-2000s, and language literacy was given more weight in the competitive College Scholastic Ability Test (CSAT), the university entrance examination. This could explain the increase in the share of top- performing students in Korea, as high-achieving students have more incentives to invest in language and reading literacy. Also, and particularly since 2010, programmes for gifted students have been expanded at the primary and secondary levels, and the secondary curriculum has been strengthened to meet the needs of these students (MEST, 2010). Education policies have been linked to macroeconomic development first through centralised planning (1962-91) then by co-ordinated and strategically oriented approaches through the National Human Resource Development Plans (one for 2001-05 and another for 2006-10, for example). They have followed a sequential approach. Prior to 1975, 65% of the education budget was spent on primary education; in the following decades, secondary education received a greater share of funding and by the late 1990s, public investment in tertiary education was expanded. In the mid-1990s, a comprehensive school reform was launched, introducing school deregulation, choice, a new curriculum and increased public expenditure. Individual schools began to assume more management responsibilities. By 2012, schools had greater autonomy, and programmes were specifically designed to assist school leaders in assuming their new roles (World Bank, 2010). The National Assessment of Educational Achievement (NAEA) programme was introduced in 1998. NAEA assesses educational achievement and trends among all 6th-, 9th- and 10th-grade students in Korean Language Arts, English, mathematics, social studies and science. Since 2010, the programme changed the grade coverage from 6th-, 9th- and 10th to 6th-, 9th- and 11th. The Subject Learning Diagnostic Test (SLDT) was introduced in 2008 and is implemented by the Nationwide Association of Superintendents of metropolitan/provincial offices of education. The previous Diagnostic Evaluation of Basic Academic Competence (DEBAC), which had tested primary school 3rd grades at the national level since 2002, was delegated to metropolitan/provincial offices of education. The Subject Learning Diagnostic Test measures basic competency in reading, writing and mathematics among 3rd, 4th-, 5th-, 7th- and 8th-grade students. Through these assessment tools, the government and metropolitan/provincial offices can monitor individual student performance levels, establish achievement benchmarks, develop an accountability system for public education, and also identify students who need support. For example, in 2008, the government established the , a national programme to ensure that all students meet basic Zero Plan for Below-Basic Students achievement criteria. The NAEA assessment was converted from a sample-based test to a census-based test to Schools for Improvement (SFI) policy identify and then support low-performing students. Also, MEST introduced a in 2009 to provide support in closing education gaps and improving achievement, also with the aim of reducing the proportion of students who do not achieve basic proficiency. The SFI supports various education programmes, including providing more resources for low-income schools and schools with a high concentration of low- performing students (Kim et al., 2012). The national curriculum was revised again in 2009, highlighting reasoning, problem solving and mathematical communication as key competencies in mathematics (MEST, 2011b). In 2012, the government announced a plan for improving mathematics education in keeping with the revised curriculum. The aim is to enhance skills in reasoning and creativity (MEST, 2012). This reform implies a profound change in the way teachers teach mathematics: up until now, teachers have largely taught to the CSAT. Reforms have also affected the teaching of language and reading. The focus of the Korean Language Arts Curriculum shifted from proficiency in grammar and literature to skills and strategies needed for creative and critical understanding and representation, similar to the approach underlying PISA. Diverse teaching methods and materials that reflected those changes were developed, and investments were made in related digital and Internet infrastructure. Schools were requested to spend a fixed share of their budgets on reading education. Training programmes for reading teachers were developed and disseminated. Parents were encouraged to participate more in school activities and were given information on how to support their children’s schoolwork. In both 2009 and 2012 Korea was among the OECD countries with the largest classes and, since 2003, Korean students have also been more likely to attend schools where the principal reported a teacher shortage. A concerted effort is underway to create more teaching posts. In 2010, more than 53 000 new jobs were assigned to the ... OECD 2014 CS S Kno W and Can d o: Student Performan C e in m 189 What Student © i e – Volume C ien C eading and S r , athemati

192 4 o A nce i n f Student Perform e A ding A Profile r education-services sector, including 2 000 English conversation lecturers, 7 000 intern teachers, who support instruction, 7 000 after-school lecturers and co-ordinators, 5 500 full-day kindergarten staff, and 5 000 special education assistants. The teacher-training system has been expanded to enable outside experts to acquire teaching certificates (MEST, 2010; 2011a). The school- and teacher-evaluation systems have also been reformed. Since 2010, the teacher-evaluation system, which was developed to improve teachers’ professional capacities, was expanded to all schools. Results from the evaluation lead to customised training programmes for teachers, depending on their results. Given the greater autonomy granted to school principals, evaluation information will be made public and regional offices of education will oversee monitoring, focusing more on output-oriented criteria. Schools will use internal assessments to measure the improvement of students who do not meet the national assessment benchmarks. School-based performance-award systems were introduced in 2011 (MEST, 2011). Fifteen-year-old students in Korea spent an average of 30 minutes less in mathematics classes in 2012 than their counterparts in 2003 did, yet a large number of Korean students participate in after-school lessons. While private lessons are common among those who can afford them, after-school group classes are often subsidised, so even disadvantaged students frequently enrol. For example, in June 2011, 99.9% of all primary and secondary schools were operating after-school programmes and about 65% of all primary and secondary students participated in after- school activities (MEST, 2011c). Many observers suspect that the high participation rates in after-school classes may be due to cultural factors and an intense focus on preparing for university entrance examinations. PISA 2006 data show that Korean students attending schools with socio-economically advantaged students are more likely to attend after-school lessons with private teachers than students in other countries; and disadvantaged students in Korea are more likely to attend after-school group lessons than disadvantaged students in other countries. In both cases, attendance in these lessons, along with other factors, is associated with better performance on PISA (OECD, 2010). Sources: Kim K., H. Kim, W. Roh, K. Sang, J. Shin, H. Jung, S. Woo, J.S. Ryoo, J. Han, S. Lauver, C. McClure, M. Cairns, A. Kanter, B. Fu, D. Yi (2012), Korea- (CRE 2012-12-2). KICE, Seoul. S bilateral study on turnaround schools u (in Korean), MEST, Seoul. Plans for advancing mathematics education Ministry of Education, Science and Technology (2012), , MEST, Seoul. Major Policies and Plans for 2011 Ministry of Education, Science and Technology (2011a), Ministry of Education, Science and Technology (2011b), Mathematical curriculum (in Korean), MEST, Seoul. Ministry of Education, Science and Technology (2011c), 2011 Analysis for after school programme (in Korean), MEST, Seoul. Ministry of Education, Science and Technology (2010), , MEST, Seoul. Major Policies and Plans for 2010 Quality Time for Students: Learning in and out of school OECD (2011), , PISA, OECD Publishing. http://dx.doi.org/10.1787/9789264087057-en World Bank (2010), Quality of Education in Colombia, Achievements and Challenges Ahead: Analysis of the Results of TIMSS 1995-2007, World Bank, Washington, D.C. Students at the different levels of proficiency in reading The seven proficiency levels used in the PISA 2012 reading assessment are the same as those established for the 2009 PISA assessment, when reading was the major area of assessment: Level 1b is the lowest described level, then Level 1a, Level 2, Level 3 and so on up to Level 6. Figure I.4.8 provides details of the nature of the reading skills, knowledge and understanding required at each level of the reading scale. The tasks related to each proficiency level are described according the three processes that students use to answer the questions. These three processes are classified as access (skills associated with finding, selecting and collecting information), (processing what integrate and interpret and retrieve is read to make sense of a text), and reflect and evaluate (drawing on knowledge, ideas or values external to the text). Figure I.4.9 shows a map of some questions in relation to their position on the reading proficiency scale. The first column shows the proficiency level within which the task is located. The second column indicates the lowest score on the task that would still be described as achieving the given proficiency level. The last column shows the name of the unit, the question number and, within parentheses, the score given for the correct response to these questions. The selected questions have been ordered according to their difficulty, with the most difficult at the top, and the least difficult at the bottom. o: Student Performan eading and S © OECD 2014 What Student S Kno W and Can d C C e in m athemati CS , i e – Volume C ien r 190

193 4 o A nce i n r e A ding f Student Perform A Profile • Figure I.4.8 • 2012 a S pi Summary description for the seven levels of proficiency in print reading in Percentage of students l able to perform tasks ower at each level or above e scor cd average) c haracteristics of tasks limit o ( evel E l 6 Tasks at this level typically require the reader to make multiple inferences, 1.1% 698 comparisons and contrasts that are both detailed and precise. They require demonstration of a full and detailed understanding of one or more texts and may involve integrating information from more than one text. Tasks may require the reader to deal with unfamiliar ideas, in the presence of prominent competing information, and to generate abstract categories for interpretations. Reflect and evaluate tasks may require the reader to hypothesise about or critically evaluate a complex text on an unfamiliar topic, taking into account multiple criteria or perspectives, and applying sophisticated understandings from beyond the text. A salient condition for access and retrieve tasks at this level is precision of analysis and fine attention to detail that is inconspicuous in the texts. 5 Tasks at this level that involve retrieving information require the reader to locate and 8.4% 626 organise several pieces of deeply embedded information, inferring which information in the text is relevant. Reflective tasks require critical evaluation or hypothesis, drawing on specialised knowledge. Both interpretative and reflective tasks require a full and detailed understanding of a text whose content or form is unfamiliar. For all aspects of reading, tasks at this level typically involve dealing with concepts that are contrary to expectations. 4 Tasks at this level that involve retrieving information require the reader to locate and 29.5% 553 organise several pieces of embedded information. Some tasks at this level require interpreting the meaning of nuances of language in a section of text by taking into account the text as a whole. Other interpretative tasks require understanding and applying categories in an unfamiliar context. Reflective tasks at this level require readers to use formal or public knowledge to hypothesise about or critically evaluate a text. Readers must demonstrate an accurate understanding of long or complex texts whose content or form may be unfamiliar. 3 Tasks at this level require the reader to locate, and in some cases recognise the 480 58.6% relationship between, several pieces of information that must meet multiple conditions. Interpretative tasks at this level require the reader to integrate several parts of a text in order to identify a main idea, understand a relationship or construe the meaning of a word or phrase. They need to take into account many features in comparing, contrasting or categorising. Often the required information is not prominent or there is much competing information; or there are other text obstacles, such as ideas that are contrary to expectation or negatively worded. Reflective tasks at this level may require connections, comparisons, and explanations, or they may require the reader to evaluate a feature of the text. Some reflective tasks require readers to demonstrate a fine understanding of the text in relation to familiar, everyday knowledge. Other tasks do not require detailed text comprehension but require the reader to draw on less common knowledge. 2 Some tasks at this level require the reader to locate one or more pieces of information, 407 82.0% which may need to be inferred and may need to meet several conditions. Others require recognising the main idea in a text, understanding relationships, or construing meaning within a limited part of the text when the information is not prominent and the reader must make low level inferences. Tasks at this level may involve comparisons or contrasts based on a single feature in the text. Typical reflective tasks at this level require readers to make a comparison or several connections between the text and outside knowledge, by drawing on personal experience and attitudes. 1a Tasks at this level require the reader to locate one or more independent pieces of 335 94.3% explicitly stated information; to recognise the main theme or author’s purpose in a text about a familiar topic, or to make a simple connection between information in the text and common, everyday knowledge. Typically the required information in the text is prominent and there is little, if any, competing information. The reader is explicitly directed to consider relevant factors in the task and in the text. 1b Tasks at this level require the reader to locate a single piece of explicitly stated 98.7% 262 information in a prominent position in a short, syntactically simple text with a familiar context and text type, such as a narrative or a simple list. The text typically provides support to the reader, such as repetition of information, pictures or familiar symbols. There is minimal competing information. In tasks requiring interpretation the reader may need to make simple connections between adjacent pieces of information. CS OECD 2014 S Kno W and Can d o: Student Performan C e in m athemati What Student , r eading and S C ien C e – Volume i © 191

194 4 A Profile f Student Perform A nce i n r e A ding o • Figure I.4.9 • m ap of selected reading questions, by proficiency level l ower scor e evel S limit unit S - Questions (position on P i l a scale) 6 ’S THE THING – Question 3 (730) THE PLA y 698 5 – Question 16 (631) LABOUR 626 4 BALLOON – Question 3.2 (595) 553 y THE PLA – Question 7 (556) ’S THE THING 3 MISER – Question 5 (548) 480 BALLOON – Question 4 (510) 2 THE PLA y – Question 4 (474) ’S THE THING 407 BALLOON – Question 3.1 (449) BALLOON – Question 6 (411) 1a MISER – Question 1 (373) 335 BALLOON – Question 8 (370) 1b – Question 7 (310) MISER 262 Figure I.4.10 shows the distribution of students among these different proficiency levels in each participating country and economy. Table I.4.1a shows the percentage of students at each proficiency level on the reading scale, with standard errors. Proficiency at Level 6 (score higher than 698 points) Tasks at Level 6 typically require the student to make multiple inferences, comparisons and contrasts that are both detailed and precise. They require demonstration of a full and detailed understanding of one or more texts and may integrating information from more than one text. Tasks may require the student to deal with unfamiliar ideas in involve . interpretations Reflect-and- the presence of prominent competing information, and to generate abstract categories for evaluate tasks may require the student to hypothesise about or critically evaluate a complex text on an unfamiliar topic, taking into account multiple criteria or perspectives, and applying sophisticated understandings from beyond the text. tasks at this level require precise analysis and fine attention to detail that is inconspicuous in the Access-and-retrieve texts. Level 6 tasks are illustrated by Question 3 from the unit THE PLAY’S THE THING (Figure I.4.14). The text is long, by PISA standards, and it may be supposed that the fictional world depicted is remote from the experience of most P HE is the beginning HING T THE S ’ LAY 15-year-olds. The introduction to the unit tells students that the stimulus of T of a play by the Hungarian dramatist Ferenc Molnár, but there is no other external orientation. The setting (“a castle by the beach in Italy”) is likely to be exotic to many, and the situation is only revealed gradually through the dialogue itself. While individual pieces of vocabulary are not particularly difficult, and the tone is often chatty, the register of the language is a little mannered. Perhaps most important, a level of unfamiliarity is introduced by the abstract theme of the discussion: a sophisticated conversation between characters about the relationship between life and art, and the challenges of writing for the theatre. The text is classified as narration because a story is told through the dialogue of the play. A high level of interpretation skills is required to define the meaning of the question’s terms. The student needs to be alert to the distinction between characters and actors. The question refers to what the characters (not the actors) were doing “just before the curtain went up”. This is potentially confusing since it requires recognition of a shift between the real world of a stage in a theatre, which has a curtain, and the imaginary world of Gal, Turai and Adam, who were in the dining room having dinner just before they entered the guest room (the stage setting). A question that assesses students’ capacity to distinguish between real and fictional worlds seems particularly appropriate in relation to a text whose theme is about just that, so that the complexity of the question is aligned with the content of the text. d C C eading and S r , e – Volume athemati m e in C o: Student Performan ien and Can W Kno S What Student OECD 2014 © i CS 192

195 4 o A nce i n f Student Perform e A ding A Profile r In addition, the information required to complete the task is in an unexpected location. The question refers to the action “before the curtain went up”, which would typically lead one to search at the opening of the scene, the beginning of the extract. But the information is actually found about half-way through the extract, when Turai reveals that he and his friends “have just arrived from the dining room”. While the scoring for the question shows that several kinds of response are acceptable, to be given full credit students must demonstrate that they have found this inconspicuous piece of information. The need to assimilate information that is contrary to expectations is characteristic of the most demanding reading tasks in PISA. Across OECD countries, around 1% of students performs at Level 6 in reading, but there is some variation among countries. Three percent of students or more perform at this level in Singapore (5.0%), Japan (3.9%), Shanghai-China (3.8%) and New Zealand (3.0%). In France, Finland and Canada between 2% and 3% of students attain proficiency Level 6. In contrast, 0.1% of students or fewer perform at Level 6 in Romania, Albania, Argentina, Thailand, Montenegro, Uruguay, Mexico, Chile, Brazil, Peru, Costa Rica, Jordan, Tunisia, Colombia, Indonesia, Kazakhstan and Malaysia (Figure I.4.10 and Table I.4.1a). Proficiency at Level 5 (score higher than 626 but lower than or equal to 698 points) retrieving information require the student to locate and organise several pieces of deeply Tasks at Level 5 that involve embedded information, inferring which information in the text is relevant. tasks require critical evaluation Reflective reflective or hypotheses, drawing on specialised knowledge. Both interpreting and tasks require a full and detailed understanding of a text whose content or form is unfamiliar. For all aspects of reading, tasks at this level typically involve dealing with concepts that are contrary to expectations. (Figure I.4.15) is an example of a task at Level 5. In fact, this task yields two levels of Question 16 in the unit LABOUR difficulty: the full-credit response category falls within Level 5, with a PISA score of 631 points; and the partial-credit access category falls within Level 3, with a PISA score of 485 points. The full-credit response category illustrates that items, like items from the other two aspect categories ( ), can reflect and evaluate and integrate and interpret and retrieve pose a significant challenge. For full credit (Level 5), students are required to locate and combine a piece of numerical information in the main body of the text (the tree diagram) with information in a footnote – that is, outside the main body of the text. In addition, students have to use this footnoted information to determine the correct number of people who fit into this category. Both of these features contribute to the difficulty of this task. For partial credit (Level 3), this task merely requires students to locate the number given in the appropriate category of the tree diagram; they are not required to use the information provided in the footnote. Even without this important information, the task is still moderately difficult. The requirement to use information found outside the main body of a text – significantly increases the difficulty of a task. This is clearly demonstrated by the two categories of this task, since the difference between full-credit and partial-credit answers involves applying – or not applying – information from a footnote to correctly identified numerical information in the body of the text. The difference in difficulty between these two categories of response is more than two proficiency levels. Across OECD countries, 8.4% of students are top performers, meaning that they are proficient at Level 5 or 6. - China has the largest proportion of top performers – 25.1% – among all participating countries and economies. Shanghai More than 15% of students in Singapore, Japan and Hong Kong-China are top performers in reading as are more than 10% of students in Korea, New Zealand, Finland, France, Canada, Belgium, Chinese Taipei, Australia, Ireland, Liechtenstein and Norway. In 15 countries and economies fewer than 1% of students perform at Level 5 or 6. With the exception of Mexico, Chile, Turkey and the Slovak Republic, more than 5% of students in every OECD country attains at least Level 5 (Figure I.4.10 and Table I.4.1a). Proficiency at Level 4 (score higher than 553 but lower than or equal to 626 points) Tasks at Level 4 that involve information require the student to locate and organise several pieces of embedded retrieving information. Some tasks at this level require the meaning of nuances of language in a section of text by interpreting taking into account the text as a whole. Other interpretative tasks require understanding and applying categories in an tasks at this level require the student to use formal or public knowledge to hypothesise Reflective unfamiliar context. about or critically evaluate a text. The student must demonstrate an accurate understanding of long or complex texts whose content or form may be unfamiliar. CS S Kno W and Can d o: Student Performan C e in m athemati What Student , r eading and S C ien C e – Volume i 193 OECD 2014 ©

196 4 f Student Perform nce i n r A A ding o A Profile e • • Figure I.4.10 p roficiency in reading P ercentage of students at each level of reading proficiency Level 3 Below Level 1b Level 2 Level 6 Level 1a Level 4 Level 5 Level 1b Shanghai-China Shanghai-China Hong Kong-China Hong Kong-China Korea Korea Estonia Estonia Viet Nam Viet Nam Ireland Ireland Japan Japan Singapore Singapore Students at Level 1a Poland Poland or below Canada Canada Finland Finland Macao-China Macao-China Chinese Taipei Chinese Taipei Liechtenstein Liechtenstein Switzerland Switzerland Netherlands Netherlands Australia Australia Germany Germany Denmark Denmark Belgium Belgium Norway Norway New Zealand New Zealand United States United States United Kingdom United Kingdom Czech Republic Czech Republic Latvia Latvia OECD average OECD average Spain Spain Croatia Croatia Portugal Portugal France France Austria Austria Italy Italy Hungary Hungary Iceland Iceland Slovenia Slovenia Lithuania Lithuania Turkey Turkey Luxembourg Luxembourg Russian Federation Russian Federation Greece Greece Sweden Sweden Israel Israel Slovak Republic Slovak Republic Costa Rica Costa Rica Thailand Thailand Chile Chile Serbia Serbia United Arab Emirates United Arab Emirates Romania Romania Bulgaria Bulgaria Mexico Mexico Montenegro Montenegro Uruguay Uruguay Brazil Brazil Tunisia Tunisia Jordan Jordan Colombia Colombia Albania Albania Malaysia Malaysia Argentina Argentina Indonesia Indonesia Students at Level 2 Kazakhstan Kazakhstan or above Qatar Qatar Peru Peru % % 20 60 80 100 40 40 20 80 100 0 60 Countries and economies are ranked in descending order of the percentage of students at Levels 2, 3, 4, 5 and 6. Source: OECD, PISA 2012 Database, Table I.4.1a. 1 http://dx.doi.org/10.1787/888932935610 2 C r OECD 2014 What Student S Kno W and Can d o: Student Performan © e in m athemati CS , eading and S C ien C e – Volume i 194

197 4 o A nce i n f Student Perform e A ding A Profile r Question 7 in the example THE PLAY’S THE THING (Figure I.4.14) requires Level 4 proficiency. In this task, the student is asked to take a global perspective, forming a broad understanding by integrating and interpreting the implications of the dialogue in the text. The task involves recognising the conceptual theme of a section of a play, where the theme is literary and abstract. The difficulty of the task largely stems from the abstract nature of the dialogue. A little under half of the students in OECD countries earned full credit for this task, with the others divided fairly evenly across the three other proposed answers. Across OECD countries, an average of around 30% of students are proficient at Level 4 or higher (that is, proficient at Level 4, 5 or 6). In Hong Kong-China, Singapore, Japan, Korea, Chinese Taipei and Finland between 40% and 50% of students attain these levels; in Shanghai-China, more than 60% of students do. In more than half of all participating countries and economies, more than one in four students performs at Level 4 or higher. However, in the partner countries and economies Kazakhstan, Indonesia, Malaysia, Jordan, Colombia, Peru, Tunisia, Argentina, Mexico and Brazil, fewer than 5% of students attain at least this level (Figure I.4.10 and Table I.4.1a). Proficiency at Level 3 (score higher than 480 but lower than or equal to 553 points) Tasks at Level 3 require the student to retrieve , and in some cases recognise the relationship between, several pieces of several tasks at this level require the student to Interpreting information that must meet multiple conditions. integrate parts of a text in order to identify a main idea, understand a relationship or construe the meaning of a word or phrase. The student needs to take into account many features in comparing, contrasting or categorising. Often the required information is not prominent or there is much competing information; or there are other obstacles in the text, such tasks at this level may require connections, as ideas that are contrary to expectation or negatively worded. Reflective comparisons and explanations, or they may require the student to evaluate a feature of the text. Some reflective tasks require the student to demonstrate a fine understanding of the text in relation to familiar, everyday knowledge. Other tasks do not require detailed text comprehension but ask the student to draw on less common knowledge. Question 5 in MISER (Figure I.4.17), a task at Level 3, requires an open-constructed response. The task sets up a dialogue between two imaginary readers representing two conflicting interpretations of the story. In fact, only the second speaker’s position is consistent with the overall implication of the text, so that in providing a supporting explanation, readers demonstrate that they have understood the “punch line” – the moral import – of the fable. The relative difficulty of the task, among the most difficult questions at Level 3, is likely to be influenced by the fact that students need to do a good deal of work to generate a full-credit response. First they must make sense of the neighbour’s speech in the story, which is expressed in a formal register. (Translators were asked to reproduce the fable-like style.) Secondly, the relationship between the question stem and the required information is not obvious: there is little or no support in the stem (“What could Speaker 2 say to support his point of view?”) to guide the reader in interpreting the task, though the reference to the stone and the neighbour by the speakers should point the reader to the end of the fable. To gain full credit, students could express, in a variety of ways, the key idea that wealth has no value unless it is used (see examples of answers in Figure I.4.17). Vague gestures at meaning, such as “the stone had a symbolic value”, are not given credit. Across OECD countries, 59% of students are proficient at Level 3 or higher (that is, proficient at Level 3, 4, 5 or 6). In Shanghai-China (86.1%), Hong Kong-China (78.9%) and Korea (76.0%) more than three out of four 15-year-olds are proficient at Level 3 or higher, and at least two out of three students attain this level in Japan, Singapore, Ireland, Chinese Taipei, Canada, Finland, Estonia, Poland and Viet Nam. In contrast, in 13 countries and economies (Kazakhstan, Indonesia, Peru, Malaysia, Colombia, Jordan, Argentina, Tunisia, Brazil, Qatar, Albania, Uruguay and Mexico) three out of four students do not attain this level (Figure I.4.10 and Table I.4.1a). Proficiency at Level 2 (score higher than 407 but lower than or equal to 480 points) Level 2 can be considered a baseline level of proficiency at which students begin to demonstrate the reading literacy competencies that will enable them to participate effectively and productively in life. The 2009 Canadian Youth in Transition Survey, which followed up students who were assessed by PISA in 2000, shows that students scoring below Level 2 face a disproportionately higher risk of poor post-secondary participation or low labour-market outcomes at age 19, and even more so at age 21, the latest age for which data from this longitudinal study are available (OECD, 2010a). retrieve one or more pieces of information that may have to be inferred and Some tasks at Level 2 require the student to may have to meet several conditions. Others require recognising the main idea in a text, understanding relationships, or CS eading and S S Kno W and Can d o: Student Performan C e in m athemati What Student , 195 OECD 2014 © i e – Volume C ien C r

198 4 o A nce i n f Student Perform e A ding A Profile r meaning within a limited part of the text when the information is not prominent and the student must make interpreting low-level inferences. Tasks at this level may involve integrating parts of the text through comparisons or contrasts based tasks at this level require the student to make a comparison or several on a single feature in the text. Typical reflective connections between the text and outside knowledge by drawing on personal experience and attitudes. Question 6 in BALLOON (Figure I.4.16), a task that corresponds to the bottom of Level 2 in difficulty, uses a multiple- because it asks about authorial intent. It focuses on a graphic choice format. This task is classified under reflect and evaluate element – the illustration of two balloons – and asks students to consider the purpose of this inclusion. In the context of the over-arching idea of the text, to describe (and celebrate) Singhania’s flight, the balloon illustration sends the message, “This is a really big balloon!”, just as the jumbo jet illustration sends the message, “This is a really high flight!”. Across OECD countries, an average of 82% of students is proficient at Level 2 or higher. In Shanghai-China, Hong Kong China, Korea, Estonia, Viet Nam, Ireland, Japan and Singapore more than 90% of students perform at or - abo ve this threshold. In Shanghai-China, fewer than 3% of students do not attain this level. In 34 participating countries and economies between 75% and 90% of students achieve the baseline level of reading proficiency, and in 14 countries and economies between 50% and 75% do so. Only in Peru, Qatar, Kazakhstan, Indonesia, Argentina, Malaysia, Albania, Colombia and Jordan, does fewer than one in two students perform at this level. In every OECD country except Mexico (58.9%), Chile (67.0%) and the Slovak Republic (71.8%), at least three out of four students perform at Level 2 or above (Figure I.4.10 and Table I.4.1a). Proficiency at Level 1a (score higher than 335 but lower than or equal to 407 points) one or more independent pieces of explicitly stated information, retrieve Tasks at Level 1a require the student to interpret reflecting on the the main theme or author’s intent in a text about a familiar topic, or make a simple connection by relationship between information in the text and common, everyday knowledge. The required information in the text is usually prominent and there is little, if any, competing information. The student is explicitly directed to consider relevant factors in the task and in the text. Question 8 in the unit BALLOON (Figure I.4.16) is typical of Level 1a tasks. The main idea of this non-continuous text is stated explicitly and prominently several times, including in the title, “Height record for hot air balloon”. Although the , with the sub-classification main idea is explicitly stated, the question is classified as forming a integrate and interpret broad understanding , because it involves distinguishing the most significant and general information from subordinate information in the text. Across OECD countries, an average of 18% of students is proficient only at or below Level 1a, and nearly 6% of students do not even attain Level 1a. Fewer than 10% of students perform at Level 1a or below in Shanghai-China, - China, Korea, Estonia, Viet Nam, Ireland, Japan and Singapore. In Shanghai-China, fewer than 1% of students Hong Kong (0.4%) do not reach Level 1a. In Estonia, Hong Kong-China, Viet Nam and Liechtenstein fewer than 2% of students do not reach Level 1a, and in Ireland, Korea, Singapore, Macao-China, Poland and Canada fewer than 3% of students do not reach this level. By contrast, in 20 participating countries and economies more then one in three students performs at Level 1a or below. In Peru, Qatar, Kazakhstan, Indonesia, Argentina, Malaysia, Albania, Colombia and Jordan more than half of all students are proficient only at or below Level 1a (Figure I.4.10 and Table I.4.1a). Proficiency at Level 1b (score higher than 262 but lower than or equal to 335 points) Tasks at Level 1b require the student to a single piece of explicitly stated information in a prominent position retrieve in a short, syntactically simple text with a familiar context and text type, such as a narrative or a simple list. The text typically provides support to the student, such as repetition of information, pictures or familiar symbols. There is minimal competing information. In tasks requiring interpretation, the student may need to make simple connections between adjacent pieces of information. Question 7 in MISER (Figure I.4.17), a task at Level 1b, requires a short response. This is one of the easiest tasks in the PISA reading assessment. The student is required to access and retrieve a piece of explicitly stated information in the opening sentence of a very short text. To gain full credit, the response can either quote directly from the text or provide a paraphrase. The formal language of the text, which is likely to have added difficulty in other tasks in the unit, is unlikely to have much impact here because the required information is located at the very beginning of the text. Although this is a very easy question, it still requires a small degree of inference: the reader must infer that there is a causal connection between the first proposition (that the miser sold all he had) and the second (that he bought gold). o: Student Performan ien © OECD 2014 What Student S Kno W and Can d C C e in m athemati CS , r eading and S i e – Volume C 196

199 4 A f Student Perform A nce i n r e o ding A Profile Across OECD countries, 1.3% of students are not proficient at Level 1b, but there are wide differences between countries. In Liechtenstein, Shanghai-China, Viet Nam, Estonia, Hong Kong-China, Ireland, Poland, Macao-China and Korea fewer than 0.5% of students perform at this level. Across all participating countries and economies, except Malaysia, Tunisia, Uruguay, Jordan, Bulgaria, Argentina, Peru, Albania and Qatar, fewer than 5% of students are not proficient at Level 1b (Figure I.4.10 and Table I.4.1a). Students with scores below 262 points – that is, below Level 1b – usually do not succeed at the most basic reading tasks that PISA measures. This does not necessarily mean that they are illiterate, but that there is insufficient information on which to base a description of their reading proficiency. Such students are likely to have serious difficulties in benefitting from further education and learning opportunities throughout life (OECD, 2010a). Trends in the percentage of low- and top-performers in reading PISA assesses the reading competencies required for students to participate fully in a knowledge-based society. These range from very complex skills that only a few students have mastered up to the baseline skills that are considered the minimum required for functioning in society. The proportion of students who do not meet this baseline proficiency (Level 2; low-performing students) and the proportion of students who are able to understand and communicate complex tasks (Levels 5 and 6; top-performing students) are important indicators of the needs and challenges faced by each country or economy and benchmarks of the level of skills development. Changes in a country’s or economy’s average performance can result from improvements or deterioration of performance at different points in the performance distribution. For example, in some countries and economies the average improvement is observed among all students, resulting in fewer students who perform below Level 2 and more students who are top performers. In other contexts, the average improvement can mostly be attributed to large improvements among low-achieving students with little or no change among high-achieving students; this may results in a smaller proportion of low-performing students, but no increase among top performers. Trends in the proportion of low- and top- performing students indicate where the changes in performance have occurred and the extent to which school systems are advancing towards providing all students with the minimum literacy skills and towards producing a larger proportion of students with the highest-level skills in reading. Countries and economies can be grouped into categories according to whether they have: simultaneously reduced the share of low performers and increased the share of top performers between any previous PISA assessment and PISA 2012; reduced the share of low performers but not increased the share of top performers between any previous PISA assessment and PISA 2012; increased the share of top performers but not reduced the share of low performers; and reduced the share of top performers or increased the share of low performers between PISA 2012 and any previous PISA assessment. The following section categorises countries and economies into these groups. Moving everyone up: Reduction in the share of low performers and increase in that of top performers Between the PISA 2000 and PISA 2012 assessments, Albania, Israel and Poland saw an increase in the share of students who meet the highest proficiency levels in PISA and a simultaneous decrease in the share of students who do not meet the baseline proficiency level. In Israel, for example, the share of students performing below Level 2 shrank by almost ten percentage points (from 33% to 24%) between 2000 and 2012, while the share of students performing at or above proficiency Level 5 grew by more than five percentage points (from 4% to 10%) (Figure I.4.11 and Table I.4.1b). The system- level improvements observed in these countries and economies have lifted students out of low performance and others into top performance. The same trend was observed in Hong Kong-China, Japan and the Russian Federation since PISA 2003; in Bulgaria, Chinese Taipei, Qatar, Serbia and Spain since PISA 2006; and in Ireland, Luxembourg, Macao-China and Singapore since PISA 2009. In Turkey, the share of low performers shrank when comparing PISA 2003 or PISA 2006 with PISA 2012, and the share of top performers increased when comparing PISA 2009 with PISA 2012 (Table I.4.1b). For many of these countries and economies, these trends in the share of low and top performers mirror how students at different levels of the performance distribution have changed their performance. Annex B4 shows how, for each country and economy, the 10th, 25th, 75th and 90th percentiles of performance have evolved across different PISA cycles. It shows, consistent with trends in the share of low- and top-performing students, that in Poland, the low-achieving students (those in the bottom 25th percentile) improved their reading performance by 61 score points and the highest-achieving students (those in the 90th percentile) also improved by more than 20 score points. Other countries that saw annualised improvements on average and among both the lowest- and highest-achieving students are Albania, Brazil, Chile, Estonia, Hong Kong-China, Hungary, Indonesia, Italy, Japan, Montenegro, Mexico, Peru, Qatar, the Russian Federation, Serbia, Portugal, Spain, Switzerland, Thailand and Tunisia (Table I.4.3d). The average annual improvement observed in these © m S Kno W and Can d o: Student Performan C 197 OECD 2014 What Student i e – Volume C ien C eading and S r , CS athemati e in

200 4 ding A nce i n r e A f Student Perform A Profile o countries is shared by high- and low-achieving students, but not all these countries were able to both increase the share of students performing at or above Level 5 and reduce the share of students performing below Level 2. Reducing underperformance: Reductions in the share of low performers but no change in that of top performers Other countries and economies have seen improvements in the performance of their low-performing students. For example, since PISA 2000, Peru, Indonesia, Chile and Latvia have reduced the share of students performing below Level 2 in reading by more than 10 percentage points with no concurrent change in the share of students who perform at or above proficiency Level 5. Liechtenstein, Germany, Portugal and Switzerland show a reduction of more than five percentage points in the share of students performing below Level 2 between 2000 and 2012. Significant reductions in the proportion of low-performing students are also observed in Italy, Mexico, Thailand and Tunisia since 2003, in Brazil, the Czech Republic, Greece, Montenegro and Norway since PISA 2006 and in Dubai (United Arab Emirates) since PISA 2009 (Figure I.4.11 and Table I.4.1b). In these countries and economies, improvements in performance have reached those students that needed it the most. Annex B4 shows the performance trajectories of these countries and economies, highlighting how the performance of their lowest achievers (those students in the 10th percentile of performance) shows greater improvements than the performance of their highest-achieving students (those in the 90th percentile). Figure I.4.11 • • p ercentage of low-performing students and top performers in reading in 2000 and 2012 2000 2012 90 Students at or above prociency Level 5 80 70 60 50 40 Percentage of students 30 20 10 0 4.4 8.4 5.5 4.1 1.1 8.6 2.1 7.3 -2.7 -4.8 -5.9 -4.3 -3.3 -3.3 -3.9 -1.2 -5.0 Italy Peru Chile Israel Spain Brazil Japan Latvia Korea France Poland Austria Ireland Greece Finland Iceland Mexico Albania Canada Norway Sweden Bulgaria Belgium Portugal Thailand Hungary Australia Romania Germany Denmark Argentina Indonesia Switzerland New Zealand Liechtenstein United States Czech Republic Hong Kong-China Russian Federation OECD average 2000 4.4 3.7 6.5 -8.1 -6.7 -1.6 -9.8 -9.6 -7.4 10.1 -12.7 -15.2 -18.0 -13.1 -13.4 -19.7 0 10 20 30 40 50 Percentage of students 60 70 80 Students below prociency Level 2 90 Notes: The chart shows only countries/economies that participated in both PISA 2000 and PISA 2012 assessments. The change between PISA 2000 and PISA 2012 in the share of students performing below Level 2 in reading is shown below the country/economy name. The change between PISA 2000 and PISA 2012 in the share of students performing at or above Level 5 in reading is shown above the country/economy name. Only statistically signicant changes are shown (see Annex A3). OECD average 2000 compares only OECD countries with comparable reading scores since 2000. Countries are ranked in descending order of the percentage of students at or above prociency Level 5 in reading in 2012. and economies Source: OECD, PISA 2012 Database, Table I.4.1b. 1 2 http://dx.doi.org/10.1787/888932935610 S What Student m e in © o: Student Performan d athemati CS , r eading and S C ien C e – Volume i and Can W C Kno OECD 2014 198

201 4 A f Student Perform A nce i n r e o ding A Profile Nurturing top performance: Increase in the share of top performers but no change in that of low performers France and Korea saw growth in the share of top-performing students in reading since PISA 2000 with no concurrent reduction in the share of low-performing students. Korea, for example, saw an increase of eight percentage points in the share of students performing at or above Level 5 (from 6% in 2000 to 14% in 2012). This trend is also observed in in Shanghai-China since PISA 2009 (Figure I.4.11 and Table I.4.1b). These countries and economies have been able to increase the share of the students who meet the highest-level skills in PISA. France saw an increase of four percentage points in the share of top performers between PISA 2000 and PISA 2012, but also an increase in the share of low performers during the same period. Annex B4 shows how, in these countries and economies, the performance of the highest-achieving students improved to a greater extent than that of the lowest-achieving students. Increase in the share of low performers or decrease in that of top performers By contrast, in some countries and economies the percentage of students who do not meet the PISA baseline proficiency level in reading increased since 2000 – or since later PISA assessments – or the share of students attaining the highest levels of proficiency shrank. This trend is observed on average across OECD countries since 2000, and in 15 countries and economies when comparing results from PISA 2012 and those from previous assessments (Figure I.4.11 and Table I.4.1b). Variation in student performance in reading The range in performance between the highest- (90th percentile) and lowest-achieving students (10th percentile) is shown in Table I.4.3a. Among the ten participating countries and economies that show the narrowest difference between the highest and lowest achievers in reading, this gap ranges between 189 and 211 points. One of the three lowest- performing PISA participants, the partner country Kazakhstan, and the highest-performing PISA participant in reading in PISA 2012, the partner economy Shanghai-China, are in this group of countries. At the other end of the spectrum, among the ten participating countries and economies that show the largest difference between the highest and lowest achievers in reading, this gap ranges from 270 to 310 points. As is true of those countries with a comparatively narrow distribution of scores among students, the group of countries with a wide range in performance is heterogeneous in mean reading proficiency. One of the lowest-performing countries, Qatar, has nearly the same gap between the highest and lowest achievers as the high-performing country, New Zealand, and both countries are included in this group. If this group is expanded to include the country with the 11th largest difference, it will include one of the five best-performing countries in reading in PISA 2012. Thus, the spread of the performance distribution does not appear to be associated with the overall level of performance. Some countries and economies perform above the OECD average and show only a narrow difference between the highest and lowest achievers in reading. Gender differences in reading performance On average across OECD countries, girls outperform boys in reading by 38 score points. While girls outperform boys in reading in every participating country and economy, the gap is much wider in some countries than in others (Figure I.4.12). As shown in PISA 2009 (OECD, 2010b), these differences are associated with differences in student attitudes and behaviours that are related to gender. Among the five highest-performing countries and economies, the gender gap in reading performance ranges from 23 to 32 score points – below the OECD average (a difference of 38 score points). Among all participating countries and economies, the narrowest gender gap – 15 score points in favour of girls – is observed in Albania. The gender gap is 25 score points or less in 11 other countries, including both low-performing countries, like Chile, Mexico, the partner countries Colombia, Peru and Costa Rica; and very high-performing countries like Korea, Japan, and the partner countries and economies Shanghai-China, Liechtenstein and Hong Kong-China. The United Kingdom, with a score around the OECD average, is also included in this group. In 14 countries, girls outperform boys by at least 50 score points. All of these countries score below the OECD average, except Finland, which performs above the OECD average in reading. In the partner country Jordan, 75 score points – the equivalent of an entire proficiency level – separate girls’ performance from boys’. With the exception of Denmark, countries in Northern Europe have wider-than-average gender gaps in performance. The most pronounced is found in Finland, where the score difference is 62 points – the largest difference observed in any OECD country. The gender-related differences in performance in East Asian countries and economies tend to cluster just below the average, with Korea, Japan, and the partner countries and economies Shanghai-China, Hong Kong-China, Viet Nam, Chinese Taipei and Macao-China all showing gender gaps of between 23 and 36 points. © m S Kno W and Can d o: Student Performan C 199 OECD 2014 What Student i e – Volume C ien C eading and S r , CS athemati e in

202 4 o A nce i n f Student Perform e A ding A Profile r Figure I.4.12 • • g ender differences in reading performance Girls Boys All students Gender differences Mean score on the reading scale (boys girls) – Jordan Qatar Bulgaria Montenegro Finland Slovenia United Arab Emirates Lithuania Thailand Latvia Sweden Iceland Greece Croatia Norway Serbia Turkey Germany Israel France Estonia Poland Romania Malaysia Russian Federation Hungary Slovak Republic Portugal OECD average Italy -38 score points Czech Republic Argentina OECD average Austria Kazakhstan Switzerland Macao-China Uruguay Canada Australia In all countries New Zealand and economies, Chinese Taipei girls outperform boys Singapore in reading Belgium Viet Nam United States Denmark Tunisia Brazil Luxembourg Spain Ireland Indonesia Netherlands Hong Kong-China Costa Rica United Kingdom Liechtenstein Japan Shanghai-China Mexico Korea Chile Peru Colombia Albania -20 450 -80 350 0 500 -40 -60 600 550 400 Mean score Score-point difference Note: All gender differences are signicant (see Annex A3). and economies girls). – are ranked in ascending order of the gender score-point difference (boys Countries OECD, PISA 2012 Database, Table I.4.3a. Source: http://dx.doi.org/10.1787/888932935610 2 1 o: Student Performan r © OECD 2014 What Student S Kno W and Can d eading and S C e in m athemati CS i e – Volume C ien C , 200

203 4 A f Student Perform A nce i n r e o ding A Profile Yet there is no obvious pattern in gender-related differences in performance among groups of countries with lower overall performance. For example, among Latin American countries, the highest-performing country (Chile) and the lowest-performing (Peru) have nearly the same, relatively small, gender gap (23 and 22 points, respectively). One of the middle-ranking countries within this group, the partner country Colombia, has the second-smallest gender gap of any country and economy, with a difference of only 19 score points between the mean scores for girls and boys. How do boys and girls differ in levels of proficiency attained? One way to determine this is to observe the highest level of proficiency attained by the largest group of girls and boys in each country and economy. As can be seen in Table I.4.2a, among all the participating countries and economies, the highest proficiency level attained by the largest group of boys (in 31 countries and economies) and girls (in 37 countries and economies) is Level 3 followed by Level 2 (the highest level attained by most boys in 17 countries and economies, and by most girls in 19 countries and economies). But while in 13 countries and economies the highest proficiency level attained by the largest group of boys is Level 1a – and in one country, Level 1b – in only one country is Level 1a the highest proficiency level attained by the largest group of girls. Level 4 is the highest proficiency level attained by the largest group of boys in only three countries, while in eight countries is the highest proficiency level attained by the largest group of girls. Around the middle of the reading scale, nearly one in two boys (49%) but only one in three girls (34%) fails to reach Level 3, which is associated with being able to perform the kinds of tasks that are commonly demanded of adults in their everyday lives. This represents a major difference in the capabilities of boys and girls at age 15. This pattern is also seen among students with particularly low levels of reading proficiency. Across OECD countries, 24% of boys do not attain Level 2, considered as the baseline level of proficiency, while only about half as many girls (12%) perform at that level. In 14 countries, more than half of all 15-year-old boys perform below Level 2 on the reading scale, but in only one country does the same proportion of girls perform at that level. Among the ten highest-performing countries in reading, the proportion of girls who perform below Level 2 is only one- quarter (in Finland) to one-half that of boys (e.g. Japan, Ireland and Singapore), while in some of the low-performing countries, such as Albania, Peru and Colombia, the proportions of girls and boys performing below Level 2 tend to be similar. Some of the differences in reading performance between boys and girls are closely related to gender differences in attitudes and behaviour, which are discussed in PISA 2009, Volume III (OECD, 2010b). Trends in gender differences in reading performance Girls have traditionally outperformed boys in reading (Buchmann et al., 2008). In PISA 2000 and on average across OECD countries, girls outperformed boys by 32 score points. That year, girls’ advantage in reading was significant in the 39 participating countries and economies, except Israel and Peru. It was largest in Albania, Finland and Latvia, at more than 50 score points and exceeded 40 points – more than the equivalent to a year of schooling – in Argentina, Bulgaria, Iceland, New Zealand, Norway and Thailand (Table I.4.3c and OECD, 2001). By 2012, the relative standing of boys had further deteriorated. In 2012 and on average across OECD countries that have comparable data in PISA 2000, girls outperformed boys by 38 PISA score points, roughly the equivalent of an academic school year. Between 2000 and 2012 the gender gap in reading performance widened in 11 countries and economies. In Bulgaria, France and Romania the gap widened by more than 15 score points. Only in Albania did the gender gap in reading performance narrow, as a result of a greater improvement in reading performance among boys (68 score points) 5 than girls (24 score points) between PISA 2000 and PISA 2012 (Figure I.4.13). Consistent with this trend, the proportion of low-performing girls shrank significantly in 16 countries and economies between PISA 2000 and PISA 2012, while the share of low-performing boys decreased in only 11 countries and economies. However, the share of low-performing boys increased in seven countries and economies, while the share of low-performing girls increased in only three countries during the period (Table I.4.2b). At the other end of the performance spectrum, the share of top-performing girls – those who perform at or above proficiency Level 5 – increased significantly between PISA 2000 and PISA 2012 in 11 countries and economies, while the share of top-performing boys increased in only seven of these countries and economies. This increase in top-performing girls was greatest in Hong Kong-China, Japan and Korea where the share of top-performing boys also grew (Table I.4.2b). i e in S Kno W and Can d o: Student Performan 201 OECD 2014 © What Student e – Volume C ien C eading and S r , CS athemati m C

204 4 o A nce i n f Student Perform e A ding A Profile r Figure I.4.13 • • c hange between 2000 and 2012 in gender differences in reading performance Gender difference in reading performance in 2012 Gender difference in reading performance in 2000 0 -10 -20 Score-point difference -30 -40 -50 -60 In all countries /economies girls perform better in reading than boys -70 -80 -6 44 -14 -22 -15 -28 -13 -11 -14 -27 -10 -14 -14 Italy Peru Chile Israel Brazil Spain Japan Latvia Korea France Poland Austria Ireland Greece Finland Iceland Mexico Albania Canada Norway Sweden Bulgaria Belgium Portugal Thailand Hungary Australia Romania Germany Denmark Argentina Indonesia Switzerland New Zealand Liechtenstein United States Czech Republic Hong Kong-China Russian Federation OECD average 2000 Notes: All gender differences in PISA 2012 are statistically signicant. Gender differences in PISA 2000 that are statistically signicant are marked in a darker tone (see Annex A3). Statistically signicant changes in the score-point difference between boys and girls in reading performance between PISA 2000 and PISA 2012 are shown next to the country/economy name. OECD average 2000 compares only OECD countries with comparable reading scores since 2000. and economies are ranked in ascending order of gender differences (boys-girls) in 2012. Countries Source: OECD, PISA 2012 Database, Table I.4.3c. http://dx.doi.org/10.1787/888932935610 2 1 e in C o: Student Performan d and Can W Kno S OECD 2014 © What Student m athemati i CS , r eading and S C ien C e – Volume 202

205 4 A Profile A nce i n f Student Perform e A ding o r S e of pi xample S S reading unit a The questions are presented in the order in which they appeared within the unit in the main survey. • • Figure I.4.14 he t the thing S lay’ p 60 who we are. Wouldn’t it be much easier to Takes place in a castle by the beach in Italy. start all this by standing up and introducing ourselves? Stands up. Good FIRST ACT ening. The three of us are guests in this ev o rnate guest room in a very nice beachside castle. We have just arrived from the castle. Doors on the right and left. Sitting 65 dining room where we had an excellent room set in the middle of the stage: couch, 5 ank two bottles of dinner and dr table, and two armchairs. Large windows at champagne. My name is Sándor TURAI, the back. Starry night. It is dark on the stage. I’m a playwright, I’ve been writing plays for When the curtain goes up we hear men thirty y ears, that’s my profession. Full stop. conversing loudly behind the door on the left. 70 Your turn. The door opens and three tuxedoed gentlemen 10 enter. ne turns the light on immediately. o GÁL They walk to the centre in silence and stand Stands up. My name is GÁL, I’m also a around the table. They sit down together, Gál playwright. I write plays as well, all of in the armchair to the left, Turai in the one on them in the company of this gentleman the right, Ádám on the couch in the middle. 15 here. We are a famous playwright duo. All 75 V ery long, almost awkward silence. ybills of good comedies and operettas pla Comfortable stretches. Silence. Then: read: written by GÁL and TURAI. Naturally, this is my profession as well. GÁL Why are you so deep in thought? GÁL and TURAI And this young man ... Together. 80 20 TURAI I’m thinking about how difficult it is to begin ÁDÁM a play. To introduce all the principal This young man is, if you allow Stands up. characters in the beginning, when it all starts. me, Albert ÁDÁM, twenty-five years old, composer. I wrote the music for these kind ÁDÁM gentlemen for their latest operetta. This is 85 25 I suppose it must be hard. my first work for the stage. These two TURAI elderly angels have discovered me and now, It is – devilishly hard. The play starts. The with their help, I’ d like to become famous. audience goes quiet. The actors enter the stage They got me invited to this castle. They got and the torment begins. It’s an eternity, my dress-coat and tuxedo made. In other 90 sometimes as much as a quarter of an hour ords, I am poor and unknown, for now. w 30 before the audience finds out who’s who and Other than that I’m an orphan and my what they are all up to. grandmother raised me. My grandmother has passed away. I am all alone in this world. I GÁL 95 have no name, I have no money. Quite a peculiar brain you’ve got. Can’t you 35 forget your profession for a single minute? TURAI But you are young. TURAI That cannot be done. GÁL And gifted. GÁL 100 Not half an hour passes without you ÁDÁM discussing theatre, actors, plays. There are 40 And I am in love with the soloist. other things in this world. TURAI TURAI You shouldn’t have added that. Everyone in There aren’t. I am a dramatist. That is my the audience would figure that out anyway. curse. 105 They all sit down. GÁL 45 TURAI You shouldn’t become such a slave to Now wouldn’t this be the easiest way to start your profession. a play? AI TUR GÁL If you do not master it, you are its slave. If we were allowed to do this, it would be 110 There is no middle ground. Trust me, it’s 50 easy to write plays. y well. It is one of the no joke starting a pla toughest problems of stage mechanics. TURAI Introducing your characters promptly. Trust me, it’s not that hard. Just think of this Let’s look at this scene here, the three of whole thing as ... 55 us. Three gentlemen in tuxedoes. Say they 115 GÁL enter not this room in this lordly castle, All right, all right, all right, just don’t start but rather a stage, just when a play begins. talking about the theatre again. I’m fed up hey would have to chat about a whole lot T with it. We’ll talk tomorrow, if you wish. of uninteresting topics until it came out “The Play’s the Thing” is the beginning of a play by the Hungarian dramatist Ferenc Molnár. Use “The Play’s the Thing” on the previous two pages to answer the questions that follow. (Note that line numbers are given in the margin of the script to help you find parts that are referred to in the questions.) eading and S C OECD 2014 C e – Volume i © 203 CS athemati , r m e in C o: Student Performan d and Can W Kno S What Student ien

206 4 o A nce i n f Student Perform e A ding A Profile r Level 6 on 3 – Qu THE PLAY’S THE THING ESTI 698 Level 5 626 Personal Situation: Level 4 553 Continuous t xt format: e Level 3 480 t e arration n xt type: Level 2 407 Integrate and interpret – Develop an interpretation spect: a Level 1a Question format: Short response 335 Level 1b d ifficulty: 730 (Level 6) 262 Below Level 1b What were the characters in the play doing the curtain went up? just before ... Scoring f ull c redit: Refers to dinner or drinking champagne. May paraphrase or quote the text directly. • T hey have just had dinner and champagne. • “W e have just arrived from the dining room where we had an excellent dinner.” [direct quotation] [direct quotation] An excellent dinner and drank two bottles of champagne.” “ • • Dinner and drinks. • Dinner . • Dr ank champagne. • Had dinner and dr ank. • T hey were in the dining room. Comment This task illustrates several features of the most difficult tasks in PISA reading. The text is long by PISA standards, and it may be supposed that the fictional world depicted is remote from the experience of most 15-year-olds. The ’ P is the beginning of a play by the HING T THE LAY S HE T introduction to the unit tells students that the stimulus of Hungarian dramatist Ferenc Molnár, but there is no other external orientation. The setting (“a castle by the beach in Italy”) is likely to be exotic to many, and the situation is only revealed gradually through the dialogue itself. While individual pieces of vocabulary are not particularly difficult, and the tone is often chatty, the register of the language is a little mannered. Perhaps most importantly a level of unfamiliarity is introduced by the abstract theme of the discussion: a sophisticated conversation between characters about the relationship between life and art, and the challenges of writing for the theatre. The text is classified as narration because this theme is dealt with as part of the play’s narrative. While all the tasks in this unit acquire a layer of difficulty associated with the challenges of the text, the cognitive demand of this task in particular is also attributable to the high level of interpretation required to define the meaning of the question’s terms, in relation to the text. The reader needs to be alert to the distinction between characters and actors. The question refers to what the characters (not the actors) were doing “just before the curtain went up”. This is potentially confusing since it requires recognition of a shift between the real world of a stage in a theatre, which has a curtain, and the imaginary world of Gal, Turai and Adam, who were in the dining room having dinner just before they entered the guest room (the stage setting). A question that assesses students’ capacity to distinguish between real and fictional worlds seems particularly appropriate in relation to a text whose theme is about just that, so that the complexity of the question is aligned with the content of the text. A further level of the task’s difficulty is introduced by the fact that the required information is in an unexpected location. The question refers to the action “before the curtain went up”, which would typically lead one to search at the opening o n the contrary, the information is actually found about half-way through the of the scene, the beginning of the extract. extract, when Turai reveals that he and his friends “have just arrived from the dining room”. While the scoring for the question shows that several kinds of response are acceptable, to be given full credit readers must demonstrate that they have found this inconspicuous piece of information. The need to assimilate information that is contrary to expectations – where the reader needs to give full attention to the text in defiance of preconceptions – is highly characteristic of the most demanding reading tasks in PISA. C eading and S r , CS athemati m ien e – Volume C e in C o: Student Performan d and Can W Kno S What Student OECD 2014 © i 204

207 4 A Profile A nce i n r e A ding o f Student Perform Level 6 ESTI – Qu THE PLAY’S THE THING on 4 698 Level 5 Situation: Personal 626 Level 4 Continuous xt format: e t 553 Level 3 xt type: e t n arration 480 spect: Integrate and interpret – Develop an interpretation a Level 2 407 Multiple choice Question format: Level 1a 335 474 (Level 2) ifficulty: d Level 1b 262 Below Level 1b “It’s an eternity, sometimes as much as a quarter of an hour ... ” (lines 29-30) According to Turai, why is a quarter of an hour “an eternity”? A. tre. It is a long time to expect an audience to sit still in a crowded thea It seems to take f B. orever for the situation to be clarified at the beginning of a play. or a dramatist to write the beginning of a play. It always seems to take a long time f C. D. It seems tha t time moves slowly when a significant event is happening in a play. Scoring c B. It seems to take forever for the situation to be clarified at the beginning of a play. f ull redit: Comment n ear the borderline between Level 2 and Level 3, this question together with the previous one illustrates the fact that questions covering a wide range of difficulties can be based on a single text. u nlike in the previous task, the stem of this task directs the reader to the relevant section in the play, even quoting the lines, thus relieving the reader of any challenge in figuring out where the necessary information is to be found. n evertheless, the reader needs to understand the context in which the line is uttered in order to respond successfully. In fact, the implication of “It seems to take forever for the situation to be clarified at the beginning of a play” underpins much of the rest of this extract, which enacts the solution of characters explicitly introducing themselves at the beginning of a play instead of waiting for the action to reveal who they are. Insofar as the utterance that is quoted in the stem prompts most of the rest of this extract, repetition and emphasis support the reader in integrating and interpreting the quotation. In that respect too, this task clearly differs from Question 3, in which the required information is only provided once, and is buried in an unexpected part of the text. Level 6 ESTI on 7 THE PLAY’S THE THING – Qu 698 Level 5 626 Situation: Personal Level 4 553 t e xt format: Continuous Level 3 480 t e xt type: n arration Level 2 407 Integrate and interpret – Form a broad understanding a spect: Level 1a Question format: Multiple choice 335 Level 1b d 556 (Level 4) ifficulty: 262 Below Level 1b Overall, what is the dramatist Molnár doing in this extract? A. t each character will solve his own problems. He is showing the way tha B. He is making his characters demonstra te what an eternity in a play is like. or a play. C. He is giving an example of a typical and traditional opening scene f D. He is using the characters to act out one of his own crea tive problems. Scoring c f ull redit: D. He is using the characters to act out one of his own creative problems. Comment by integrating and interpreting form a broad understanding In this task the reader is asked to take a global perspective, the implications of the dialogue across the text. The task involves recognising the conceptual theme of a section of a play, where the theme is literary and abstract. This relatively unfamiliar territory for most 15-year-olds is likely to constitute the difficulty of the task, which is located at Level 4. A little under half of the students in ECD countries gained full credit o for this task, with the others divided fairly evenly across the three distractors. e in C 205 d and Can W Kno S What Student eading and S r , CS athemati m OECD 2014 C ien C e – Volume i © o: Student Performan

208 4 o A nce i n r e A ding A Profile f Student Perform Figure I.4.15 • • l abour The tree diagram below shows the structure of a country’s labour force or “working-age population”. total population of the country in 1995 was about 3.4 million. The 1 The labour force structure, year ended 31 March 1995 (000s) 2 Working-age population 2656.5 3 In labour force Not in labour force 1706.5 64.2% 949.9 35.8% Employed Unemployed 1578.4 92.5% 128.1 7.5% Part-time Full-time 341.3 21.6% 1237.1 78.4% Seeking part-time Seeking full-time work work 26.5 20.7% 101.6 79.3% Seeking Not seeking full-time work full-time work 23.2 6.8% 318.1 93.2% 1. Numbers of people are given in thousands (000s). 2. The working-age population is defined as people between the ages of 15 and 65. 3. People “Not in labour force” are those not actively seeking work and/or not available for work. , ESA Publications, Box 9453, Newmarker, Auckland, NZ, p. 64. Source: D. Miller, Form 6 Economics Level 6 698 Level 5 626 ESTI on 16 L ABOUR – Qu Level 4 553 Level 3 Reading for education Situation: 480 Level 2 n Text format: on-continuous 407 Level 1a Retrieving information Aspect: 335 485 – Difficulty: Percentage of correct answers (OECD countries): 64.9% Level 1b 262 Percentage of correct answers (OECD countries): 27.9% 631 – Below Level 1b How many people of working age were not in the labour force? (Write the number of people, not the percentage.) Comment The question presented here yields responses at two levels of difficulty, with the partial-credit response category falling within Level 3 with a score of 485 and the full-credit category within Level 5 with a score of 631. For full credit (Level 5) students are required to locate and combine a piece of numerical information in the main body of the text (the tree diagram) with information in a footnote – that is, outside the main body of the text. In addition, students have to apply this footnoted information in determining the correct number of people fitting into this category. Both of these features contribute to the difficulty of this task, which is one of the most difficult retrieving information tasks in the PISA reading assessment. For partial credit (Level 3) this task merely requires students to locate the number given in the appropriate category of the tree diagram. They are not required to use the conditional information provided in the footnote to receive partial credit. Even without this important information the task is still moderately difficult. o: Student Performan e – Volume ien C eading and S i , CS athemati m e in C C d and Can W Kno S What Student OECD 2014 © r 206

209 4 A Profile f Student Perform A nce i n r o A ding e • Figure I.4.16 • alloon b eight record for hot air balloons h he Indian pilot Vijaypat Singhania beat the height record for hot air balloons on November 26, 2005. T He was the first person to fly a balloon 21 000 metres above sea level. ecord height r Side slits 21 000 m can be opened xygen o to let out only 4% of what is available hot air for Size of at ground level descent. conventional hot air balloon Earlier record Height 19 800 m 49 m ature t emper 95° C – The balloon abric f went out Jumbo jet Nylon towards the sea. 10 000 m When it met the nflation i jet stream it was 2.5 hours taken back over the land again. Size 3 453 000 m 3 ) (normal hot air balloon 481 m New Delhi Approximate Weight landing area 1 800 kg 483 km Gondola Height: 2.7 m Width: 1.3 m Mumbai Enclosed pressure cabin with insulated windows Aluminium construction, like airplanes Vijaypat Singhania wore a space suit during the trip. © MCT/Bulls Use “Balloon” on the previous page to answer the questions that follow. Level 6 BALLOON on 8 ESTI – Qu 698 Level 5 626 Situation: Educational Level 4 553 n on-continuous e xt format: t Level 3 480 Description xt type: e t Level 2 407 spect: a Integrate and interpret – Form a broad understanding Level 1a Question format: Multiple choice 335 Level 1b 370 (Level 1a) d ifficulty: 262 Below Level 1b What is the main idea of this text? A. Singhania was in danger during his balloon trip. B. Singhania set a new world record. C. Singhania travelled over both sea and land. D. Singhania’s balloon was enormous. r , 207 CS athemati m © o: Student Performan OECD 2014 i e in e – Volume C ien C eading and S d and Can W Kno S What Student C

210 4 o nce i n r e A ding f Student Perform A Profile A Scoring f redit: B. Singhania set a new world record. c ull Comment The main idea of this non-continuous text is stated explicitly and prominently several times, including in the title, “Height record for hot air balloon”. The prominence and repetition of the required information helps to explains its easiness: it is located in the lower half of Level 1a. Although the main idea is explicitly stated, the question is classified as integrate and interpret, with the sub-classification forming a broad understanding , because it involves distinguishing the most significant and general from subordinate information in the text. The first option – “Singhania was in danger during his balloon trip” – is a plausible speculation, but it is not supported by anything in the text, and so cannot qualify as a main idea. The third option – “Singhania travelled over both sea and land” – accurately paraphrases information from the text, but it is a detail rather than the main idea. The fourth option – “Singhania’s balloon was enormous” – refers to a conspicuous graphic feature in the text but, again, it is subordinate to the main idea. Level 6 ESTI on 3 – Qu BALLOON 698 Level 5 626 Educational Situation: Level 4 553 e xt format: n on-continuous t Level 3 480 xt type: t e Description Level 2 407 a Access and retrieve – Retrieve information spect: Level 1a 335 Short response Question format: Level 1b 262 ifficulty: d Full credit 595 (Level 4); Partial credit 449 (Level 2) Below Level 1b Vijaypat Singhania used technologies found in two other types of transport. Which types of transport? 1. ... 2. ... Scoring f redit: ull Refers to BOTH airplanes AND spacecraft (in either order, can include both answers on one line). For c example: • craft 1. Air 2. Spacecr aft • 1. Airplanes 2. Space ships 1. Air travel • 2. Space tr avel 1. Planes • 2. Space roc kets 1. J ets • 2. Roc kets Partial Refers to EITHER airplanes OR spacecraft. For example: redit: c Spacecraft • • avel Space tr • Space roc kets • Rockets • Aircraft Airplanes • • Air tr avel Jets • , r eading and S C e – Volume ien C athemati m CS e in C o: Student Performan d and Can W Kno S What Student OECD 2014 © i 208

211 4 f Student Perform nce i n r A A ding o A Profile e Comment In this task full credit is given for responses that lists the two required types of transport, and partial credit is given to responses that listed one type. The scoring rules reproduced above demonstrate that credit is available for several different paraphrases of the terms “airplanes” and “spacecraft”. The partial credit score is located in the upper half of Level 2 while the full credit score is located at Level 4, illustrating the fact that questions can create a significant challenge. The difficulty of the task is particularly influenced access and retrieve by a number of features of the text. The layout, with several different kinds of graphs and multiple captions, is quite a common type of non-continuous presentation often seen in magazines and modern textbooks, but because it does not have a conventional ordered structure (unlike, for example, a table or graph), finding specific pieces of discrete information is relatively inefficient. Captions (“Fabric”, “Record height”, and so on) give some support to the reader in navigating the text, but the information specific required for this task does not have a caption, so that readers have to generate their own categorisation of the relevant information as they search. Having once found the required information, inconspicuously located at the bottom left-hand corner of the diagram, the reader needs to recognise that the “aluminium construction, like airplanes” and the “space suit” are associated with categories of transport. In order to obtain credit for this question, the response needs to refer to a form or forms of transport, rather than simply transcribing an approximate section of text. Thus “space travel” is credited, but “space suit” is not. A significant piece of competing information in the text constitutes a further difficulty: many students referred to a “jumbo jet” in their answer. Although “air travel” or “airplane” or “jet” is given credit, “jumbo jet” is deemed to refer specifically to the image and caption on the right of the diagram. This answer is not given credit as the jumbo jet in the illustration is not included in the material with reference to technology used for Singhania’s balloon. Level 6 BALLOON – Qu ESTI on 4 698 Level 5 626 Situation: Educational Level 4 553 e t on-continuous n xt format: Level 3 480 t Description xt type: e Level 2 spect: a Reflect and evaluate – Reflect on and evaluate the content of a text 407 Level 1a o Question format: pen Constructed Response 335 Level 1b ifficulty: d 510 (Level 3) 262 Below Level 1b What is the purpose of including a drawing of a jumbo jet in this text? ... ... Scoring f ull c redit: Refers explicitly or implicitly to the height of the balloon OR to the record. May refer to comparison between the jumbo jet and the balloon. T • o show how high the balloon went. T • o emphasise the fact that the balloon went really, really high. • T o show how impressive his record really was – he went higher than jumbo jets! As a point of reference regarding height. • • T o show how impressive his record really was. [minimal] Comment The main idea of the text is to describe the height record set by Vijaypat Singhania in his extraordinary balloon. The diagram on the right-hand side of the graphic, which includes the jumbo jet, implicitly contributes to the “wow!” factor of the text, showing just how impressive the height achieved by Singhania was by comparing it with what we usually associate with grand height: a jumbo jet’s flight. In order to gain credit for this task, students must recognise the persuasive intent of including the illustration of the jumbo jet. For this reason the task is classified as reflect and evaluate, with the sub-category reflect on and evaluate the content of a text . At the upper end of Level 3, this question is moderately difficult. C r o: Student Performan e in m athemati CS , OECD 2014 d and Can W Kno S What Student eading and S C ien C e – Volume i © 209

212 4 o A nce i n r e f Student Perform ding A Profile A Level 6 ESTI – Qu BALLOON on 6 698 Level 5 626 Situation: Educational Level 4 553 n t e xt format: on-continuous Level 3 480 e xt type: Description t Level 2 407 a Reflect and evaluate – Reflect on and evaluate the content of a text spect: Level 1a Multiple choice Question format: 335 Level 1b ifficulty: 411 (Level 2) d 262 Below Level 1b Size of conventional hot air balloon eight h 49 m Why does the drawing show two balloons? o compare the size of Singhania’s balloon before and after it was inflated. A. T B. To compare the size of Singhania’s balloon with that of other hot air balloons. C. To show that Singhania’s balloon looks small from the ground. D. To show that Singhania’s balloon almost collided with another balloon. Scoring B. To compare the size of Singhania’s balloon with that of other hot air balloons. redit: c ull f Comment It is important for readers to be aware that texts are not randomly occurring artefacts, but are constructed deliberately and with intent, and that part of the meaning of a text is found in the elements that authors choose to include. Like the previous task, this task is classified under reflect and evaluate because it asks about authorial intent. It focuses on a graphic element – here the illustration of two balloons – and asks students to consider the purpose of this inclusion. In the context of the over-arching idea of the text, to describe (and celebrate) Singhania’s flight, the balloon illustration sends the message, “This is a really big balloon!”, just as the jumbo jet illustration sends the message, “This is a really high flight!” The caption on the smaller balloon (“Size of a conventional hot air balloon”) makes it obvious that this is a o ption D has different balloon to Singhania’s, and therefore, for attentive readers, renders options A and C implausible. no support in the text. With a difficulty near the bottom of Level 2, this is a rather easy task. e – Volume CS , r eading and S C o: Student Performan e in m athemati d and Can W Kno S What Student OECD 2014 © C C ien i 210

213 4 A Profile f Student Perform A nce i n r o A ding e • Figure I.4.17 • i m er S h S ND a GOLD T r E M i SE hi A fable b y Aesop A miser sold all that he had and bought a lump of gold, which he buried in a hole in the ground by the side of an old wall. He went to look at it daily. One of his workmen observed the miser’s frequent visits to the spot and decided to watch his movements. The workman soon discovered the secret of the hidden treasure, and digging down, came to the lump of gold, and stole it. The miser, on his next visit, found the hole empty and began to tear his hair and to make loud lamentations. A neighbour, seeing him overcome with grief and learning the cause, said, “Pray do not grieve so; but go and take a stone, and place it in the hole, and fancy that the gold is still lying there. It will do you quite the same service; for when the gold was there, you had it not, as you did not make the slightest use of it.” Use the fable “The Miser and his Gold” on the previous page to answer the questions that follow. Level 6 on 1 MISER – Qu ESTI 698 Level 5 626 Situation: Personal Level 4 553 xt format: t e Continuous Level 3 480 n e arration t xt type: Level 2 407 Integrate and interpret – Develop an interpretation a spect: Level 1a Closed constructed response Question format: 335 Level 1b 373 (Level 1a) ifficulty: d 262 Below Level 1b Read the sentences below and number them according to the sequence of events in the text. The miser decided to turn all his money into a lump of gold. A man stole the miser’ s gold. The miser dug a hole and hid his treasure in it. The miser’ s neighbour told him to replace the gold with a stone. Scoring c redit: All four correct: 1, 3, 2, 4 in that order. ull f Comment Fables are a popular and respected text type in many cultures and they are a favourite text type in reading assessments for similar reasons: they are short, self-contained, morally instructive and have stood the test of time. While perhaps not ECD countries they are nevertheless likely to be familiar from the most common reading material for young adults in o childhood, and the pithy, often acerbic observations of a fable can pleasantly surprise even a blasé 15-year-old. MISER is typical of its genre: it captures and satirises a particular human weakness in a neat economical story, executed in a single paragraph. are defined as referring to properties of objects in time, typically answering “when” questions, it is Since narrations appropriate to include a task based on a narrative text that asks for a series of statements about the story to be put into the correct sequence. With such a short text, and with statements in the task that are closely matched with the terms of o n the other hand, the language of the text is rather formal the story, this is an easy task, around the middle of Level 1a. and has some old-fashioned locutions. (Translators were asked to reproduce the fable-like style of the source versions.) This characteristic of the text is likely to have added to the difficulty of the question. athemati e in 211 OECD 2014 m i e – Volume C ien C eading and S r , CS C o: Student Performan d and Can W Kno S What Student ©

214 4 nce r e A ding n o f Student Perform A A Profile i Level 6 – Qu MISER on 7 ESTI 698 Level 5 626 Personal Situation: Level 4 553 e Continuous xt format: t Level 3 480 t e xt type: n arration Level 2 407 a Access and retrieve – Retrieve information spect: Level 1a 335 Question format: Short response Level 1b 262 d 310 (Level 1b) ifficulty: Below Level 1b How did the miser get a lump of gold? ... Scoring f ull redit: States that he sold everything he had. May paraphrase or quote directly from the text. c • He sold all he had. • He sold all his stuff. • He bought it. [implicit connection to selling everything he had] Comment This is one of the easiest tasks in PISA reading, with a difficulty in the middle of Level 1b. The reader is required to access and retrieve a piece of explicitly stated information in the opening sentence of a very short text. To gain full credit, the response can either quote directly from the text – “He sold all that he had” – or provide a paraphrase such as “He sold all his stuff”. The formal language of the text, which is likely to have added difficulty in other tasks in the unit, is unlikely to have much impact here because the required information is located at the very beginning of the text. Although this is an extremely easy question in PISA’s frame of reference, it still requires a small degree of inference, beyond the absolutely literal: the reader must infer that there is a causal connection between the first proposition (that the miser sold all he had) and the second (that he bought gold). on 5 ESTI – Qu MISER Level 6 698 Level 5 626 Situation: Personal Level 4 553 xt format: t e Continuous Level 3 480 t e xt type: n arration Level 2 a spect: Integrate and interpret – Develop an interpretation 407 Level 1a o pen constructed response Question format: 335 Level 1b d 548 (Level 3) ifficulty: 262 Below Level 1b Here is part of a conversation between two people who read “The Miser and his Gold”. The neighbour was nasty. No he couldn’t. He could have The stone was recommended important in replacing the the story. gold with something better than a stone. Speaker 2 Speaker 1 What could Speaker 2 say to support his point of view? ... ... C i OECD 2014 What Student S Kno W and Can d o: Student Performan © e in m e – Volume athemati CS , r eading and S C ien C 212

215 4 n f Student Perform A nce i o r e A ding A Profile Scoring c ull f redit Recognises that the message of the story depends on the gold being replaced by something useless or worthless. y something worthless to make the point. It needed to be replaced b • T he stone is important in the story, because the whole point is he might as well have buried a stone for all the good • the gold did him. If y • ou replaced it with something better than a stone, it would miss the point because the thing buried needs to be something really useless. • A stone is useless, but for the miser , so was the gold! ould be something he could use – he didn’t use the gold, that’s what the guy was pointing out. • Something better w Because stones can be found an ywhere. The gold and the stone are the same to the miser. • [“can be found anywhere” implies that the stone is of no special value] Comment This task takes the form of setting up a dialogue between two imaginary readers, to represent two conflicting interpretations of the story. In fact only the second speaker’s position is consistent with the overall implication of the text, so that in providing a supporting explanation readers demonstrate that they have understood the “punch line” – the moral import – of the fable. The relative difficulty of the task, near the top of Level 3, is likely to be influenced by the fact that readers needs to do a good deal of work to generate a full credit response. First they must make sense of the neighbour’s speech in the story, which is expressed in a formal register. (As noted, translators were asked to reproduce the fable-like style.) Secondly, the relationship between the question stem and the required information is not obvious: there is little or no support in the stem (“What could Speaker 2 say to support his point of view?”) to guide the reader in interpreting the task, though the reference to the stone and the neighbour by the speakers should point the reader to the end of the fable. As shown in examples of responses, to gain full credit, students could express,in a variety of ways, the key idea that wealth has no value unless it is used. Vague gestures at meaning, such as “the stone had a symbolic value”, are not given credit. C o: Student Performan S Kno W and Can 213 OECD 2014 © i e – Volume What Student ien C eading and S r , CS athemati m e in C d

216 4 A f Student Perform A nce i n r e o ding A Profile Notes 1. Of the 64 countries and economies that have trend data up to 2012, 30 participated in PISA 2012 and have comparable results for every assessment since PISA 2000; 14 countries and economies have comparable data for 2012 and three other PISA assessments; 13 have comparable data for 2012 and two other PISA assessments; and 7 have comparable data for 2012 and one additional PISA assessment. 2. As described in more detail in Annex A5, the annualised change takes into account the specific year in which the assessment took place. In the case of reading, this is especially relevant for the 2009 assessment as Costa Rica, Malaysia and the United Arab Emirates (excl. Dubai) implemented the assessment in 2010 as part of PISA+ and the 2000 assessment as Chile and the partner countries and economies Albania, Argentina, Bulgaria, Hong Kong-China, Indonesia, Peru and Thailand implemented the assessment in 2001, Israel and Romania in 2002 as part of PISA+. 3. As described in Annex A5, the annualised change considers the case of countries and economies that implemented PISA 2000 in 2001 or 2002 and those that implemented PISA 2009 in 2010 as part of PISA+. 4. By accounting for students’ gender, age, socio-economic status, migration background and language spoken at home, the adjusted trends allow for a comparison of changes in performance assuming no alteration in the underlying population or the effective samples’ average socio-economic status, age and percentage of girls, students with an immigrant background or students that speak a language at home that is different from the language of assessment. 5. Israel shows a seven percentage-point decline in the weighted percentage of girls assessed by PISA. The sampling design for Israel in the PISA 2000 assessment did not account for the gender composition of schools, despite the different participation rates between boys and girls in Israel due to the fact that some boys’ schools refused to take part in the assessment. The gender distribution in the PISA 2000 data for Israel was subject to a relatively large sampling variance due to an inefficient sampling design. The section on adjusted trends takes this into account by adjusting results for 2000 so that the gender distribution is comparable to that observed in 2012. Nevertheless, trends in the socio-economic status of students and in the percentage of students with an immigrant background – which are also taken into account in the adjusted trends – also played an important role in the observed performance changes in Israel. References and Buchmann, C., T. DiPrete (2008), “Gender Inequalities in Education”, A. McDaniel 337. Annual Review of Sociology , Vol. 34, pp. 319 - (2010), Pathways to Success: How Knowledge and Skills at Age 15 Shape Future Lives in Canada, PISA, OECD Publishing. OECD http://dx.doi.org/10.1787/9789264081925-en OECD (2010b), PISA 2009 Results: Learning to Learn, Student Engagement, Strategies and Practices (Volume III), PISA, OECD Publishing. http://dx.doi.org/10.1787/9789264083943-en OECD (2009), PISA 2009 Assessment Framework: Key Competencies in Reading, Mathematics and Science, PISA, OECD Publishing. http://dx.doi.org/10.1787/9789264062658-en OECD (2001), Knowledge and Skills for Life: First Results from PISA 2000, PISA, OECD Publishing. http://dx.doi.org/10.1787/9789264195905-en ien and Can © OECD 2014 What Student S Kno i e – Volume C d C eading and S r , CS athemati m e in C o: Student Performan W 214

217 5 A Profile of Student Performance in Science This chapter examines student performance in science in PISA 2012. It provides examples of assessment questions, relating them to each PISA proficiency level, discusses gender differences in student performance, compares countries’ and economies’ performance in science, and highlights trends in science performance up to 2012. © e in athemati CS , r eading and S What Student ien C e – Volume i m OECD 2014 215 C o: Student Performan d and Can W Kno S C

218 5 nce in Science A Profile of Student Perform A 15-y ear-old students do in science? This chapter describes how PISA 2012 measures student achievement What can in science around the world, at the country and regional levels, among boys and girls, and also compares outcomes of PISA 2012 with those of the previous PISA cycles. It provides a few examples of the questions asked in the science assessment. An understanding of science and technology is central to a young person’s preparedness for life in modern society, not least because it empowers individuals to participate in determining public policy where issues of science and technology affect their lives. PISA defines scientific literacy as an individual’s scientific knowledge, and use of that knowledge, to identify questions, acquire new knowledge, explain scientific phenomena and draw evidence-based conclusions about science-related issues; understanding of the characteristic features of science as a form of human knowledge and enquiry; awareness of how science and technology shape our material, intellectual and cultural environments; and willingness to engage in science-related issues, and with the ideas of science, as a reflective citizen (OECD, 2007). Science was the focus of the PISA 2006 survey and a minor domain in PISA 2009 and 2012. Less time was allocated during these latter two assessments than in PISA 2006. Ninety minutes of the assessment time were devoted to science in the last two cycles, allowing for only an update on overall performance rather than the kind of in-depth analysis of knowledge and skills shown in the PISA 2006 report (OECD, 2007). What the data tell us • Nineteen of 64 countries and economies with comparable data show an average annual improvement, 37 show no change, and 8 show a deterioration in their science performance throughout their participation in PISA. Hong K ong-China, Ireland, Japan, Korea and Poland performed at or above the OECD average in science in 2006 • and by 2012 showed an improvement in science performance of more than two score points per year. Estonia also performed above the OECD average in science in 2006, and between 2009 and 2012 improved its score by 14 points. • Estonia, Isr ael, Italy, Poland, Qatar and Singapore reduced the share of students who do not attain the baseline level of proficiency and simultaneously increased the share of top-performing students in science. ys and girls perform similarly in science and, on average, that remained true in 2012. But in Colombia, Japan Bo • and Spain, while there was no gender gap in science performance in 2006, a gender gap in favour of boys was observed in 2012. S Student performance in cience In PISA 2006 the mean science score for OECD countries was initially set at 500 points (for 30 OECD countries), then was re-set at 498 points after taking into account the four newest OECD countries. To help interpret what students’ scores mean in substantive terms, the scale is divided into levels of proficiency that indicate the kinds of tasks that students at those levels are capable of completing successfully (OECD, 2006). Average performance in science One way to summarise student performance and to compare the relative standing of countries in science is through countries’ mean performance, both relative to each other and to the OECD mean. For PISA 2012, the mean in science for OECD countries increased to 501 points. This establishes the benchmark against which each country’s and economy’s science performance in PISA 2012 is compared. When interpreting mean performance, only those differences among countries and economies that are statistically significant should be taken into account. Figure I.5.1 shows each country’s/economy’s mean score and also for which pairs of countries/economies the differences between the means are statistically significant. For each country/economy shown in the middle column, the countries/economies whose mean scores are not statistically significantly different are listed in the right column. In all other cases, country/economy A scores higher than country/economy B if country/ economy A is situated above country/economy B in the middle column, and scores lower if country/economy A is situated - China, below country/economy B. For example: Shanghai-China ranks first on the PISA science scale, but Hong Kong which appears second on the list, cannot be distinguished with confidence from Singapore and Japan, which appear third and fourth, respectively. o: Student Performan r © OECD 2014 What Student S Kno W and Can d eading and S C e in m athemati CS i e – Volume C ien C , 216

219 5 nce in Science A A Profile of Student Perform • Figure I.5.1 • omparing countries’ and economies’ performance in science c the OECD average above Statistically significantly Not statistically significantly different from the OECD average below the OECD average Statistically significantly omparison country/ c ean m statistically significantly different from that comparison country’s/economy’s score ountries/economies whose mean score is not economy score c 580 Shanghai-China 555 Hong Kong-China Singapore, Japan Singapore 551 Hong Kong-China, Japan 547 Japan Hong Kong-China, Singapore, Finland, Estonia, Korea Japan, Estonia, Korea 545 Finland Estonia Japan, Finland, Korea 541 Japan, Finland, Estonia, Viet Nam 538 Korea 528 Korea, Poland, Canada, Liechtenstein, Germany, Chinese Taipei, Netherlands, Ireland, Australia, Macao-China Viet Nam 526 Poland Viet Nam, Canada, Liechtenstein, Germany, Chinese Taipei, Netherlands, Ireland, Australia, Macao-China Viet Nam, Poland, Liechtenstein, Germany, Chinese Taipei, Netherlands, Ireland, Australia 525 Canada Liechtenstein Viet Nam, Poland, Canada, Germany, Chinese Taipei, Netherlands, Ireland, Australia, Macao-China 525 Viet Nam, Poland, Canada, Liechtenstein, Chinese Taipei, Netherlands, Ireland, Australia, Macao-China 524 Germany Chinese Taipei 523 Viet Nam, Poland, Canada, Liechtenstein, Germany, Netherlands, Ireland, Australia, Macao-China Viet Nam, Poland, Canada, Liechtenstein, Germany, Chinese Taipei, Ireland, Australia, Macao-China, New Zealand, Switzerland, Netherlands 522 United Kingdom Viet Nam, Poland, Canada, Liechtenstein, Germany, Chinese Taipei, Netherlands, Australia, Macao-China, New Zealand, Switzerland, 522 Ireland United Kingdom Viet Nam, Poland, Canada, Liechtenstein, Germany, Chinese Taipei, Netherlands, Ireland, Macao-China, Switzerland, United Kingdom Australia 521 521 Viet Nam, Poland, Liechtenstein, Germany, Chinese Taipei, Netherlands, Ireland, Australia, Switzerland, United Kingdom Macao-China New Zealand 516 Netherlands, Ireland, Switzerland, Slovenia, United Kingdom Switzerland 515 Netherlands, Ireland, Australia, Macao-China, New Zealand, Slovenia, United Kingdom, Czech Republic Slovenia 514 New Zealand, Switzerland, United Kingdom, Czech Republic 514 United Kingdom Netherlands, Ireland, Australia, Macao-China, New Zealand, Switzerland, Slovenia, Czech Republic, Austria 508 Czech Republic Switzerland, Slovenia, United Kingdom, Austria, Belgium, Latvia 506 United Kingdom, Czech Republic, Belgium, Latvia, France, Denmark, United States Austria 505 Belgium Czech Republic, Austria, Latvia, France, United States Czech Republic, Austria, Belgium, France, Denmark, United States, Spain, Lithuania, Norway, Hungary 502 Latvia Austria, Belgium, Latvia, Denmark, United States, Spain, Lithuania, Norway, Hungary, Italy, Croatia France 499 Austria, Latvia, France, United States, Spain, Lithuania, Norway, Hungary, Italy, Croatia Denmark 498 497 Austria, Belgium, Latvia, France, Denmark, Spain, Lithuania, Norway, Hungary, Italy, Croatia, Luxembourg, Portugal United States Latvia, France, Denmark, United States, Lithuania, Norway, Hungary, Italy, Croatia, Portugal Spain 496 496 Latvia, France, Denmark, United States, Spain, Norway, Hungary, Italy, Croatia, Luxembourg, Portugal Lithuania Norway 495 Latvia, France, Denmark, United States, Spain, Lithuania, Hungary, Italy, Croatia, Luxembourg, Portugal, Russian Federation Latvia, France, Denmark, United States, Spain, Lithuania, Norway, Italy, Croatia, Luxembourg, Portugal, Russian Federation 494 Hungary 494 France, Denmark, United States, Spain, Lithuania, Norway, Hungary, Croatia, Luxembourg, Portugal Italy 491 Croatia France, Denmark, United States, Spain, Lithuania, Norway, Hungary, Italy, Luxembourg, Portugal, Russian Federation, Sweden Luxembourg United States, Lithuania, Norway, Hungary, Italy, Croatia, Portugal, Russian Federation 491 United States, Spain, Lithuania, Norway, Hungary, Italy, Croatia, Luxembourg, Russian Federation, Sweden 489 Portugal Russian Federation 486 Norway, Hungary, Croatia, Luxembourg, Portugal, Sweden Sweden 485 Croatia, Portugal, Russian Federation Iceland Sweden, Slovak Republic, Israel 478 Iceland Slovak Republic Iceland, Israel, Greece, Turkey 471 470 Israel Iceland, Slovak Republic, Greece, Turkey 467 Greece Slovak Republic, Israel, Turkey 463 Turkey Slovak Republic, Israel, Greece 448 United Arab Emirates Bulgaria, Chile, Serbia, Thailand 1, 2 446 United Arab Emirates, Chile, Serbia, Thailand, Romania, Cyprus Bulgaria 445 Chile United Arab Emirates, Bulgaria, Serbia, Thailand, Romania Serbia 445 United Arab Emirates, Bulgaria, Chile, Thailand, Romania Thailand United Arab Emirates, Bulgaria, Chile, Serbia, Romania 444 1, 2 Romania 439 Bulgaria, Chile, Serbia, Thailand, Cyprus 1, 2 Bulgaria, Romania Cyprus 438 429 Kazakhstan Costa Rica Costa Rica, Malaysia Kazakhstan 425 Kazakhstan, Uruguay, Mexico 420 Malaysia Malaysia, Mexico, Montenegro, Jordan Uruguay 416 Malaysia, Uruguay, Jordan Mexico 415 410 Montenegro Uruguay, Jordan, Argentina Jordan Uruguay, Mexico, Montenegro, Argentina, Brazil 409 Montenegro, Jordan, Brazil, Colombia, Tunisia, Albania 406 Argentina Jordan, Argentina, Colombia, Tunisia Brazil 405 Colombia 399 Argentina, Brazil, Tunisia, Albania Argentina, Brazil, Colombia, Albania Tunisia 398 397 Albania Argentina, Colombia, Tunisia 384 Qatar Indonesia Indonesia Qatar, Peru 382 373 Peru Indonesia 1. Note by Turkey: The information in this document with reference to ”Cyprus” relates to the southern part of the Island. There is no single authority representing both Turkish and Greek Cypriot people on the Island. Turkey recognises the Turkish Republic of Northern Cyprus (TRNC). Until a lasting and equitable solution is found within the context of the United Nations, Turkey shall preserve its position concerning the “Cyprus issue”. 2. Note by all the European Union Member States of the OECD and the European Union: The Republic of Cyprus is recognised by all members of the United Nations with the exception of Turkey. The information in this document relates to the area under the effective control of the Government of the Republic of Cyprus. Source: OECD, PISA 2012 Database. http://dx.doi.org/10.1787/888932935629 12 CS OECD 2014 S Kno W and Can d o: Student Performan C e in m athemati What Student , r eading and S C ien C e – Volume i © 217

220 5 nce in Science A Profile of Student Perform A Moreover, countries and economies are divided into three broad groups: those whose mean scores are statistically around the OECD mean (highlighted in dark blue), those whose mean scores are above the OECD mean (highlighted in pale blue), and those whose mean scores are below the OECD mean (highlighted in medium blue). As shown in Figure I.5.1, five countries and economies outperform all other countries and economies in science in PISA 2012 by about half a standard deviation above the average or more: Shanghai-China (580 points), Hong Kong - China (555 points), Singapore (551 points), Japan (547 points) and Finland (545 points). Shanghai-China has a mean score of 580, which is more than three-quarters of a proficiency level above the average of 501 score points in PISA 2012. Other countries with mean performances above the average include Estonia, Korea, Viet Nam, Poland, Canada, Liechtenstein, Germany, Chinese Taipei, the Netherlands, Ireland, Australia, Macao-China, New Zealand, Switzerland, Slovenia, the United Kingdom and the Czech Republic. Countries that performed around the average include Austria, Belgium, Latvia, France, Denmark and the United States. Thirty-seven participating countries and economies have a mean score that is below the OECD average. The gap in performance between the highest- and the lowest-performing OECD countries is 132 score points. That is, while the average score of the highest-performing OECD country, Japan (547), is slightly more than half a standard deviation above the OECD average, the average score of the lowest-performing OECD country, Mexico (415 points) is more than three-quarters of one standard deviation below the OECD average. But the performance difference observed among partner countries and economies is even larger, with a 207 score-point difference between Shanghai-China (580 points) and Peru (373 points). Because the figures are derived from samples, it is not possible to determine a country’s/economy’s precise ranking among all participating countries and economies. However, it is possible to determine with confidence a range of rankings in which the country’s/economy’s performance level lies (Figure I.5.2). For entities other than those for which full samples were drawn (i.e. Shanghai-China, Hong Kong-China, Chinese Taipei and Macao-China) is not possible to calculate a rank order but the mean score provides a possibility to position subnational entities against the performance of the countries and economies. For example Western Australia shows a score just below the performance of top-performer Korea. Trends in average science performance The change in a school system’s average performance over time indicates how and to what extent the system is progressing towards achieving the goal of providing all students with the knowledge and skills needed to become full participants in a knowledge-based society. PISA 2012 science results can be compared with those from PISA 2009 and PISA 2006, when science was first a major domain. PISA 2012 results for 54 countries and economies can be compared with data from both PISA 2009 and PISA 2006; trends for nine countries and economies can be observed using data from PISA 2009 and PISA 2012; and trends for one country can be observed using data from PISA 2006 and PISA 2012. The following trends in average performance are presented as the annualised change for these 64 countries and economies – the average yearly change in science performance observed in a country or economy throughout its participation in 1 PISA. (For further details on the estimation of the annualised change, see Annex A5). On average across OECD countries, science performance has remained broadly stable since 2006. Among the 64 countries and economies with annualised change, 19 countries and economies saw improvements in their science performance. Figure I.5.3 shows that the annualised change was largest in Kazakhstan (at an annual increase of eight score points per year), Turkey (six score points per year), Qatar and Poland (five and four points per year, respectively), Thailand, Romania, Singapore and Italy (three points per year). For example, the average 15-year-old student in Turkey scored 424 points in the PISA 2006 science assessment; three years later, the average student scored 454 points and, in 2012, he or she scored 463 points. Similarly, in Poland in 2006, the average student scored at the OECD average of 498 points in science, improved to 508 points in 2009, then improved again to score 526 points in 2012 (Table I.5.3b). Improvements of more than two score points per year were observed in Israel, Korea, Japan, Dubai (United Arab Emirates), Portugal, Brazil, Ireland, Tunisia, Hong Kong-China and Latvia. Annualised improvement in science was also seen in Macao-China. The average change observed over successive PISA cycles does not capture the extent to which this change is steady, or whether it is decelerating or accelerating. The rate of acceleration of improvement may be steady, in which case the science skills of a country’s/economy’s students improved at a steady pace between 2006 and 2012. The rate may also be accelerating, in which case the improvement between 2009 and 2012 is greater than that between 2006 and 2009; or the rate could be decelerating, in which case there was less of an improvement observed between 2009 and 2012 than between 2006 and 2009. o: Student Performan m © OECD 2014 What Student S Kno W and Can d athemati C i e – Volume C ien C eading and S r , CS e in 218

221 5 A nce in Science A Profile of Student Perform • ] Part 1/3 • Figure I.5.2 [ 2012 participants, at national and regional levels Science performance among pi S a Science scale ange of ranks r countries o a ll countries/economies cd E ean score m ower rank l pper rank u ower rank l pper rank u 1 580 1 Shanghai-China Hong Kong-China 555 2 3 4 2 551 Singapore Japan 6 3 3 1 547 6 4 3 1 545 Finland Estonia 7 5 4 2 541 538 Korea 2 4 5 8 535 Western Australia (Australia) 534 Australian Capital Territory (Australia) Trento (Italy) 533 Friuli Venezia Giulia (Italy) 531 Veneto (Italy) 531 Lombardia (Italy) 529 15 528 7 Viet Nam 527 u nited States) Massachusetts ( 16 8 5 526 Poland 9 ew South Wales (Australia) 526 n 8 Canada 525 5 8 14 Liechtenstein 17 8 525 10 Germany 524 5 8 17 17 9 Chinese Taipei 523 Netherlands 8 11 5 522 18 10 18 11 522 6 Ireland 11 11 7 521 Australia 18 521 nited States) u Connecticut ( 521 13 17 Macao-China Castile and Leon (Spain) 519 Bolzano (Italy) 519 519 Queensland (Australia) Flemish community (Belgium) 518 518 Victoria (Australia) Madrid (Spain) 517 Asturias (Spain) 517 nited Kingdom) u 516 England ( 21 17 14 516 New Zealand 10 22 515 10 15 17 Switzerland 18 Slovenia 514 11 14 21 514 n avarre (Spain) 10 United Kingdom 514 15 16 22 513 u nited Kingdom) Scotland ( South Australia (Australia) 513 512 Emilia Romagna (Italy) 512 Galicia (Spain) 510 La Rioja (Spain) 509 Piemonte (Italy) 14 508 Czech Republic 25 21 17 Valle d’Aosta (Italy) 508 German-speaking community (Belgium) 508 507 nited Kingdom) u orthern Ireland ( n 507 Marche (Italy) 26 Austria 506 18 22 15 506 Basque Country (Spain) 25 22 18 15 505 Belgium Aragon (Spain) 504 Latvia 29 23 502 mbria (Italy) 501 u 501 Liguria (Italy) Toscana (Italy) 501 501 Cantabria (Spain) Tasmania (Australia) 500 22 31 France 499 17 24 24 Denmark 17 23 32 498 United States 497 17 25 24 35 OECD countries are shown in bold black. Partner countries are shown in bold blue. Participating economies and subnational entities that are not included in national results Notes: are shown in bold blue italics. Regions are shown in black italics (OECD countries) or blue italics (partner countries). 1. Note by Turkey: The information in this document with reference to “Cyprus” relates to the southern part of the Island. There is no single authority representing both Turkish and Greek Cypriot people on the Island. Turkey recognises the Turkish Republic of Northern Cyprus (TRNC). Until a lasting and equitable solution is found within the context of the United Nations, Turkey shall preserve its position concerning the “Cyprus issue”. 2. Note by all the European Union Member States of the OECD and the European Union: The Republic of Cyprus is recognised by all members of the United Nations with the exception of Turkey. The information in this document relates to the area under the effective control of the Government of the Republic of Cyprus. Countries, economies and subnational entities are ranked in descending order of mean science performance. OECD, PISA 2012 Database. Source: http://dx.doi.org/10.1787/888932935629 12 CS OECD 2014 S Kno W and Can d o: Student Performan C e in m athemati What Student , r eading and S C ien C e – Volume i © 219

222 5 nce in Science A A Profile of Student Perform Part 2/3 • • Figure I.5.2 [ ] 2012 participants, at national and regional levels a Science performance among pi S Science scale r ange of ranks cd E o ll countries/economies a countries ean score m u l ower rank pper rank l ower rank u pper rank 26 23 496 Spain 18 33 496 34 26 Lithuania 36 19 495 26 Norway 26 Hungary 36 27 26 19 494 494 Alentejo (Portugal) Italy 494 20 26 28 35 492 Catalonia (Spain) 491 Croatia 38 29 491 23 26 32 36 Luxembourg Wales ( 491 u nited Kingdom) Portugal 489 22 27 30 38 French community (Belgium) 487 486 Russian Federation 38 34 Andalusia (Spain) 486 nited States) u 485 Florida ( 485 Sweden 26 28 36 39 Lazio (Italy) 484 Puglia (Italy) 483 n orthern Territory (Australia) 483 483 Balearic Islands (Spain) 483 Extremadura (Spain) Abruzzo (Italy) 482 Perm Territory region (Russian Federation) 480 Murcia (Spain) 479 40 478 29 38 28 Iceland Dubai ( u nited Arab Emirates) 474 Sardegna (Italy) 473 28 471 Slovak Republic 42 39 31 Israel 28 32 39 43 470 468 Molise (Italy) Greece 467 29 32 40 43 Basilicata (Italy) 465 32 30 41 43 Turkey 463 457 Campania (Italy) Sicilia (Italy) 454 Sharjah ( 450 u nited Arab Emirates) 47 448 44 United Arab Emirates 49 446 44 Bulgaria 48 33 44 33 Chile 445 49 Serbia 44 445 44 49 444 Thailand u Abu Dhabi ( 440 nited Arab Emirates) 50 439 47 Romania 1, 2 438 48 50 Cyprus Jalisco (Mexico) 436 435 n uevo León (Mexico) 435 Aguascalientes (Mexico) 432 Querétaro (Mexico) 431 nited Arab Emirates) u Ras Al Khaimah ( 431 Calabria (Italy) 429 Colima (Mexico) 51 429 Costa Rica 52 429 Chihuahua (Mexico) Manizales (Colombia) 429 428 Espírito Santo (Brazil) Distrito Federal (Mexico) 427 425 nited Arab Emirates) u Fujairah ( 425 Morelos (Mexico) 425 Kazakhstan 51 53 425 Ciudad Autónoma de Buenos Aires (Argentina) 423 Puebla (Mexico) 423 Durango (Mexico) Federal District (Brazil) 423 OECD countries are shown in bold black. Partner countries are shown in bold blue. Participating economies and subnational entities that are not included in national results Notes: are shown in bold blue italics. Regions are shown in black italics (OECD countries) or blue italics (partner countries). 1. Note by Turkey: The information in this document with reference to “Cyprus” relates to the southern part of the Island. There is no single authority representing both Turkish and Greek Cypriot people on the Island. Turkey recognises the Turkish Republic of Northern Cyprus (TRNC). Until a lasting and equitable solution is found within the context of the United Nations, Turkey shall preserve its position concerning the “Cyprus issue”. 2. Note by all the European Union Member States of the OECD and the European Union: The Republic of Cyprus is recognised by all members of the United Nations with the exception of Turkey. The information in this document relates to the area under the effective control of the Government of the Republic of Cyprus. Countries, economies and subnational entities are ranked in descending order of mean science performance. OECD, PISA 2012 Database. Source: http://dx.doi.org/10.1787/888932935629 12 o: Student Performan e – Volume © OECD 2014 What Student S Kno W and Can d C C e in m athemati CS , r eading and S C ien i 220

223 5 A nce in Science A Profile of Student Perform • ] Part 3/3 • Figure I.5.2 [ 2012 participants, at national and regional levels Science performance among pi S a Science scale ange of ranks r countries o a ll countries/economies cd E ean score m ower rank l pper rank u ower rank l pper rank u Coahuila (Mexico) 421 421 Mexico (Mexico) u 420 Ajman ( nited Arab Emirates) Minas Gerais (Brazil) 420 52 420 Malaysia 55 419 Rio Grande do Sul (Brazil) 418 Baja California Sur (Mexico) 418 Santa Catarina (Brazil) Medellin (Colombia) 418 417 Baja California (Mexico) São Paulo (Brazil) 417 Quintana Roo (Mexico) 416 San Luis Potosí (Mexico) 416 56 Uruguay 53 416 Paraná (Brazil) 416 mm Al Quwain ( u nited Arab Emirates) 415 u 415 Yucatán (Mexico) Mexico 56 54 34 34 415 Mato Grosso do Sul (Brazil) 415 414 Tamaulipas (Mexico) 412 Tlaxcala (Mexico) 412 Paraíba (Brazil) 411 Bogota (Colombia) 411 Hidalgo (Mexico) Montenegro 56 58 410 59 409 55 Jordan 408 Sinaloa (Mexico) ayarit (Mexico) 407 n 61 Argentina 406 56 Campeche (Mexico) 405 60 405 57 Brazil Guanajuato (Mexico) 404 403 Piauí (Colombia) Zacatecas (Mexico) 402 402 Cali (Brazil) 401 Veracruz (Mexico) 401 Rio de Janeiro (Brazil) 62 59 399 Colombia 59 Tunisia 62 398 60 397 Albania 62 Goiás (Brazil) 396 394 Sergipe (Brazil) 391 Tabasco (Mexico) 390 Bahia (Brazil) Rondônia (Brazil) 389 Rio Grande do orte (Brazil) 387 n Ceará (Brazil) 386 63 384 Qatar 64 382 Amapá (Brazil) 64 382 63 Indonesia 381 Mato Grosso (Brazil) 380 Acre (Brazil) Tocantins (Brazil) 378 Chiapas (Mexico) 377 Pará (Brazil) 377 376 Amazonas (Brazil) Roraima (Brazil) 375 374 Pernambuco (Brazil) 65 Peru 373 65 Guerrero (Mexico) 372 359 Maranhão (Brazil) Alagoas (Brazil) 346 Notes: OECD countries are shown in bold black. Partner countries are shown in bold blue. Participating economies and subnational entities that are not included in national results are shown in bold blue italics. Regions are shown in black italics (OECD countries) or blue italics (partner countries). 1. Note by Turkey: The information in this document with reference to “Cyprus” relates to the southern part of the Island. There is no single authority representing both Turkish and Greek Cypriot people on the Island. Turkey recognises the Turkish Republic of Northern Cyprus (TRNC). Until a lasting and equitable solution is found within the context of the United Nations, Turkey shall preserve its position concerning the “Cyprus issue”. 2. Note by all the European Union Member States of the OECD and the European Union: The Republic of Cyprus is recognised by all members of the United Nations with the exception of Turkey. The information in this document relates to the area under the effective control of the Government of the Republic of Cyprus. Countries, economies and subnational entities are ranked in descending order of mean science performance. OECD, PISA 2012 Database. Source: 12 http://dx.doi.org/10.1787/888932935629 CS OECD 2014 S Kno W and Can d o: Student Performan C e in m athemati What Student , r eading and S C ien C e – Volume i © 221

224 5 nce in Science A Profile of Student Perform A • Figure I.5.3 • a a S pi nnualised change in science performance throughout participation in Science scor e-point difference associated with one calendar year 10 8 6 4 2 0 Annualised change in science performance -2 -4 3 2 3 2 3 3 3 3 3 3 3 3 3 3 3 3 3 2 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 2 3 3 3 3 3 2 3 3 3 2 3 3 2 3 3 3 3 3 3 2 2 3 3 3 3 3 3 2 3 3 Italy Peru Chile Israel Brazil Spain Japan Qatar Latvia Korea Serbia Jordan France Turkey Tunisia Poland Austria Ireland Estonia Greece Finland Iceland Mexico Croatia Albania Canada Norway Sweden Bulgaria Belgium Portugal Slovenia Uruguay Malaysia Thailand Hungary Australia Romania Lithuania Germany Denmark Argentina Indonesia Colombia Singapore Costa Rica Kazakhstan Switzerland Dubai (UAE) Netherlands Montenegro Luxembourg New Zealand Liechtenstein United States Macao-China Chinese Taipei Czech Republic Slovak Republic Shanghai-China United Kingdom Hong Kong-China Russian Federation OECD average 2006 United Arab Emirates* * United Arab Emirates excluding Dubai. Statistically signicant score point changes are marked in a darker tone (see Annex A3). Notes: The number of comparable science scores used to calculate the annualised change is shown in next to the country/economy name. The annualised change is the average annual change in PISA score points from a country’s/economy‘s earliest participation in PISA to PISA 2012. It is calculated taking into account all of a country’s/economy‘s participation in PISA. For more details on the calculation of the annualised change, see Annex A5. OECD average 2006 compares only OECD countries with comparable science scores since 2006. Countries and economies are ranked in descending order of the annualised change in science performance. OECD, PISA 2012 Database, Table I.5.3b. Source: 2 1 http://dx.doi.org/10.1787/888932935629 Results on the rate of acceleration of a country’s/economy’s improvement can be calculated only for the 54 countries and economies that participated in PISA 2006, PISA 2009 and PISA 2012, 16 of which saw an annualised improvement in science performance during the period. Of these 16 countries, Macao-China shows greater improvement between 2009 and 2012 than between 2006 and 2009. Improvements in science performance decelerated in Brazil, Portugal, Qatar, Tunisia and Turkey, where the observed improvement between 2009 and 2012 was smaller than that observed between 2006 and 2009. For the remaining countries, the annualised improvement is relatively similar between the 2006-09 and 2009-12 periods. Other countries and economies show no overall average annual improvement in performance, but do show notable improvements in science performance between PISA 2009 and PISA 2012. Such is the case of Estonia, where science performance improved by 14 score points as well as Luxembourg and Montenegro (Figure I.5.4). At any point in time, countries and economies share similar levels of performance in science with other countries and economies. But as time passes and school systems evolve, some countries and economies may improve their performance while others may not. Figure I.5.5 shows, for each country and economy with comparable results in 2006 and 2012, those other countries and economies that had similar performance in 2006 but whose performance improved or deteriorated by 2012. For example, in 2006, Japan was similar in science performance to New Zealand, Chinese Taipei, Australia, Canada, the Netherlands, Liechtenstein, Hong Kong-China, Estonia and Korea; but after its annualised improvement of 2.6 score points per year, it scored higher in science than New Zealand, Chinese Taipei, Australia, Canada, the Netherlands and Liechtenstein in 2012. In 2006, Germany had lower scores in science than New Zealand, Chinese Taipei and Canada; but by 2012, its performance was similar to those countries’ performance. Along the same lines, Romania had similar levels of performance as Uruguay, Jordan, Montenegro, Mexico, Thailand and Bulgaria in 2006. By 2012, Romania showed better performance than Uruguay, Jordan, Montenegro and Mexico, and had attained similar levels of performance as Chile and Serbia, both of which had higher scores in science than Romania did in 2006. Estonia’s improvement in PISA and recent educational policies and programmes is outlined in Box I.5.1. C Kno S What Student OECD 2014 r © eading and S ien C e – Volume i , CS athemati m e in C o: Student Performan d and Can W 222

225 5 A A Profile of Student Perform nce in Science • • Figure I.5.4 c assessments a S pi urvilinear trajectories of average science performance across formance (quadratic term) Rate of acceleration or deceleration in per Accelerating Steadily changing Decelerating PISA science score PISA science score PISA science score 2012 2009 2009 2006 2006 2012 2009 2012 2006 Countries/economies Hong Kong-China Macao-China Korea Brazil Ireland Latvia Portugal with positive annualised change Israel Poland Qatar Italy Romania Tunisia Japan Thailand Turkey PISA science score PISA science score PISA science score 2012 2009 2009 2006 2006 2012 2006 2009 2012 Countries/economies Czech Republic Argentina Croatia Indonesia Serbia Norway Estonia Australia Denmark Liechtenstein Spain Luxembourg Belgium France Lithuania Switzerland Montenegro Bulgaria Germany Mexico United Kingdom with no signicant annualised change Slovenia Chile Greece Netherlands United States Chinese Taipei Colombia Hungary Russian Fed. PISA science score PISA science score PISA science score 2006 2012 2009 2006 2012 2006 2009 2012 2009 Countries/economies Iceland Canada New Zealand Finland with negative annualised change Slovak Republic Jordan Sweden Uruguay Notes: Figures are for illustrative purposes only. Countries and economies are grouped according to the direction and signicance of their annualised change and their rate of acceleration. Countries and economies with data from only one PISA assessments other than 2012 are excluded. Source: OECD, PISA 2012 Database, Table I.5.3b. 1 http://dx.doi.org/10.1787/888932935629 2 athemati CS , r eading and S C ien C and Can i © OECD 2014 223 o: Student Performan C d e in m W Kno S What Student e – Volume

226 5 nce in Science A A Profile of Student Perform • ] Part 1/4 [ Figure I.5.5 • ultiple comparisons of science performance between 2006 and 2012 m countries/economies with countries/economies with countries/economies with countries/economies with Science Science ountries/economies with similar c higher performance in 2006 ountries/economies with similar c higher performance in 2006 ountries/economies with similar c lower performance in 2006 Science lower performance in 2006 Science performance performance performance in 2006 but lower performance performance in 2006 but with similar performance performance in 2006 but higher performance performance but similar performance performance in 2006 in 2012 but higher performance in 2012 in 2012 and similar performance in 2012 in 2012 but lower performance in 2012 in 2012 in 2012 in 2012 in 2006 Hong Kong 555 Chinese T aipei, Canada - China Finland 555 542 542 Hong Kong-China Japan 547 Finland China, Estonia, Korea - Japan Hong Kong New Zealand, Chinese Taipei, Australia, 547 531 Japan 531 Canada, Netherlands, Liechtenstein 545 563 545 Hong Kong-China Estonia, Japan, Korea Finland Finland 563 Japan, Korea Finland 541 531 Estonia Estonia 531 541 New Zealand, Chinese Taipei, Australia, Canada, Netherlands, Liechtenstein Canada Finland Poland New Zealand, United Kingdom, 538 522 522 Korea Korea 538 Estonia, Japan, Liechtenstein Germany, Austria, Czech Republic, Chinese Taipei, Australia, Netherlands, Switzerland, Slovenia 526 United States, Croatia, Latvia, 498 Poland New Zealand, Ireland 526 Poland 498 Austria, Czech Republic, United Kingdom, Germany, Slovak Republic, Lithuania, France, Belgium, Slovenia Sweden, Hungary, Spain, Denmark, Chinese Taipei, Australia, Iceland Canada, Macao-China, Netherlands, Switzerland, Liechtenstein, Korea Chinese Taipei, Australia - New Zealand 525 Poland, Germany, Ireland, Hong Kong China, Estonia, Japan 534 Canada Korea 525 Canada 534 Macao-China, Netherlands, Liechtenstein Canada 525 Poland, Macao-China Austria, Czech Republic, Belgium, 522 Liechtenstein Liechtenstein 522 525 New Zealand, United Kingdom, Estonia, Japan Slovenia Germany, Chinese Taipei, Australia, Ireland, Netherlands, Switzerland, Korea New Zealand, Chinese Taipei, Germany Austria, Czech Republic, Hungary, 524 516 516 United Kingdom, Australia, Ireland, 524 Poland Germany Korea Canada China, Netherlands, Switzerland, - Macao Belgium, Slovenia Liechtenstein Chinese Taipei Hong Kong - China, Estonia, Japan, Korea Poland, United Kingdom, New Zealand, Australia, Canada, 523 532 Chinese Taipei 532 523 Netherlands, Liechtenstein Germany, Ireland, Macao-China, Switzerland Czech Republic 522 525 Netherlands Poland, United Kingdom, Estonia, Japan, Korea New Zealand, Germany, Chinese Taipei, 525 Canada 522 Netherlands Australia, Slovenia, Liechtenstein Ireland, Macao-China, Switzerland New Zealand, Chinese Taipei, Austria, Czech Republic, Sweden, Poland, United Kingdom, Germany, 508 522 Ireland Ireland Slovenia 522 508 - Hungary, Belgium Australia, Canada, Macao China, Switzerland, Liechtenstein Netherlands Australia 527 521 New Zealand, Germany, Chinese Taipei, Poland, United Kingdom, Estonia, Japan, Korea 521 Australia 527 Canada, Netherlands, Liechtenstein Ireland, Macao-China, Switzerland China - Macao New Zealand, Chinese Taipei, Poland United Kingdom, Germany, Ireland, ustria, Czech Republic, Hungary, A 521 511 Macao-China 511 521 Slovenia Switzerland Belgium Australia, Canada, Netherlands, Liechtenstein 530 Estonia, Japan, Canada, Korea Chinese Taipei, Australia, Netherlands, New Zealand Poland, United Kingdom, 516 New Zealand 516 530 Liechtenstein Germany, Czech Republic, Ireland, Macao-China, Switzerland, Slovenia 515 512 Switzerland Switzerland Poland New Zealand, Chinese Taipei, United Kingdom, Germany, Austria, 515 512 Sweden, Hungary, Belgium Korea Czech Republic, Ireland, Macao Australia, Netherlands China, - Slovenia, Liechtenstein 519 514 519 Poland, Ireland, Macao-China Germany, Liechtenstein, Korea Slovenia United Kingdom, Czech Republic, New Zealand 514 Slovenia Austria Netherlands, Switzerland United Kingdom 515 514 Germany, Austria, Czech Republic, 515 514 United Kingdom New Zealand, Chinese Taipei, Poland, Latvia Korea Australia, Netherlands - Ireland, Macao China, Belgium, Switzerland, Slovenia, Liechtenstein Sweden, Hungary 513 508 New Zealand Poland United States, Latvia, France, China, - Germany, Ireland, Macao Czech Republic Czech Republic 513 508 United Kingdom, Austria, Belgium, Netherlands, Liechtenstein, Korea Switzerland, Slovenia Denmark Sweden, Hungary 506 United States, Latvia, Austria 511 United Kingdom, Czech Republic, Germany, Ireland, Macao - China, Austria 506 Poland 511 Belgium, Switzerland Slovenia, Liechtenstein, Korea Lithuania, France, Denmark, Norway 510 United States, Latvia, France, - China, 505 Poland Belgium Germany, Ireland, Macao United Kingdom, Austria, Belgium Sweden, Hungary 505 510 Switzerland, Liechtenstein Czech Republic Denmark 502 Slovak Republic, Luxembourg, Iceland, Latvia United States, Croatia, Lithuania, France, 490 Poland Italy United Kingdom, Austria, Sweden Latvia 490 502 Russian Federation Spain, Denmark, Norway Czech Republic, Hungary, Belgium Portugal, Italy 495 France 499 495 Poland Slovak Republic, Sweden, Iceland 499 Austria, Czech Republic, United States, Croatia, Latvia, Lithuania, France Hungary, Spain, Denmark, Norway Belgium Poland United States, Croatia, Latvia, Lithuania, Slovak Republic, Sweden, Iceland 498 496 Denmark 496 498 Denmark Luxembourg, Portugal, Italy Austria, Czech Republic, Belgium France, Hungary, Spain, Norway United States 489 497 United States 489 Austria, Czech Republic, Portugal, Italy 497 Slovak Republic, Iceland Croatia, Latvia, Luxembourg, Lithuania, Poland France, Spain, Denmark, Norway, Sweden, Hungary, Belgium Russian Federation Note: Only countries and economies that participated in the PISA 2006 and PISA 2012 assessments are shown. Countries and economies are ranked in descending order of their mean science performance in PISA 2012. OECD, PISA 2012 Database, Table I.5.3b. Source: 12 http://dx.doi.org/10.1787/888932935629 o: Student Performan e – Volume © OECD 2014 What Student S Kno W and Can d C C e in m athemati CS , r eading and S C ien i 224

227 5 nce in Science A A Profile of Student Perform • ] Part 2/4 [ • Figure I.5.5 ultiple comparisons of science performance between 2006 and 2012 m ountries/economies with c c ountries/economies with c ountries/economies with ountries/economies with c Science Science countries/economies with similar higher performance in 2006 countries/economies with similar higher performance in 2006 countries/economies with similar lower performance in 2006 Science wer performance in 2006 lo Science performance performance performance in 2006 but lower performance performance in 2006 but with similar performance performance in 2006 but higher performance performance but similar performance performance in 2006 in 2012 but higher performance in 2012 in 2012 and similar performance in 2012 in 2012 but lower performance in 2012 in 2012 in 2012 in 2012 in 2006 Japan Finland 542 Hong Kong 555 - China 555 Hong Kong-China 542 Chinese Taipei, Canada New Zealand, Chinese Taipei, Australia, 547 531 Japan Japan 531 547 Finland Hong Kong-China, Estonia, Korea Canada, Netherlands, Liechtenstein - 563 545 Estonia, Japan, Korea Hong Kong Finland China 545 563 Finland Japan, Korea Estonia 531 541 New Zealand, Chinese Taipei, Australia, Estonia 531 Finland 541 Canada, Netherlands, Liechtenstein Korea Korea 522 538 New Zealand, United Kingdom, 538 522 Canada Finland Poland Estonia, Japan, Liechtenstein Germany, Austria, Czech Republic, Chinese Taipei, Australia, Netherlands, Switzerland, Slovenia 526 498 Poland New Zealand, United States, Croatia, Latvia, Ireland Poland Austria, Czech Republic, 526 498 Slovak Republic, Lithuania, France, United Kingdom, Germany, Belgium, Slovenia Chinese Taipei, Australia, Sweden, Hungary, Spain, Denmark, Canada, Macao Iceland China, - Netherlands, Switzerland, Liechtenstein, Korea Chinese Taipei, Australia Canada 534 525 New Zealand Poland, Germany, Ireland, Hong Kong-China, Estonia, Japan Korea 525 534 Canada Macao - China, Netherlands, Liechtenstein - China Canada 525 522 Poland, Macao Liechtenstein Liechtenstein 522 525 Austria, Czech Republic, Belgium, New Zealand, United Kingdom, Estonia, Japan Slovenia Germany, Chinese Taipei, Australia, Ireland, Netherlands, Switzerland, Korea New Zealand, Chinese Taipei, 516 Poland United Kingdom, Australia, Ireland, Austria, Czech Republic, Hungary, 524 516 Germany 524 Germany Korea Canada Belgium, Slovenia Macao-China, Netherlands, Switzerland, Liechtenstein Chinese Taipei Hong Kong-China, Estonia, Japan, Korea New Zealand, Australia, Canada, 532 523 Poland, United Kingdom, 523 532 Chinese Taipei Germany, Ireland, Netherlands, Liechtenstein Macao - China, Switzerland 522 525 Netherlands Poland, United Kingdom, Estonia, Japan, Korea New Zealand, Germany, Chinese Taipei, Czech Republic Canada 522 525 Netherlands Australia, Slovenia, Liechtenstein China, - Ireland, Macao Switzerland 522 Austria, Czech Republic, Sweden, Poland, United Kingdom, Germany, Ireland New Zealand, Chinese Taipei, 508 Slovenia 522 508 Ireland Australia, Canada, Hungary, Belgium Macao-China, Switzerland, Liechtenstein Netherlands Australia 527 521 New Zealand, Germany, Chinese Taipei, Estonia, Japan, Korea Poland, United Kingdom, 527 521 Australia - China, Ireland, Macao Canada, Netherlands, Liechtenstein Switzerland Macao-China 521 Austria, Czech Republic, Hungary, 511 United Kingdom, Germany, Ireland, Poland New Zealand, Chinese Taipei, China - Slovenia 521 511 Macao Australia, Canada, Switzerland Belgium Netherlands, Liechtenstein New Zealand 530 516 Chinese Taipei, Australia, Netherlands, Poland, United Kingdom, Estonia, Japan, Canada, Korea 516 530 New Zealand Liechtenstein Germany, Czech Republic, Ireland, Macao - China, Switzerland, Slovenia Poland Sweden, Hungary, Belgium 515 United Kingdom, Germany, Austria, 515 New Zealand, Chinese Taipei, 512 Switzerland 512 Switzerland Korea Australia, Netherlands Czech Republic, Ireland, Macao-China, Slovenia, Liechtenstein Austria China - Poland, Ireland, Macao Germany, Liechtenstein, Korea United Kingdom, Czech Republic, 514 519 Slovenia venia Slo 519 514 New Zealand Netherlands, Switzerland Poland, Latvia United Kingdom 515 514 New Zealand, Chinese Taipei, United Kingdom 515 514 Germany, Austria, Czech Republic, Korea Ireland, Macao-China, Belgium, Australia, Netherlands Switzerland, Slovenia, Liechtenstein Poland United States, Latvia, France, New Zealand 508 Czech Republic Czech Republic 513 508 Sweden, Hungary United Kingdom, Austria, Belgium, Germany, Ireland, Macao-China, 513 Denmark Netherlands, Liechtenstein, Korea Switzerland, Slovenia 506 511 Austria Germany, Ireland, Macao-China, United Kingdom, Czech Republic, United States, Latvia, Sweden, Hungary Poland 506 511 Austria Belgium, Switzerland Slovenia, Liechtenstein, Korea Lithuania, France, Denmark, Norway Poland Germany, Ireland, Macao-China, Belgium 510 505 Sweden, Hungary United Kingdom, Austria, Belgium 510 United States, Latvia, France, 505 Czech Republic Switzerland, Liechtenstein Denmark 502 490 United States, Croatia, Lithuania, France, Slovak Republic, Luxembourg, Iceland, Latvia Poland Italy United Kingdom, Austria, 502 Latvia Sweden 490 Czech Republic, Hungary, Spain, Denmark, Norway Russian Federation Belgium 499 United States, Croatia, Latvia, Lithuania, 495 France Poland Portugal, Italy Austria, Czech Republic, Slovak Republic, Sweden, Iceland France 499 495 Belgium Hungary, Spain, Denmark, Norway United States, Croatia, Latvia, Lithuania, Slovak Republic, Sweden, Iceland 498 496 Denmark Denmark 496 Austria, Czech Republic, Luxembourg, Portugal, Italy 498 Poland Belgium France, Hungary, Spain, Norway United States 489 United States 489 497 497 Austria, Czech Republic, Portugal, Italy Slovak Republic, Iceland Croatia, Latvia, Luxembourg, Lithuania, Poland France, Spain, Denmark, Norway, Sweden, Hungary, Belgium Russian Federation Note: Only countries and economies that participated in the PISA 2006 and PISA 2012 assessments are shown. Countries and economies are ranked in descending order of their mean science performance in PISA 2012. OECD, PISA 2012 Database, Table I.5.3b. Source: 12 http://dx.doi.org/10.1787/888932935629 CS OECD 2014 S Kno W and Can d o: Student Performan C e in m athemati What Student , r eading and S C ien C e – Volume i © 225

228 5 A nce in Science A Profile of Student Perform • Part 3/4 [ Figure I.5.5 • ] m ultiple comparisons of science performance between 2006 and 2012 countries/economies with countries/economies with countries/economies with countries/economies with Science Science ountries/economies with similar c higher performance in 2006 ountries/economies with similar c higher performance in 2006 ountries/economies with similar c lower performance in 2006 Science lower performance in 2006 Science performance performance performance in 2006 but lower performance performance in 2006 but with similar performance performance in 2006 but higher performance performance but similar performance performance in 2006 in 2012 but higher performance in 2012 in 2012 and similar performance in 2012 in 2012 but lower performance in 2012 in 2012 in 2012 in 2012 in 2006 Spain 496 Slovak Republic, Iceland, 488 United States, Croatia, Latvia, Spain 488 496 Portugal, Italy Hungary Sweden Poland Luxembourg, Lithuania, France, Russian Federation Denmark, Norway 488 496 Slovak Republic, Iceland United States, Croatia, Latvia, Lithuania Lithuania 488 Portugal, Italy Austria, Hungary Sweden 496 Poland Luxembourg, France, Spain, Denmark, Norway, Russian Federation Austria, Sweden, Hungary Portugal, Italy United States, Croatia, Latvia, Norway 487 Norway 487 495 Slovak Republic, Iceland 495 Luxembourg, Lithuania, France, Spain, Denmark, Russian Federation Hungary 504 494 France, Sweden, Denmark Poland, Germany, Austria, United States, Croatia, Latvia, 504 494 Hungary China, - Czech Republic, Ireland, Macao Luxembourg, Lithuania, Belgium, Switzerland Spain, Portugal, Norway, Russian Federation, Italy Italy 475 494 Greece Portugal, Russian Federation United States, Croatia, Latvia, Slovak Republic, Iceland 494 475 Italy Luxembourg, Lithuania, France, Sweden, Hungary, Spain, Denmark, Norway Luxembourg, Portugal, Croatia 493 491 Sweden, Hungary Croatia 493 491 Slovak Republic, Iceland United States, Latvia, Lithuania, France, Poland Spain, Denmark, Norway Russian Federation, Italy Luxembourg 486 491 Croatia, Sweden, Hungary, Luxembourg 486 491 Slovak Republic, Iceland United States, Lithuania, Spain, Norway, Portugal, Italy Latvia Russian Federation Denmark Portugal United States, Croatia, Russian Federation, Italy Greece 489 474 Slovak Republic Portugal 474 489 Luxembourg, Lithuania, France, Sweden, Hungary, Spain, Denmark, Iceland, Norway Russian Federation Russian Federation 479 486 Greece, Slovak Republic United States, Luxembourg, Lithuania, 479 Latvia, Spain Croatia, Sweden, Hungary, 486 Iceland Portugal, Norway, Italy United States, Croatia, Sweden 503 485 Hungary Poland, Austria, Czech Republic, France, Latvia, Lithuania, Spain 485 503 Sweden Ireland, Belgium, Denmark, Switzerland Luxembourg, Israel, Iceland, Portugal, Norway, Russian Federation, Italy United States, Poland, Croatia, Latvia, Slovak Republic 478 491 Iceland Iceland 491 478 Sweden Italy Israel, Portugal, Luxembourg, Lithuania, France, Spain, Russian Federation Denmark, Norway Portugal, Italy 471 United States, Poland, Croatia, Latvia, 488 Slovak Republic Greece, Turkey, Israel Iceland 471 488 Slovak Republic Luxembourg, Lithuania, France, Spain, Denmark, Norway, Russian Federation 454 470 Chile Turkey Greece, Slovak Republic, Israel 470 454 Israel Sweden, Iceland Slovak Republic 467 Portugal, Russian Federation, Italy Turkey, Israel 473 467 473 Greece Greece Uruguay, Thailand, Jordan, Chile, 424 463 463 Turkey Bulgaria Greece, Slovak Republic, 424 Turkey Israel Serbia, Romania 434 446 Uruguay, Jordan Bulgaria 446 434 Bulgaria Thailand, Turkey, Chile, Serbia, Romania Chile 438 445 Uruguay Bulgaria, Serbia Turkey, Israel Thailand, Romania 445 438 Chile Serbia 436 445 Uruguay Bulgaria, Chile Turkey Thailand, Romania 445 436 Serbia Thailand 444 Chile, Serbia Turkey 421 Thailand 421 444 Uruguay, Jordan Bulgaria, Romania Chile, Serbia 439 418 Turkey Romania Romania 418 439 Uruguay, Jordan, Montenegro, Mexico Thailand, Bulgaria 416 Argentina, Montenegro, Thailand, Turkey, Bulgaria, Chile, Jordan Uruguay Uruguay 416 428 428 Mexico Serbia, Romania 410 Mexico 410 415 Indonesia, Montenegro Romania Argentina Uruguay, Jordan 415 Mexico Montenegro 412 410 Mexico, Romania Brazil, Argentina Uruguay, Jordan 410 412 Montenegro Jordan Brazil, Argentina, Thailand, Turkey, Bulgaria, Romania Uruguay 422 409 422 409 Jordan Montenegro, Tunisia, Colombia, Mexico Uruguay, Jordan, 391 406 Indonesia Brazil, Tunisia, Colombia Argentina Argentina 406 391 Montenegro, Mexico Indonesia Brazil 405 390 Argentina, Tunisia, Colombia Jordan, Montenegro 405 390 Brazil Colombia 388 399 Indonesia Brazil, Argentina, Tunisia Jordan 399 388 Colombia Tunisia Indonesia 398 386 Brazil, Argentina, Colombia 398 Jordan Tunisia 386 349 Qatar Qatar 349 384 Indonesia 384 382 393 382 Brazil, Argentina, Tunisia, Colombia, Indonesia Indonesia 393 Qatar Mexico Note: Only countries and economies that participated in the PISA 2006 and PISA 2012 assessments are shown. Countries and economies are ranked in descending order of their mean science performance in PISA 2012. Source: OECD, PISA 2012 Database, Table I.5.3b. http://dx.doi.org/10.1787/888932935629 12 What Student e – Volume d o: Student Performan C e in m athemati CS W Kno S and Can OECD 2014 © C , r eading and S C ien i 226

229 5 A nce in Science A Profile of Student Perform • [ Figure I.5.5 • ] Part 4/4 m ultiple comparisons of science performance between 2006 and 2012 ountries/economies with c ountries/economies with c ountries/economies with c c ountries/economies with Science Science countries/economies with similar higher performance in 2006 countries/economies with similar higher performance in 2006 countries/economies with similar lower performance in 2006 Science lo wer performance in 2006 Science performance performance performance in 2006 but lower performance performance in 2006 but with similar performance performance in 2006 but higher performance performance but similar performance performance in 2006 in 2012 but higher performance in 2012 in 2012 and similar performance in 2012 in 2012 but lower performance in 2012 in 2012 in 2012 in 2012 in 2006 488 496 Slovak Republic, Iceland, Spain United States, Croatia, Latvia, Spain 496 Sweden Portugal, Italy Hungary 488 Poland Russian Federation Luxembourg, Lithuania, France, Denmark, Norway 488 Slovak Republic, Iceland United States, Croatia, Latvia, Lithuania Lithuania 496 Portugal, Italy Austria, Hungary Sweden 496 488 Poland Luxembourg, France, Spain, Denmark, Norway, Russian Federation Slovak Republic, Iceland Norway 487 495 Austria, Sweden, Hungary Norway 487 495 Portugal, Italy United States, Croatia, Latvia, Luxembourg, Lithuania, France, Spain, Denmark, Russian Federation 504 494 Hungary France, Sweden, Denmark United States, Croatia, Latvia, Poland, Germany, Austria, 494 504 Hungary Luxembourg, Lithuania, Czech Republic, Ireland, Macao-China, Spain, Portugal, Norway, Belgium, Switzerland Russian Federation, Italy 494 Greece Portugal, Russian Federation United States, Croatia, Latvia, Italy 475 Italy Slovak Republic, Iceland 494 475 Luxembourg, Lithuania, France, Sweden, Hungary, Spain, Denmark, Norway Luxembourg, Portugal, 493 Slovak Republic, Iceland United States, Latvia, Lithuania, France, Croatia Croatia 491 493 Sweden, Hungary 491 Poland Russian Federation, Italy Spain, Denmark, Norway Portugal, Italy Luxembourg Luxembourg 486 491 Croatia, Sweden, Hungary, 486 491 Slovak Republic, Iceland United States, Lithuania, Spain, Norway, Latvia Russian Federation Denmark Portugal 474 489 Greece Russian Federation, Italy United States, Croatia, Portugal 474 489 Slovak Republic Luxembourg, Lithuania, France, Sweden, Hungary, Spain, Denmark, Iceland, Norway Croatia, Sweden, Hungary, Russian Federation Russian Federation 479 486 Latvia, Spain 479 486 Greece, Slovak Republic United States, Luxembourg, Lithuania, Portugal, Norway, Italy Iceland United States, Croatia, 503 Poland, Austria, Czech Republic, France, Hungary 485 Sweden Sweden 503 485 Latvia, Lithuania, Spain Ireland, Belgium, Denmark, Switzerland Luxembourg, Israel, Iceland, Portugal, Norway, Russian Federation, Italy United States, Poland, Croatia, Latvia, Slovak Republic 478 491 Iceland Iceland 491 478 Sweden Italy Israel, Portugal, Luxembourg, Lithuania, France, Spain, Russian Federation Denmark, Norway Portugal, Italy United States, Poland, Croatia, Latvia, 471 Slovak Republic 488 Iceland 471 488 Slovak Republic Greece, Turkey, Israel Luxembourg, Lithuania, France, Spain, Denmark, Norway, Russian Federation 470 Chile 470 Greece, Slovak Republic, Israel 454 454 Israel Turkey Sweden, Iceland 467 Portugal, Russian Federation, Italy Turkey, Israel 473 467 473 Greece Greece Slovak Republic 424 463 Uruguay, Thailand, Jordan, Chile, Turkey Bulgaria Greece, Slovak Republic, 463 424 Turkey Israel Serbia, Romania Bulgaria 446 Uruguay, Jordan Thailand, Turkey, Chile, Serbia, 446 434 Bulgaria 434 Romania Chile 438 445 Uruguay Bulgaria, Serbia Turkey, Israel Thailand, Romania 445 438 Chile Serbia 436 445 Uruguay Bulgaria, Chile Turkey Thailand, Romania 445 436 Serbia Chile, Serbia 421 Thailand 444 Thailand 421 444 Uruguay, Jordan Bulgaria, Romania Turkey Chile, Serbia Turkey Thailand, Bulgaria Uruguay, Jordan, Montenegro, Mexico Romania 418 439 439 418 Romania 428 Uruguay 416 Jordan Thailand, Turkey, Bulgaria, Chile, 428 Argentina, Montenegro, Uruguay 416 Mexico Serbia, Romania 415 Mexico 410 Indonesia, Montenegro Romania Argentina Uruguay, Jordan 415 410 Mexico Montenegro 412 Montenegro 412 410 Mexico, Romania Brazil, Argentina Uruguay, Jordan 410 409 Brazil, Argentina, Thailand, Turkey, Bulgaria, Romania Uruguay 422 Jordan 409 422 Jordan Montenegro, Tunisia, Colombia, Mexico Argentina 391 Argentina Indonesia Brazil, Tunisia, Colombia Uruguay, Jordan, 391 406 406 Montenegro, Mexico Brazil 390 405 Indonesia Argentina, Tunisia, Colombia Jordan, Montenegro 405 390 Brazil Colombia 388 399 Indonesia Brazil, Argentina, Tunisia Jordan 399 388 Colombia Indonesia Jordan Brazil, Argentina, Colombia Tunisia 398 386 398 Tunisia 386 Qatar 349 384 Indonesia 384 349 Qatar 393 382 Brazil, Argentina, Tunisia, Colombia, Qatar 382 Indonesia Indonesia 393 Mexico Note: Only countries and economies that participated in the PISA 2006 and PISA 2012 assessments are shown. Countries and economies are ranked in descending order of their mean science performance in PISA 2012. Source: OECD, PISA 2012 Database, Table I.5.3b. http://dx.doi.org/10.1787/888932935629 12 and Can OECD 2014 m athemati CS , r eading and S C C o: Student Performan d e in W Kno S What Student ien C e – Volume i © 227

230 5 A nce in Science A Profile of Student Perform Figure I.5.6 shows the relationship between each country’s/economy’s average science performance in 2006 and their 2 annualised change between 2006 and 2012. The correlation between performance in PISA 2006 and the annualised change is -0.39, signalling that countries and economies that had lower performance in their first PISA science assessment are more likely to be those that improve the fastest. To put it another way, 15% of the variation in countries’/economies’ annualised change in science performance can be explained by its initial performance in PISA (Table I.5.3b). Of the 19 countries and economies that saw an improvement in science performance since PISA 2006, nine had an average initial score of 470 score points, well below the OECD average. Figure I.5.6 • • elationship between annualised change in science performance r pi a 2006 science scores and average S PISA 2006 performance PISA 2006 performance OECD average OECD average below above 7 1. Spain Turkey Lithuania 2. 6 3. Austria 4. Belgium Qatar Performance improved 5 Poland Thailand 4 Romania Italy 3 Israel Korea Japan Annualised change in science performance Argentina Portugal Hong Kong-China Ireland Latvia Tunisia Bulgaria 2 Brazil Macao-China Germany United States Norway Serbia Colombia Estonia Russian 2 1 Chile 1 OECD average 2006 Federation Luxembourg Switzerland Mexico France Liechtenstein Denmark 0 United Kingdom Performance deteriorated Croatia Netherlands Montenegro Slovenia 3 4 Greece -1 Australia Czech Republic Canada Indonesia Iceland -2 Hungary Chinese Taipei Jordan Uruguay New Zealand -3 Slovak Republic Finland Sweden -4 600 500 350 400 450 550 325 570 525 475 425 375 Mean score in science in PISA 2006 Notes: Annualised score point change in science that are statistically signicant are indicated in a darker tone (see Annex A3). The annualised change is the average annual change in PISA score points from a country/economy’s earliest participation in PISA to PISA 2012. It is calculated taking into account all of a country’s/economy’s participation in PISA. For more details on the calculation of the annualised change, see Annex A5. OECD average 2006 considers only those countries with comparable data since PISA 2006. The correlation between a country’s/economy’s mean score in 2006 and its annualised performance is -0.39. Source: OECD, PISA 2012 Database, Tables I.5.3b. 1 2 http://dx.doi.org/10.1787/888932935629 Yet it is not inevitable that only countries and economies that perform below the OECD average show improvements over time. Japan, for example, performed significantly above the OECD average in science in 2006 (at 531 points) and by 2012, shows an annualised improvement in science performance of around two score points per year. Estonia had similar levels of performance to Japan in PISA 2006 and improved, in the three years between PISA 2009 and PISA 2012 by 14 score points. Similarly, among the countries and economies that scored around the OECD average in science in 2006, m C o: Student Performan d and Can W Kno S What Student OECD 2014 e in © i athemati CS r eading and S C ien C e – Volume , 228

231 5 nce in Science A Profile of Student Perform A Poland and Ireland saw improvements by 2012 but Sweden and Hungary did not. The Russian Federation, Italy, Portugal and Greece, for example, all showed similar levels of performance in science in 2006 (around 475 points), but while Italy and Portugal improved their performance by 2012, the Russian Federation and Greece did not. Also telling is that among countries that performed below the OECD average in 2006, eight countries saw no improvement up until 2012. This underscores the fact that all countries and economies can improve their science performance, irrespective of how well they perform in science (Figure I.5.6). Trends in science performance adjusted for sampling and demographic changes There are many reasons why a country’s or economy’s science performance may change over time. Improvements may be the result of specific education policies or changes in the demographic characteristics of the population. For example, because of trends in migration, the characteristics of the PISA reference population – 15-year-olds enrolled in school – may have shifted; or, as a result of economic, cultural and social development, the environments in which students live can better promote student learning. By asking students about their after-school experiences and backgrounds, PISA can identify whether the socio-economic conditions of students have changed and whether more students had an immigrant background in 2012 than did in previous years. These differences in the characteristics of the reference population may 3 be driving the observed trends in some countries but not in others. Adjusted trends shed light on those trends in science performance that are not due to changes in the demographic and economic characteristics of the student population. Figure I.5.7 presents the adjusted annualised change after - socio assuming that the average age and socio-economic status of students in 2006 and 2009 is the same as that of students who took part in PISA 2012. This adjusted trend also assumes that the proportion of girls, students with an immigrant background and students who speak a language at home that is different from that of the assessment is identical in previous cycles to those observed in PISA 2012. In short, it assumes that the population and sample characteristics observed in 2012 have not changed since 2006. Countries and economies that see a difference between the adjusted trends and the observed trends, particularly when the observed trend is more negative than the adjusted trend (non-negative), can consider these changes in the student population as a challenge that needs to be addressed by the school system, as it is the observed trends, not the adjusted trends, that measure the quality and the real-life outcome of school systems. After accounting for differences in the sampling and population characteristics, 11 countries and economies show an improvement in science performance. For these countries and economies, the annualised change in performance observed throughout their participation in PISA is not completely attributable to changes in the background characteristics of the students who take part in PISA. This means that, in these countries and economies, either the background characteristic of students haven’t changed during the period, that any changes that may have taken place have not brought about differences in average performance, or that improved education services have offset any negative effect on average science performance related to changes in the population. On average across OECD countries, for example, the observed overall annualised improvement in science performance is no longer observed after changes in students’ demographic characteristics are taken into account. This means that, on average across OECD countries, improvements in science performance can be explained by changes in the background characteristics of the student population. Similarly, the annualised improvement observed in Brazil, Hong Kong-China, Ireland, Korea, Latvia, Portugal and Tunisia is no longer apparent when comparing students with similar characteristics across the different PISA assessments. By contrast, less than 20% of the improvement observed in Dubai (United Arab Emirates), Israel, Italy, Kazakhstan and Turkey can be attributed to changes in the demographic profile of the student population. In these countries and economies, improvements in science performance remain after accounting for students’ background characteristics. Although an important part of the annualised improvement observed in Japan, Poland, Qatar, Romania, Singapore and Thailand is explained by changes in the demographic characteristics of the student population, improvements are still observed when comparing students with similar characteristics in 2012 and previous PISA assessments. In these countries and economies, only part of the observed annualised trend can be attributed to changing country demographics. In Japan, for example, there was an average annual improvement in science performance of 2.6 points; but after accounting for changes in students’ background characteristics, this annualised improvement remains but decreases to 2.0 science score points per year. In Macao-China, the observed annualised improvement between PISA 2006 and PISA 2012 becomes negative after accounting for demographic changes in the population. CS eading and S S Kno W and Can d o: Student Performan C e in m athemati What Student , 229 OECD 2014 © i e – Volume C ien C r

232 5 A A Profile of Student Perform nce in Science Figure I.5.7 • • djusted and observed annualised performance change in average a science scores pi S a After accounting for social and demographic changes Before accounting for social and demographic changes 9 8 7 6 5 4 3 2 1 Annualised change in science performance 0 -1 -2 -3 -4 Italy Peru Chile Israel Brazil Spain Japan Qatar Latvia Korea Serbia Jordan France Turkey Tunisia Poland Austria Ireland Estonia Greece Iceland Finland Mexico Croatia Canada Norway Sweden Bulgaria Belgium Portugal Slovenia Uruguay Malaysia Thailand Hungary Australia Romania Lithuania Germany Denmark Argentina Indonesia Colombia Singapore Costa Rica Kazakhstan Switzerland Dubai (UAE) Netherlands Montenegro Luxembourg New Zealand Liechtenstein United States Macao-China Chinese Taipei Czech Republic Slovak Republic Shanghai-China United Kingdom Hong Kong-China Russian Federation OECD average 2006 United Arab Emirates* * United Arab Emirates excluding Dubai. Notes: Statistically signicant values are marked in a darker tone (see Annex A3). The annualised change is the average annual change in PISA score points. It is calculated taking into account all of a country’s/economy’s participation in PISA. For more details on the calculation of the annualised change, see Annex A5. The annualised change adjusted for demographic changes assumes that the average age and as well as PISA index of social, cultural and economic status, the percentage of female students, those with an immigrant background and those who speak a language other than the assessment at home is the same in previous assessments as those observed in 2012. For more details on the calculation of the adjusted annualised change, see Annex A5. OECD average 2006 considers only those countries with comparable science scores since PISA 2006. Countries and economies are ranked in descending order of the annualised change after accounting for demographic changes. Source: OECD, PISA 2012 Database, Tables I.5.3b and I.5.4. 2 1 http://dx.doi.org/10.1787/888932935629 Informative as they may be, adjusted trends are merely hypothetical scenarios that help to understand the source of changes in students’ performance over time. Observed trends depicted in Figure I.5.7 and throughout this chapter summarise the overall evolution of a school system, highlighting the challenges that countries and economies face in improving students’ and schools’ science performance. Students at the different levels of proficiency in science When science was the major domain in PISA 2006, six proficiency levels were defined on the science scale. These same proficiency levels are used for reporting science results in PISA 2012. The process used to produce proficiency levels in science is similar to that used to produce proficiency levels in mathematics, as described in Chapter 2. Figure I.5.8 presents a description of the scientific knowledge and skills that students possess at the various proficiency levels. Figure I.5.9 shows a map of some questions in relation to their position on the science proficiency scale. The first column shows the proficiency level within which the task is located. The second column indicates the lowest score on the task that would still be described as achieving the given proficiency level. The last column shows the name of the unit and the task number. The score given for the correct response to these questions is shown between parentheses. The selected questions have been ordered according to their difficulty, with the most difficult at the top, and the least difficult at the bottom. d and Can C Kno S What Student OECD 2014 © ien C e – Volume eading and S r , i CS athemati m e in C o: Student Performan W 230

233 5 A Profile of Student Perform A nce in Science Figure I.5.8 • • 2012 a S pi Summary description for the six levels of proficiency in science in Percentage of students ower able to perform tasks l e at each level or above scor limit E o ( What students can typically do evel l average) cd 6 At Level 6, students can consistently identify, explain and apply scientific knowledge 708 1.2% and knowledge about science in a variety of complex life situations. They can link different information sources and explanations and use evidence from those sources to justify decisions. They clearly and consistently demonstrate advanced scientific thinking and reasoning, and they use their scientific understanding in support of solutions to unfamiliar scientific and technological situations. Students at this level can use scientific knowledge and develop arguments in support of recommendations and decisions that centre on personal, social or global situations. 5 At Level 5, students can identify the scientific components of many complex life 8.4% 633 situations, apply both scientific concepts and knowledge about science to these situations, and can compare, select and evaluate appropriate scientific evidence for responding to life situations. Students at this level can use well-developed inquiry abilities, link knowledge appropriately, and bring critical insights to situations. They can construct explanations based on evidence and arguments based on their critical analysis. 4 At Level 4, students can work effectively with situations and issues that may involve 559 28.9% explicit phenomena requiring them to make inferences about the role of science or technology. They can select and integrate explanations from different disciplines of science or technology and link those explanations directly to aspects of life situations. Students at this level can reflect on their actions and they can communicate decisions using scientific knowledge and evidence. 3 At Level 3, students can identify clearly described scientific issues in a range of 484 57.7% contexts. They can select facts and knowledge to explain phenomena and apply simple models or inquiry strategies. Students at this level can interpret and use scientific concepts from different disciplines and can apply them directly. They can develop short statements using facts and make decisions based on scientific knowledge. 2 At Level 2, students have adequate scientific knowledge to provide possible 409 82.2% explanations in familiar contexts or draw conclusions based on simple investigations. They are capable of direct reasoning and making literal interpretations of the results of scientific inquiry or technological problem solving. 1 At Level 1, students have such limited scientific knowledge that it can only be applied 95.2% 335 to a few, familiar situations. They can present scientific explanations that are obvious and follow explicitly from given evidence. • Figure I.5.9 • ap of selected science questions, by proficiency level m l ower scor e a evel limit unit S – Questions (position on P i scale) S l 6 GREENHOUSE – Question 5 (709) 708 5 GREENHOUSE – Question 4.2 (659) (full credit) 633 4 GREENHOUSE – Question 4.1 (568) (partial credit) 559 – Question 1 (567) CLOTHES 3 MAR y MONTAGU – Question 4 (507) 484 2 MONTAGU y – Question 2 (436) MAR 409 – Question 3 (431) MAR y MONTAGU y GENETICALL MODIFIED CROPS – Question 3 (421) 1 SICAL EXERCISE y PH – Question 3 (386) 335 Figure I.5.10 shows the distribution of students among these different proficiency levels in each participating country or economy. Table I.5.1a provides figures for the percentage of students at each proficiency level on the science scale with standard errors. C W d o: Student Performan C e in m athemati CS , What Student eading and S and Can ien C e – Volume i © Kno OECD 2014 231 S r

234 5 A nce in Science A Profile of Student Perform • Figure I.5.10 • roficiency in science p ercentage of students at each level of science proficiency P Level 3 Level 5 Level 2 Below Level 1 Level 1 Level 6 Level 4 Shanghai-China Shanghai-China Estonia Estonia Hong Kong-China Hong Kong-China Korea Korea Viet Nam Viet Nam Finland Finland Japan Japan Macao-China Macao-China Students at Level 1 Poland Poland or below Singapore Singapore Chinese Taipei Chinese Taipei Liechtenstein Liechtenstein Canada Canada Ireland Ireland Germany Germany Latvia Latvia Switzerland Switzerland Slovenia Slovenia Netherlands Netherlands Australia Australia Czech Republic Czech Republic United Kingdom United Kingdom Spain Spain Austria Austria Lithuania Lithuania New Zealand New Zealand Denmark Denmark Croatia Croatia Belgium Belgium OECD average OECD average Hungary Hungary United States United States Italy Italy France France Russian Federation Russian Federation Portugal Portugal Norway Norway Luxembourg Luxembourg Sweden Sweden Iceland Iceland Greece Greece Turkey Turkey Slovak Republic Slovak Republic Israel Israel Thailand Thailand Chile Chile Serbia Serbia United Arab Emirates United Arab Emirates Bulgaria Bulgaria Romania Romania Costa Rica Costa Rica Kazakhstan Kazakhstan Malaysia Malaysia Uruguay Uruguay Mexico Mexico Jordan Jordan Montenegro Montenegro Argentina Argentina Albania Albania Brazil Brazil Tunisia Tunisia Colombia Colombia Students at Level 2 Qatar Qatar or above Indonesia Indonesia Peru Peru % % 60 40 20 0 20 40 60 80 100 100 80 Countries and economies are ranked in descending order of the percentage of students at Levels 2, 3, 4, 5 and 6. Source: OECD, PISA 2012 Database, Table I.5.1a. 1 2 http://dx.doi.org/10.1787/888932935629 What Student C o: Student Performan d and Can W Kno S C OECD 2014 e – Volume i C eading and S r , CS athemati m © e in ien 232

235 5 nce in Science A Profile of Student Perform A Proficiency at Level 6 (scores higher than 708 points) At Level 6, students can consistently identify, explain and apply scientific knowledge and knowledge about science in a variety of complex life situations. They can link different information sources and explanations and use evidence from those sources to justify decisions. They clearly and consistently demonstrate advanced scientific thinking and reasoning, and they use their scientific understanding in support of solutions to unfamiliar scientific and technological situations. Students at this level can use scientific knowledge and develop arguments in support of recommendations and decisions that centre on personal, social or global situations. Question 5 of GREENHOUSE (Figure I.5.14) is an example of task at Level 6 and of the competency explaining phenomena scientifically. In this question, students must analyse a conclusion to account for other factors that could influence the greenhouse effect. This question combines aspects of the two skills: identifying scientific issues and explaining phenomena scientifically. The student needs to understand the necessity of controlling factors outside the change and measured variables and to recognise those variables. The student must have sufficient knowledge of “Earth systems” to be able to identify at least one of the factors that should be controlled. The latter criterion is considered the explaining phenomena scientifically critical scientific skill involved, so this question is categorised as . The effects of this environmental issue are global, which defines the setting. As a first step in gaining credit for this question the student must be able to identify the change and measured variables and have sufficient understanding of methods of investigation to recognise the influence of other factors. However, the student also needs to recognise the scenario in context and identify its major components. This involves a number of abstract concepts and their relationships in determining what “other” factors might affect the relationship between the Earth’s temperature and the amount of carbon dioxide emissions into the atmosphere. This locates the question near the category. This question requires a short boundary between Levels 5 and 6 in the explaining phenomena scientifically open-constructed response. Across OECD countries, an average of 1.1% of students perform at Level 6. Between 3% and 6% of the students are at this level in Singapore (5.8%), Shanghai-China (4.2%), Japan (3.4%) and Finland (3.2%). In New Zealand, Australia, Canada, the United Kingdom, Hong Kong-China, Estonia, Poland, Germany and Ireland between 1.5% and 2.7% of students perform at the highest proficiency level. By contrast, in the majority of participating countries the share of students at proficiency Level 6 is below 1%. Around zero percent of students on average reach this level in Albania, Argentina, Brazil, Chile, Colombia, Costa Rica, Indonesia, Jordan, Kazakhstan, Malaysia, Mexico, Montenegro, Peru, Romania, Tunisia, Turkey and Uruguay (Figure I.5.10 and Table I.5.1a). Proficiency at Level 5 (scores higher than 633 but lower than or equal to 708 points) At Level 5, students can identify the scientific components of many complex life situations, apply both scientific concepts and knowledge about science to these situations, and can compare, select and evaluate appropriate scientific evidence for responding to life situations. Students at this level can use well-developed inquiry abilities, link knowledge appropriately, and bring critical insights to situations. They can construct explanations based on evidence and arguments based on their critical analysis. Question 4 of GREENHOUSE (Figure I.5.14), an example of task at Level 5, requires an open-constructed response. using scientific evidence This task centres on the skill and asks students to identify a portion of a graph that does not provide evidence supporting a conclusion. This question requires the student to look for specific differences that vary from positively correlated general trends in these two graphical datasets. Students must locate a portion where both curves are not ascending or descending and provide this finding as part of a justification for a conclusion. As a result, the task involves a greater amount of insight and analytical skill than is required for Question 3. Rather than provide a generalisation about the relation between the graphs, the student is asked to explain the difference in the nominated period in order to gain full credit. The question is located at Level 5 because it requires the ability to compare the details of two datasets and to criticise a given conclusion. If the student understands what the question requires of them and correctly identifies a difference in the two graphs, but is unable to explain this difference, the student gains partial credit for the question and is identified at Level 4 of the scientific proficiency scale. The skill required is to interpret data graphically presented, so the question belongs in the scientific explanations category. Across OECD countries, 8.4% of students are proficient at Level 5 or 6. Students scoring at Level 5 or 6 are considered as top performers. More than 15% of students attain one of these levels in Shanghai-China (27.2%), Singapore (22.7%), CS C S Kno W and Can d o: Student Performan C e in m athemati What Student , r eading and S C 233 OECD 2014 © i e – Volume ien

236 5 nce in Science A Profile of Student Perform A Japan (18.2%), Finland (17.1%) and Hong Kong-China (16.7%). In 11 countries and economies between 10% and 15% of students are top performers in science. Some countries have virtually no top performers in science: in two partner countries, Indonesia and Peru, fewer than 0.1% of students reaches Level 5 or 6, and in Tunisia, Colombia, Mexico, Kazakhstan, Costa Rica, Argentina, Jordan, Brazil, Malaysia, Montenegro and Albania, fewer than 0.5% of students attains Level 5 or 6 (Figure I.5.10 and Table I.5.1a). Proficiency at Level 4 (scores higher than 559 but lower than or equal to 633 points) At Level 4, students can work effectively with situations and issues that may involve explicit phenomena requiring them to make inferences about the role of science or technology. They can select and integrate explanations from different disciplines of science or technology and link those explanations directly to aspects of life situations. Students at this level can reflect on their actions and they can communicate decisions using scientific knowledge and evidence. Question 1 in the unit CLOTHES (Figure I.5.15), which typifies a Level 4 question, requires the student to identify the change and measured variables associated with testing a claim about clothing. It also involves an assessment of whether there are techniques to quantify the measured variable and whether other variables can be controlled. This process then needs to be accurately applied for all four claims. The issue of “intelligent” clothes is in the category frontiers of science and technology and is a community issue addressing a need for disabled children; therefore, the setting is social. The scientific skills applied involve the nature of investigation, which places the question in the scientific enquiry category. The need to identify change and measured variables, together with an appreciation of what would be involved in carrying out measurement and controlling variables, locates the question at Level 4. Students are required to answer in a complex multiple-choice format. Across OECD countries, an average of 29% of students is proficient at Level 4 or higher (Level 4, 5 or 6). In seven countries and economies, at least 40% of students attain this level, including between 40% and 50% of students in Japan, Finland, Korea, Estonia and in the partner country Singapore, slightly more than 50% in Hong Kong-China, and more than 60% of students in Shanghai-China. In contrast, fewer than 5% of students reach Level 4, 5 or 6 in Indonesia, Peru, Tunisia, Colombia, Mexico, Brazil, Argentina, Jordan, Kazakhstan, Costa Rica, Albania, Malaysia and Montenegro (Figure I.5.10 and Table I.5.1a). Proficiency at Level 3 (scores higher than 484 but lower than or equal to 559 points) At Level 3, students can identify clearly described scientific issues in a range of contexts. They can select facts and knowledge to explain phenomena and apply simple models or inquiry strategies. Students at this level can interpret and use scientific concepts from different disciplines and can apply them directly. They can develop short statements using facts and make decisions based on scientific knowledge. An example of a question at Level 3 is Question 4 from MARY MONTAGU (Figure I.5.16). This question requires the student to identify why young children and old people are more at risk of the effects of influenza than others in the population. Directly, or by inference, the reason is attributed to the weaker immune systems among young children and old people. The issue is community control of disease, so the setting is social. A correct explanation involves applying several pieces of knowledge that are well established in the community. The question stem also provides a clue to the groups’ different levels of resistance to disease. Students have to answer with an open-constructed response. Across OECD countries, 58% of students are proficient at Level 3 or higher (Level 3, 4, 5 or 6) on the science scale. In the partner economies Shanghai-China and Hong Kong-China, more than 80% of students perform at least at this level. In the OECD countries Estonia, Finland, Korea and Japan, more than three out of four 15-year-olds are proficient at Level 3 or higher, and at least two out of three students in Singapore, Viet Nam, Chinese Taipei, Macao-China, Canada, Poland, Liechtenstein, Germany, Ireland and the Netherlands perform at least at this level (Figure I.5.10 and Table I.5.1a). Proficiency at Level 2 (scores higher than 409 but lower than or equal to 484 points) In 2007, following a detailed analysis of the questions from the main study, the international PISA Science Expert Group, which guided the development of the science framework and questions, identified Level 2 as the baseline proficiency level. This level does not establish a threshold for scientific illiteracy. Rather, the baseline level of proficiency defines the level of achievement on the PISA scale at which students begin to demonstrate the science competencies that will enable them to participate effectively and productively in life situations related to science and technology. At Level 2, students have adequate scientific knowledge to provide possible explanations in familiar contexts or draw conclusions based on simple investigations. They are capable of direct reasoning and making literal interpretations of the results of scientific inquiry or technological problem solving. o: Student Performan m © OECD 2014 What Student S Kno W and Can d athemati C i e – Volume C ien C eading and S r , CS e in 234

237 5 nce in Science A Profile of Student Perform A Question 3 from the unit GENETICALLY MODIFIED CROPS (Figure I.5.17) is typical of Level 2 tasks. It asks a simple question about varying conditions in a scientific investigation and students are required to demonstrate knowledge about the design of science experiments. To answer this question correctly in the absence of cues, the student needs to be aware that the effect of the treatment (different herbicides) on the outcome (insect numbers) could depend on environmental factors. Thus, by repeating the test in 200 locations, the chance of a specific set of environmental factors giving rise to a spurious outcome can be accounted for. Since the question focuses on the methodology of the investigation it is categorised as scientific enquiry frontiers of science . The application area of genetic modification places this at the and technology and given its restriction to one country, it can be said to have a social setting. In the absence of cues, this question has the characteristics of Level 4, i.e. the student shows an awareness of the need to account for varying environmental factors and is able to recognise an appropriate way of dealing with that issue. However, because of the cues given in three distracters, and the fact that most students will easily eliminate these as options, the question actually scale. identifying scientific issues sits at Level 2 of the Across OECD countries, 82% of students, on average, are proficient at Level 2 or higher In Estonia, Hong Kong-China, Korea, Viet Nam, Finland, Japan, Macao-China, Poland, Singapore and Chinese Taipei between 90% and 95% of students perform at or above this threshold. In the partner economy Shanghai-China, only 3% of students are below this level. In every country except the three partner countries Peru, Indonesia and Qatar, at least 40% of students are at Level 2 or above (Figure I.5.10 and Table I.5.1a). Proficiency at Level 1 (scores higher than 335 but lower than or equal to 409 points) or below At Level 1, students have such limited scientific knowledge that it can only be applied to a few, familiar situations. They can present scientific explanations that are obvious and follow explicitly from given evidence. x ERCISE (Figure I.5.18) is an example of task at Level 1. To gain credit for this Question 3 in the unit PHYSICAL E question, the student has to correctly recall knowledge about the operation of muscles and about the formation of fat in the body, i.e. students must have knowledge of the scientific fact that more blood flows through active muscles and that fats are not formed when muscles are exercised. This enables the student to accept the first explanation of this complex multiple-choice question and reject the second explanation. The two simple factual explanations contained in the question are not related to each other. Each is accepted or rejected as an effect of the exercise of muscles. Since this is common knowledge, the question is located at the very bottom of the explaining phenomena scale. scientifically Students who score below 335 points – that is, below Level 1 – usually do not succeed at the most basic levels of science that PISA measures. Such students are more likely to have serious difficulties in using science to benefit from further education and learning opportunities and in participating in life situations related to science and technology (OECD, 2010). Across OECD countries, 18% of students perform at or below Level 1– more precisely, 13% perform at Level 1 and 5% perform below Level 1. In Shanghai-China, Estonia, Hong Kong-China, Korea, Viet Nam, Finland, Japan, Macao-China, Poland, Singapore and Chinese Taipei, fewer than 10% of students perform at Level 1 or below. In all of these countries and economies, except in Singapore (2.2%), 2% of students or fewer score below Level 1. In OECD countries, the proportion of students performing below Level 1 ranges from 2% in Japan to less than 13% in Mexico. In some countries, the share of students at proficiency Level 1 or below Level 1 is substantial, notably in Peru, Indonesia, Qatar, Colombia, Tunisia, Brazil, Albania, Argentina and Montenegro where more than half of all 15-year-olds perform at proficiency Level 1 or below. In the partner countries Qatar, Peru, Indonesia, Albania and Tunisia, more than 20% of students perform below Level 1 (Figure I.5.10 and Table I.5.1a). Trends in the percentage of low- and top-performers in science PISA’s science assessments gauge the extent to which a country’s or economy’s students have acquired the knowledge and skills in science that will allow them to participate fully in a knowledge-based society. These skills range from basic notions of science (related to proficiency Level 2) to understanding of more complex scientific concepts and processes (related to proficiency Levels 5 and 6). Changes in a country’s or economy’s average performance can result from improvements or deterioration at different points in the performance distribution. For example, in some countries and economies the average improvement may be observed among all students, resulting in fewer students performing below Level 2 and more students becoming CS i S Kno W and Can d o: Student Performan C e in m athemati What Student , r eading and S C ien C 235 OECD 2014 © e – Volume

238 5 nce in Science A Profile of Student Perform A top performers. In other contexts, the average improvement can be attributed to large improvements among low- achieving students with little or no change among high-achieving students; this may result in a smaller share of low-performing students, but no increase in the share of top performers. From a trends perspective, countries and economies succeed when they reduce the share of students who perform below proficiency Level 2 (low performers) or when they increase the share of students who perform at or above proficiency Level 5 (top performers) as they provide more opportunities for students to begin to show scientific literacy or to have the highest level competencies in science. Countries and economies can be grouped into categories according to whether they have: simultaneously reduced the share of low performers and increased the share of top performers between any previous PISA assessment and PISA 2012; reduced the share of low performers but not increased the share of top performers between any previous PISA assessment and PISA 2012; increased the share of top performers but not reduced the share of low performers; and reduced the share of top performers or increased the share of low performers between PISA 2012 and any previous PISA assessment. The following section categorises countries and economies into these groups. Moving everyone up: Reduction in the share of low performers and increase in that of top performers Between PISA 2006 and PISA 2012, Poland, Qatar and Italy saw a reduction in the share of students who perform below proficiency Level 2 in science and an increase in the share of students who perform at or above proficiency Level 5. In Poland, for example, the share of students who perform below Level 2 in science dropped from 17% in 2006 to 9% in 2012, while the share of students who perform at or above Level 5 in science increased from 7% to 11%. In Italy, 25% of students were considered low performers in 2006; by 2012, that percentage had decreased to 19%. During the same period, the proportion of top performers in Italy increased from 5% to 6% (Figure I.5.11). As shown in Table I.5.1b, the same was observed in Singapore, Estonia and Israel between the PISA 2009 and PISA 2012 assessments. The reduction in the share of low performers and increase in the share of top performers in these countries and economies mirrors the changes in how students at different points of the distribution have improved since 2006. Annex B4 shows, for each country and economy, the trajectories of the 10th, 25th, 75th and 90th percentiles of science performance. These are the lowest-, low-, high- and highest-achieving students. Consistent with the changes in the shares of low and top performers, it shows how overall average improvements in Poland and Italy are also seen among their low- and high-achieving students. In Poland, for example, the lowest-achieving students improved their science performance by 5.6 score points per year (from 381 points in 2006 to 415 points in 2012), and the highest- achieving students also improved their performance by an average of 3.7 points per year (from 615 points in 2006 to 637 points in 2012), resulting in a decrease in the share of students performing below Level 2 and an increase in the share of students performing at Level 5. Similar improvements in science performance among low- and high- achieving students are observed in Italy and Portugal. Reducing underperformance: Reduction in the share of low performers but no change in the share of top performers While relatively few countries and economies succeeded in increasing the share of top performers while simultaneously reducing the share of students who do not meet the baseline proficiency in science, many reduced the share of low performers between PISA 2006 and PISA 2012. Turkey, Thailand, Romania, Tunisia, Brazil, the United States, Portugal, Latvia, Korea, Ireland, Lithuania, Spain, Japan, Switzerland and Hong Kong-China saw a reduction in the share of students performing below proficiency Level 2 between 2006 and 2012, thus raising the number of students who demonstrate science literacy. Similarly, the Czech Republic, Slovenia, Dubai (United Arab Emirates) and Kazakhstan reduced the share of low performers between PISA 2009 and PISA 2012. Latvia, Portugal, the United States, Brazil, Tunisia, Romania, Thailand and Turkey, for example, reduced the share of students performing below proficiency Level 2 by more than five percentage points between 2006 and 2012 (Figure I.5.11). Many of the countries and economies that reduced the share of low-performing students are those that show average improvements in science, and concentrate this improvement among their low-achieving students). Annex B4 shows the trajectories of low- and high-achieving students for all countries and economies, highlighting how, in Turkey, Korea, Romania, Brazil, Chile, Estonia, Switzerland, Spain, Tunisia and Lithuania, for example, while the lowest-achieving students improved their science performance by at least two score points per year between PISA 2006 and PISA 2012, the highest-achieving students saw no change in science performance. i C © OECD 2014 What Student S Kno W and Can d e in e – Volume C ien C eading and S r , CS athemati m o: Student Performan 236

239 5 nce in Science A Profile of Student Perform A Nurturing top performance: Increase in the share of high-performers but no change in that of low performers Top-performing students in science are those who perform at or above proficiency Level 5. Luxembourg and Serbia saw an increase in the share of top-performing students while the share of low-performing students remained unchanged between 2006 and 2012. Similar improvements were observed in Albania and Macao-China. Between PISA 2009 and PISA 2012. In Luxembourg, for example, the share of top performers increased from 6% in 2006 to 8% in 2012 (Figure I.5.11 and Table I.5.1b). Increase in the share of low performers or decrease in that of high performers By contrast, in 13 countries and economies the percentage of students who do not meet the baseline proficiency in science in PISA increased since 2006 – or since more recent PISA cycles – or the share of students who perform at the highest levels of proficiency decreased (Figure I.5.11 and Table I.5.1b). • • Figure I.5.11 p ercentage of low-performing students and top performers in science in 2006 and 2012 2006 2012 80 Students at or above prociency Level 5 70 60 50 40 Percentage of students 30 20 10 0 2.3 0.9 1.1 4.1 1.5 -4.0 -2.6 -3.1 -4.2 -3.9 -0.9 -6.3 -0.5 -3.3 Italy Chile Israel Spain Brazil Japan Qatar Latvia Korea Serbia Jordan France Turkey Tunisia Poland Austria Ireland Estonia Greece Iceland Finland Mexico Croatia Canada Norway Sweden Bulgaria Belgium Portugal Slovenia Uruguay Thailand Hungary Australia Romania Lithuania Germany Denmark Argentina Indonesia Colombia Switzerland Netherlands Montenegro Luxembourg New Zealand Liechtenstein United States Macao-China Chinese Taipei Czech Republic Slovak Republic United Kingdom Hong Kong-China Russian Federation OECD average 2006 5.2 3.6 5.9 3.4 6.7 4.8 -2.6 -4.6 -8.0 -4.4 -3.2 -7.3 -6.2 -7.4 -6.6 -7.3 -3.6 -4.3 -2.1 -3.9 -5.5 -5.1 -9.6 -3.2 0 -16.5 -20.2 -12.5 10 20 30 40 Percentage of students 50 60 70 Students below prociency Level 2 80 Notes: The chart shows only countries/economies that participated in both PISA 2006 and PISA 2012 assessments. The change between PISA 2006 and PISA 2012 in the share of students performing below Level 2 in science is shown below the country/economy name. The change between PISA 2006 and PISA 2012 in the share of students performing at or above Level 5 in science is shown above the country/economy name. Only statistically signicant changes are shown (see Annex A3). OECD average 2006 compares only OECD countries with comparable science scores since 2006. Countries are ranked in descending order of the percentage of students at or above prociency Level 5 in science in 2012. and economies OECD, PISA 2012 Database, Table I.5.1b. Source: 1 2 http://dx.doi.org/10.1787/888932935629 athemati 237 © i C C eading and S r e – Volume , CS OECD 2014 m e in C o: Student Performan d and Can W Kno S What Student ien

240 5 nce in Science A Profile of Student Perform A i mproving in pi S a : e stonia Box I.5.1. Estonia’s performance in PISA improved significantly since it first participated in PISA in 2006: by an average of 2.4 score points per year in reading and and science scores improved 14 points between PISA 2009 and PISA 2012. Its performance in reading improved from 501 points in PISA 2006 to 516 points in PISA 2012, and science performance improved from 531 points in PISA 2006 to 541 points in PISA 2012. This improvement came in a challenging educational context. A significant demographic shift in Estonia’s population of 1.3 million resulted in a 25% reduction in the number of students in general education between 2004 and 2012. Municipal schools in peripheral areas closed and repercussions are still being felt in teacher- training and retention systems, in higher education and in the labour market. High dropout rates further reduce the number of upper secondary and tertiary-level graduates. In addition, Estonia – as other OECD countries – faces the challenge of encouraging the best teachers to teach in remote and disadvantaged schools. In response to the changing student population, the government changed its school funding model from a per capita to a per class criteria in 2008, allowing for a more equitable distribution of funds to rural schools, and, to reduce dropout rates, also began to promote vocational training. The change in financing recognises that not all of a schools’ operational costs are variable, thus allowing many rural schools to keep functioning because in a per capita financing scheme they would have closed on budgetary reasons (Estonian Ministry of Education and Research, 2008). To encourage newly qualified teachers to teach in small towns and rural areas, and for teachers with command of the Estonian language to teach in schools where Russian is the language of instruction, new teachers are offered an allowance of more than 12 750 EUR during the first three years of teaching. Higher education institutions providing pre-service teacher training have formulated common competency standards for teachers and articulated a development plan for the teacher-training system (European Commission, 2010). Other policy initiatives have promoted the use of assessments for self-monitoring purposes. In 2006, the Ministry of Education and Research introduced compulsory internal assessments for all pre-primary child-care institutions, general education schools and vocational training institutions, shifting supervisory functions from the state to the individual school level. Schools are offered support from the state to conduct their internal assessment (Estonian Ministry of Education and Research, 2008). Since 2009, Estonia, through the Tiger Leap Foundation, has been promoting ICT use at all levels of education and in a wide range of study programmes, including science, mathematics, embroidery and robotics. The introduction of ICT equipment is combined with teacher training and new learning materials. For example, for mathematics projects, teachers are taught to use mathematics-learning software and funding is provided to schools to acquire computer-based algebra software (European Commission, 2010). Based on the “Development Plan for the General Education System for 2007–2013”, the national curriculum for basic and upper secondary schools was updated in January 2010 and the Basic Schools and Upper Secondary Schools Act was amended. As a result of these specifications, the volume of compulsory subjects in upper secondary schools was reduced from 72 to 63 courses and more elective courses are offered (Government of the Republic of Estonia, 2011a, 2011b). The new national curriculum aims to offer more opportunities for a diverse student population in order to reduce grade repetition and dropout (Government of the Republic of Estonia, 2011a, 2011b). It is oriented towards learning, rather than teaching, and recognises the greater role students – and student engagement – take in the learning process. For example, in language-of-instruction classes, composition is emphasised; in natural science classes, research-based learning is promoted; in foreign-language classes, real-life situations are used to prompt responses in the language concerned. Certain topics in science and mathematics have been shifted from primary to secondary schools to ensure that they are taught in appropriate depth (Government of the Republic of Estonia, 2011a, 2011b). Sources: The Development of Education, Estonian Ministry of Education and Research, Estonian Ministry of Education and Research (2008), Tallinn. verviews on Education Systems in Europe and ngoing Reforms: Estonia 2010 o European Commission (2010), o n ational Systems , Eurydice, Brussels. Edition Government of the Republic of Estonia (2011a), Tallinn. n ational Curriculum for Basic Schools, Tallinn. pper Secondary Schools, u ational Curriculum for n Government of the Republic of Estonia (2011b), C © OECD 2014 What Student S Kno W and Can d o: Student Performan C e in m athemati CS , r e – Volume eading and S C ien i 238

241 5 nce in Science A Profile of Student Perform A Variation in student performance in science The difference in performance between students within countries and economies is shown in Table I.5.3a. Within countries, the difference in scores between the highest- (90th percentile) and lowest-achieving students (10th percentile) ranges from 174 to 281 points, with an OECD average of 239 points. Some of the lower-performing countries have among the narrowest gaps between the highest- and lowest-achieving students: Indonesia (with a gap of 174 points), Mexico (with a gap of 180 points), Colombia (with a gap of 196 points), Peru (with a gap of 200 points) and Tunisia (with a gap of 201 points). However, Viet Nam performs well above the OECD average and shows one of the ten narrowest gaps (197 points). Shanghai-China shows the best performance in science and a difference of only 209 points between the highest- and lowest-achieving students. At the other end of the spectrum, among the ten participating countries and economies that show the largest difference between the highest and lowest achievers in science, this gap ranges from between 257 to 281 points. One of the lowest-performing countries, Qatar (with a gap of 275 points), has nearly the same gap between the highest- and lowest-achieving students as one of the highest-performing countries, New Zealand (272 points). As in mathematics and reading, some countries perform well without having large differences between their highest- and lowest-achieving students. Among the eight best-performing countries in science, this is the case in Estonia, Korea, and in the partner countries and economies Viet Nam, Shanghai-China and Hong Kong-China, where the differences are around 30 points smaller than the OECD average. Gender differences in science performance Across OECD countries, differences in science performance related to gender tend to be small compared with the large gender gap in reading performance and the more moderate gender differences in mathematics performance. As shown in Figure I.5.12, in more than half of the countries assessed, differences in the average score for boys and girls are not statistically significant. This indicates that gender equality is more prevalent in science performance than in mathematics or reading performance. In 2006, when science was the main focus of the PISA assessment, gender differences were observed in two of the science processes being assessed. Across OECD countries, girls scored higher in the area of identifying scientific issues, while boys outscored girls in explaining phenomena scientifically. The shorter assessment time for science in 2012 did not allow for an update of this finding. The largest gender differences in favour of boys are observed in Colombia (18 score points) and in Luxembourg, the United Kingdom, Costa Rica, Japan and Denmark, where there is a 10-to-15 score-point difference between boys and girls. In Spain, Chile, Mexico and Switzerland, boys outperform girls in science by six to seven score points. By contrast, in Jordan, Qatar, United Arab Emirates, girls outperform boys in science by 43, 35 and 28 score points, respectively. In Bulgaria, Thailand, Montenegro, Finland, Latvia, Lithuania, Greece, Malaysia and Turkey, girls outperform boys in science by from 20 to 10 score points (Figure I.5.12 and Table I.5.3a). How do boys and girls differ in levels of proficiency? One way to determine this is to observe the highest level of proficiency attained by the largest group of girls and boys in each country and economy. As can be seen in Table I.5.2a, among all the participating countries and economies, the highest proficiency level attained by the largest group of boys (in 36 countries and economies) and girls (in 33 countries and economies) is Level 3 followed by Level 2 (the highest level attained by the largest group of boys in 15 countries and economies and by most girls in 21 countries and economies). But while in nine countries the highest proficiency level attained by the largest group of boys is Level 1 – and in one country, below Level 1 – in six countries, Level 1 is the highest proficiency level attained by the largest group of girls. In only four countries is Level 4 the highest proficiency level attained by the largest group of boys and in five countries, the highest proficiency level attained by the largest group of girls. On average across OECD countries, 18.6% of boys do not attain the baseline level of proficiency in science, Level 2, and 16.9% of girls do not attain this level – 5.3% of boys and 4.2% of girls do not even attain Level 1. The gender gap in the proportion of boys and girls performing below Level 2 is particularly pronounced in Jordan, the United Arab Emirates, Thailand, Qatar and Bulgaria. The share of girls performing below Level 2 is at least 10 percentage points smaller than that of boys. The largest difference is found in Jordan where more than 60% of boys perform at or below Level 1 compared to 39% of girls. The opposite pattern can be observed in several countries and economies. The five countries and economies with the largest gender gap, in favour of boys, among students performing below proficiency Level 2 are Colombia, Costa Rica, Liechtenstein, Luxembourg and Mexico. There appears to be no relation between overall science performance and this gender gap as these countries and economies vary considerably in overall science performance. CS r S Kno W and Can d o: Student Performan C e in m athemati What Student 239 OECD 2014 © i e – Volume C ien C eading and S ,

242 5 nce in Science A Profile of Student Perform A • • Figure I.5.12 ender differences in science performance g Girls Boys All students Gender differences Mean score on the science scale – (boys girls) Jordan Qatar United Arab Emirates Bulgaria Thailand Montenegro Finland Latvia Lithuania Greece Boys perform Girls perform Malaysia better better Turkey Slovenia Kazakhstan Sweden Albania Argentina Russian Federation Romania Serbia Norway Indonesia Iceland Poland France Estonia Croatia Portugal United States Macao-China Uruguay Israel Singapore Germany OECD average Czech Republic 1 score point Chinese Taipei Tunisia Viet Nam OECD average Brazil Italy Canada Hungary Netherlands Korea Belgium Ireland New Zealand Australia Shanghai-China Peru Switzerland Mexico Hong Kong-China Chile Slovak Republic Spain Austria Denmark Japan Costa Rica United Kingdom Luxembourg Liechtenstein Colombia 20 40 500 550 600 -60 -40 -20 450 0 400 350 Mean score Score-point difference Note: Statistically signicant gender differences are marked in a darker tone (see Annex A3). Countries and economies are ranked in ascending order of the score-point difference (boys girls). – Source: OECD, PISA 2012 Database, Table I.5.3a. 2 1 http://dx.doi.org/10.1787/888932935629 d C e – Volume i W Kno © S What Student OECD 2014 ien C eading and S r , CS athemati m e in C o: Student Performan and Can 240

243 5 nce in Science A Profile of Student Perform A Not only do fewer girls than boys perform at the lowest proficiency levels, but fewer girls than boys perform at the highest proficiency levels on the science scale as well. Across OECD countries, 9.3% of boys are top performers in science (performing at Level 5 or 6), but only 7.4% of girls are. In Japan, Liechtenstein, Hong Kong-China and Shanghai-China, all of which are among the highest-performing countries and economies in science and have relatively large shares of students performing at the highest proficiency levels, the share of top performers among boys is at least four percentage points larger than that among girls. Trends in gender differences in science performance In 37 of the 54 countries and economies that participated in PISA 2006 (and also took part in PISA 2012) there was no gender gap in science. A gender gap favouring boys was observed in eight countries (and largest in Chile, at 22 score points), and in ten countries, girls outperformed boys (Table I.5.3c and OECD, 2007). Between PISA 2006 and PISA 2012, and on average across OECD countries, the gender gap in science performance remained unchanged. However, in those countries and economies where the magnitude of the gender gap in science did change, the change always favoured girls. This was the case in Finland, Montenegro, Sweden and the Russian Federation where, while there was no gender gap in science in PISA 2006, a gender gap in favour of girls was observed in PISA 2012. In the Russian Federation this is the result of an improvement in science performance among girls between PISA 2006 and PISA 2012 that was not observed among boys. In Finland, Montenegro and Sweden, the observed gender gap in science in favour of girls is the result of a greater deterioration in science performance among boys than among girls. In Chile the gender gap that favoured boys in PISA 2006 was weaker in 2012, and was no longer present in Brazil as girls’ science performance has improved more rapidly than boys’ (Figure I.5.13). • Figure I.5.13 • c hange between 2006 and 2012 in gender differences in science performance Gender differences in science performance in 2012 Gender differences in science performance in 2006 30 Boys perform better 20 10 Score-point difference 0 -10 -20 Girls perform better -30 -40 -50 -7 -9 28 -8 -13 -15 -15 -15 Italy Chile Israel Brazil Spain Japan Qatar Latvia Korea Serbia Jordan France Turkey Tunisia Poland Austria Ireland Estonia Greece Iceland Finland Mexico Croatia Canada Norway Sweden Bulgaria Belgium Portugal Slovenia Uruguay Thailand Hungary Australia Romania Lithuania Germany Denmark Argentina Indonesia Colombia Switzerland Netherlands Montenegro Luxembourg New Zealand Liechtenstein United States Macao-China Chinese Taipei Czech Republic Slovak Republic United Kingdom Hong Kong-China Russian Federation OECD average 2006 Gender differences in PISA 2006 and PISA 2012 that are statistically signicant are marked in a darker tone (see Annex A3). Notes: Statistically signicant changes in the score-point difference between boys and girls in science performance between PISA 2006 and PISA 2012 are shown next to the country/economy name. OECD average 2006 compares only OECD countries with comparable science scores since 2006. Countries and economies are ranked in ascending order of gender differences (boys-girls) in 2012. Source: OECD, PISA 2012 Database, Table I.5.3c. http://dx.doi.org/10.1787/888932935629 2 1 e in OECD 2014 Kno W and Can d o: Student Performan C S m athemati CS , r eading and S C ien C e – Volume i © What Student 241

244 5 nce in Science A Profile of Student Perform A S Science unit a S pi of S xample e The questions are presented in the order in which they appeared within the unit in the main survey. Figure I.5.14 • • reenhou S e g Read the texts and answer the questions that follow. HE T ? FICTION OR FACT : EFFECT GREENHOUSE Living things need energy to survive. The energy that sustains life on the Earth comes from the Sun, which radiates energy into space because it is so hot. A tiny proportion of this energy reaches the Earth. The Earth’s atmosphere acts like a protective blanket over the surface of our planet, preventing the variations in temperature that would exist in an airless world. Most of the radiated energy coming from the Sun passes through the Earth’s atmosphere. The Earth absorbs some of this energy, and some is reflected back from the Earth’s surface. Part of this reflected energy is absorbed by the atmosphere. As a result of this the average temperature above the Earth’s surface is higher than it would be if there were no atmosphere. The Earth’s atmosphere has the same effect as a greenhouse, hence the term greenhouse effect. The greenhouse effect is said to have become more pronounced during the twentieth century. It is a fact that the average temperature of the Earth’s atmosphere has increased. In newspapers and periodicals the increased carbon dioxide emission is often stated as the main source of the temperature rise in the twentieth century. A student named André becomes interested in the possible relationship between the average temperature of the Earth’s atmosphere and the carbon dioxide emission on the Earth. In a library he comes across the following two graphs. Average temperature Carbon dioxide emission (thousand million of the Earth’s of tonnes per year) atmosphere (°c) 15.4 20 15.0 10 14.6 0 Years Years 1920 1870 1880 1890 1900 1910 1920 1930 1940 1950 1960 1970 1980 1990 1930 1940 1950 1960 1970 1980 1990 1860 1860 1870 1880 1890 1900 1910 André concludes from these two graphs that it is certain that the increase in the average temperature of the Earth’s atmosphere is due to the increase in the carbon dioxide emission. Level 6 GREENHOUSE – Qu ESTI on 4 708 Level 5 633 Question type: o pen-constructed response Level 4 C sing scientific evidence u ompetency: 559 Level 3 Knowledge category: “Scientific explanations” (knowledge about science) 484 Level 2 “Environment” Application area: 409 Level 1 Global Setting: 335 Below Level 1 Difficulty: Full credit 659; Partial credit 568 Percentage of correct answers (OECD countries): 34.5% Another student, Jeanne, disagrees with André’s conclusion. She compares the two graphs and says that some parts of the graphs do not support his conclusion. Give an example of a part of the graphs that does not support André’s conclusion. Explain your answer. e – Volume C e in m athemati CS , r eading and S i ien o: Student Performan d and Can W Kno S What Student OECD 2014 © C C 242

245 5 nce in Science A Profile of Student Perform A Scoring redit: c ull f Refers to one particular part of the graphs in which the curves are not both descending or both climbing and gives the corresponding explanation. For example: In 1900–1910 (about) CO • was increasing, whilst the temperature was going down. 2 In 1980–1983 carbon dioxide went do • wn and the temperature rose. T • he temperature in the 1800s is much the same but the first graph keeps climbing. • Between 1950 and 1980 the temper ature didn’t increase but the CO did. 2 rom 1940 until 1975 the temperature stays about the same but the carbon dioxide emission shows a sharp rise. • F • ature is a lot higher than in 1920 and they have similar carbon dioxide emissions. In 1940 the temper Partial c redit: Mentions a correct period, without any explanation. For example: 1930–1933. • • before 1910. Mentions only one particular y ear (not a period of time), with an acceptable explanation. For example: In 1980 the emissions were do wn but the temperature still rose. • ote: n [ Gives an example that doesn’t support André’s conclusion but makes a mistake in mentioning the period. There should be evidence of this mistake – e.g. an area clearly illustrating a correct answer is marked on the graph and then a For example: mistake made in transferring this information to the text.] • Between 1950 and 1960 the temper ature decreased and the carbon dioxide emission increased. Refers to differences between the two curves, without mentioning a specific period. For example: • At some places the temper ature rises even if the emission decreases. Earlier there w • as little emission but nevertheless high temperature. When there is a stead • y increase in graph 1, there isn’t an increase in graph 2, it stays constant. [ n ote: It stays constant “overall”.] Because at the start the temper • ature is still high where the carbon dioxide was very low. Refers to an irregularity in one of the graphs. For example: It is about 1910 w • hen the temperature had dropped and went on for a certain period of time. aph there is a decrease in temperature of the Earth’s atmosphere just before 1910. • In the second gr Indicates difference in the graphs, but explanation is poor. For example: • In ote: The explanation is very poor, but the the 1940s the heat was very high but the carbon dioxide very low. [ n difference that is indicated is clear.] Comment Another example from GREE ou H n SE centres on the competency using scientific evidence and asks students to identify a portion of a graph that does not provide evidence supporting a conclusion. This question requires the student to look for specific differences that vary from positively correlated general trends in these two graphical datasets. Students must locate a portion where curves are not both ascending or descending and provide this finding as part of a justification for a conclusion. As a consequence it involves a greater amount of insight and analytical skill than is required for Question 3. Rather than a generalisation about the relation between the graphs, the student is asked to accompany the nominated period of difference with an explanation of that difference in order to gain full credit. The ability to effectively compare the detail of two datasets and give a critique of a given conclusion locates the full scale. If the student understands what the question requires of them credit question at Level 5 of the scientific literacy and correctly identifies a difference in the two graphs, but is unable to explain this difference, the student gains partial scale. credit for the question and is identified at Level 4 of the scientific literacy global This environmental issue is which defines the setting. The skill required by students is to interpret data graphically presented so the question belongs in the “Scientific explanations” category. CS OECD 2014 S Kno W and Can d o: Student Performan C e in m athemati What Student , r eading and S C ien C e – Volume i © 243

246 5 A Profile of Student Perform nce in Science A Level 6 ESTI – Qu GREENHOUSE on 5 708 Level 5 633 o pen-constructed response Question type: Level 4 C ompetency: Explaining phenomena scientifically 559 Level 3 Knowledge category: “Earth and space systems” (knowledge of science) 484 Level 2 “Environment” Application area: 409 Level 1 Setting: Global 335 Below Level 1 709 Difficulty: Percentage of correct answers (OECD countries): 18.9% André persists in his conclusion that the average temperature rise of the Earth’s atmosphere is caused by the increase in the carbon dioxide emission. But Jeanne thinks that his conclusion is premature. She says: “Before accepting this conclusion you must be sure that other factors that could influence the greenhouse effect are constant”. Name one of the factors that Jeanne means. Scoring redit: c ull f Gives a factor referring to the energy/radiation coming from the Sun. For example: he sun heating and maybe the earth changing position. • T • [Assuming that by “Earth” the student means “the ground”.] k from Earth. Energy reflected bac Gives a factor referring to a natural component or a potential pollutant. For example: ater vapour in the air. • W Clouds. • • he things such as volcanic eruptions. T • Atmospheric pollution (gas, fuel). he amount of exhaust gas. T • • CFCs. • he number of cars. T • Ozone (as a component of air). omment C Question 5 of GREE n H ou SE is an example of Level 6 and of the competency explaining phenomena scientifically . In this question, students must analyse a conclusion to account for other factors that could influence the greenhouse effect. This question combines aspects of the two competencies identifying scientific issues and explaining phenomena scientifically. The student needs to understand the necessity of controlling factors outside the change and measured variables and to recognise those variables. The student must possess sufficient knowledge of “Earth systems” to be able to identify at least one of the factors that should be controlled. The latter criterion is considered the critical scientific skill involved so this question is categorised as explaining phenomena scientifically. The effects of this environmental issue are global , which defines the setting. As a first step in gaining credit for this question the student must be able to identify the change and measured variables and have sufficient understanding of methods of investigation to recognise the influence of other factors. However, the student also needs to recognise the scenario in context and identify its major components. This involves a number of abstract concepts and their relationships in determining what “other” factors might affect the relationship between the Earth’s temperature and the amount of carbon dioxide emissions into the atmosphere. This locates the question near the boundary between Level 5 and 6 in the explaining phenomena scientifically category. CS , r eading and S C e in e – Volume ien C C m athemati o: Student Performan d and Can W Kno S What Student OECD 2014 © i 244

247 5 A nce in Science A Profile of Student Perform • Figure I.5.15 • S lothe c Read the text and answer the questions that follow. text S clothe A team of British scientists is developing “intelligent” clothes that will give disabled children the power of Children wearing waistcoats made of a unique electrotextile, linked to a speech synthesiser, will be “speech”. able to make themselves understood simply by tapping on the touch-sensitive material. The material is made up of normal cloth and an ingenious mesh of carbon-impregnated fibres that can conduct electricity. When pressure is applied to the fabric, the pattern of signals that passes through the conducting fibres is altered and a computer chip can work out where the cloth has been touched. It then can trigger whatever electronic device is attached to it, which could be no bigger than two boxes of matches. “The smart bit is in how we weave the fabric and how we send signals through it – and we can weave it into existing fabric designs so you cannot see it’s in there,” says one of the scientists. Without being damaged, the material can be washed, wrapped around objects or scrunched up. The scientist also claims it can be mass-produced cheaply. Source: Farrer, S., “Interactive fabric promises a material gift of the garb”, The Australian , 10 August 1998. LOTHES ESTI on 1 C – Qu Level 6 708 Question type : Complex multiple choice Level 5 633 Competency: Identifying scientific issues Level 4 559 Knowledge category : “Scientific enquiry” (knowledge about science) Level 3 “Frontiers of science and technology” Application area: 484 Level 2 Social Setting: 409 Level 1 567 Difficulty: 335 Below Level 1 Percentage of correct answers (OECD countries): 47.9% Can these claims made in the article be tested through scientific investigation in the laboratory? Circle either “Yes” or “No” for each. Can the claim be tested through scientific investigation in the laboratory? The material can be Yes / No washed without being damaged. wrapped around objects without being damaged. Yes / No Yes / No scrunched up without being damaged. mass-produced cheaply. Yes / No Scoring ull c redit: Yes, Yes, Yes, No, in that order. f Comment The question requires the student to identify the change and measured variables associated with testing a claim about the clothing. It also involves an assessment of whether there are techniques to quantify the measured variable and whether other variables can be controlled. This process then needs to be accurately applied for all four claims. The issue of “intelligent” clothes is in the category “Frontiers of science and technology” and is a community issue addressing a need for disabled . The scientific skills applied are concerned with the nature of investigation which places the social children so the setting is question in the “Scientific enquiry” category. The need to identify change and measured variables, together with an appreciation of what would be involved in carrying out measurement and controlling variables, locates the question at Level 4. e in C 245 OECD 2014 © m e – Volume C ien C eading and S r , CS athemati o: Student Performan d and Can W Kno S What Student i

248 5 A Profile of Student Perform A nce in Science Figure I.5.16 • • m m ontagu ary Read the following newspaper article and answer the questions that follow. THE TION ACCINA V OF HISTORY Mary Montagu was a beautiful woman. She survived an attack of smallpox in 1715 but she was left covered with scars. While living in Turkey in 1717, she observed a method called inoculation that was commonly used there. This treatment involved scratching a weak type of smallpox virus into the skin of healthy young people who then became sick, but in most cases only with a mild form of the disease. Mary Montagu was so convinced of the safety of these inoculations that she allowed her son and daughter to be inoculated. In 1796, Edward Jenner used inoculations of a related disease, cowpox, to produce antibodies against smallpox. Compared with the inoculation of smallpox, this treatment had less side effects and the treated person could not infect others. The treatment became known as vaccination. ESTI on 2 MARY MONTAGU – Qu Level 6 708 Question type: Multiple choice Level 5 633 Competency: Explaining phenomena scientifically Level 4 category: “Living s ystems” ( Knowledge ) knowledge of science 559 Level 3 “Health” Application area: 484 Level 2 Setting: Social 409 Level 1 Difficulty: 436 335 Below Level 1 74.9% Percentage of correct answers (OECD countries): What kinds of diseases can people be vaccinated against? A. Inherited diseases like haemophilia. B. t are caused by viruses, like polio. Diseases tha Diseases from the malfunctioning of the body , like diabetes. C. t has no cure. Any sort of disease tha D. Scoring B. Diseases that are caused by viruses, like polio. f ull c redit: Comment To gain credit the student must recall a specific piece of knowledge that vaccination helps prevent diseases, the cause for which is external to normal body components. This fact is then applied in the selection of the correct explanation and the rejection of other explanations. The term “virus” appears in the stimulus text and provides a hint for students. This lowered the difficulty of the question. Recalling an appropriate, tangible scientific fact and its application in a relatively simple context locates the question at Level 2. on 3 ESTI MARY MONTAGU – Qu Level 6 708 Question type: Multiple choice Level 5 633 Explaining phenomena scientifically Competency: Level 4 559 ) ( “Living systems” category: Knowledge knowledge of science Level 3 Application area: “Health” 484 Level 2 Social Setting: 409 Level 1 431 Difficulty: 335 Below Level 1 75.1% Percentage of correct answers (OECD countries): m d and Can W Kno S What Student OECD 2014 © e in athemati CS , r eading and S ien C e – Volume i C C o: Student Performan 246

249 5 A nce in Science A Profile of Student Perform If animals or humans become sick with an infectious bacterial disease and then recover, the type of bacteria that caused the disease does not usually make them sick again. What is the reason for this? t may cause the same kind of disease. The body has killed all bacteria tha A. t kill this type of bacteria before they multiply. The body has made antibodies tha B. t may cause the same kind of disease. C. The red blood cells kill all bacteria tha D. The red blood cells capture and get rid of this type of bacteria from the body . Scoring f c redit: B. The body has made antibodies that kill this type of bacteria before they multiply. ull Comment To correctly answer this question the student must recall that the body produces antibodies that attack foreign bacteria, the cause of bacterial disease. Its application involves the further knowledge that these antibodies provide resistance to subsequent infections of the same bacteria. The issue is community control of disease, so the setting is social. In selecting the appropriate explanation the student is recalling a tangible scientific fact and applying it in a relatively simple context. Consequently, the question is located at Level 2. MARY MONTAGU ESTI on 4 – Qu Level 6 708 Question type: pen-constructed response o Level 5 633 ompetency: Explaining phenomena scientifically C Level 4 559 Knowledge ( “Living s category: ystems” ) knowledge of science Level 3 484 “Health” Application area: Level 2 409 Setting: Social Level 1 Difficulty: 507 335 Below Level 1 61.7% Percentage of correct answers (OECD countries): Give one reason why it is recommended that young children and old people, in particular, should be vaccinated against influenza (flu). Scoring f c redit: Responses referring to young and/or old people having weaker immune systems than other people, or ull similar. For example: These people have less resistance to getting sick. The young and old can’t fight off disease as easily as others. They are more likely to catch the flu. If they get the flu the effects are worse in these people. Because organisms of young children and older people are weaker. Old people get sick more easily. Comment This question requires the student to identify why young children and old people are more at risk of the effects of influenza than others in the population. Directly, or by inference, the reason is attributed to young children and old people having weaker immune systems. The issue is community control of disease, so the setting is social . A correct explanation involves applying several pieces of knowledge that are well established in the community. The question stem also provides a cue to the groups having different resistance to disease. This puts the question at Level 3. C e in m athemati OECD 2014 CS Kno and Can d o: Student Performan W S What Student , r eading and S C ien C e – Volume i © 247

250 5 A nce in Science A Profile of Student Perform • Figure I.5.17 • odified crop m enetically g S GM C ORN HOULD B E B ANNED S Wildlife conservation groups are demanding that a new genetically modified (GM) corn be banned. This GM corn is designed to be unaffected by a powerful new herbicide that kills conventional corn plants. This new herbicide will kill most of the weeds that grow in cornfields. The conservationists say that because these weeds are feed for small animals, especially insects, the use of the new herbicide with the GM corn will be bad for the environment. Supporters of the use of the GM corn say that a scientific study has shown that this will not happen. Here are details of the scientific study mentioned in the above article: as planted in 200 fields across the country. • Corn w • Eac h field was divided into two. The genetically modified (GM) corn treated with the powerful new herbicide was grown in one half, and the conventional corn treated with a conventional herbicide was grown in the other half. • T he number of insects found in the GM corn, treated with the new herbicide, was about the same as the number of insects in the conventional corn, treated with the conventional herbicide. M ODIFIED CROPS G ESTI on 3 ENETICALLY – Qu Level 6 Multiple choice Question type: 708 Level 5 Competency: Identifying scientific issues 633 Level 4 Knowledge category: “Scientific enquiry” (knowledge about science) 559 Level 3 “Frontiers of science and technology” Application area: 484 Setting: Social Level 2 409 421 Difficulty: Level 1 335 Percentage of correct answers (OECD countries): 73.6% Below Level 1 Corn was planted in 200 fields across the country. Why did the scientists use more than one site? t many farmers could try the new GM corn. A. So tha B. T o see how much GM corn they could grow. C. o cover as much land as possible with the GM crop. T o include various growth conditions for corn. D. T Scoring redit: f ull c D. To include various growth conditions for corn. Comment Towards the bottom of the scale, typical questions for Level 2 are exemplified by Question 3 from the unit GE n ETICALLY identifying scientific issues . Question 3 asks a simple question about PS, which is for the competency o DIFIED CR o M varying conditions in a scientific investigation and students are required to demonstrate knowledge about the design of science experiments. To answer this question correctly in the absence of cues, the student needs to be aware that the effect of the treatment (different herbicides) on the outcome (insect numbers) could depend on environmental factors. Thus, by repeating the test in 200 locations the chance of a specific set of environmental factors giving rise to a spurious outcome can be accounted for. Since the question focuses on the methodology of the investigation it is categorised as “Scientific enquiry”. The application area of genetic modification places this at the “Frontiers of science and technology” and given its restriction to one country it can be said to have a setting. social In the absence of cues this question has the characteristics of Level 4, the student shows an awareness of the need i.e. to account for varying environmental factors and is able to recognise an appropriate way of dealing with that issue. However, the question actually performed at Level 2. This can be accounted for by the cues given in the three distractors. Students likely are able to easily eliminate these as options thus leaving the correct explanation as the answer. The effect is to reduce the difficulty of the question. o: Student Performan and Can i d C ien C eading and S r , CS athemati m e in C W Kno S What Student OECD 2014 © e – Volume 248

251 5 A nce in Science A Profile of Student Perform • Figure I.5.18 • hy p e S xerci e ical S Regular but moderate physical exercise is good for our health. Level 6 PHYSICAL EXERCISE on 3 ESTI – Qu 708 Level 5 633 Question type: Complex multiple choice Level 4 559 Competency: Explaining phenomena scientifically Level 3 484 ) ( Knowledge category: “Living systems” knowledge of science Level 2 409 Application area: “Health” Level 1 Setting: Personal 335 Below Level 1 Difficulty: 386 82.4% Percentage of correct answers (OECD countries): What happens when muscles are exercised? Circle “Yes” or “No” for each statement. Yes or No? Does this happen when muscles are exercised? Yes / No Muscles get an increased flow of blood. Yes / No Fats are formed in the muscles. Scoring f redit: Both correct: Yes, No, in that order. ull c Comment For this question, to gain credit a student has to correctly recall knowledge about the operation of muscles and about the i.e. students must have knowledge of the science fact that active muscles get an increased formation of fat in the body, flow of blood and that fats are not formed when muscles are exercised. This enables the student to accept the first explanation of this complex multiple-choice question and reject the second explanation. The two simple factual explanations contained in the question are not related to each other. Each is accepted or rejected as an effect of the exercise of muscles and the knowledge has widespread currency. This question is located at Level 1, explaining phenomena scientifically. at the very bottom of the scale for the competency eading and S , 249 OECD 2014 r © i e – Volume C ien C CS athemati m e in C o: Student Performan d and Can W Kno S What Student

252 5 nce in Science A Profile of Student Perform A Notes 1. As described in more detail in Annex A5, the annualised change takes into account the specific year in which the assessment took place. In the case of science, this is especially relevant for the 2009 assessment as Costa Rica, Malaysia and the United Arab Emirates (excluding Dubai) implemented the assessment in 2010 as part of PISA+. 2. As described in Annex A5, the annualised change considers the case of countries and economies that implemented PISA 2009 in 2010 as part of PISA 2009+. 3. By accounting for students’ gender, age, socio-economic status, migration background and language spoken at home, the adjusted trends allow for a comparison of trends in performance assuming no change in the underlying population or the effective samples’ average socio-economic status, age and percentage of girls, students with an immigrant background or students that speak a language at home that is different than the language of assessment. See Annex A5 for more details on the calculation of adjusted trends. References The Development of Education, Estonian Ministry of Education and Research (2008), Estonian Ministry of Education and Research, Tallinn. (2010), European Commission , ngoing Reforms: Estonia 2010 Edition o verviews on Education Systems in Europe and o ational Systems n Eurydice, Brussels. Tallinn. ational Curriculum for Basic Schools, n (2011a), Government of the Republic of Estonia Government of the Republic of Estonia Tallinn. pper Secondary Schools, u ational Curriculum for n (2011b), , OECD Publishing. PISA n Line Digital Technologies and Performance (Volume VI), o OECD ), PISA 2009 Results: Students (2011 http://dx.doi.org/10.1787/9789264112995-en (2010), Pathways To Success: How Knowledge And Skills At Age 15 Shape Future Lives In Canada, OECD PISA, OECD Publishing. http://dx.doi.org/10.1787/9789264081925-en (2007), PISA 2006: Science Competencies for Tomorrow’s World OECD , PISA, OECD Publishing. http://dx.doi.org/10.1787/9789264040014-en PISA, OECD Publishing. OECD (2006), Assessing Scientific, Reading and Mathematical Literacy: A Framework for PISA 2006, http://dx.doi.org/10.1787/9789264026407-en o: Student Performan e – Volume © OECD 2014 What Student S Kno W and Can d C C e in m athemati CS , r eading and S C ien i 250

253 6 Policy Implications of Student Performance in PISA 2012 The PISA 2012 assessment dispels the notion that achievement in mathematics is mainly a product of innate ability rather than hard work. Results also suggest that improvement is possible among high performers as well as among low performers. This chapter considers how education policies of school systems and individual schools are associated with student performance and with gender differences in performance. OECD 2014 athemati , r eading and S C What Student C e – Volume i © CS 251 m e in C o: Student Performan d and Can W Kno S ien

254 6 Policy m P lications of s tudent Performance in P isa 2012 i OECD countries invest over USD 230 billion each year in mathematics education in schools. While this is a major investment, the returns are many times larger. Countries that have conducted longitudinal studies of student performance, including performance in PISA, have shown that proficiency in mathematics is a strong predictor of positive outcomes for young adults, influencing their ability to participate in post-secondary education and their expected future earnings. The new Survey of Adult Skills (OECD, 2013) also found that foundation skills in mathematics have a major impact on individuals’ life chances. The survey shows that poor mathematics skills severely limit people’s access to better-paying and more-rewarding jobs; at the aggregate level, inequality in the distribution of mathematics skills across populations is closely related to how wealth is shared within nations. Beyond that, the survey shows that people with strong skills in mathematics are also more likely to volunteer, see themselves as actors rather than as objects of political processes, and are even more likely to trust others. Fairness, integrity and inclusiveness in public policy thus also hinge on the skills of citizens. PISA 2012 provides the most comprehensive picture of the mathematics skills developed in schools that has ever been available, looking not just at what students know in the different domains of mathematics, but also at what they can do with what they know. The results show wide differences between countries in the mathematics knowledge and skills of 15-year-olds. The equivalent of almost six years of schooling, 245 score points on the PISA mathematics scale, separates the highest and lowest average performances of the countries that took part in the PISA 2012 mathematics assessment. However, differences between countries represent only a fraction of the overall variation in student performance. The countries is generally even greater, with over 300 points – the equivalent difference in mathematics performances within of more than seven years of schooling – often separating the highest and the lowest performers in a country. Addressing the education needs of such diverse populations and narrowing the observed gaps in student performance remains a formidable challenge for all countries. The results show that a surprisingly small proportion of the performance variation among countries is explained by the wealth of nations (21% among all countries and economies, 12% among OECD countries) or expenditure per student (30% among all countries and economies, 17% among OECD countries), suggesting that the world is no longer divided into rich and well-educated nations, and poor and badly educated ones. Even more important, the PISA 2012 assessment dispels the widespread notion that mathematics achievement is mainly a product of innate ability rather than hard work. On average across all countries, 32% of 15-year-olds do not reach the baseline Level 2 on the PISA mathematics scale (24% across OECD countries), meaning that those students can perform – at best – routine mathematical procedures following direct instructions. But in Japan and Korea, fewer than 10% of students – and in Shanghai-China, fewer than 4% of students – do not reach this level of proficiency. In these education systems, high expectations for all students are not a mantra but a reality; students who start to fall behind are identified quickly, their problems are promptly and accurately diagnosed, and the appropriate course of action for improvement is quickly taken. Everyone knows what is required to earn a given qualification, in terms of both the content studied and the level of performance to be demonstrated. As discussed in Volume III, the observed variation in mathematics performance is closely related to students’ beliefs about the importance of self-concept, effort and persistence for their performance in mathematics. The fact that those beliefs vary significantly across schools and countries suggests that they can be shaped by education policy and practice. These findings should inspire education policy makers to move away from the notion that only a few students can achieve in mathematics towards one that embraces the proposition that all students can. i mproving average performance It is possible to evaluate trends in performance for countries that participated in PISA 2012 and at least one previous assessment. Trends are analysed for 64 countries and economies, 40 of which improved their average performance in at least one of the three subjects. Countries and economies that improve in PISA are diverse: they are countries and economies from all parts of the world, with education systems that organise their schooling in different ways, and that, when they began their participation in PISA, performed below, at or above the OECD average. The diversity of improving countries and economies shows that improvement in performance in all subjects – or in one particular subject – is possible for all school systems. Some contend that the observed performance differences among countries are mainly the product of culture or socio- economic status. However, PISA 2012 results show that many countries and economies have improved their performance, whatever their culture or socio-economic status. For some of the countries that improved their performance in one or more of the domains assessed, improvements are observed among all students: everyone “moved up”. Other countries concentrated their improvements among their low-achieving students, increasing the share of students who begin to e – Volume o: Student Performan © OECD 2014 What Student S Kno W and Can i C C ien C eading and S r , CS athemati m e in d 252

255 6 m P lications of s tudent Performance in P isa 2012 Policy i show literacy in mathematics, reading or science. Improvement in other countries, by contrast, is concentrated among high-achieving students, so the share of top-performing students grew. Some of the highest-performing education systems were able to extend their lead, while others with very low performance have been catching up. This suggests that improvement is possible, whatever the starting point for students, schools and education systems. Brazil, Dubai (United Arab Emirates), Hong Kong-China, Israel, Macao-China, Poland, Portugal, Qatar, Singapore, Tunisia science during their participation in PISA, and and Turkey improved their average performance in mathematics, reading showing that broad improvement in performance is possible, even in a short time span. Improvements in mathematics and reading were observed in Albania, Chile, Germany, Mexico, Montenegro, Serbia and Shanghai-China. Improvements in mathematics and science were observed in Italy, Kazakhstan and Romania, while improvements in reading and science were observed in Japan, Korea, Latvia and Thailand. Improvements in mathematics (but not in reading or science) were observed in Bulgaria, Greece, Malaysia and the United Arab Emirates (ex. Dubai) while improvements in science (but not in mathematics or reading) were observed only in Ireland. Improvements in reading (but not in mathematics or science) were observed in Chinese Taipei, Colombia, Estonia, Hungary, Indonesia, Liechtenstein, Luxembourg, Peru, the Russian Federation and Switzerland. Even though different countries and economies face significantly different challenges in education and operate in different contexts that privilege certain policies and practices over others, the reform trajectories of improving countries are remarkably consistent with those attributes and policies that, throughout the analyses in Volumes II, III and IV of 1 the PISA results, are related to higher mathematics performance. Throughout these volumes, case studies examine in greater detail the policy reforms adopted by some countries that have improved in PISA. Poland (see Box IV.2.1 in Volume IV), for example, reformed its education system by delaying the age of selection into different programmes, and schools in Germany (see Box II.3.2 in Volume II) are also moving towards reducing the levels of stratification across education programmes. Estonia (see Box I.5.1), Poland (see Box IV.2.1 in Volume IV), Brazil (see Box I.2.4), Colombia (see Box IV.4.3 in Volume IV), Japan (see Box III.3.1 in Volume III), Mexico (see Box II.2.4 in Volume II) and Israel (see Box IV.1.4 in Volume IV) for example, have focused certain policies on improving the quality of their teaching staff by increasing the requirements to earn a teaching license, providing incentives for high-achieving students to enter the profession, raising salaries to make the profession more attractive and to retain more teachers, by offering incentives for teachers to engage in in-service teacher-training programmes or by changing the criteria and benefits associated with teachers’ career advancement. Israel (see Box IV.1.4 in Volume IV), Germany (see Box II.3.2 in Volume II), Mexico (see Box II.2.4 in Volume II), Turkey (see Box I.2.5) and Brazil (see Box I.2.4) have implemented targeted policies to improve performing schools or students, or implemented systems to distribute more resources to those - the performance of low regions and schools that need them the most. Some countries, like Colombia (see Box IV.4.3 in Volume IV), Poland (see Box IV.2.1 in Volume IV) and Korea (see Box I.4.1), have given schools and local authorities more autonomy but have recognised that autonomy works only in the context of collaboration and accountability. Others, like Portugal (see Box III.4.1 in Volume III), have reshaped the organisation of schools to facilitate collaboration and economies of scale between individual schools by creating school clusters. Many low-performing countries that have improved their performance (e.g. Brazil, Box I.2.4, Turkey, Box I.2.5, Colombia, Box IV.4.3 in Volume IV, Tunisia, Box III.3.2 in Volume III and Mexico, Box II.2.4 in Volume II) have focused on ensuring that all 15-year-olds are enrolled and attend school, and have increased the amount of financial resources devoted to the school system. Poland (see Box IV.2.1 in Volume IV), Mexico (see Box II.2.4 in Volume II) and Colombia (see Box IV.4.3 in Volume IV) have expanded the information infrastructure of the education system in support of schools’ and local authorities’ accountability arrangements. Recognising that a positive learning environment is key to promoting positive attitudes among students which, in turn, promote learning, Japan (see Box III.3.1 in Volume III) and Portugal (see Box III.4.1 in Volume III) have improved their students’ attitudes, dispositions and self-beliefs towards school in general, and towards mathematics in particular, by, for example, reforming their curricula so that they are better aligned with students’ interests and 21st century skills. As described further in Volume II of this series, of the countries that improved, and among those that also participated in PISA 2003, Germany, Mexico, Poland and Turkey also reduced the relationship between students’ performance and their socio-economic status, showing that simultaneous improvement in performance and equity is possible. p ur S uing excellence In most countries and economies, only a small proportion of students attains the highest levels and can be called top performers in reading, mathematics or science. Even fewer are the academic all-rounders, those students who achieve CS C S Kno W and Can d o: Student Performan C e in m athemati What Student , r 253 OECD 2014 © i e – Volume C ien eading and S

256 6 m P lications of s tudent Performance in P isa 2012 Policy i proficiency Level 5 or higher in all three subjects. Nurturing excellence in mathematics, reading or science, or in all three domains, is crucial for a country’s development as these students will be the vanguard of a competitive, knowledge-based global economy. Results from the PISA 2012 assessment show that nurturing top performance and tackling low performance need not be mutually exclusive. Some high-performing countries in PISA 2012, like Estonia and Finland, also show small variations in student scores, proving that high performance is possible for all students. Equally important, since their first participations in PISA, France, Hong Kong-China, Italy, Japan, Korea, Luxembourg, Macao-China, Poland, Portugal and the Russian Federation have been able to increase the share of top performers in mathematics, reading or science, indicating that education systems can pursue and promote academic excellence whether they perform at or above the OECD average (e.g. Japan, Korea) or below the OECD average (e.g. Italy, Portugal, the Russian Federation). Only a handful of countries and economies can promote performance at the highest levels and can claim that more than one in ten students are all-rounders. The fact that some countries and economies have a large proportion of all-rounders, that others attain top performance in one subject, and that yet others achieve excellence among all students, suggests that there is untapped potential – and a need for policies and practices to develop this potential – in all countries and economies. performance W ling lo K ac t Countries with large numbers of students who struggle to master basic reading skills at age 15 are likely to be held back in the future, when those students become adults who lack the skills needed to function effectively in the workplace and in society. Among students who fail to reach the baseline level of performance (Level 2) in mathematics, reading or science, most can be expected not to continue with education beyond compulsory schooling, and therefore risk facing difficulties using mathematics, reading and using science concepts throughout their lives. Students who do not reach Level 2 in mathematics, for example, have difficulties with questions involving unfamiliar contexts or requiring information from different sources. The proportion of 15-year-old students at this level varies widely across countries, from fewer than one student in ten in four countries and economies, to the majority of students in 15 countries. Even in the average OECD country, where more than one in five students does not reach Level 2, tackling such low performance is a major challenge. Reducing the proportion of students who perform below Level 2 also has an important economic dimension. According to one estimate, if all students attained Level 2 proficiency in mathematics the combined economic output of OECD countries would be boosted by around USD 200 trillion (OECD, 2010). While such estimates are never wholly certain, they do suggest that the cost of improving education outcomes is just a fraction of the high cost of low student performance. To tackle poor performance and also to increase the share of top-performing students, countries need to look at the barriers posed by social background (examined in Volume II of this series), the relationship between performance and students’ attitudes towards learning (examined in Volume III), and schools’ organisation, resources and learning environment (examined in Volume IV). S S S ind K e SS ne K ea W and S trength S ing SS e of mathematic aSS in different Mathematics performance does not only vary widely among students, but in many countries it also varies between different areas of mathematical processes and content. Now that computer technology is accessible to virtually all and is increasingly capable of carrying out routine processes, jobs that do not require mathematical skills are becoming scarcer. formulate problems mathematically It is now clear that students’ mastery of mathematics must include the capacity to interpret results, as students – and adults – are required to “translate” a real-life situation into mathematical terms and and interpret the results as they apply to this real-life situation. For students to succeed in mathematics and use mathematics during their lives, their daily encounters with the subject at school need to involve more than solving of already- formulated mathematical tasks; they must learn how to formulate and interpret these concepts and tasks. Of course, all countries and economies need to make curricular choices based on their national contexts and priorities; but they can use the results of their students’ performance in PISA’s mathematics subscales to see where their strengths and weaknesses lie to inform policy development in pedagogical orientations and curricular content. Success in mathematics in PISA does not necessarily result in the same level of success in all process and content subscales. For example, within countries and economies there is wide variation in student performance in the and space and shape o: Student Performan i © OECD 2014 What Student S Kno W and Can d C C e in m athemati CS , r eading and S C ien e – Volume 254

257 6 m P lications of s tudent Performance in P isa 2012 Policy i uncertainty and data subscales: countries that succeed in developing students’ ability in space and shape the do not necessarily develop their students’ ability in . uncertainty and data These differences in performance are likely a reflection of the different emphases countries and economies give to the mathematics topics related to these scales (such as geometry for and probability and statistics for space and shape uncertainty and data ). They also offer an opportunity for countries and economies to reflect on whether their weaknesses result from a lack of exposure to content or the way this content is taught in the classroom. What content is covered and how it is covered has implications for students’, and also for country’s/economy’s performance in PISA. PISA 2012 measures, for the first time, the relationship between students’ opportunities to learn mathematics and students’ mathematics literacy. Students who are exposed to formal and applied mathematics perform better in mathematics. PISA finds that exposure only or mostly to applied mathematics is not associated with higher levels of performance. Higher levels of performance are found among those students who are exposed to formal mathematics combined with some exposure to applied mathematics problems. These relationships are strong, which underscores the importance of school in the development of mathematics literacy, and the need for balance in the way mathematics is taught, so that students can master both mathematics concepts and content and how these are applied to real-life problems and situations. S ual opportunitie Q roviding e p S and girl S for boy Boys and girls show different levels of performance in mathematics, reading and science, but performance differences within the genders are significantly larger than those between them. This suggests that the gender gap can be narrowed considerably as both boys and girls in all countries and economies show that they can succeed in all three subjects. Marked gender differences in mathematics performance – in favour of boys – are observed in many countries and economies, but with a number of exceptions and to varying degrees. Among girls, the greatest hurdle is in reaching the top: girls are under-represented among the highest achievers in most countries and economies, which poses a serious challenge to achieving gender parity in science, technology, engineering and mathematics occupations in the future. Some countries succeeded in narrowing the gender gap in mathematics, but strategies for improving the level of engagement, dispositions, self-beliefs and performance among girls need to be continually reviewed and strengthened, particularly those that promote top performance. At the same time, there is evidence that in many countries and economies more boys than girls are among the lowest-performing students, and in some of these more should be done to engage boys in mathematics. In addition, the size of the gender gap in mathematics varies, depending on the particular processes and content of formulating and in the content subscale mathematics. In general, boys’ advantage is most marked in the process subscale employing and space and shape. Girls’ disadvantage in mathematics seems to be narrowest in the process subscale interpreting These gender differences in performance across subscales and in the content subscale uncertainty and data. indicate potential areas for policy development to close the gender gap in mathematics. They also show that overall gender gaps in mathematics can be narrowed, since these are related to particular content and processes. As Volume III in this series highlights, gender differences are also observed in boys’ and girls’ drive towards mathematics and self- beliefs in mathematics: even when boys and girls have the same level of performance, girls are more likely to show signs of anxiety towards mathematics and lower levels of mathematics self-efficacy and self-beliefs. Evidence suggest that actions to close the gender gap in mathematics performance should be targeted at youth and, indeed, children, and should include activities to improve students’ attitudes and self-beliefs towards mathematics. By contrast, in almost all countries and economies, girls outperform boys in reading. This gender gap is particularly large in some high-performing countries, where almost all underperformance in reading is seen only among boys. Low- performing boys face a particularly large disadvantage as they are heavily over-represented among those who fail to show basic levels of reading literacy. These low levels of performance tend to be coupled with low levels of engagement with school and – as observed in PISA 2009 – with low levels of engagement and commitment to reading. To close the gender gap in reading performance policy makers need to promote boys’ engagement with reading and ensure that more boys begin to show the basic level of proficiency that will allow them to participate fully and productively in life. CS i S Kno W and Can d o: Student Performan C e in m athemati What Student , r eading and S C ien C 255 OECD 2014 © e – Volume

258 6 m P lications of s tudent Performance in P isa 2012 Policy i Note 1. As PISA is a series of cross-sectional studies, it is impossible to infer which, if any, of these policy initiatives are at the centre of these countries’ improvement in PISA. The examples described in the country-specific boxes throughout the volumes of the PISA 2009 report provide a description of the challenges and the policy trajectories of the countries that have improved their PISA performance; they do not provide causal evidence that the performance improvement is the result of any particular policy. Reference utlook: First Results from the Survey of Adult Skills, ECD Skills o OECD Publishing. o (2013), OECD http://dx.doi.or g/10.1787/9789264204256-en o PISA, OECD (2010), The High Cost of Low Educational Performance: The Long-Run Economic Impact of Improving PISA utcomes, OECD Publishing. http://dx.doi.org/10.1787/9789264077485-en o: Student Performan e – Volume C ien C r , CS athemati m e in C i d and Can W Kno S What Student OECD 2014 © eading and S 256

259 Annex A i S 2012 P a tE chnical back G round All figures and tables in Annex A are available on line hool and parent context a nnex a 1 : Indices from the student, sc questionnaires http://dx.doi.or g/10.1787/888932937073 2 he PISA target population, the PISA samples nnex a : T a and the definition of schools g/10.1787/888932937092 http://dx.doi.or T : a nnex a 3 echnical notes on analyses in this volume ance Quality assur : 4 a nnex a nnex a 5 a echnical details of trends analyses : T http://dx.doi.or g/10.1787/888932937054 a nnex elopment of the PISA assessment instruments 6 : Dev a 7 a nnex a T : echnical note on Brazil http://dx.doi.or g/10.1787/888932935743 n otes regarding c yprus The information in this document with reference to “Cyprus” relates to the southern part of the Island. There is no single authority Note by Turkey: representing both Turkish and Greek Cypriot people on the Island. Turkey recognises the Turkish Republic of Northern Cyprus (TRNC). Until a lasting and equitable solution is found within the context of the United Nations, Turkey shall preserve its position concerning the “Cyprus issue”. The Republic of Cyprus is recognised by all members of Note by all the European Union Member States of the OECD and the European Union: the United Nations with the exception of Turkey. The information in this document relates to the area under the effective control of the Government of the Republic of Cyprus. note regarding i srael a he statistical data for Israel are supplied by and under the responsibility of the relevant Israeli authorities. The use of such data by the OECD is T without prejudice to the status of the Golan Heights, East Jerusalem and Israeli settlements in the West Bank under the terms of international law. 257 , OECD 2014 © i e – Volume C ien C eading and S r S CS athemati m e in C o: Student Performan d and Can W Kno What Student

260 I Annex A1 A nd p A rent context quest ces from the student, school onn AI res : Ind I nnex a 1 a S S S ue Q chool and parent context S tudent, tionnaire from the S ndice i Explanation of the indices This section explains the indices derived from the student and school context questionnaires used in PISA 2012. Several PISA measures reflect indices that summarise responses from students, their parents or school representatives (typically principals) to a series of related questions. The questions were selected from a larger pool of questions on the basis of theoretical considerations (OECD, 2013) provides an in-depth description of this PISA 2012 Assessment and Analytical Framework and previous research. The conceptual framework. Structural equation modelling was used to confirm the theoretically expected behaviour of the indices and to validate their comparability across countries. For this purpose, a model was estimated separately for each country and collectively for PISA 2012 Technical Report all OECD countries. For a detailed description of other PISA indices and details on the methods, see the (OECD, forthcoming). There are two types of indices: simple indices and scale indices. Simple indices are the variables that are constructed through the arithmetic transformation or recoding of one or more items, in exactly the same way across assessments. Here, item responses are used to calculate meaningful variables, such as the recoding of the four-digit ISCO-08 codes into “Highest parents’ socio-economic index (HISEI)” or, teacher-student ratio based on information from the school questionnaire. Scale indices are the variables constructed through the scaling of multiple items. Unless otherwise indicated, the index was scaled using a weighted likelihood estimate (WLE) (Warm, 1989), using a one-parameter item response model (a partial credit model was used in the PISA 2012 Technical Report case of items with more than two categories). For details on how each scale index was constructed see the (OECD, forthcoming). In general, the scaling was done in three stages: • The item parameters were estimated from equal-sized subsamples of students from all participating countries and economies. The estimates were computed for all students and all schools by anchoring the item parameters obtained in the preceding step. • The indices were then standardised so that the mean of the index value for the OECD student population was zero and the standard • deviation was one (countries being given equal weight in the standardisation process). Sequential codes were assigned to the different response categories of the questions in the sequence in which the latter appeared in the student, school or parent questionnaires. Where indicated in this section, these codes were inverted for the purpose of constructing indices or scales. Negative values for an index do not necessarily imply that students responded negatively to the underlying questions. A negative value merely indicates that the respondents answered less positively than all respondents did on average across OECD countries. Likewise, a positive value on an index indicates that the respondents answered more favourably, or more positively, than respondents did, on average, across OECD countries. Terms enclosed in brackets < > in the following descriptions were replaced in the national versions of the student, school and parent questionnaires by the appropriate national equivalent. For example, the term was translated in the United States into “Bachelor’s degree, post-graduate certificate program, Master’s degree program or first professional degree program”. Similarly the term in Luxembourg was translated into “German classes” or “French classes” depending on whether students received the German or French version of the assessment instruments. In addition to simple and scaled indices described in this annex, there are a number of variables from the questionnaires that correspond to single items not used to construct indices. These non-recoded variables have prefix of “ST” for the questionnaire items in the student questionnaire, “SC” for the items in the school questionnaire, and “PA” for the items in the parent questionnaire. All the context www.pisa.oecd.org . questionnaires as well as the PISA international database, including all variables, are available through Scaling of questionnaire indices for trend analyses In PISA, to gather information about students’ and schools’ characteristics, both students and schools complete a background questionnaire. In PISA 2003 and PISA 2012 several questions were kept untouched, enabling the comparison of responses to these questions over time. In this report, only questions that maintained an exact wording are used for trends analyses. Questions with subtle word changes or questions with major word changes were not compared across time because it is impossible to discern whether observed changes in the response are due to changes in the construct they are measuring or to changes in the way the construct is being measured. Also, in PISA, as described in Annex A1, questionnaire items are used to construct indices. Whenever the questions used in the construction of indices remains intact in PISA 2003 and PISA 2012, the corresponding indices are compared. Two types of indices are used in PISA: simple indices and scale indices. Simple indices recode a set of responses to questionnaire items. For trends analyses, the values observed in PISA 2003 are compared directly to PISA 2012, just as simple responses to questionnaire items are. This is the case of indices like student-teacher ratio and ability grouping in mathematics. o: Student Performan e – Volume © OECD 2014 What Student S Kno W and Can d C C e in m athemati CS , r eading and S C ien i 258

261 a Ind onna I res: I nnex a 1 I ces from the student, school and parent context quest Scale indices, on the other hand, imply WLE estimates which require rescaling in order to be comparable across PISA cycles. Scale , index of attitudes towards school the , index of sense of belonging , the social and cultural status , PISA index of economic indices, like the , the index of intrinsic motivation to learn mathematics the index of instrumental motivation to learn mathematics , the index of mathematics index of quality of index of mathematics anxiety the , the index of teacher shortage , the self-efficacy , the index of mathematics self-concept , the , index of disciplinary climate the , index of quality of schools’ educational resources the , physical infrastructure index of teacher-student , the , relations index of teacher morale the index of student-related factors affecting school climate and the index of teacher-related factors , were scaled, in PISA 2012 to have an OECD average of 0 and a standard deviation of 1, on average, across affecting school climate OECD countries. These same scales were scaled, in PISA 2003, to have an OECD average of 0 and a standard deviation of 1. Because they are on different scales, values reported in Learning for Tomorrow’s World: First Results from PISA 2003 (OECD, 2004) cannot be compared with those reported in this volume. To make these scale indices comparable, values for 2003 have been rescaled to the 2012 scale, using the PISA 2012 parameter estimates. . They can be merged to the corresponding PISA 2003 dataset using the These re-scaled indices are available at www.pisa.oecd.org country names, school and student-level identifiers. The rescaled PISA index of economic, social and cultural status is also available to be merged with the PISA 2000, PISA 2006 and PISA 2009 dataset. Student-level simple indices Age The variable AGE is calculated as the difference between the middle month and the year in which students were assessed and their month and year of birth, expressed in years and months. Study programme In PISA 2012, study programmes available to 15-year-old students in each country were collected both through the student tracking form and the student questionnaire. All study programmes were classified using ISCED (OECD, 1999). In the PISA international database, all national programmes are indicated in a variable (PROGN) where the first six digits refer to the national centre code and the last two digits to the national study programme code. The following internationally comparable indices were derived from the data on study programmes: • Programme level (ISCEDL) indicates whether students are (1) primary education level (ISCED 1); (2) lower-secondary education level (ISCED 2); or (3) upper secondary education level (ISCED 3). • Programme designation (ISCEDD) indicates the designation of the study programme: (1) = “A” (general programmes designed to give access to the next programme level); (2) = “B” (programmes designed to give access to vocational studies at the next programme level); (3) = “C” (programmes designed to give direct access to the labour market); or (4) = “M” (modular programmes that combine any or all of these characteristics). Programme orientation (ISCEDO) indicates whether the programme’s curricular content is (1) general; (2) pre-vocational; (3) • vocational; or (4) modular programmes that combine any or all of these characteristics. Occupational status of parents Occupational data for both a student’s father and a student’s mother were obtained by asking open-ended questions in the student questionnaire. The responses were coded to four-digit ISCO codes (ILO, 1990) and then mapped to the SEI index of Ganzeboom et al. (1992). Higher scores of SEI indicate higher levels of occupational status. The following three indices are obtained: • Mother’s occupational status (OCOD1). Father’s occupational status (OCOD2). • • The highest occupational level of parents (HISEI) corresponds to the higher SEI score of either parent or to the only available parent’s SEI score. Education level of parents The education level of parents is classified using ISCED (OECD, 1999) based on students’ responses in the student questionnaire. As in PISA 2000, 2003, 2006 and 2009, indices were constructed by selecting the highest level for each parent and then assigning them to the following categories: (0) None, (1) ISCED 1 (primary education), (2) ISCED 2 (lower secondary), (3) ISCED 3B or 3C (vocational/pre-vocational upper secondary), (4) ISCED 3A (upper secondary) and/or ISCED 4 (non-tertiary post-secondary), (5) ISCED 5B (vocational tertiary), (6) ISCED 5A, 6 (theoretically oriented tertiary and post-graduate). The following three indices with these categories are developed: Mother’s education level (MISCED). • Father’s education level (FISCED). • Highest education level of parents (HISCED) corresponds to the higher ISCED level of either parent. • Highest education level of parents was also converted into the number of years of schooling (PARED). For the conversion of level of education into years of schooling, see Table A1.1. CS OECD 2014 S Kno W and Can d o: Student Performan C e in m athemati What Student , r eading and S C ien C e – Volume i © 259

262 : Ind Annex A1 rent context quest I onn AI res A I A ces from the student, school nd p ] Part 1/1 [ a able t evels of parental education converted into years of schooling l 1.1 S level d E c i ompleted c c S i ompleted c E d c E d S i ompleted c (upper secondary 3 a or 3 b vels 3 le c (university a vel 5 le education providing y (upper secondar ompleted c level tertiary education) E and a 5 d c S i access to education providing c ompleted b level 5 d E c i S E c i S d level 6 or programmes) and/ 5 b direct access to the S level 2 i d E c ompleted c versity (non-uni (advanced research c d level 4 (non- S i E or labour market or to (lower secondary d c S level 1 i E tertiary education) programmes) tertiary post-secondary) b 5 d c S programmes) E i education) (primary education) ustr alia a 10.0 11.0 12.0 15.0 14.0 6.0 9.0 4.0 ustria a 15.0 17.0 12.5 12.0 OECD 1 15.0 17.0 12.0 9.0 12.0 6.0 elgium b 15.0 12.0 17.0 c anada 6.0 9.0 12.0 16.0 17.0 12.0 8.0 6.0 hile c 12.0 13.0 c zech r epublic 5.0 9.0 11.0 16.0 16.0 16.0 18.0 13.0 13.0 d enmark 7.0 10.0 15.0 12.0 12.0 16.0 Estonia 6.0 9.0 f 16.5 12.0 14.5 9.0 6.0 inland 12.0 15.0 12.0 14.0 12.0 9.0 5.0 ance r f 10.0 4.0 Germany 13.0 15.0 18.0 13.0 6.0 9.0 11.5 12.0 17.0 15.0 Greece 8.0 h ungary 4.0 16.5 10.5 12.0 13.5 i celand 7.0 10.0 13.0 14.0 18.0 16.0 14.0 16.0 i reland 6.0 9.0 12.0 12.0 srael i 6.0 15.0 15.0 12.0 12.0 9.0 i taly 5.0 8.0 12.0 13.0 17.0 16.0 6.0 9.0 14.0 Japan 16.0 12.0 12.0 k or 14.0 16.0 ea 6.0 9.0 12.0 12.0 12.0 9.0 17.0 6.0 16.0 uxembourg l 13.0 16.0 12.0 12.0 9.0 6.0 exico m 14.0 13.0 10.0 15.0 6.0 etherlands n 12.0 16.0 14.0 ew Zealand 5.5 10.0 11.0 12.0 15.0 n 16.0 n orway 6.0 9.0 12.0 12.0 14.0 11.0 8.0 a Poland 15.0 16.0 12.0 Portugal 12.0 12.0 17.0 15.0 6.0 9.0 2 18.0 16.0 4.0 9.0 12.0 13.0 epublic r Slovak 4.0 8.0 11.0 12.0 16.0 15.0 Slovenia 5.0 8.0 Spain 12.0 16.5 13.0 10.0 Sweden 6.0 9.0 11.5 12.0 16.0 14.0 Switzerland 9.0 12.5 12.5 17.5 14.5 6.0 t urke y 5.0 8.0 11.0 11.0 15.0 13.0 u 15.0 nited k ingdom (exclud. Scotland) 6.0 9.0 12.0 13.0 16.0 u 11.0 k 15.0 17.0 13.0 9.0 nited 7.0 ingdom (Scotland) u 6.0 9.0 a 12.0 16.0 14.0 nited States a lbania 6.0 9.0 12.0 12.0 16.0 16.0 a rgentina 6.0 10.0 12.0 12.0 17.0 14.5 9.0 14.0 a zerbaijan 4.0 11.0 11.0 17.0 Partners razil 4.0 8.0 11.0 11.0 16.0 14.5 b b ulgaria 4.0 8.0 10.0 12.0 17.5 15.0 c olombia 5.0 9.0 11.0 11.0 15.5 14.0 16.0 11.0 12.0 14.0 6.0 9.0 c osta r ica 17.0 c 12.0 11.0 8.0 4.0 roatia 15.0 6.0 14.0 ong ong- c hina h 9.0 11.0 13.0 16.0 k 6.0 14.0 15.0 12.0 12.0 9.0 ndonesia i Jordan 10.0 12.0 12.0 16.0 14.5 6.0 14.0 azakhstan 4.0 9.0 11.5 12.5 15.0 k atvia 4.0 8.0 11.0 11.0 16.0 14.0 l 13.0 14.0 iechtenstein 5.0 11.0 9.0 17.0 l ithuania 3.0 8.0 11.0 11.0 16.0 15.0 l 15.0 acao- c hina 6.0 9.0 11.0 12.0 16.0 m m alaysia 6.0 9.0 11.0 13.0 15.0 16.0 16.0 12.0 8.0 4.0 11.0 15.0 m ontenegro 14.0 17.0 11.0 11.0 6.0 Peru 9.0 6.0 9.0 12.0 12.0 16.0 15.0 Qatar r 14.0 omania 4.0 8.0 11.5 12.5 16.0 11.5 9.0 4.0 ation a 15.0 12.0 r ussian f eder 4.0 8.0 11.0 12.0 17.0 14.5 Serbia 9.0 Shanghai- c hina 6.0 12.0 12.0 16.0 15.0 Singapore 6.0 8.0 10.0 11.0 16.0 13.0 6.0 12.0 14.0 12.0 16.0 9.0 c hinese t aipei 16.0 12.0 14.0 hailand t 12.0 6.0 9.0 16.0 17.0 13.0 12.0 9.0 6.0 unisia t u 5.0 rab Emirates a 15.0 nited 16.0 12.0 12.0 9.0 u ruguay 6.0 9.0 12.0 12.0 17.0 15.0 v 12.0 a iet 5.0 9.0 am 12.0 17.0 n 1. In Belgium the distinction between universities and other tertiary schools doesn’t match the distinction between ISCED 5A and ISCED 5B. 2. In the Slovak Republic, university education (ISCED 5A) usually lasts five years and doctoral studies (ISCED 6) lasts three more years. Therefore, university graduates will have completed 18 years of study and graduates of doctoral programmes will have completed 21 years of study. 2 http://dx.doi.org/10.1787/888932937073 1 C OECD 2014 What Student S Kno W and Can d o: Student Performan C e in m athemati CS , r eading and S C e – Volume © ien i 260

263 a Ind onna I res: I nnex a 1 I ces from the student, school and parent context quest Immigration and language background Information on the country of birth of students and their parents is collected in a similar manner as in PISA 2000, PISA 2003 and PISA 2006 by using nationally specific ISO coded variables. The ISO codes of the country of birth for students and their parents are available in the PISA international database (COBN_S, COBN_M, and COBN_F). The index on immigrant background (IMMIG) has the following categories: (1) native students (those students born in the country of assessment, or those with at least one parent born in that country; students who were born abroad with at least one parent born in the country of assessment are also classified as ‘native’ students), (2) second-generation students (those born in the country of assessment but whose parents were born in another country) and (3) first-generation students (those born outside the country of assessment and whose parents were also born in another country). Students with missing responses for either the student or for both parents, or for all three questions have been given missing values for this variable. Students indicate the language they usually speak at home. The data are captured in nationally-specific language codes, which were recoded into variable LANGN with the following two values: (1) language at home is the same as the language of assessment, and (2) language at home is a different language than the language of assessment. Relative grade Data on the student’s grade are obtained both from the student questionnaire and from the student tracking form. As with all variables that are on both the tracking form and the questionnaire, inconsistencies between the two sources are reviewed and resolved during data-cleaning. In order to capture between-country variation, the relative grade index (GRADE) indicates whether students are at the modal grade in a country (value of 0), or whether they are below or above the modal grade level (+ x grades, - x grades). The relationship between the grade and student performance was estimated through a multilevel model accounting for the following PISA index of economic, social and cultural status the ii) ; PISA index of economic, social and cultural status the i) background variables: the school mean of the iii) squared; PISA index of economic, social and cultural status; iv) an indicator as to whether students were v) the percentage of first-generation students in the school; and vi) foreign-born first-generation students; students’ gender. Table A1.2 presents the results of the multilevel model. Column 1 in Table A1.2 estimates the score-point difference that is associated with one grade level (or school year). This difference can be estimated for the 32 OECD countries in which a sizeable number of 15-year-olds in the PISA samples were enrolled in at least two different grades. Since 15-year-olds cannot be assumed to be distributed at random across the grade levels, adjustments had to be made for the above-mentioned contextual factors that may relate to the assignment of students to the different grade levels. These adjustments are documented in columns 2 to 7 of the table. While it is possible to estimate the typical performance difference among students in two adjacent grades net of the effects of selection and contextual factors, this difference cannot automatically be equated with the progress that students have made over the last school year but should be interpreted as a lower boundary of the progress achieved. This is not only because different students were assessed but also because the content of the PISA assessment was not expressly designed to match what students had learned in the preceding school year but more broadly to assess the cumulative outcome of learning in school up to age 15. For example, if the curriculum of the grades in which 15-year-olds are enrolled mainly includes material other than that assessed by PISA (which, in turn, may have been included in earlier school years) then the observed performance difference will underestimate student progress. Student-level scale indices For this cycle, in order to obtain trends for all cycles from 2000 to 2012, the computation of the indices WEALTH, HEDRES, CULTPOSS and HOMEPOS was based on data from all cycles from 2000 to 2012. HOMEPOS is of particular importance as it is used in the computation of ESCS. These were then standardised on 2012 so that the OECD mean is 0 and the standard deviation is 1. This means that the indices calculated on the previous cycle will be on the 2012 scale and thus not directly comparable to the indices in the database for the previously released cycles. To estimate item parameters for scaling, a calibration sample from all cycles was used, consisting of 500 students from all countries in the previous cycles, and 750 from 2012. The items used in the computation of the indices have changed to some extent from cycle to cycle, thought they have remained much the same from 2006 to 2012. The earlier cycle are in general missing a few items that are present in the later cycles, but it was felt leaving out items only present in the later cycles would give too much weight to the earlier cycles. So a superset of all items (except country specific items) in the five cycles was used, and international item parameters derived from this set. The second step was to estimate WLEs for the indices, anchoring on the international item set while estimating the country specific items. This is the same procedure used in previous cycles. A description of the 2012 items used for these indices is given below. Family wealth (WEALTH) is based on students’ responses on whether they had the following at home: a room of their own, The index of family wealth a link to the Internet, a dishwasher (treated as a country-specific item), a DVD player, and three other country-specific items; and their responses on the number of cellular phones, televisions, computers, cars and the number of rooms with a bath or shower. CS © S Kno W and Can d o: Student Performan C e in m athemati What Student , r eading and S C ien C e – Volume 261 OECD 2014 i

264 Annex A1 I A nd p A rent context quest I onn AI res : Ind ces from the student, school Part 1/1 ] [ a able multilevel model to estimate grade effects in mathematics accounting for some background variables 1.2 t a 1 , accounting for: ultilevel model to estimate grade effects in mathematics performance m percentage school mean of i index a S P of first- index S a i the P of economic, S P i a index generation of economic, social and of economic, students at the student social and first-generation cultural status social and school level is a female intercept cultural status students squared cultural status grade S.E. c oeff S.E. c oeff S.E. c oeff S.E. c oeff S.E. c oeff S.E. c c S.E. c oeff S.E. oeff oeff a alia 35 (2.3) 20 (1.4) 1 (1.1) 68 (7.1) 6 (3.9) 0 (0.2) -12 (2.9) 481 (4.1) ustr ustria (8.2) (2.7) 11 (1.8) -2 (1.6) 62 a -9 (6.5) 0 (0.3) -28 (3.3) 526 (5.8) 36 OECD -16 (2.3) 9 (1.4) 2 (0.9) 86 (9.3) 52 (4.4) 0 (0.4) -21 (2.0) 529 (5.4) b elgium anada 44 (2.5) 19 (1.5) 3 (1.1) c (6.8) 6 (3.7) 0 (0.1) -13 (1.9) 506 (4.0) 29 c 33 (1.8) 9 (1.5) 1 (0.7) 37 (3.6) -2 (10.2) -1 (1.1) -29 (2.1) 469 (4.7) hile zech r 47 (3.5) 13 (2.0) -3 (2.0) 111 (9.3) 1 (9.1) -2 (0.9) -24 (2.9) 502 (4.2) c epublic (5.3) 34 26 (2.2) 2 (1.6) 44 (8.0) -34 (3.9) 0 (0.5) -18 (2.2) 483 (5.4) d enmark 41 (2.7) 16 (2.0) 2 (2.3) 25 (6.7) -20 (17.0) -4 (0.6) -7 (2.5) 530 (3.3) Estonia inland f (4.4) 22 (2.1) 6 (1.9) 38 (13.2) -38 (8.7) -1 (0.8) 1 (3.1) 501 (7.7) 52 r (9.5) 49 (4.8) 16 (2.3) 2 (1.7) 60 f -6 (5.8) 0 (0.4) -18 (2.7) 509 (6.3) ance -20 41 5 (1.5) 1 (1.4) 108 (8.3) (2.1) (7.9) -2 (0.7) -28 (2.6) 487 (5.6) Germany (6.3) 8 (6.8) (4.5) 458 (2.6) -15 (0.2) Greece 41 (6.3) 17 (1.7) 1 (1.2) 29 0 h ungary (3.0) 7 (1.8) 3 (1.2) 64 (8.6) 42 (23.9) -1 (0.5) -27 (2.5) 494 (5.6) 32 i celand c 19 (3.2) 3 (1.9) 24 (9.4) -31 (11.0) -1 (0.5) 7 (3.5) 454 (8.4) c i reland (1.8) 24 (1.7) 1 (1.8) 60 (6.1) 18 (4.8) 0 (0.3) -15 (3.0) 491 (4.4) 10 i srael 35 (4.2) 21 (2.6) 3 (1.5) 91 (14.8) -12 (7.7) 1 (0.8) -11 (4.2) 446 (9.7) i taly (1.9) 3 (0.9) -1 (0.7) 54 (5.5) 35 (3.4) 0 (0.1) -23 (1.7) 495 (3.1) -13 Japan c c 3 (2.1) 1 (2.2) 156 (13.3) c c c c -14 (3.2) 548 (5.5) k or ea 40 (14.6) 25 (4.7) 5 (3.0) 75 (20.8) c c c c -10 (5.8) 555 (6.2) l 481 uxembourg 50 (2.3) 12 (1.8) 0 (0.8) 55 (5.4) -7 (4.3) 0 (0.1) -23 (2.7) (4.7) m exico (1.8) 8 (1.1) 2 (0.4) 17 (2.0) 26 (6.0) (0.5) -14 (1.5) 451 (3.1) -44 -1 etherlands 35 (2.6) 6 (1.6) 0 (1.1) 108 (22.6) -14 (9.4) -1 (1.1) -19 n 480 (8.1) (2.1) n 35 (5.6) 31 (2.5) -1 (1.8) 60 ew Zealand -1 (4.4) 0 (0.4) -10 (3.2) 502 (9.6) (8.4) n orway 36 (17.8) 24 (2.5) -2 (1.7) 29 (29.3) -21 (7.8) -1 (0.8) 3 (4.0) 474 (18.0) Poland 80 26 (2.1) -2 (1.8) 37 (6.9) c c c c -5 (3.7) 539 (4.5) (7.0) (7.1) Portugal 17 (1.5) 2 (0.9) 27 (4.0) 10 (2.9) 0 (0.5) -17 (2.2) 540 (4.3) 51 Slovak r epublic 42 (3.8) 21 (2.2) -1 (1.4) 39 (7.5) c c c c -20 (3.0) 530 (4.4) (12.9) Slovenia (6.2) 1 (1.7) 4 (1.5) 72 24 -34 (6.7) 0 (0.8) -25 (2.9) 484 (5.2) (0.2) (3.0) -24 (1.5) 531 (2.4) -16 0 Spain 64 (1.5) 14 (0.9) 2 (0.7) 21 (3.0) 461 (3.0) 3 (0.2) 0 (8.0) -21 (7.8) 29 (1.4) 2 (2.1) 27 (6.7) 67 Sweden (4.6) Switzerland 52 20 (1.8) -2 (1.2) 20 (7.9) -29 (4.5) -1 (0.3) -20 (2.4) 528 (4.3) (3.0) c urke (2.9) 1 (2.4) -1 (1.0) 47 (9.1) c 29 c c -22 (2.7) 553 (17.0) t y u k ingdom 23 (5.4) 20 (2.3) 3 (1.8) 88 nited 4 (6.2) 0 (0.3) -9 (3.2) 465 (4.9) (8.2) u (8.0) 41 21 (1.8) 7 (1.5) 51 (9.4) nited States (3.3) 1 (0.4) -12 (3.5) 457 (6.5) 9 o E cd average 41 (1.0) 16 (0.4) 1 (0.3) 56 (1.9) -10 (1.6) 0 (0.1) -15 (0.5) 498 (1.2) c a (3.9) m m m m m m 6 c c c 0 (4.1) 395 (4.0) lbania a rgentina 31 (1.7) 9 (1.7) 2 (0.9) 38 (7.1) 1 (12.1) -2 (1.0) -18 (2.3) 446 (5.3) (4.3) b 31 (1.2) 5 (2.1) 0 (0.7) 26 razil -49 (19.1) 0 (1.4) -25 (1.8) 432 (7.3) Partners b (1.6) ulgaria 30 (4.2) 12 1 (1.1) 25 (12.6) c c c c -10 (2.6) 429 (8.0) c olombia (1.3) 7 (2.4) 1 (0.7) 26 (4.1) c c c c -30 (2.0) 444 (5.7) 25 c osta ica 26 (1.3) 8 (1.6) 1 (0.6) 25 (4.2) -7 (8.0) 0 (0.8) -29 (2.3) 447 (7.5) r -10 roatia (2.8) 9 (1.9) -1 (1.3) 71 (13.7) 21 (7.6) -1 (0.9) -24 (2.9) 504 (8.1) c c yprus* 39 (6.0) 18 (1.8) 2 (1.1) 61 (8.7) -5 (5.5) 0 (0.2) -14 (2.4) 439 (5.3) 48 h ong- c hina 36 (2.2) 4 (2.6) 1 (1.2) k (14.5) 26 (4.3) 0 (1.0) -22 (3.3) 613 (18.1) ong i ndonesia 17 (2.7) 6 (2.3) 1 (0.6) 27 (5.6) c c c c -6 (1.9) 438 (10.9) (14.9) J 37 (5.3) 12 (2.1) 2 (0.8) 22 ordan 6 (6.6) 2 (1.0) 9 (11.7) 393 (11.4) k 459 -4 (2.2) (5.2) 0 azakhstan 16 (2.5) 14 (2.4) 0 (1.5) 36 (10.3) -5 (5.0) (0.3) l 2 (1.9) 18 (4.0) 53 atvia 25 (3.8) 510 (3.0) -7 c c c c (5.9) (1.8) l iechtenstein (8.9) 8 (4.1) -5 (2.7) 107 (25.4) -10 (9.3) -2 (1.0) -27 (5.2) 543 (20.9) 40 l ithuania (3.4) 17 (1.8) -2 (1.5) 47 (6.9) c c c c -7 (2.6) 483 (4.1) 32 m (12.2) hina 50 (1.7) 7 (2.9) 2 (1.4) 8 acao- 24 (3.0) -1 (0.5) -26 (2.3) 544 (14.2) c m alaysia 79 (7.0) 15 (2.3) 2 (0.9) 53 (7.2) c c c c 2 (2.1) 466 (6.5) m ontenegro (3.1) 13 (1.9) 1 (1.0) 76 (15.6) 16 (7.0) -2 (1.1) -11 (3.2) 437 (8.6) 9 c 25 8 (2.1) 1 (0.6) 36 (3.8) (1.3) c c c -28 (2.5) 434 (6.4) Peru Qatar 28 (2.2) 6 (1.4) 1 (0.7) 26 (7.9) 32 (3.3) 1 (0.1) 2 (4.1) 310 (5.4) r (7.4) -5 (5.6) 20 (2.3) 5 (1.0) 51 omania c c c c -7 (2.8) 475 (9.6) eder r (4.7) 487 (2.2) ussian f (2.6) ation 34 (2.5) 22 -2 -1 (1.5) 21 (9.6) -16 (6.4) -1 (0.5) -11 Serbia 8 (2.1) -1 (1.7) 81 (11.8) (10.4) (11.5) 0 (0.9) -26 (3.9) 480 (8.0) 33 Shanghai- c hina 43 (5.5) 6 (2.4) -3 (1.4) 52 (6.5) -27 (16.1) -1 (1.0) -14 (2.6) 674 (7.6) 29 Singapore (3.3) 21 (2.2) 0 (1.2) 81 (12.6) 44 (4.8) -1 (0.3) -1 (2.7) 608 (9.4) (9.6) c (9.8) 638 aipei 47 (13.2) 21 (3.8) -6 (2.1) 114 (4.1) c c c c 3 t hinese c hailand (3.9) 13 (3.0) 3 (1.1) -22 t 16 c c c 2 (3.5) 418 (17.5) (10.8) t unisia 36 (1.7) 7 (2.0) 2 (0.7) 12 (7.0) c c c c -26 (1.7) 429 (11.5) u 23 a rab Emirates 33 (1.5) 9 (1.3) 3 (0.8) nited (7.4) 31 (2.1) 1 (0.1) -2 (4.7) 387 (4.1) u (2.3) 480 (4.7) -19 ruguay 39 (2.1) 15 (2.0) 3 (0.9) 35 (4.3) c c c c v (15.1) c c c c 36 -22 (4.4) 550 (32.4) n iet (4.8) 12 (4.1) 3 (1.1) 26 am Values that are statistically significant are indicated in bold (see Annex A3). Note: 1. Multilevel regression model (student and school levels): Mathematics performance is regressed on the variables of school policies and practices presented in this table. * See note at the beginning of this Annex. 1 http://dx.doi.org/10.1787/888932937073 2 o: Student Performan e – Volume © OECD 2014 What Student S Kno W and Can d C C e in m athemati CS , r eading and S C ien i 262

265 a Ind onna I res: I nnex a 1 I ces from the student, school and parent context quest Home educational resources The index of home educational resources (HEDRES) is based on the items measuring the existence of educational resources at home including a desk and a quiet place to study, a computer that students can use for schoolwork, educational software, books to help with students’ school work, technical reference books and a dictionary. Cultural possessions (CULTPOSS) is based on the students’ responses to whether they had the following at home: classic index of cultural possessions The literature, books of poetry and works of art. Economic, social and cultural status highest occupational The PISA index of economic, social and cultural status (ESCS) was derived from the following three indices: in years of education according to ISCED (PARED), and highest education level of parents (HISEI), home possessions status of parents index of home possessions (HOMEPOS). The (HOMEPOS) comprises all items on the indices of WEALTH, CULTPOSS and HEDRES, as well as books in the home recoded into a four-level categorical variable (0-10 books, 11-25 or 26-100 books, 101-200 or 201-500 books, more than 500 books). The PISA index of economic, social and cultural status (ESCS) was derived from a principal component analysis of standardised variables (each variable has an OECD mean of zero and a standard deviation of one), taking the factor scores for the first principal component as measures of the PISA index of economic, social and cultural status . Principal component analysis was also performed for each participating country to determine to what extent the components of the index operate in similar ways across countries. The analysis revealed that patterns of factor loading were very similar across countries, with all three components contributing to a similar extent to the index (for details on reliability and factor loadings, see the PISA 2012 Technical Report (OECD, forthcoming). The imputation of components for students missing data on one component was done on the basis of a regression on the other two PISA index of economic, social and cultural status (ESCS) variables, with an additional random error component. The final values on the for PISA 2012 have an OECD mean of 0 and a standard deviation of one. ESCS was computed for all students in the five cycles, and ESCS indices for trends analyses were obtained by applying the parameters used to derive standardised values in 2012 to the ESCS components for previous cycles. These values will therefore not be directly comparable to ESCS in the databases for previous cycles, though the differences are not large for the 2006 and 2009 cycles. ESCS in earlier cycles were computed using different algorithms, so for 2000 and 2003 the differences are larger. Changes to the computation of socio-economic status for PISA 2012 While the computation of socio-economic status followed what had been done in previous cycles, PISA 2012 undertook an important upgrade with respect to the coding of parental occupation. Prior to PISA 2012, the 1988 International Standard Classification of Occupations (ISCO-88) was used for the coding of parental occupation. By 2012, however, ISCO-88 was almost 25 years old and it 1 It was therefore decided to use its replacement, ISCO-08, was no longer tenable to maintain its use as an occupational coding scheme. for occupational coding in PISA 2012. The change from ISCO-88 to ISCO-08 required an update of the International Socio-Economic Index (ISEI) of occupation codes. PISA 2012 therefore used a modified quantification scheme for ISCO-08 (referred to as ISEI-08), as developed by Harry Ganzeboom omen with valid education, occupation and (personal) 500 men and w (2010). ISEI-08 was constructed using a database of 198 incomes derived from the combined 2002-07 datasets of the International Social Survey Programme (ISSP) (Ganzeboom, 2010). The methodology used for this purpose was similar to the one employed in the construction of ISEI for ISCO-68 and ISCO-88 described in 2 different publications (Ganzeboom et al., 1992; Ganzeboom and Treiman, 1996; Ganzeboom and Treiman, 2003). The main differences with regard to the previous ISEI construction are the following: • A new database was used which is more recent, larger and cross-nationally more diverse than the one used earlier. The new ISEI was constructed using data for women and men, while previously only men were used to estimate the scale. The data • on income were corrected for hours worked to adjust the different prevalence of part-time work between men and women in many countries. A range of validation activities accompanied the transition from ISCO-88/ISEI-88 to ISCO-08/ISEI-08, including a comparison of (a) the distributions of ISEI-88 with ISEI-08 in terms of range, mean and standard deviations for both mothers’ and fathers’ occupations and (b) correlations between the two ISEI indicators and performance, again separately undertaken for mothers’ and fathers’ occupation. 1. The update from ISCO-88 to ISCO-08 mainly involved (a) more adequate categories for IT-related occupations, (b) distinction of military ranks and (c) a revision of the categories classifying different managers. http://www.ilo.org/public/english/bureau/stat/isco/index.htm and http://home.fsw.vu.nl/hbg. 2. Information on ISCO08 and ISEI08 is included from ganzeboom/isco08 CS OECD 2014 S Kno W and Can d o: Student Performan C e in m athemati What Student , r eading and S C ien C e – Volume i © 263

266 onn Annex A1 nd p A rent context quest I A AI res : Ind I ces from the student, school The rotated design of the student questionnaire A major innovation in PISA 2012 is the rotated design of the student questionnaire. One of the main reasons for a rotated design, which has previously been implemented for the cognitive assessment, was to extend the content coverage of the student questionnaire. Table A1.3 provides an overview of the rotation design and content of questionnaire forms for the main survey. able Student questionnaire rotation design 1.3 a t Question Set 3 – Opportunity to Learn / Common Question Set (all forms) Form A Question Set 1 – Mathematics Attitudes / Learning Strategies Problem Solving Form B Common Question Set (all forms) Question Set 2 – School Climate / Attitudes Question Set 1 – Mathematics Attitudes / towards School / Anxiety Problem Solving Form C Common Question Set (all forms) Question Set 3 – Opportunity to Learn / Question Set 2 – School Climate / Attitudes towards School / Anxiety Learning Strategies ote: For details regarding the questions in each question set, please refer to PISA 2012 Technical Report (OECD, forthcoming). n The (OECD, forthcoming) provides all details regarding the rotated design of the student questionnaire PISA 2012 Technical Report in PISA 2012, including its implications in terms of (a) proficiency estimates, (b) international reports and trends, (c) further analyses, (d) structure and documentation of the international database, and (e) logistics have been discussed elsewhere. The rotated design has negligible implications for proficiency estimates and correlations of proficiency estimates with context constructs. The international database (available at ) contains all background variables included for each student whereby ones that s/he has www.pisa.oecd.org answered reflecting his or her responses and the ones that s/he was not administered showing a distinctive missing code by design. Rotation allows the estimation of a full co-variance matrix which means that all variables can be correlated with all other variables. It does not affect conclusions in terms of whether or not an effect would be considered significant in multilevel models. References Ganzeboom, H.B.G. (2010), “A new international socio-economic index [ISEI] of occupational status for the International Standard Classification of Occupation 2008 [ISCO-08] constructed with data from the ISSP 2002-2007; with an analysis of quality of occupational measurement in ISSP”, Paper presented at Annual Conference of International Social Survey Programme, Lisbon, 1 May 2010. Ganzeboom, H. B.G. and (2003), “Three Internationally Standardised Measures for Comparative Research on Occupational D. J. Treiman ational Comparison: A European Working Book for n Advances in Cross- Status”, in Jürgen H.P. Hoffmeyer-Zlotnik and C. Wolf (eds.), Demographic and Socio-Economic Variables , Kluwer Academic Press, New York, pp. 159-193. Ganzeboom, H.B.G. and D.J. Treiman (1996), “Internationally Comparable Measures of Occupational Status for the 1988 International (25), pp. 201-239. Standard Classification of Occupations”, Social Science Research Ganzeboom, H.B.G., P. De Graaf, (1992), “A Standard International Socio-Economic Index of D.J. Treiman (with J. De Leeuw) and Social Science Research Occupational Status”, (21-1), pp. 1-56. Ganzeboom, H.B.G., R. Luijkx and D.J. Treiman (1989),”Intergenerational Class Mobility in Comparative Perspective”, Research in (8), pp. 3-79. Social Stratification and Mobility o ILO (1990), ISC o -88: International Standard Classification of International Labour Office, Geneva. ccupations, t, PISA, OECD Publishing. (forthcoming), OECD PISA 2012 Technical Repor (2013), OECD PISA 2012 Assessment and Analytical Framework: Mathematics, Reading, Science, Problem Solving and Financial Literacy , PISA, OECD Publishing. http://dx.doi.org/10.1787/9789264190511-en OECD (2004), Learning for Tomorrow’s World: First Results from PISA 2003 , PISA, OECD Publishing. http://dx.doi.org/10.1787/9789264006416-en OECD Classifying Educational Programmes: Manual for ISCED-97 Implementation in (1999), o ECD Countries . www.oecd.org/education/skills-beyond-school/1962350.pdf 450. - Warm, T.A. (1989), “Weighted likelihood estimation of ability in item response theory”, Psychometrika, Volume 54, Issue 3, pp 427 http://dx.doi.org/10.1007/BF02294627 o: Student Performan e – Volume © OECD 2014 What Student S Kno W and Can d C C e in m athemati CS , r eading and S C ien i 264

267 THE PISA TARGET POPULATION, THE PISA SAMPLES AND THE DEFINITION OF SCHOOLS: NNE x A2 A Annex A2 chool S on of ITI n I he def T nd A S le P m SA he PISA T on, ATI ul P o S The PISA TA rge T P Definition of the PISA tar get population PISA 2012 provides an assessment of the cumulative yield of education and learning at a point at which most young adults are still enrolled in initial education. A major challenge for an international survey is to ensure that international comparability of national target populations is guaranteed in such a venture. Differences between countries in the nature and extent of pre-primary education and care, the age of entry into formal schooling and the institutional structure of education systems do not allow the definition of internationally comparable grade levels of schooling. Consequently, international comparisons of education performance typically define their populations with reference to a target age group. Some previous international assessments have defined their target population on the basis of the grade level that provides maximum coverage of a particular age cohort. A disadvantage of this approach is that slight variations in the age distribution of students across grade levels often lead to the selection of different target grades in different countries, or between education systems within countries, raising serious questions about the comparability of results across, and at times within, countries. In addition, because not all students of the desired age are usually represented in grade-based samples, there may be a more serious potential bias in the results if the unrepresented students are typically enrolled in the next higher grade in some countries and the next lower grade in others. This would exclude students with potentially higher levels of performance in the former countries and students with potentially lower levels of performance in the latter. In order to address this problem, PISA uses an age-based definition for its target population, i.e. a definition that is not tied to the institutional structures of national education systems. PISA assesses students who were aged between 15 years and 3 (complete) months and 16 years and 2 (complete) months at the beginning of the assessment period, plus or minus a 1 month allowable variation, and who were enrolled in an educational institution with Grade 7 or higher, regardless of the grade levels or type of institution in which they were enrolled, and regardless of whether they were in full-time or part-time education. Educational institutions are generally referred to as schools in this publication, although some educational institutions (in particular, some types of vocational education establishments) may not be termed schools in certain countries. As expected from this definition, the average age of students across OECD countries was 15 years and 9 months. The range in country means was 2 months and 5 days (0.18 years), from the minimum country mean of 15 years and 8 months to the maximum country mean of 15 years and 10 months. Given this definition of population, PISA makes statements about the knowledge and skills of a group of individuals who were born within a comparable reference period, but who may have undergone different educational experiences both in and outside of schools. In PISA, these knowledge and skills are referred to as the yield of education at an age that is common across countries. Depending on countries’ policies on school entry, selection and promotion, these students may be distributed over a narrower or a wider range of grades across different education systems, tracks or streams. It is important to consider these differences when comparing PISA results across countries, as observed differences between students at age 15 may no longer appear as students’ educational experiences converge later on. If a country’s scale scores in reading, scientific or mathematical literacy are significantly higher than those in another country, it cannot automatically be inferred that the schools or particular parts of the education system in the first country are more effective than those in the second. However, one can legitimately conclude that the cumulative impact of learning experiences in the first country, starting in early childhood and up to the age of 15, and embracing experiences both in school, home and beyond, have resulted in higher outcomes in the literacy domains that PISA measures. The PISA target population did not include residents attending schools in a foreign country. It does, however, include foreign nationals attending schools in the country of assessment. To accommodate countries that desired grade-based results for the purpose of national analyses, PISA 2012 provided a sampling option to supplement age-based sampling with grade-based sampling. Population coverage All countries attempted to maximise the coverage of 15-year-olds enrolled in education in their national samples, including students enrolled in special educational institutions. As a result, PISA 2012 reached standards of population coverage that are unprecedented in international surveys of this kind. The sampling standards used in PISA permitted countries to exclude up to a total of 5% of the relevant population either by excluding schools or by excluding students within schools. All but eight countries, Luxembourg (8.40%), Canada (6.38%), Denmark (6.18%), Norway (6.11%), Estonia (5.80%), Sweden (5.44%), the United Kingdom (5.43%) and the United States (5.35%), achieved this standard, and in 30 countries and economies, the overall exclusion rate was less than 2%. When language exclusions were accounted for (i.e. removed from the overall exclusion rate), Norway , Sweden, the United Kingdom and the United States no longer had an exclusion rate greater than 5%. For details, see www.pisa.oecd.org. CS OECD 2014 S Kno W and Can d o: Student Performan C e in m athemati What Student , r eading and S C ien C e – Volume i © 265

268 OF SCHOOLS Annex A2 e T POPULATIO n , TH e PISA SAMPL e S A n D TH e D e FI n ITIO n PISA TARG : TH e Exclusions within the above limits include: At the school level: • i) schools that were geographically inaccessible or where the administration of the PISA assessment was not considered feasible; and ii) schools that provided teaching only for students in the categories defined under “within-school exclusions”, such as schools for the blind. The percentage of 15-year-olds enrolled in such schools had to be less than 2.5% of the nationally desired target population [0.5% maximum for i) and 2% maximum for ii) ]. The magnitude, nature and justification of school-level exclusions are documented in the (OECD, forthcoming). PISA 2012 Technical Report At the student level: students with limited iii) students with a functional disability; ii) • students with an intellectual disability; i) assessment language proficiency; other – a category defined by the national centres and approved by the international centre; iv) and v) students taught in a language of instruction for the main domain for which no materials were available. Students could not be excluded solely because of low proficiency or common discipline problems. The percentage of 15-year-olds excluded within schools had to be less than 2.5% of the nationally desired target population. Table A2.1 describes the target population of the countries participating in PISA 2012. Further information on the target population and the implementation of PISA sampling standards can be found in the (OECD, forthcoming). PISA 2012 Technical Report • Column 1 shows the total number of 15-year-olds according to the most recent available information, which in most countries meant the year 2011 as the year before the assessment. Column 2 • shows the number of 15-year-olds enrolled in schools in Grade 7 or above (as defined above), which is referred to as the eligible population . national desired target population • . Countries were allowed to exclude up to 0.5% of students a priori from Column 3 shows the the eligible population, essentially for practical reasons. The following a priori exclusions exceed this limit but were agreed with the PISA Consortium: Belgium excluded 0.23% of its population for a particular type of student educated while working; Canada excluded 1.14% of its population from Territories and Aboriginal reserves; Chile excluded 0.04% of its students who live in Easter Island, Juan Fernandez Archipelago and Antarctica; Indonesia excluded 1.55% of its students from two provinces because of operational reasons; Ireland excluded 0.05% of its students in three island schools off the west coast; Latvia excluded 0.08% of its students in distance learning schools; and Serbia excluded 2.11% of its students taught in Serbian in Kosovo. either umber of students enrolled in schools that were excluded from the national desired target population shows the n Column 4 • from the sampling frame or later in the field during data collection. shows the • Column 5 size of the national desired target population after subtracting the students enrolled in excluded schools . This is obtained by subtracting Column 4 from Column 3. • Column 6 shows the percentage of students enrolled in excluded schools . This is obtained by dividing Column 4 by Column 3 and multiplying by 100. shows the . Note that in some cases this number does not account for number of students participating in PISA 2012 Column 7 • 15-year-olds assessed as part of additional national options. shows the , i.e. the number of students in the nationally defined target population weighted number of participating students Column 8 • that the PISA sample represents. • Each country attempted to maximise the coverage of the PISA target population within the sampled schools. In the case of each sampled school, all eligible students, namely those 15 years of age, regardless of grade, were first listed. Sampled students who were to be excluded had still to be included in the sampling documentation, and a list drawn up stating the reason for their exclusion. Column 9 total number of excluded students , which is further described and classified into specific categories in Table A2.2. indicates the • Column 10 indicates the , i.e. the overall number of students in the nationally defined target weighted number of excluded students population represented by the number of students excluded from the sample, which is also described and classified by exclusion i) students with an intellectual disability – the categories in Table A2.2. Excluded students were excluded based on five categories: student has a mental or emotional disability and is cognitively delayed such that he/she cannot perform in the PISA testing situation; ii) students with a functional disability – the student has a moderate to severe permanent physical disability such that he/she cannot students with a limited assessment language proficiency – the student is unable to read or iii) perform in the PISA testing situation; speak any of the languages of the assessment in the country and would be unable to overcome the language barrier in the testing situation (typically a student who has received less than one year of instruction in the languages of the assessment may be excluded); other – a category defined by the national centres and approved by the international centre; and iv) v) students taught in a language of instruction for the main domain for which no materials were available. . This is calculated as the weighted number of excluded percentage of students excluded within schools shows the Column 11 • students (Column 10), divided by the weighted number of excluded and participating students (Column 8 plus Column 10), then multiplied by 100. • overall exclusion rate , which represents the weighted percentage of the national desired target population shows the Column 12 excluded from PISA either through school-level exclusions or through the exclusion of students within schools. It is calculated as the school-level exclusion rate (Column 6 divided by 100) plus within-school exclusion rate (Column 11 divided by 100) multiplied by 1 minus the school-level exclusion rate (Column 6 divided by 100). This result is then multiplied by 100. Eight countries, Canada, Denmark, Estonia, Luxembourg, Norway, Sweden, the United Kingdom and the United States, had exclusion rates higher than 5%. When language exclusions were accounted for (i.e. removed from the overall exclusion rate), Norway, Sweden, the United Kingdom and the United States no longer had an exclusion rate greater than 5%”. o: Student Performan e – Volume © OECD 2014 What Student S Kno W and Can d C C e in m athemati CS , r eading and S C ien i 266

269 THE PISA TARGET POPULATION, THE PISA SAMPLES AND THE DEFINITION OF SCHOOLS: x A2 NNE A Part 1/2 [ ] a S a target populations and samples 2.1 able pi t Population and sample information otal in national t desired target olled otal enr t population after all otal in t population of Weighted number umber of n School-level school exclusions and hool- otal sc t national 15-year-olds otal t of participating participating exclusion rate before within-school level desired target at Grade 7 or population students students (%) exclusions exclusions population above of 15-year-olds (1) (2) (4) (5) (6) (7) (8) (3) 17 774 1.98 282 457 5 702 250 779 a ustr alia 291 967 288 159 288 159 106 a 93 537 89 073 89 073 ustria 88 967 0.12 4 756 82 242 OECD 117 912 elgium 123 469 121 493 121 209 1 324 119 885 1.09 9 690 b 348 070 anada 417 873 409 453 21 548 2 936 401 831 0.73 404 767 c 6 857 1.06 249 938 229 199 c hile 274 803 252 733 252 625 2 687 93 214 c r epublic 96 946 93 214 zech 1 577 91 637 1.69 6 535 82 101 72 310 65 642 enmark 70 854 70 854 1 965 68 889 2.77 7 481 d Estonia 12 649 12 438 12 438 442 11 996 3.55 5 867 11 634 61 672 60 047 8 829 0.84 f inland 62 523 62 195 62 195 523 755 447 792 983 ance r f 701 399 5 682 3.63 728 044 27 403 755 447 Germany 798 136 798 136 10 914 787 222 1.37 5 001 756 907 798 136 Greece 110 521 105 096 105 096 1 364 103 732 1.30 5 125 96 640 1 725 h 111 761 108 816 108 816 ungary 107 091 1.59 4 810 91 179 10 i celand 4 505 4 491 4 491 3 508 4 481 0.22 4 169 59 296 57 979 57 952 0 57 952 0.00 5 016 54 010 reland i 107 745 srael 118 953 113 278 113 278 2 784 110 494 2.46 6 061 i 521 288 taly 605 490 566 973 i 8 498 558 475 1.50 38 142 566 973 1 188 657 26 099 1 214 756 1 214 756 6 351 1 128 179 2.15 Japan 1 241 786 603 632 672 101 3 053 669 048 0.45 672 101 5 033 ea 687 104 k or 6 187 6 082 6 082 151 5 931 2.48 5 260 5 523 uxembourg l m exico 2 114 745 1 472 875 1 472 875 7 307 1 465 568 0.50 33 806 1 326 025 7 546 194 000 193 190 193 190 etherlands 185 644 3.91 4 460 196 262 n 59 118 53 414 ew Zealand 59 118 60 940 579 58 539 0.98 5 248 n 750 64 917 64 777 64 777 orway 64 027 1.16 4 686 59 432 n Poland 425 597 410 700 410 700 6 900 403 800 1.68 5 662 379 275 Portugal 108 728 127 537 127 537 0 127 537 0.00 5 722 96 034 Slovak 57 887 2.49 5 737 54 486 59 367 59 367 1 480 r epublic 59 723 0.61 7 229 18 303 19 471 Slovenia 115 18 935 18 820 18 935 374 266 25 335 0.50 402 343 2 031 404 374 404 374 423 444 Spain 102 027 102 027 102 087 Sweden 94 988 4 739 1.67 100 322 1 705 Switzerland 85 239 2 479 82 760 2.91 11 234 79 679 87 200 85 239 866 681 y 1 266 638 965 736 965 736 10 387 955 349 1.08 4 848 urke t nited k ingdom 738 066 745 581 688 236 19 820 725 761 2.66 12 659 745 581 u 3 536 153 nited States 3 985 714 4 074 457 4 074 457 41 142 4 033 315 1.01 6 111 u 42 466 lbania 76 910 50 157 50 157 56 50 101 0.11 4 743 a rgentina 684 879 637 603 637 603 3 995 633 608 0.63 5 908 545 942 a 2 470 804 20 091 b razil 3 574 928 2 786 064 2 786 064 34 932 2 751 132 1.25 Partners 70 188 54 255 5 282 59 684 2.41 ulgaria 58 247 59 684 1 437 b 11 173 560 805 620 422 620 422 olombia 620 418 0.00 c 889 729 4 40 384 osta r ica 81 489 64 326 64 326 0 64 326 0.00 4 602 c 45 502 roatia 48 155 46 550 c 417 46 133 0.90 6 153 46 550 yprus* 9 650 9 956 9 956 9 955 128 9 827 1.29 5 078 c 1.04 ong k ong- c hina 84 200 77 864 77 864 813 77 051 h 4 670 70 636 ndonesia 4 174 217 3 599 844 3 544 028 8 039 3 535 989 0.23 5 622 2 645 155 i 125 192 0.11 7 038 111 098 Jordan 129 492 125 333 125 333 141 258 716 azakhstan 208 411 239 674 7 374 5 808 247 048 247 048 2.98 k atvia 18 789 16 054 18 389 l 655 17 720 3.56 5 276 18 375 383 383 417 iechtenstein 314 293 0.26 382 1 l 35 041 35 567 35 567 526 38 524 1.48 4 618 33 042 l ithuania acao- c hina 6 600 5 416 5 416 6 5 410 0.11 5 335 5 366 m 225 544 302 457 999 457 999 alaysia 457 774 0.05 5 197 m 432 080 m ontenegro 8 600 8 600 8 600 18 8 582 0.21 4 744 7 714 Peru 508 969 508 969 263 508 706 0.05 6 035 419 945 584 294 Qatar 11 667 11 532 11 532 202 11 330 1.75 10 966 11 003 omania 146 243 146 243 146 243 5 091 141 152 3.48 5 074 140 915 r 1 268 814 1 272 632 f 1 172 539 6 418 1.40 1 251 014 17 800 1 268 814 ussian ation eder r 80 089 Serbia 74 272 1 987 72 285 2.67 4 684 67 934 75 870 c 1 252 108 056 90 796 90 796 Shanghai- 89 544 1.38 6 374 85 127 hina 53 637 52 163 52 163 293 51 870 0.56 5 546 51 088 Singapore t hinese c aipei 328 356 328 336 328 336 1 747 326 589 0.53 6 046 292 542 t 982 080 784 897 784 897 9 123 775 774 1.16 6 606 703 012 hailand 120 784 unisia 132 313 132 313 132 313 169 132 144 0.13 4 407 t a 971 48 446 48 446 48 824 rab Emirates 40 612 11 500 2.00 47 475 u nited 39 771 5 315 0.03 46 428 14 46 442 46 442 54 638 ruguay u n 1 091 462 1 091 462 7 729 1 083 733 0.71 4 959 956 517 v am 1 717 996 iet Notes: (OECD, forthcoming). The figure for total national population of PISA 2012 Technical Report For a full explanation of the details in this table please refer to the olds enrolled in Column 2 may occasionally be larger than the total number of 15-year-olds in Column 1 due to differing data sources. year - 15 - Information for the adjudicated regions is available on line. * See note at the beginning of this Annex. 2 http://dx.doi.org/10.1787/888932937092 1 , Kno W and Can d o: Student Performan C e in m athemati CS What Student r eading and S C ien C e – Volume i OECD 2014 S © 267

270 n Annex A2 n , TH e PISA SAMPL e S A n D TH e D e FI n ITIO T POPULATIO OF SCHOOLS : TH e e PISA TARG [ Part 2/2 ] a 2.1 target populations and samples a S pi t able overage indices c Population and sample information c overage index 3: c overage index 2: c overage index 1: c overage of overage of c overage of c verall o Within-school Weighted number umber n 15-year-old national enrolled national desired exclusion rate exclusion rate of of population population population (%) (%) excluded students excluded students (9) (10) (11) (12) (13) (14) (15) 4.00 a 0.859 0.960 5 282 2.06 0.960 505 alia ustr 1.33 ustria 0.879 0.987 a 0.987 46 1 011 1.21 OECD 1.40 0.986 0.955 0.31 367 39 elgium b 0.984 6.38 0.926 c anada 1 796 21 013 5.69 0.833 0.936 1.30 0.834 c hile 18 548 0.24 0.987 0.987 1.83 c 0.982 0.847 0.14 118 15 epublic r zech 0.982 6.18 d 0.908 0.938 0.938 3.50 2 381 368 enmark 5.80 0.942 0.942 0.920 2.33 277 143 Estonia 1.91 f 225 653 1.08 0.981 0.981 0.960 inland 4.42 r ance 52 5 828 0.82 f 0.956 0.956 0.885 1.54 Germany 8 1 302 0.17 0.985 0.985 0.948 3.60 Greece 136 2 304 2.33 0.964 0.964 0.874 2.58 h 27 928 1.01 0.974 0.974 0.816 ungary 3.81 celand 155 156 3.60 i 0.962 0.962 0.925 4.47 i reland 271 2 524 4.47 0.955 0.955 0.911 4.13 i 0.906 0.959 srael 114 1 884 1.72 0.959 3.33 741 taly 0.861 0.967 0.967 i 1.86 9 855 2.15 0.979 0.00 Japan 0.979 0 0.909 0 0.82 or k 17 2 238 0.37 ea 0.992 0.992 0.879 8.40 l uxembourg 357 357 6.07 0.872 0.916 0.893 0.74 exico 0.627 0.993 m 0.993 58 0.24 3 247 4.42 1.012 etherlands 27 1 056 0.54 0.956 0.956 n 4.61 n ew Zealand 2 030 3.66 255 0.954 0.954 0.876 6.11 0.939 0.939 n orway 278 3 133 5.01 0.916 4.59 212 Poland 0.954 0.954 0.891 2.96 11 566 1.60 0.883 0.984 0.984 1.60 1 560 124 Portugal 2.93 0.45 epublic r 0.971 0.971 0.912 29 246 Slovak 1.58 0.940 0.984 0.984 0.98 181 84 Slovenia 4.32 3.84 0.884 959 Spain 0.957 0.957 14 931 5.44 Sweden 3 789 3.84 0.946 0.946 0.930 201 4.22 256 1 093 1.35 Switzerland 0.958 0.958 0.914 1.49 0.42 t urke y 21 3 684 0.684 0.985 0.985 5.43 u 2.85 ingdom 486 20 173 nited 0.946 0.946 k 0.932 5.35 u nited States 319 162 194 4.39 0.887 0.946 0.946 0.14 0.999 1 0.999 0.552 0.02 a lbania 10 0.74 rgentina a 0.797 0.993 0.993 641 0.12 12 1.45 b 4 900 0.691 0.986 0.986 0.20 razil 44 Partners 2.55 b 0.15 0.974 0.974 0.773 ulgaria 6 80 0.14 c 23 olombia 0.630 0.999 0.999 789 0.14 0.03 12 osta r ica 2 c 0.03 1.000 1.000 0.496 2.24 0.945 c roatia 0.978 627 1.36 91 0.978 3.29 0.969 c yprus* 157 200 2.03 0.967 0.967 1.76 c ong- 0.982 0.839 k 0.73 518 ong 38 0.982 hina h 0.26 i 0.03 0.634 0.982 0.997 860 2 ndonesia 0.39 0.858 0.27 0.996 0.996 19 304 Jordan 3.43 k azakhstan 951 0.45 0.806 0.966 0.966 25 4.02 l 14 76 0.47 0.960 0.959 0.854 atvia 4.22 iechtenstein 13 3.97 0.958 0.958 0.753 13 l 4.00 ithuania 130 867 2.56 0.858 0.960 0.960 l 0.17 3 0.813 acao- c 3 0.998 0.06 hina 0.998 m 0.18 0.998 7 554 0.13 alaysia 0.998 0.794 m 0.31 m 4 8 0.10 ontenegro 0.997 0.997 0.897 0.18 Peru 8 549 0.13 0.719 0.998 0.998 2.51 85 0.943 Qatar 0.77 0.975 0.975 85 3.48 0.964 0.965 0.965 omania 0.00 0 0 r 2.40 ussian f eder ation 69 r 1.01 11 940 0.976 0.976 0.921 2.87 0.951 Serbia 10 136 0.20 0.848 0.971 1.50 hina 0.788 0.985 0.985 0.13 107 8 c Shanghai- 1.17 Singapore 33 315 0.61 0.988 0.988 0.952 1.22 0.988 hinese t aipei 44 2 029 0.69 c 0.988 0.891 1.32 0.716 0.16 1 144 0.987 0.987 t hailand 12 0.24 unisia t 0.11 130 0.913 5 0.998 0.998 2.09 11 0.979 0.832 0.09 37 0.979 rab Emirates a nited u 0.28 15 ruguay u 0.997 0.997 0.728 0.25 99 0.73 n am 1 198 0.02 iet 0.993 0.993 0.557 v PISA 2012 Technical Report Notes: (OECD, forthcoming). The figure for total national population of For a full explanation of the details in this table please refer to the 15 year olds enrolled in Column 2 may occasionally be larger than the total number of 15-year-olds in Column 1 due to differing data sources. - - Information for the adjudicated regions is available on line. * See note at the beginning of this Annex. 2 http://dx.doi.org/10.1787/888932937092 1 C OECD 2014 What Student S Kno W and Can d o: Student Performan C e in m athemati CS , r eading and S C e – Volume © ien i 268

271 THE PISA TARGET POPULATION, THE PISA SAMPLES AND THE DEFINITION OF SCHOOLS: NNE A x A2 Part 1/1 ] [ a xclusions t 2.2 e able Student exclusions (unweighted) Student exclusions (weighted) Weighted umber n number Weighted Weighted of excluded of excluded umber n n umber number number students students Weighted umber n of n umber Weighted of of excluded of excluded because of because of number of excluded of number excluded students students t otal t otal no materials no materials of excluded excluded students of excluded excluded students with with number weighted available in available in students students with students students with functional number of of the language the language for other for other because of because of functional intellectual intellectual disability excluded excluded of instruction of instruction reasons reasons language language disability disability disability c ode 1) ( students students ode 5) c c ode 5) ( ( c ode 4) c ode 4) ( ( ode 3) ode 3) c c ( ( c ode 1) ( ode 2) c ode 2) c ( ( (2) (3) (4) (5) (6) (1) (8) (9) (10) (11) (12) (7) a alia 39 395 71 0 0 505 471 3 925 886 0 0 ustr 5 282 a 11 24 11 0 0 ustria 332 438 241 0 0 1 011 46 OECD elgium b 12 0 0 39 24 154 189 0 0 367 5 22 18 682 anada 121 0 0 1 796 981 1 593 1 350 0 0 21 013 82 c hile 3 15 0 0 0 18 74 474 0 0 0 548 c c r epublic 1 8 6 0 0 15 1 84 34 0 0 118 zech enmark 2 381 204 112 42 0 368 44 1 469 559 310 0 10 d Estonia 134 2 0 0 143 14 260 3 0 0 277 7 43 f 80 101 15 24 225 5 363 166 47 35 653 inland 0 0 ance 5 828 0 0 0 0 5 828 52 0 52 0 r f 0 4 0 0 8 0 705 597 0 0 1 302 Germany 4 348 3 111 0 136 49 4 91 1 816 0 2 304 Greece 18 ungary 928 15 2 9 0 27 36 568 27 296 0 1 h 5 celand 105 27 18 0 156 5 105 27 18 0 155 i 2 524 reland 13 159 33 66 0 271 121 1 521 283 599 0 i 114 9 91 14 0 0 srael 133 1 492 260 0 0 1 884 i 9 855 taly 64 566 111 0 0 741 596 7 899 1 361 0 0 i Japan 0 0 0 0 0 0 0 0 0 0 0 0 357 uxembourg 6 261 90 0 0 357 6 261 90 0 0 l 0 45 2 390 812 58 0 0 1 36 3 247 0 21 exico m 27 5 21 1 0 0 etherlands 188 819 50 0 0 1 056 n n ew Zealand 27 118 99 0 11 255 235 926 813 0 57 2 030 orway 192 11 n 0 0 278 75 2 180 832 0 0 3 133 120 11 566 oland 23 89 6 88 6 212 1 470 5 187 177 4 644 89 P 1 560 0 0 94 860 124 0 0 7 48 605 Portugal 69 17 k 2 15 0 0 0 ea 223 2 015 0 0 0 2 238 or Slovak r epublic 2 14 0 13 0 29 22 135 0 89 0 246 0 13 27 44 Slovenia 0 84 23 76 81 0 0 181 618 Spain 224 0 0 959 679 11 330 2 984 0 0 14 931 56 3 789 120 0 81 0 0 201 2 218 0 1 571 0 0 Sweden 1 093 346 706 0 0 150 99 0 0 256 41 Switzerland 7 2 556 757 21 0 0 2 14 5 y 3 684 0 0 371 urke t k ingdom 40 405 41 0 0 486 1 468 15 514 3 191 0 0 20 173 u nited 37 319 63 0 0 nited States 18 399 113 965 29 830 0 0 162 194 u 219 0 a 0 1 0 0 1 0 0 10 0 0 10 lbania a rgentina 1 11 0 0 0 12 84 557 0 0 0 641 1 792 razil 27 0 0 0 44 17 3 108 0 0 0 4 900 b Partners ulgaria 6 0 0 0 0 6 80 0 0 0 0 80 b 397 12 10 1 0 0 23 olombia 378 14 0 0 789 c 0 osta r ica 0 2 0 c 0 2 0 12 0 0 0 12 69 roatia 3 0 0 91 78 539 19 0 0 627 10 c yprus* 72 54 60 35 0 157 9 64 8 55 0 200 c 0 k h c hina 4 33 1 ong 0 38 57 446 15 0 0 518 ong- i ndonesia 1 0 1 0 0 2 426 0 434 0 0 860 109 Jordan 5 0 0 19 6 72 122 0 0 304 8 951 azakhstan 9 16 0 0 0 25 317 634 0 0 0 k 76 atvia 3 7 4 0 l 14 8 45 24 0 0 0 0 5 7 1 13 0 0 13 5 iechtenstein 1 7 0 l 120 10 ithuania 0 0 0 801 130 66 867 0 0 0 l acao- hina 0 1 2 0 0 3 0 1 2 0 0 3 c m alaysia 4 0 0 0 7 3 279 0 0 0 554 274 m 0 ontenegro 3 1 0 0 0 4 7 1 0 8 0 m Peru 5 0 0 0 8 3 280 0 0 0 269 549 Qatar 23 43 19 0 0 85 23 43 19 0 0 85 0 r 0 0 0 0 0 omania 0 0 0 0 0 0 660 11 940 0 0 6 934 4 345 69 0 ussian f 4 ation 25 40 0 eder r 2 4 4 Serbia 136 0 0 28 55 10 53 0 0 Shanghai- 107 hina 1 6 1 0 0 8 14 80 14 0 0 c 33 Singapore 17 11 0 0 5 50 157 109 0 0 315 t hinese 296 70 2 029 0 c 0 1 664 aipei 6 36 2 0 0 44 12 1 144 2 10 0 0 0 hailand 13 1 131 0 0 0 t 4 t unisia 1 0 0 0 5 104 26 0 0 0 130 9 26 2 0 0 37 u nited a rab Emirates 3 7 1 0 0 11 33 66 15 0 0 0 6 9 ruguay 99 u 0 0 0 1 iet n 198 0 0 0 198 0 am 0 0 0 1 0 v Exclusion codes: Code 1 Functional disability – student has a moderate to severe permanent physical disability. Code 2 vely delayed or is considered in the professional opinion of Intellectual disability – student has a mental or emotional disability and has either been tested as cogniti qualified staff to be cognitively delayed. Code 3 Limited assessment language proficienc y – student is not a native speaker of any of the languages of the assessment in the country and has been resident in the country for less than one year. Code 4 Other reasons defined by the national centres and approved by the international centre. No materials available in the language of instruction. Code 5 (OECD, forthcoming). Note: For a full explanation of the details in this table please refer to the PISA 2012 Technical Report Information for the adjudicated regions is available on line. * See note at the beginning of this Annex. 1 http://dx.doi.org/10.1787/888932937092 2 CS OECD 2014 S Kno W and Can d o: Student Performan C e in m athemati What Student , r eading and S C ien C e – Volume i © 269

272 n Annex A2 PISA TARG e T POPULATIO n , TH e PISA SAMPL e S A n D TH e D e FI e ITIO n OF SCHOOLS : TH • Column 13 presents an index of the extent to which the national desired target population is covered by the PISA sample . Canada, Denmark, Estonia, Luxembourg, Norway, Sweden, the United Kingdom and the United States were the only countries where the coverage is below 95%. . The index • Column 14 presents an index of the extent to which 15-year-olds enrolled in schools are covered by the PISA sample measures the overall proportion of the national enrolled population that is covered by the non-excluded portion of the student sample. The index takes into account both school-level and student-level exclusions. Values close to 100 indicate that the PISA sample represents the entire education system as defined for PISA 2012. The index is the weighted number of participating students (Column 8) divided by the weighted number of participating and excluded students (Column 8 plus Column 10), times the nationally defined target population (Column 5) divided by the eligible population (Column 2). presents an • Column 15 index of the coverage of the 15-year-old population . This index is the weighted number of participating students (Column 8) divided by the total population of 15-year-old students (Column 1). This high level of coverage contributes to the comparability of the assessment results. For example, even assuming that the excluded students would have systematically scored worse than those who participated, and that this relationship is moderately strong, an exclusion rate in the order of 5% would likely lead to an overestimation of national mean scores of less than 5 score points (on a scale with an international mean of 500 score points and a standard deviation of 100 score points). This assessment is based on the following calculations: if the correlation between the propensity of exclusions and student performance is 0.3, resulting mean scores would likely be overestimated by 1 score point if the exclusion rate is 1%, by 3 score points if the exclusion rate is 5%, and by 6 score points if the exclusion rate is 10%. If the correlation between the propensity of exclusions and student performance is 0.5, resulting mean scores would be overestimated by 1 score point if the exclusion rate is 1%, by 5 score points if the exclusion rate is 5%, and by 10 score points if the exclusion rate is 10%. For this calculation, a model was employed that assumes a bivariate normal distribution for performance (OECD, forthcoming). and the propensity to participate. For details, see the PISA 2012 Technical Report Sampling procedures and response rates The accuracy of any survey results depends on the quality of the information on which national samples are based as well as on the sampling procedures. Quality standards, procedures, instruments and verification mechanisms were developed for PISA that ensured that national samples yielded comparable data and that the results could be compared with confidence. Most PISA samples were designed as two-stage stratified samples (where countries applied different sampling designs, these are documented in the [OECD, forthcoming]). The first stage consisted of sampling individual schools in which PISA 2012 Technical Report 15-year-old students could be enrolled. Schools were sampled systematically with probabilities proportional to size, the measure of size being a function of the estimated number of eligible (15-year-old) students enrolled. A minimum of 150 schools were selected in each country (where this number existed), although the requirements for national analyses often required a somewhat larger sample. As the schools were sampled, replacement schools were simultaneously identified, in case a sampled school chose not to participate in PISA 2012. In the case of Iceland, Liechtenstein, Luxembourg, Macao-China and Qatar, all schools and all eligible students within schools were included in the sample. Experts from the PISA Consortium performed the sample selection process for most participating countries and monitored it closely in those countries that selected their own samples. The second stage of the selection process sampled students within sampled schools. Once schools were selected, a list of each sampled school’s 15-year-old students was prepared. From this list, 35 students were then selected with equal probability (all 15-year-old students were selected if fewer than 35 were enrolled). The number of students to be sampled per school could deviate from 35, but could not be less than 20. Data-quality standards in PISA required minimum participation rates for schools as well as for students. These standards were established to minimise the potential for response biases. In the case of countries meeting these standards, it was likely that any bias resulting from non-response would be negligible, i.e. typically smaller than the sampling error. A minimum response rate of 85% was required for the schools initially selected. Where the initial response rate of schools was between 65% and 85%, however, an acceptable school response rate could still be achieved through the use of replacement schools. This procedure brought with it a risk of increased response bias. Participating countries were, therefore, encouraged to persuade as many of the schools in the original sample as possible to participate. Schools with a student participation rate between 25% and 50% were not regarded as participating schools, but data from these schools were included in the database and contributed to the various estimations. Data from schools with a student participation rate of less than 25% were excluded from the database. PISA 2012 also required a minimum participation rate of 80% of students within participating schools. This minimum participation rate had to be met at the national level, not necessarily by each participating school. Follow-up sessions were required in schools in which too few students had participated in the original assessment sessions. Student participation rates were calculated over all original schools, and also over all schools, whether original sample or replacement schools, and from the participation of students in both the original assessment and any follow-up sessions. A student who participated in the original or follow-up cognitive sessions was regarded as a participant. Those who attended only the questionnaire session were included in the international database and contributed to the statistics presented in this publication if they provided at least a description of their father’s or mother’s occupation. o: Student Performan athemati © OECD 2014 What Student S Kno W and Can d CS C e in i e – Volume C ien C eading and S r , m 270

273 THE PISA TARGET POPULATION, THE PISA SAMPLES AND THE DEFINITION OF SCHOOLS: x A2 NNE A Part 1/2 [ ] a 2.3 esponse rates t able r f hool replacement nitial sample – before school replacement i inal sample – after sc Weighted Weighted number number of Weighted of schools sampled n umber of schools sampled number of Weighted school (responding and Weighted number Weighted school responding and n umber of (responding and responding participation non-responding) of responding participation rate non-responding responding non-responding) schools rate before (weighted also schools (weighted after replacement schools schools (weighted also (weighted also replacement by enrolment) also by enrolment) (%) (unweighted) (unweighted) by enrolment) by enrolment) (%) (1) (3) (4) (5) (6) (7) (8) (2) 274 432 268 631 98 790 757 274 432 a ustr alia 98 268 631 191 a 100 88 967 88 967 191 ustria 100 88 967 88 967 OECD b elgium 84 100 482 119 019 246 294 97 115 004 119 006 c anada 91 362 178 396 757 828 907 93 368 600 396 757 c 239 370 236 576 99 224 200 hile 92 220 009 239 429 c 88 797 zech r epublic 98 87 238 88 884 292 297 100 88 447 d enmark 87 61 749 71 015 311 366 96 67 709 70 892 206 206 12 046 12 046 100 Estonia 100 12 046 12 046 60 323 59 912 99 313 310 60 323 59 740 99 inland f f r 703 458 728 401 223 97 97 703 458 728 401 ance 231 98 735 944 753 179 227 233 98 737 778 753 179 Germany 99 93 102 087 176 192 95 107 100 892 102 053 Greece h ungary 98 99 317 101 751 198 208 99 101 187 101 751 i 99 4 395 4 424 133 140 99 4 395 4 424 celand i reland 99 56 962 57 711 182 185 99 57 316 57 711 i 109 895 srael 91 99 543 109 326 166 186 94 103 075 522 686 1 104 536 921 478 317 536 821 97 1 232 i taly 89 96 Japan 1 175 794 173 200 1 015 198 1 123 211 1 175 794 86 k or ea 100 661 575 662 510 156 157 100 661 575 662 510 l 42 100 5 931 5 931 42 uxembourg 100 5 931 5 931 m 1 442 234 exico 92 1 323 816 1 442 242 1 431 1 562 95 1 374 615 199 n 139 709 185 468 148 75 89 165 635 185 320 etherlands n ew Zealand 81 47 441 58 676 156 197 89 52 360 58 616 63 642 n 85 54 201 63 653 177 208 95 60 270 orway Poland 85 343 344 402 116 159 188 98 393 872 402 116 122 713 195 128 050 122 238 186 96 Portugal 95 128 129 Slovak epublic r 57 353 236 99 57 599 202 58 201 87 50 182 18 680 18 329 98 353 335 18 680 18 329 98 Slovenia 403 999 402 604 100 Spain 403 999 402 604 100 904 902 Sweden 99 99 726 207 211 100 99 536 99 767 98 645 94 82 032 83 450 397 422 98 Switzerland 83 424 78 825 170 urke 97 921 643 945 357 165 t 100 944 807 945 357 y u nited k ingdom 80 564 438 705 011 477 550 89 624 499 699 839 u 67 2 647 253 3 945 575 139 207 77 3 040 661 3 938 077 nited States a lbania 100 49 632 49 632 204 204 100 49 632 49 632 229 a 95 578 723 606 069 218 rgentina 96 580 989 606 069 b razil 803 2 747 688 93 2 545 863 2 745 045 886 95 2 622 293 Partners b 99 57 101 57 574 186 188 100 57 464 ulgaria 57 574 c 87 530 553 612 605 323 363 97 596 557 612 261 olombia 191 c ica 99 64 235 64 920 r 193 99 64 235 64 920 osta c roatia 99 45 037 45 636 161 164 100 45 608 45 636 c 97 9 485 9 821 117 131 97 9 485 9 821 yprus* h ong 76 567 ong- c hina 79 60 277 76 589 123 156 94 72 064 k 98 2 951 028 i ndonesia 95 2 799 943 2 950 696 199 210 2 892 365 119 147 119 147 100 Jordan 119 147 119 147 100 233 233 239 767 100 239 767 239 767 218 218 100 239 767 azakhstan k 17 488 15 371 atvia l 88 17 448 17 428 100 213 186 100 382 382 12 12 100 382 382 iechtenstein l l ithuania 98 33 989 34 614 211 216 100 34 604 34 604 5 410 m acao- c hina 100 45 5 410 45 100 5 410 5 410 m 100 455 543 455 543 164 164 100 455 543 455 543 alaysia 8 540 ontenegro 100 8 540 8 540 51 51 100 8 540 m 507 602 243 514 574 Peru 98 503 915 514 574 238 99 164 157 11 340 11 333 100 11 340 11 333 100 Qatar 139 597 omania 100 139 597 139 597 178 100 139 597 178 r 1 243 564 100 f eder ation 100 ussian 1 243 564 227 227 r 1 243 564 1 243 564 72 752 Serbia 90 65 537 72 819 143 160 95 69 433 100 89 832 Shanghai- c hina 100 89 832 89 832 155 155 89 832 98 50 415 51 687 170 176 98 50 945 51 896 Singapore t hinese c 100 aipei 324 667 324 667 163 163 100 324 667 324 667 772 654 772 452 t hailand 98 757 516 772 654 235 240 100 153 152 130 141 129 229 99 unisia t 130 141 129 229 99 u nited 99 460 453 46 748 46 469 99 rab Emirates 46 748 a 46 469 ruguay u 100 180 45 736 46 009 46 009 99 179 46 009 1 068 462 iet 1 068 462 1 068 462 100 162 162 n 1 068 462 100 am v Information for the adjudicated regions is available on line. * See note at the beginning of this Annex. 1 http://dx.doi.org/10.1787/888932937092 2 CS OECD 2014 S Kno W and Can d o: Student Performan C e in m athemati What Student , r eading and S C ien C e – Volume i © 271

274 OF SCHOOLS Annex A2 e PISA SAMPL e S A n D TH e D e FI n ITIO n , TH : TH e n T POPULATIO e PISA TARG [ Part 2/2 ] t able a 2.3 r esponse rates f inal sample – after sc f hool replacement inal sample – students within sc hools after school replacement n umber of students n umber of students umber n sampled sampled Weighted student of responding and umber n (assessed umber of students n (assessed n umber of students participation rate responding - non of responding and absent) assessed and absent) assessed after replacement schools schools (unweighted) (unweighted) (weighted) (weighted) (%) (unweighted) (unweighted) (9) (10) (11) (12) (13) (14) (15) 87 20 799 213 495 17 491 790 757 alia 246 012 a ustr 92 ustria 191 191 a 75 393 82 242 4 756 5 318 OECD b 9 649 103 914 91 294 282 elgium 114 360 10 595 c 324 328 anada 840 907 81 261 928 20 994 25 835 c hile 221 224 95 214 558 226 689 6 857 7 246 90 73 536 7 222 81 642 295 6 528 epublic 297 c zech r 366 339 enmark d 8 496 7 463 62 988 56 096 89 Estonia 206 93 10 807 11 634 5 867 6 316 206 f inland 311 313 91 54 126 59 653 8 829 9 789 89 f ance 223 231 r 605 371 676 730 5 641 6 308 93 Germany 228 233 742 416 692 226 4 990 5 355 Greece 192 97 92 444 95 580 5 125 5 301 188 h ungary 204 208 93 84 032 90 652 4 810 5 184 i celand 133 140 85 3 503 4 135 3 503 4 135 i 45 115 5 016 5 977 reland 183 185 84 53 644 i 186 172 srael 6 727 6 061 101 288 91 181 90 i 1 186 1 232 93 473 104 510 005 38 084 41 003 taly Japan 191 200 96 1 034 803 1 076 786 6 351 6 609 k or ea 156 157 99 595 461 603 004 5 033 5 101 l 42 42 uxembourg 5 523 95 5 260 5 523 5 260 m 1 468 1 562 94 1 193 866 1 271 639 33 786 35 972 exico n etherlands 177 199 85 148 432 174 697 4 434 5 215 6 206 n ew Zealand 177 197 85 40 397 47 703 5 248 91 208 197 orway 4 686 n 56 286 5 156 51 155 5 629 371 434 325 389 88 188 182 Poland 6 452 92 395 5 608 6 426 187 87 80 719 195 Portugal 6 106 53 912 50 544 94 5 737 231 epublic r 236 Slovak 90 353 335 Slovenia 7 921 7 211 17 849 16 146 Spain 902 90 334 382 372 042 26 443 29 027 904 209 4 739 92 87 359 94 784 Sweden 5 141 211 410 Switzerland 92 72 116 78 424 11 218 12 138 422 t urke y 169 170 98 850 830 866 269 4 847 4 939 u nited ingdom 505 550 86 k 613 736 12 638 14 649 528 231 u nited States 161 207 89 2 429 718 2 734 268 6 094 6 848 a lbania 204 204 92 39 275 42 466 4 743 5 102 a 219 5 804 519 733 229 6 680 457 294 rgentina 88 b razil 837 886 90 2 133 035 2 368 438 19 877 22 326 Partners b ulgaria 187 188 96 51 819 54 145 5 280 5 508 c olombia 352 363 93 507 178 544 862 11 164 12 045 r c osta ica 191 193 89 35 525 39 930 4 582 5 187 c 163 164 92 41 912 45 473 6 153 6 675 roatia c yprus* 117 131 93 8 719 9 344 5 078 5 458 66 665 4 659 93 5 004 62 059 h ong k ong- c hina 147 156 95 210 206 ndonesia 5 579 2 605 254 2 478 961 i 5 885 J 7 402 ordan 233 95 105 493 111 098 7 038 233 218 azakhstan k 206 053 5 874 5 808 208 411 99 218 l 213 91 14 579 16 039 5 276 5 785 211 atvia iechtenstein 12 12 93 293 314 293 314 l l ithuania 216 216 92 30 429 33 042 4 618 5 018 m 5 366 c hina 45 acao- 99 5 335 5 366 5 335 45 5 529 m alaysia 164 164 94 405 983 432 080 5 197 m ontenegro 5 117 4 799 7 714 7 233 94 51 51 6 291 Peru 240 243 96 398 193 414 728 6 035 Qatar 10 996 10 966 10 966 100 164 157 10 996 omania 178 178 98 137 860 140 915 5 074 5 188 r r eder ussian f ation 227 227 97 1 141 317 1 172 539 6 418 6 602 5 017 4 681 64 658 93 160 152 60 366 Serbia 6 467 155 98 155 83 821 85 127 6 374 c hina Shanghai- 5 887 172 176 94 47 465 50 330 5 546 Singapore c 6 279 6 046 aipei 163 163 96 281 799 292 542 t hinese 6 606 6 681 239 240 99 695 088 702 818 t hailand 119 917 4 391 4 857 90 153 152 unisia t 108 342 u 40 384 38 228 95 460 11 460 453 rab Emirates a nited 12 148 u 180 90 39 771 5 315 5 904 180 35 800 ruguay v 162 4 966 4 959 956 517 955 222 100 iet 162 am n Information for the adjudicated regions is available on line. * See note at the beginning of this Annex. 1 http://dx.doi.org/10.1787/888932937092 2 o: Student Performan e – Volume © OECD 2014 What Student S Kno W and Can d C C e in m athemati CS , r eading and S C ien i 272

275 THE PISA TARGET POPULATION, THE PISA SAMPLES AND THE DEFINITION OF SCHOOLS: NNE A x A2 Table A2.3 shows the response rates for students and schools, before and after replacement. . This is obtained by dividing Column 2 by Column 3, shows the Column 1 • weighted participation rate of schools before replacement multiply by 100. weighted number of responding schools before school replacement shows the Column 2 (weighted by student enrolment). • (including both responding and non- weighted number of sampled schools before school replacement shows the 3 Column • responding schools, weighted by student enrolment). Column 4 shows the unweighted number of responding schools before school replacement • . • Column 5 shows the unweighted number of responding and non-responding schools before school replacement. • Column 6 shows the weighted participation rate of schools after replacement . This is obtained by dividing Column 7 by Column 8, multiply by 100. • Column 7 shows the weighted number of responding schools after school replacement (weighted by student enrolment). weighted number of schools sampled after school replacement • Column 8 shows the (including both responding and non-responding schools, weighted by student enrolment). Column 9 shows the unweighted number of responding schools after school replacement . • unweighted number of responding and non-responding schools after school replacement. • Column 10 shows the shows the Column 11 • . This is obtained by dividing Column 12 by Column 13, weighted student participation rate after replacement multiply by 100. • . weighted number of students assessed shows the Column 12 • (including both students who were assessed and students who were weighted number of students sampled shows the Column 13 absent on the day of the assessment). shows the unweighted number of students assessed. • Note that any students in schools with student-response rates less Column 14 than 50% were not included in these rates (both weighted and unweighted). • Column 15 shows the unweighted number of students sampled (including both students that were assessed and students who were absent on the day of the assessment). Note that any students in schools where fewer than half of the eligible students were assessed were not included in these rates (neither weighted nor unweighted). Definition of schools In some countries, sub-units within schools were sampled instead of schools and this may affect the estimation of the between-school variance components. In Austria, the Czech Republic, Germany, Hungary, Japan, Romania and Slovenia, schools with more than one study programme were split into the units delivering these programmes. In the Netherlands, for schools with both lower and upper secondary programmes, schools were split into units delivering each programme level. In the Flemish Community of Belgium, in the case of multi-campus schools, implantations (campuses) were sampled, whereas in the French Community, in the case of multi-campus schools, the larger administrative units were sampled. In Australia, for schools with more than one campus, the individual campuses were listed for sampling. In Argentina, Croatia and Dubai (United Arab Emirates), schools that had more than one campus had the locations listed for sampling. In Spain, the schools in the Basque region with multi-linguistic models were split into linguistic models for sampling. Grade levels Students assessed in PISA 2012 are at various grade levels. The percentage of students at each grade level is presented by country and economy in Table A2.4a and by gender within each country and economy in Table A2.4b. CS OECD 2014 S Kno W and Can d o: Student Performan C e in m athemati What Student , r eading and S C ien C e – Volume i © 273

276 Annex A2 PISA TARG T POPULATIO n , TH e PISA SAMPL e S A n D TH e D e FI n ITIO n OF SCHOOLS : TH e e Part 1/1 [ ] a p ercentage of students at each grade level t 2.4a able a ll students 8th grade 9th grade 10th grade 11th grade 12th grade and above 7th gr ade % % % S.E. % S.E. S.E. S.E. % S.E. S.E. % a ustr alia 0.0 (0.0) 0.1 (0.0) 10.8 (0.5) 70.0 (0.6) 19.1 (0.4) 0.0 (0.0) a 43.3 0.3 (0.1) 5.4 (0.7) ustria (0.9) 51.0 (1.0) 0.1 (0.0) 0.0 c OECD b (0.1) 6.4 (0.5) 30.9 (0.6) 0.9 (0.6) 1.0 (0.1) 0.0 (0.0) elgium 60.8 anada 0.1 (0.0) 1.1 (0.1) 13.2 (0.6) c (0.6) 1.0 (0.1) 0.1 (0.0) 84.6 c 1.4 (0.3) 4.1 (0.6) 21.7 (0.8) hile (1.2) 6.7 (0.3) 0.0 c 66.1 c zech r epublic 0.4 (0.1) 4.5 (0.4) 51.1 (1.2) 44.1 (1.3) 0.0 c 0.0 c d enmark (0.0) 18.2 (0.8) 80.6 (0.8) 1.0 (0.2) 0.0 c 0.0 c 0.1 (0.3) 0.6 (0.7) 75.4 (0.7) 1.9 22.1 0.0 c 0.0 c Estonia (0.2) inland 0.7 (0.2) 14.2 (0.4) 85.0 (0.4) 0.0 f 0.1 (0.1) 0.0 c c f ance 0.0 (0.0) 1.9 (0.3) 27.9 r 66.6 (0.7) 3.5 (0.3) 0.1 (0.1) (0.7) Germany 0.6 (0.1) 10.0 (0.6) 51.9 (0.8) 36.7 (0.9) 0.8 (0.4) 0.0 c Greece 0.3 1.2 (0.3) 4.0 (0.7) 94.5 (1.0) 0.0 c 0.0 c (0.1) 20.6 h (0.5) 8.7 (0.9) 67.8 (0.9) 2.8 (0.6) 0.0 c 0.0 c ungary i celand 0.0 c 0.0 c 0.0 c 100.0 c 0.0 c 0.0 c (0.8) i 0.0 (0.0) 1.9 (0.2) 60.5 reland 24.3 (1.2) 13.3 (1.0) 0.0 c 0.0 (0.3) 0.8 (0.9) 81.7 (0.9) 17.1 (0.1) 0.3 (0.0) c i srael 0.0 i taly (0.1) 1.7 (0.2) 16.8 (0.6) 78.5 (0.7) 2.6 (0.2) 0.0 (0.0) 0.4 0.0 0.0 c 0.0 c 100.0 c 0.0 c 0.0 c Japan c or k 0.0 c 0.0 c 5.9 (0.8) ea (0.8) 0.2 (0.1) 0.0 c 93.8 l 0.7 (0.1) 10.2 (0.2) 50.7 (0.1) uxembourg (0.1) 0.5 (0.1) 0.0 c 38.0 m exico 1.1 (0.1) 5.2 (0.3) 30.8 (1.0) 60.8 (1.1) 2.1 (0.3) 0.1 (0.0) n etherlands c 3.6 (0.4) 46.7 (1.0) 49.2 (1.1) 0.5 (0.1) 0.0 c 0.0 6.2 n c 0.0 c 0.1 (0.1) 0.0 (0.4) 88.3 (0.5) 5.4 (0.4) ew Zealand n orway 0.0 c 0.0 c 0.4 (0.1) 99.4 (0.1) 0.2 (0.0) 0.0 c 0.5 Poland 4.1 (0.4) 94.9 (0.4) (0.1) (0.2) 0.0 c 0.0 c 0.5 0.0 2.4 (0.3) 8.2 (0.7) 28.6 (1.6) 60.5 (2.1) 0.3 (0.1) c Portugal c r epublic 1.7 (0.3) 4.5 (0.5) 39.5 (1.5) 52.7 (1.4) 1.6 (0.5) 0.0 Slovak Slovenia c 0.3 (0.2) 5.1 (0.8) 90.7 (0.8) 3.9 (0.2) 0.0 c 0.0 (0.6) 0.1 (0.5) 24.1 (0.4) 66.0 9.8 0.0 (0.0) Spain (0.0) c 0.0 eden 0.0 (0.0) 3.7 (0.3) 94.0 (0.6) c (0.5) 0.0 c 0.0 2.2 Sw 25.6 0.6 12.9 (0.8) 60.6 Switzerland (0.1) (1.0) 0.2 (0.1) 0.0 c (1.0) t urke y 0.5 (0.2) 2.2 (0.3) 27.6 (1.2) 65.5 (1.2) 4.0 (0.3) 0.3 (0.1) u nited ingdom 0.0 c 0.0 c 0.0 (0.0) 1.3 (0.3) 95.0 (0.3) 3.6 (0.1) k 71.2 nited States c 0.3 (0.1) 11.7 (1.1) 0.0 (1.1) 16.6 (0.8) 0.2 (0.1) u o E cd average 0.5 (0.0) 4.9 (0.1) 34.7 (0.1) 51.9 (0.2) 7.7 (0.1) 0.3 (0.0) (2.4) a 0.1 (0.1) 2.2 (0.3) 39.4 lbania 58.0 (2.5) 0.3 (0.1) 0.0 c a (0.6) 1.1 (0.7) (0.5) (1.2) 22.6 (1.4) 59.4 2.0 (2.1) 12.0 2.8 rgentina (0.7) 13.5 (0.5) 6.9 c 0.0 razil b (0.2) 2.6 (1.0) 42.0 (1.0) 34.9 Partners b (0.4) 0.9 (0.5) 89.5 (0.7) 4.9 4.6 0.0 (0.0) 0.0 c ulgaria (0.2) c 40.2 5.5 12.1 (0.7) 21.5 (0.8) (0.6) (0.9) 20.7 (1.0) 0.0 c olombia c osta r ica 7.4 (0.9) 13.7 (0.9) 39.6 (1.3) 39.1 (1.8) 0.2 (0.1) 0.0 c c roatia c 0.0 c 79.8 (0.4) 0.0 (0.4) 0.0 c 0.0 c 20.2 (0.0) yprus* 0.0 (0.0) 0.5 (0.1) 4.5 (0.1) 94.3 (0.1) 0.7 (0.0) 0.0 c 25.9 c 0.0 (1.4) 1.5 (0.9) 65.0 (0.7) (0.4) 6.5 (0.1) 1.1 h ong k ong- c hina i ndonesia (0.4) 8.3 (0.8) 37.7 (2.6) 1.9 (3.0) 3.9 (0.6) 0.6 (0.6) 47.7 Jordan 0.1 (0.0) 1.1 (0.1) 6.0 (0.4) 92.9 (0.4) 0.0 c 0.0 c k (0.1) 0.2 (0.1) 4.9 (0.5) 67.2 azakhstan 27.4 (2.0) 0.2 0.1 (0.1) (1.9) l atvia 2.1 (0.4) 14.8 (0.7) 80.0 (0.8) 3.0 (0.4) 0.0 (0.0) 0.0 c l iechtenstein (0.7) 14.2 (1.5) 66.3 (1.3) 4.9 (0.2) 0.0 c 0.0 c 14.6 l 12.4 ithuania (0.1) 6.2 (0.6) 81.2 (0.7) 0.2 (0.7) 0.0 (0.0) 0.0 c m hina (0.2) 16.4 44.6 (0.1) 0.4 (0.1) 0.0 (0.0) (0.1) (0.2) 33.2 acao- c 5.4 m 4.0 (0.0) c 0.0 alaysia 0.1 c 0.0 (0.0) 0.0 (0.5) 96.0 (0.5) m ontenegro c 0.1 (0.0) 79.5 (0.1) 0.0 (0.1) 0.0 c 0.0 c 20.4 Peru 2.7 (0.4) 7.8 (0.5) 18.1 (0.7) 47.7 (0.9) 23.7 (0.8) 0.0 c (0.1) Qatar 0.9 (0.0) 3.1 (0.1) 13.8 (0.1) 64.8 17.1 (0.1) 0.3 (0.0) r (0.4) (0.1) 7.4 (0.5) 87.2 (0.6) 5.1 0.2 0.0 c 0.0 c omania r eder f c ation 0.6 (0.1) 8.1 (0.5) 73.8 (1.6) 17.4 (1.8) 0.1 (0.1) 0.0 ussian 0.0 (0.2) 1.7 (0.7) (0.7) c c 0.0 Serbia 0.1 (0.1) 1.5 96.7 Shanghai- (1.5) (0.1) 0.1 (0.1) 0.6 (1.3) 54.2 39.6 (0.6) 4.5 (0.2) 1.1 hina c Singapore (0.1) 2.0 (0.2) 8.0 (0.3) 0.4 89.6 (0.3) 0.1 (0.1) 0.0 c c c hinese t aipei 0.0 c 0.2 (0.1) 36.2 (0.7) 63.6 (0.7) 0.0 0.0 c c 0.0 (0.5) 2.9 (1.1) 76.0 (1.0) (0.1) 20.7 0.1 t hailand 0.3 (0.0) (1.4) (1.3) unisia 5.0 (0.6) 11.8 20.6 t 56.7 (2.7) 5.9 (0.5) 0.0 c 61.9 (0.2) (1.0) 22.2 (0.7) 0.9 2.8 (0.2) 11.3 (0.8) 0.9 nited a rab Emirates (0.2) u 1.3 (1.5) 0.0 57.3 (1.0) 22.4 (0.6) 12.2 (0.8) 6.9 ruguay c (0.2) u iet n c 0.0 c 0.0 (2.3) 88.6 (1.7) am 8.3 (0.7) 2.7 (0.2) 0.4 v Information for the adjudicated regions is available on line. * See note at the beginning of this Annex. 1 2 http://dx.doi.org/10.1787/888932937092 e in © C ien C eading and S r , CS athemati i m e – Volume C o: Student Performan d and Can W Kno S What Student OECD 2014 274

277 THE PISA TARGET POPULATION, THE PISA SAMPLES AND THE DEFINITION OF SCHOOLS: x A2 NNE A Part 1/2 ] [ a ercentage of students at each grade level, by gender 2.4b p t able b oys 8th grade 9th grade 10th grade 11th grade 12th grade and above 7th gr ade % % % S.E. % S.E. S.E. S.E. % S.E. S.E. % a ustr alia 0.0 c 0.1 (0.0) 13.1 (0.9) 69.2 (0.9) 17.5 (0.6) 0.0 (0.0) a 0.3 (0.1) 6.0 (0.9) 44.8 ustria 48.9 (1.5) 0.0 c 0.0 c (1.4) OECD 57.1 elgium (0.1) 7.1 (0.6) 33.8 (0.9) 1.0 (1.0) 1.0 (0.2) 0.0 (0.0) b c anada 0.1 (0.1) 1.3 (0.2) 14.8 (0.8) 82.7 (0.8) 0.9 (0.1) 0.1 (0.1) c hile (0.4) 5.0 (0.9) 24.2 (1.0) 63.1 (1.6) 6.4 (0.4) 0.0 c 1.4 (2.0) zech epublic 0.7 (0.2) 5.5 (0.6) 54.9 c 39.0 (2.1) 0.0 c 0.0 c r d enmark 0.1 (0.0) 23.4 (1.0) 75.7 (1.0) 0.8 (0.3) 0.0 c 0.0 c 1.7 Estonia 25.7 (1.0) 71.7 (1.1) (0.3) (0.4) 0.0 c 0.0 c 0.8 f inland 0.9 (0.4) 16.2 (0.6) 82.8 (0.7) 0.0 c 0.1 (0.1) 0.0 c 30.8 f ance 0.1 (0.1) 2.3 (0.4) r (0.9) 63.5 (1.0) 3.2 (0.5) 0.1 (0.1) 33.2 (1.1) 53.6 (0.7) 11.6 c 0.0 (0.3) 0.7 (1.2) Germany 0.9 (0.2) Greece 0.4 1.8 (0.6) 4.8 (1.0) 93.0 (1.4) 0.0 c 0.0 c (0.2) ungary 17.0 (0.6) 12.1 (1.5) 67.1 (1.3) h (0.8) 0.0 c 0.0 c 3.9 100.0 i c 0.0 c 0.0 c 0.0 c 0.0 c 0.0 c celand i reland 0.0 c 2.4 (0.3) 63.6 (1.0) 21.1 (1.4) 13.0 (1.3) 0.0 c 79.6 i (0.1) 0.3 (0.1) 18.9 (1.3) 0.1 (1.3) 1.2 (0.5) 0.0 c srael i taly 0.5 (0.2) 2.1 (0.3) 19.3 (0.7) 75.8 (0.7) 2.3 (0.2) 0.0 c c Japan c 0.0 c 0.0 0.0 100.0 c 0.0 0.0 c c k or ea 0.0 c 0.0 c 6.4 (1.2) 93.4 (1.2) 0.2 (0.1) 0.0 c l uxembourg (0.1) 10.7 (0.2) 51.1 (0.2) 37.0 (0.2) 0.6 (0.1) 0.0 c 0.7 57.2 m (0.2) 6.3 (0.3) 33.0 (1.1) 1.3 (1.2) 2.1 (0.5) 0.0 (0.0) exico n etherlands 0.0 c 4.4 (0.6) 49.5 (1.1) 45.7 (1.2) 0.4 (0.1) 0.0 c 7.0 n c 0.0 c 0.2 (0.1) 0.0 (0.5) 88.0 (0.7) 4.8 (0.5) ew Zealand n orway 0.0 c 0.0 c 0.6 (0.1) 99.1 (0.1) 0.3 (0.0) 0.0 c (0.6) Poland (0.2) 5.7 (0.6) 93.0 0.9 0.4 (0.2) 0.0 c 0.0 c 57.0 (2.2) 0.4 (0.2) 0.0 c 9.9 30.1 (1.7) (0.9) Portugal 2.6 (0.5) 1.5 (2.1) 51.5 (2.0) 40.1 (0.8) 5.4 (0.3) 1.5 epublic r c 0.0 (0.5) Slovak Slovenia 0.4 (0.3) 6.3 (1.0) 90.2 (1.0) 3.1 (0.4) 0.0 c 0.0 c 0.1 (0.1) 25.8 (0.6) 62.2 (0.7) (0.6) (0.1) 0.1 Spain 11.8 c 0.0 0.1 (0.1) 4.6 (0.5) 93.7 (0.8) 1.7 eden 0.0 c 0.0 c (0.6) Sw (0.1) 13.9 (0.9) 60.6 (1.7) 0.5 (2.0) 0.2 (0.1) 0.0 c Switzerland 24.7 urke y 0.3 (0.1) 2.6 (0.5) 33.2 (1.5) 60.3 (1.5) 3.2 (0.4) 0.3 (0.1) t 1.7 nited 0.0 c 0.0 c 0.0 (0.0) ingdom (0.4) 94.7 (0.4) 3.7 (0.2) k u nited States 0.0 c 0.4 (0.2) 14.6 (1.1) 69.8 (1.1) 14.9 (0.9) 0.3 (0.2) u (0.2) E average 0.6 (0.1) 5.9 (0.1) 35.6 cd 50.1 (0.2) 7.5 (0.1) 0.3 (0.1) o 53.8 a lbania 0.1 (0.1) 2.9 (0.4) 42.9 (2.7) c (2.8) 0.2 (0.1) 0.0 a rgentina (0.8) 15.0 (1.7) 25.8 (1.9) 52.6 (2.6) 3.0 (0.9) 0.8 (0.5) 2.8 36.1 b c 9.0 (0.7) 15.8 (0.8) 0.0 (1.1) 37.2 (1.0) 1.9 (0.2) razil Partners ulgaria b (0.3) 5.8 (0.7) 88.2 (1.0) 4.6 (0.4) 0.0 c 0.0 c 1.3 c olombia (0.8) 13.5 (1.0) 22.1 (1.0) 7.4 (1.4) 18.2 (1.2) 0.0 c 38.8 c osta r ica 9.3 (1.3) 16.4 (1.2) 38.5 (1.5) 35.7 (2.0) 0.0 (0.0) 0.0 c 18.0 c c 0.0 c 82.0 (0.6) 0.0 (0.6) 0.0 c 0.0 c roatia c yprus* 0.0 (0.0) 0.5 (0.1) 4.7 (0.1) 94.0 (0.2) 0.7 (0.1) 0.0 c (0.5) h k ong- c hina 1.2 (0.2) 6.9 ong 27.5 (0.7) 63.0 (1.0) 1.4 (1.3) 0.0 c i (3.7) 45.5 (3.0) 38.5 (1.1) (0.6) 0.6 (0.6) 3.1 ndonesia 2.3 (0.4) 10.0 Jordan 0.8 (0.1) 0.1 c c 0.0 (0.6) 93.4 0.0 (0.6) 5.7 (0.2) k azakhstan 5.5 (0.6) 68.4 (2.4) 25.4 (2.6) 0.2 (0.1) 0.2 (0.2) 0.3 (0.1) (0.3) atvia 18.0 (0.9) 76.4 (1.3) 2.0 (0.8) 0.0 (0.0) 0.0 c l 3.6 l 4.5 (1.2) 16.5 (2.1) 69.4 (2.2) 9.6 (0.6) 0.0 c 0.0 c iechtenstein l ithuania (0.1) 7.3 (0.6) 82.2 (0.9) 0.2 (0.8) 0.0 (0.0) 0.0 c 10.4 m 40.0 acao- 7.1 (0.2) 19.3 (0.2) 33.3 (0.2) hina (0.2) 0.2 (0.1) 0.0 (0.0) c m alaysia 0.0 c 0.1 (0.1) 5.1 (0.7) 94.7 (0.7) 0.0 c 0.0 c m c 0.0 c 0.1 (0.1) 82.0 ontenegro 17.9 (0.3) 0.0 c 0.0 (0.3) c Peru 3.1 (0.5) 9.1 (0.8) 19.5 (0.7) 46.2 (1.0) 22.1 (0.9) 0.0 16.1 (0.1) (0.0) 0.4 (0.2) (0.2) 64.6 Qatar 1.2 (0.1) 3.6 (0.1) 14.0 r omania 6.5 (0.6) 88.7 (0.7) 4.5 (0.4) (0.2) c 0.0 c 0.3 0.0 eder r 0.0 ation 0.7 (0.2) 8.9 (0.7) ussian (1.5) 16.7 (1.8) 0.1 (0.1) f c 73.7 Serbia (0.1) 1.9 (0.9) 96.7 (1.0) 0.1 (0.2) 0.0 c 0.0 c 1.4 Shanghai- c hina 1.3 (0.3) 5.3 (0.8) 41.6 (1.6) 51.2 (1.4) 0.6 (0.1) 0.0 (0.0) Singapore 0.4 (0.1) 2.0 (0.3) 8.3 (0.4) 89.3 (0.5) (0.0) 0.0 c 0.0 (1.5) 0.2 aipei 0.0 c c (0.2) 37.4 c 62.4 (1.5) 0.0 c 0.0 t hinese t hailand 0.1 (0.1) 0.4 (0.2) 22.9 (1.3) 74.1 (1.5) 2.5 (0.5) 0.0 c (1.6) t 6.3 (0.8) 14.6 (1.6) 21.9 unisia 52.3 (3.0) 4.9 (0.5) 0.0 c (0.9) 60.3 21.8 (0.1) (1.0) 0.6 (1.2) u nited a rab Emirates 1.3 (0.3) 3.1 (0.3) 12.9 (1.1) 24.0 (0.8) 13.1 (1.3) 9.4 ruguay u 0.0 (0.2) 1.2 (1.9) 52.4 c (2.2) iet n am c 0.0 c 0.0 (2.8) 85.3 0.7 10.5 (0.3) (0.8) 3.5 v Information for the adjudicated regions is available on line. * See note at the beginning of this Annex. 1 2 http://dx.doi.org/10.1787/888932937092 CS OECD 2014 S Kno W and Can d o: Student Performan C e in m athemati What Student , r eading and S C ien C e – Volume i © 275

278 Annex A2 PISA TARG T POPULATIO n , TH e PISA SAMPL e S A n D TH e D e FI n ITIO n OF SCHOOLS : TH e e Part 2/2 [ ] a able ercentage of students at each grade level, by gender 2.4b t p Girls 8th grade 10th grade 11th grade 12th grade and above 7th grade 9th grade % S.E. % S.E. % S.E. % S.E. % S.E. % S.E. a ustr alia 0.0 (0.0) 0.2 (0.1) 8.3 (0.3) 70.8 (0.6) 20.7 (0.6) 0.0 (0.0) a (1.3) 0.3 (0.1) 4.7 (0.7) 41.8 ustria 53.1 (1.4) 0.1 (0.1) 0.0 c OECD b (0.1) 5.7 (0.5) 28.0 (0.7) 0.9 (0.8) 1.0 (0.2) 0.0 c elgium 64.4 anada 0.1 (0.0) 0.9 (0.1) 11.5 (0.5) c (0.5) 1.2 (0.1) 0.0 (0.0) 86.4 c 1.3 (0.3) 3.3 (0.6) 19.3 (1.0) hile (1.2) 7.1 (0.4) 0.0 c 69.0 c zech r epublic 0.1 (0.1) 3.5 (0.5) 47.1 (2.0) 49.4 (2.1) 0.0 c 0.0 c d enmark (0.0) 13.0 (0.9) 85.6 (0.9) 1.3 (0.3) 0.0 c 0.0 c 0.1 (0.4) 0.3 (0.8) 79.0 (0.9) 2.2 18.6 0.0 c 0.0 c Estonia (0.1) inland 0.5 (0.1) 12.0 (0.4) 87.3 (0.4) 0.0 f 0.2 (0.1) 0.0 c c f ance 0.0 c 1.6 (0.3) 25.1 r 69.4 (1.1) 3.8 (0.4) 0.1 (0.1) (1.1) Germany 0.3 (0.1) 8.2 (0.6) 50.2 (1.0) 40.4 (1.1) 0.8 (0.4) 0.0 c Greece 0.3 0.5 (0.1) 3.1 (0.7) 96.1 (0.8) 0.0 c 0.0 c (0.1) 24.1 h (0.7) 5.7 (0.8) 68.4 (1.1) 1.8 (0.8) 0.0 c 0.0 c ungary i celand 0.0 c 0.0 c 0.0 c 100.0 c 0.0 c 0.0 c (1.0) i 0.1 (0.1) 1.4 (0.2) 57.3 reland 27.6 (1.4) 13.7 (1.2) 0.0 c (0.0) c 0.0 (0.1) 0.4 (1.0) 83.8 (1.0) 15.5 (0.1) 0.2 i srael 0.0 i taly (0.1) 1.2 (0.2) 14.0 (0.6) 81.5 (0.8) 3.0 (0.3) 0.0 (0.0) 0.3 0.0 0.0 c 0.0 c 100.0 c 0.0 c 0.0 c Japan c or k 0.0 c 0.0 c 5.4 (1.1) ea (1.1) 0.2 (0.1) 0.0 c 94.4 l 0.7 (0.1) 9.7 (0.2) 50.2 (0.2) uxembourg (0.2) 0.4 (0.1) 0.0 c 39.0 m exico 0.8 (0.1) 4.1 (0.3) 28.7 (1.0) 64.2 (1.1) 2.1 (0.3) 0.1 (0.1) n etherlands c 2.7 (0.4) 43.8 (1.1) 53.0 (1.1) 0.5 (0.2) 0.0 c 0.0 5.3 n c 0.0 c 0.1 (0.1) 0.0 (0.4) 88.6 (0.6) 5.9 (0.6) ew Zealand n orway 0.0 c 0.0 c 0.2 (0.1) 99.8 (0.1) 0.0 c 0.0 c 0.6 Poland 2.6 (0.3) 96.7 (0.4) (0.1) (0.2) 0.0 c 0.0 c 0.2 0.0 2.2 (0.3) 6.6 (0.7) 27.2 (1.6) 63.8 (2.2) 0.2 (0.1) c Portugal c r epublic 1.9 (0.5) 3.5 (0.5) 38.8 (1.9) 54.0 (1.9) 1.8 (0.5) 0.0 Slovak Slovenia c 0.2 (0.2) 3.8 (0.9) 91.2 (1.0) 4.7 (0.5) 0.0 c 0.0 (0.8) 0.1 (0.5) 22.3 (0.7) 69.9 7.8 0.0 (0.0) Spain (0.0) c 0.0 eden 0.0 c 2.8 (0.3) 94.4 (0.6) c (0.6) 0.0 c 0.0 2.8 Sw 26.6 0.6 11.9 (1.0) 60.7 Switzerland (0.2) (1.8) 0.2 (0.1) 0.0 c (1.7) t urke y 0.7 (0.3) 1.7 (0.3) 21.9 (1.2) 70.8 (1.1) 4.8 (0.4) 0.2 (0.1) u nited ingdom 0.0 c 0.0 c 0.0 (0.0) 1.0 (0.3) 95.4 (0.3) 3.6 (0.2) k 72.7 nited States c 0.1 (0.1) 8.8 (1.2) 0.0 (1.3) 18.3 (0.9) 0.2 (0.1) u o E cd average 0.4 (0.0) 3.9 (0.1) 33.7 (0.2) 53.8 (0.2) 7.9 (0.1) 0.3 (0.1) (2.6) a 0.1 (0.1) 1.4 (0.4) 35.7 lbania 62.5 (2.6) 0.3 (0.1) 0.0 c a (0.4) 1.4 (0.8) (0.3) (0.9) 19.7 (1.3) 65.8 1.2 (1.9) 9.1 2.7 rgentina (1.0) 33.8 (0.7) 11.5 (0.4) 5.0 c 0.0 razil b (0.2) 3.3 (1.1) 46.4 Partners b (0.5) 0.5 (0.5) 90.9 (0.7) 5.2 3.3 0.0 (0.0) 0.0 c ulgaria (0.2) c 41.4 3.9 10.8 (0.7) 21.0 (0.9) (0.6) (1.1) 22.9 (1.1) 0.0 c olombia c osta r ica 5.7 (0.8) 11.3 (0.8) 40.5 (1.3) 42.1 (1.7) 0.4 (0.2) 0.0 c c roatia c 0.0 c 77.5 (0.6) 0.0 (0.6) 0.0 c 0.0 c 22.5 (0.0) yprus* 0.0 c 0.5 (0.1) 4.2 (0.2) 94.6 (0.2) 0.7 (0.1) 0.0 c 24.2 c 0.0 (1.5) 1.6 (1.0) 67.3 (0.8) (0.6) 6.0 (0.2) 0.9 h ong k ong- c hina i ndonesia (0.4) 6.4 (0.8) 36.8 (2.9) 1.5 (3.0) 4.7 (0.8) 0.5 (0.5) 50.0 Jordan 0.0 (0.0) 1.3 (0.2) 6.3 (0.5) 92.4 (0.6) 0.0 c 0.0 c k (0.1) 0.1 (0.1) 4.4 (0.5) 65.9 azakhstan 29.3 (2.1) 0.2 0.0 c (1.9) l atvia 0.6 (0.2) 11.6 (0.8) 83.7 (1.1) 4.1 (0.7) 0.0 c 0.0 c l iechtenstein (1.3) 11.5 (1.9) 62.8 (1.9) 5.3 (0.8) 0.0 c 0.0 c 20.4 l 14.4 ithuania (0.1) 5.2 (0.6) 80.2 (0.9) 0.1 (0.8) 0.0 (0.0) 0.0 c m hina (0.3) 13.3 49.5 (0.3) 0.7 (0.2) 0.0 c (0.1) (0.2) 33.1 acao- c 3.5 m 2.9 c c 0.0 alaysia 0.0 c