12 Risks with infinite impact

Transcript

1 Global System Major Asteroid Global Artificial Future Bad Extreme Collapse Impact Pandemic Intelligence Global Governance Climate Change Global System Major Asteroid Global Artificial Future Bad Extreme Collapse Impact Pandemic Intelligence Global Governance Climate Change Global Ecological Unknown Super-volcano Synthetic Nuclear War Nanotechnology Challenges Catastrophe Consequences Biology Global System Major Asteroid Global Artificial Future Bad Extreme Ecological Unknown Super-volcano Synthetic Nuclear War Nanotechnology Collapse Impact Pandemic Intelligence Global Governance Climate Change Catastrophe Risks that threaten Consequences Biology Global System Major Asteroid Global Artificial Future Bad Extreme 12 human civilisation Collapse Impact Pandemic Intelligence Global Governance Climate Change Ecological Unknown Super-volcano Synthetic Nuclear War Nanotechnology Catastrophe Consequences Biology Ecological Unknown Super-volcano Synthetic Nuclear War Nanotechnology Catastrophe Consequences Biology Global System Major Asteroid Global Artificial Future Bad Extreme Collapse Impact Pandemic Intelligence Global Governance Climate Change Global System Major Asteroid Global Artificial Future Bad Extreme Collapse Impact Pandemic Intelligence Global Governance Climate Change Global System Major Asteroid Global Artificial Future Bad Extreme Collapse Impact Pandemic Intelligence Global Governance Climate Change Global System Major Asteroid Global Artificial Future Bad Extreme Ecological Synthetic Super-volcano Unknown Nuclear War Nanotechnology Collapse Impact Pandemic Intelligence Global Governance Climate Change Catastrophe Biology Consequences Ecological Super-volcano Synthetic Unknown Nanotechnology Nuclear War Catastrophe Biology Consequences Global System Major Asteroid Global Artificial Future Bad Extreme Ecological Synthetic Super-volcano Unknown Nanotechnology Nuclear War Collapse Impact Pandemic Intelligence Global Governance Climate Change Catastrophe Biology Consequences Ecological Super-volcano Synthetic Unknown Nanotechnology Nuclear War The case for a new risk category Catastrophe Biology Consequences Ecological Synthetic Super-volcano Unknown Nanotechnology Nuclear War Catastrophe Biology Consequences Global System Major Asteroid Global Artificial Future Bad Extreme Collapse Impact Pandemic Intelligence Global Governance Climate Change Global System Major Asteroid Global Artificial Future Bad Extreme Collapse Impact Pandemic Intelligence Global Governance Climate Change Ecological Unknown Super-volcano Synthetic Nuclear War Nanotechnology Catastrophe Consequences Biology Ecological Unknown Synthetic Super-volcano Nuclear War Nanotechnology Catastrophe Consequences Biology Global System Major Asteroid Global Artificial Future Bad Extreme Collapse Impact Pandemic Intelligence Global Governance Climate Change Ecological Synthetic Super-volcano Unknown Nuclear War Nanotechnology Catastrophe Biology Consequences

2 Published February 2015 by Global Challenges Foundation. The opinions expressed in this report are those of the authors and do not necessarily reflect those of Global Challenges Foundation. Any inaccuracies in the report remain the responsibility of the authors. The material and the geographical designations in this report do not imply the expression of any opinion whatsoever on the part of Global Challenges Foundation concerning the legal status of any country, territory, or area, or concerning the delimitation of its frontiers or boundaries. For comments and/or queries on this report, please contact the co-authors: Dennis Pamlin Executive Project Manager Global Risks Global Challenges Foundation [email protected] globalchallenges.org or Stuart Armstrong James Martin Research Fellow Future of Humanity Institute Oxford Martin School University of Oxford [email protected] fhi.ox.ac.uk

3 Patrick McSharry , important role. collaborative approach where many people The main authors of this report are Dennis Head of Smith School’s Catastrophe have provided invaluable contributions. Pamlin , Executive Project Manager, Global Risk Financing research area, provided Challenges Foundation and Dr Stuart invaluable input regarding complex The authors would therefore like to thank Armstrong , James Martin Research Fellow, systems and ways that the economic a few people in particular. First and Future of Humanity Institute, Oxford Martin system can respond to infinite impacts. László Szombatfalvy foremost , Chairman School & Faculty of Philosophy, University also played a key part as Alex Kirby of the Global Challenges Foundation, of Oxford. Dr Stuart Armstrong wrote he did so much more than proofread whose work is the basis for this report the chapter covering the twelve global the text; the report would hardly be and whose guidance on all levels has challenges, under the direction of Dennis possible to read without his help. Various been invaluable. The rest of the board Pamlin who served as project manager additional edits and changes were made of the Global Challenges Foundation and himself wrote and edited the rest of Peter Brietbart by . have also contributed in many different , Executive Director Seth Baum the report. ways, in particular, has Johan Rockström of the Global Catastrophic Risk Institute Others that must be mentioned, including provided important input regarding the and affiliate researcher at the Center for those who participated in the workshop structure and methodology. Outside the Research on Environmental Decisions, on 14 January 2014, at the Future of foundation Prof Nick Bostrom , Professor Columbia University, also played an Humanity Institute (FHI), University of & Director of the Future of Humanity important role as he helped develop Oxford and the workshop at the Munich Institute, Oxford Martin School & Faculty the methodology chapter regarding the RE office in London on 15 January 2014, of Philosophy, University of Oxford, who selection of the global challenges with and helped provide input regarding the initiated the possibility of working with potentially infinite impacts as well as economic and finance aspects, include the Future of Humanity Institute at the providing helpful input throughout the (in alphabetical order): University of Oxford, played a particularly process. The report is the result of a Dr Nick Beckstead , Research Fellow, Future of Humanity Institute, Oxford Martin School & Faculty of Philosophy, University of Oxford Kennette Benedict , Executive Director and Publisher of the Bulletin of the Atomic Scientists Oliver Bettis , Pricing Actuary, Munich RE and Fellow of the Chartered Insurance Institute and the Institute & Faculty of Actuaries Dr Eric Drexler , Academic Visitor, Future of Humanity Institute, Oxford Martin School & Faculty of Philosophy, University of Oxford Madeleine Enarsson , Transformative Catalyst, 21st Century Frontiers Dr Seán Ó hÉigeartaigh , Senior Academic Manager, Future of Humanity Institute, University of Oxford and Executive Director, Centre for the Study of Existential Risk, University of Cambridge Martin Hellman , Professor Emeritus of Electrical Engineering, Stanford University Pan Jiahua , Director of the Institute for Urban and Environmental Studies, Chinese Academy of Social Sciences (CASS); Professor of economics at CASS; Vice-President Chinese Society for Ecological Economics; Member of the National Expert Panel on Climate Change and National Foreign Policy Advisory Committee, China Aled Jones , Director of the Global Sustainability Institute (GSI) at Anglia Ruskin University Nick Mabey , Chief Executive and Founding Director of E3G (Third Generation Environmentalism) , Founder & Co-Convener, The Finance Lab Jennifer Morgan Prof Vincent Müller , James Martin Research Fellow, Future of Humanity Institute, Oxford Martin School & Faculty of Philosophy, University of Oxford Robert de Neufville , Professional Associate, Global Catastrophic Risk Institute Prof Toby Ord , James Martin Research Fellow, Future of Humanity Institute, Oxford Martin School & Faculty of Philosophy, University of Oxford Jules Peck , Founding Partner, Jericho Chambers; Trustee, New Economics Foundation Dr Anders Sandberg , James Martin Research Fellow, Future of Humanity Institute, Oxford Martin School & Faculty of Philosophy, University of Oxford Nick Silver , Director of Callund Consulting and founder and director of the Climate Bonds Initiative (CBI) Andrew Simms , Author, Fellow at the New Economics Foundation and Chief Analyst at Global Witness Andrew Snyder-Beattie , Academic Project Manager, Future of Humanity Institute, Oxford Martin School & Faculty of Philosophy, University of Oxford James Taplan , Principal Sustainability Advisor, Forum for the Future Raj Thamotheram , CEO, Preventable Surprises Nathan Wolfe , Director of Global Viral and the Lorry I. Lokey Visiting Professor in Human Biology at Stanford University , Investment Consultant at Towers Watson Liang Yin Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 1

4 Contents Contents Executive Summary 4 26 Preface 28 1. Twelve risks that threaten human civilisation 30 2. Risks with infinite impact: A new category of risks 2.1 Report structure ... 37 38 2.2 Goals ... 2.3 Global challenges and infinite impact ... 40 40 2.3.1 Definition of infinite impact ... 49 2.4 Methodology... 50 2.4.1 A scientific review of key literature ... 52 2.4.2 A review of organisations working on global challenges... 55 2.4.3 Workshops ... 55 2.5 The list of global risks... 56 2.5.1 Risks not included ... 2.6 Relationship between impact levels beyond the infinite threshold ... 58 3. Twelve Global Challenges 60 3.1 Current risks / 3.1.1 Extreme Climate Change ... 62 3.1.2 Nuclear War ... 70 3.1.3 Ecological Catastrophe ... 78 3.1.4 Global Pandemic ... 84 3.1.5 Global System Collapse ... 90 3.2 Exogenic risk / 3.2.1 Major Asteroid Impact ... 96 3.2.2 Super-volcano... 102 108 3.3 Emerging risk / 3.3.1 Synthetic Biology ... 3.3.2 Nanotechnology ... 114 3.3.3 Artificial Intelligence ... 120 3.3.4 Unknown Consequences ... 126 132 3.4 Global policy risk / 3.4.1 Future Bad Global Governance ... Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 2

5 Contents 4. Relations between global risks 138 139 4.1 General relations between global risks and their potential impacts ... 141 4.2 Specific relations between global risks ... 5. Probabilities and uncertainties – an initial overview 142 170 6. Underlying trends of key importance 171 6.1 Poverty ... 173 6.2 Population growth ... 174 6.3 Technological development... 6.4 Demographic changes... 175 176 7. Possible ways forward ... 184 Endnotes Appendix 1 – Global Challenges Bibliography 198 Appendix 2 – Workshops 206 Notes 208 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 3

6 Executive Summary Executive Summary This is the executive summary of a report about a limited number of global risks that pose a threat to human civilisation, or even possibly to all human life. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 4

7 Executive Summary With such a focus it may surprise some readers to find that the report’s essential aim is to inspire action and dialogue as well as an increased use of the methodologies used for risk assessment. The real focus is not on the almost unimaginable impacts of the risks the report outlines. Its fundamental purpose is to encourage global collaboration and to use this new category of risk as a driver for innovation. The idea that we face a number of global challenges threatening the very basis of our civilisation at the beginning of the 21st century is well accepted in the scientific community, I However, and is studied at a number of leading universities. there is still no coordinated approach to address this group of challenges and turn them into opportunities. I http://en.wikipedia.org/wiki/Global_catastrophic_risk It is only 70 years ago that Robert Oppenheimer, who History: Edward Teller, one of the led the Manhattan Project to develop the nuclear bomb, greatest physicists of his time, the LA-602 halted the project to see with his back-of-the-envelope calculations, produced results whether Teller’s calculations document were correct. The resulting that differed drastically from all that had gone before. His document, LA-602: Ignition of the Atmosphere with Nuclear calculations showed that the Bombs, concluded that Teller explosion of a nuclear bomb – a was wrong. But the sheer creation of some of the brightest minds on the planet, including complexity drove the assessors Teller himself – could result in to end their study by writing a chain reaction so powerful that “further work on the subject [is] highly desirable”. that it would ignite the world’s The LA-602 document can atmosphere, thereby ending human life on Earth. be seen as the first global challenge report addressing a category of risks where the worst possible impact in all practical senses is infinite. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 5

8 Executive Summary This report has, to the best of the authors’ knowledge, 12 Global created the first list of global risks with impacts that for all practical purposes can be called infinite. It is risks also the first structured overview of key events related to such challenges and has tried to provide initial rough quantifications for the probabilities of these impacts. In the next phase of the project, these placeholder estimates will be improved and refined by a variety of methods (expert elicitation, fault trees, simulations, etc.) appropriate to each specific risk. It is worth noting that complex The report conducts its exploration systems are often stable only within within carefully defined bounds, resulting in a list of twelve risks with certain boundaries outside which potentially infinite outcomes. the system can collapse and rapidly change to a new stable state. Such a There were many challenges which collapse can trigger a process where change continues for a long time until might have been included on the a new stable state is found. None of list because of their ability to pose the risks in this report are likely to severe damage to humanity. They result directly in an infinite impact, were excluded for one or more of three reasons: and some cannot do so physically. All the risks however are big enough to 1. Limited impact reach a threshold where the social and – tsunamis, for example, and chemical pollution. ecological systems become so unstable that an infinite impact could ensue. No effective countermeasures – 2. the report focuses on promoting This is a report about two extremes, effective interventions and so not one. It is about how a better understanding of the magnitude of ignores challenges where nothing useful can be done to prevent or the challenges can help the world mitigate the impact, as with to address the risks it faces, and nearby gamma-ray bursts. can help to create a path towards more sustainable development. It – is a scientific assessment about 3. Included in other challenges the possibility of oblivion, certainly, many challenges are already but more than that it is a call for covered by others, or are very action based on the assumption that similar to them. Population growth, humanity is able to rise to challenges for one, is significant for climate change and ecosystem and turn them into opportunities. We are confronted with possibly catastrophe, but without direct the greatest challenge ever and large-scale impacts of its own. our response needs to match this through global collaboration in new and innovative ways. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 6

9 Global System Major Asteroid Global Artificial Future Bad Extreme Executive Summary Collapse Impact Pandemic Intelligence Global Governance Climate Change Global System Major Asteroid Global Artificial Future Bad Extreme Collapse Impact Pandemic Intelligence Global Governance Climate Change Global System Ecological Major Asteroid Global System Global Artificial Future Bad Extreme Major Asteroid Synthetic Unknown Super-volcano Nuclear War Nanotechnology Global Artificial Future Bad Extreme Collapse Catastrophe Impact Collapse Pandemic Intelligence Global Governance Climate Change Impact Consequences Biology Pandemic Intelligence Global Governance Climate Change Global System Global System Global System Major Asteroid Major Asteroid Major Asteroid Global Global Global Artificial Artificial Future Bad Future Bad Future Bad Artificial Extreme Extreme Extreme Collapse Collapse Collapse Impact Impact Impact Pandemic Pandemic Pandemic Intelligence Intelligence Global Governance Global Governance Global Governance Intelligence Climate Change Climate Change Climate Change Global System Ecological Global System Major Asteroid Global Artificial Future Bad Extreme Major Asteroid Super-volcano Synthetic Unknown Nanotechnology Nuclear War Global Artificial Future Bad Extreme Collapse Catastrophe Collapse Impact Pandemic Intelligence Global Governance Climate Change Impact Biology Consequences Pandemic Intelligence Global Governance Climate Change Ecological Ecological Super-volcano Synthetic Unknown Nuclear War Nanotechnology Super-volcano Synthetic Unknown Nanotechnology Nuclear War Catastrophe Catastrophe Biology Consequences Biology Consequences Global System Major Asteroid Global Artificial Future Bad Extreme Collapse Impact Pandemic Intelligence Global Governance Climate Change Ecological Ecological Ecological Synthetic Super-volcano Unknown Super-volcano Super-volcano Synthetic Unknown Synthetic Nanotechnology Nuclear War Nanotechnology Nuclear War Nuclear War Nanotechnology Unknown Catastrophe Catastrophe Catastrophe Biology Consequences Biology Biology Consequences Consequences Ecological Ecological Super-volcano Synthetic Unknown Nuclear War Nanotechnology Synthetic Super-volcano Unknown Nanotechnology Nuclear War Catastrophe Catastrophe Biology Consequences Biology Consequences Ecological Global System Global System Major Asteroid Major Asteroid Synthetic Super-volcano Unknown Nuclear War Nanotechnology Global Global Artificial Future Bad Artificial Future Bad Extreme Extreme Catastrophe Collapse Collapse Impact Impact Biology Consequences Pandemic Pandemic Intelligence Global Governance Intelligence Global Governance Climate Change Climate Change Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 7 Ecological Ecological Super-volcano Synthetic Unknown Super-volcano Synthetic Unknown Nuclear War Nanotechnology Nuclear War Nanotechnology Catastrophe Catastrophe Biology Consequences Biology Consequences

10 Executive Summary The four main goals of this report are to acknowledge, The goals inspire, connect and deliver. of the report to connect The first of the report’s goals – The third goal is , different groups at every level acknowledging the existence of risks with potentially infinite impact – so that leaders in different sectors connect with each other to seeks to help key stakeholders to acknowledge the existence of the encourage collaboration. This will category of risks that could result need a specific focus on financial and security policy, where significant in infinite impact, and to show them risks combine to demand action that we can reduce or even eliminate beyond the incremental. most of them. The fourth goal is to deliver The second goal is actual to inspire by showing the practical action that strategies and initiatives that is taking place today produce actual results. The report . This report seeks to show that helping to meet is a first step and its success will these global challenges is perhaps ultimately be measured only on how the most important contribution it contributes to concrete results. anyone can make today, and The report will have achieved its highlights concrete examples to inspire a new generation of leaders. goals when key decision-makers recognise the magnitude of the possible risks and our ability to reduce or even eliminate most of them. The goals 1. to acknowledge the existence of risks with potentially infinite impact. 2. to inspire by showing the practical action that is taking place today. 3. to connect different groups at every level. 4. to deliver actual strategies and initiatives that produce actual results. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 8

11 Executive Summary The first part of the report introduces and defines the Report global challenges and includes the methodology for selecting them. structure The third part of the report discusses the relationship between the different The second part is an overview of the twelve challenges and key challenges, as action to address one can increase the risk of events that illustrate strategic work to address them. It also lists for each another. Many solutions can also address multiple challenges, so challenge five important factors that there are significant benefits from influence its probability or impact. understanding how they are linked. The challenges are divided into four different categories: The fourth part is an overview, the first – current challenges includes those ever to the authors’ knowledge, of the probabilities of global challenges with which currently threaten humanity because of its economic and potentially infinite impacts. technological development; The fifth part presents some of the are those exogenic challenges – most important underlying trends where the basic probability of an that influence the challenges, which event is beyond human control, but often build up slowly to a threshold where the probability and magnitude where very rapid changes can of the impact can be influenced; ensue. could both emerging challenges – The sixth part presents an overview help reduce the risks associated of possible ways forward. with current challenges and also result in infinite impacts; – the last of the twelve challenges are global policy challenges , threats arising from future global governance as it resorts to destructive policies in response to the categories of challenge listed above. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 9

12 Executive Summary The idea that there may be risks where the impact can be A new category described as infinite, defined as the end of human civilisation or even human life, is not new. However, it excites relatively of global risk little political or academic interest, and the way it is treated in popular culture makes a serious discussion more difficult. The review of literature indicates For several reasons the potentially infinite impacts of the challenges in that, under a business as usual scenario, new risks with potential this report are not as well known as infinite impact are probably they should be. One reason is the way that extreme impacts are often inseparable from the rapid masked by most of the theories and technological development in models used by governments and areas like synthetic biology, business today. nanotechnology and AI. Most risks are linked to increased Climate change is a good example, where almost all of the focus is on knowledge, economic and technical the most likely scenarios, and there development that has brought many are few public studies that include benefits. E.g. climate change is a the low-probability high-impact result from the industrial revolution scenarios. In most reports about and fossil fuel based development. climate impacts, those caused by The increased potential for global warming beyond five or six degrees pandemics is one consequence of Celsius are omitted from tables an integrated global economy where and graphs. Other aspects that goods and services move quickly contribute to this relative invisibility internationally. Similar challenges can include the fact that extreme be expected for synthetic biology, nanotechnology and AI. impacts are difficult to translate into monetary terms, as they have There are remedies, including a global scope and often require a time-horizon of a century or more. technological and institutional, They cannot be understood simply by for all risks. But they will require collaboration of a sort humanity has linear extrapolation of current trends, and they lack historical precedents. not achieved before, and the creation There is also the fact that the of systems which can deal with measures required to significantly problems pre-emptively. It is important reduce the probability of infinite to understand that much of the knowledge and many tools that we impacts will be radical compared to have, and will develop, can be a business-as-usual scenario. both a risk and a solution to risks depending on context. A scientific approach requires us to base our decisions on the whole probability distribution. x = Impact Probability Risk Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 10 New Category Normal Risks Requires new measures and tools Traditional measures and tools applicable Threshold probability impact 0

13 Executive Summary There is a clear ethical dimension to the concept of infinite Infinite impacts impact, because a very small group alive today can take decisions that will fundamentally affect all future generations. and thresholds events that could result first in a Using traditional economic tools civilisation collapse, and then later is problematic and can generate = x Probability Risk Impact result in an infinite impact. Such disagreement over issues such as discounting, which the report thresholds are especially important to recognise in a complex and examines in some detail, considering for example the role of tipping points. interconnected society where resilience is decreasing. The report distinguishes between the A collapse of civilisation is defined concepts of infinite impact – where civilisation collapses to a state of as a drastic decrease in human population size and political/ great suffering and does not recover, economic/social complexity, globally or a situation where all human life and for an extended time. ends – and infinite impact threshold – an impact that can trigger a chain of New Category Normal Risks Requires new measures and tools Traditional measures and tools applicable Threshold probability impact 0 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 11

14 Executive Summary In order to establish a list of global challenges with Methodology potentially infinite impact, a methodological triangulation was used, consisting of: – A quantitative assessment of Two workshops were arranged relevant literature. where the selection of challenges was discussed, one with risk experts – A strategic selection of relevant in Oxford at the Future of Humanity organisations and their priorities. Institute and the other in London with experts from the financial sector. – A qualitative assessment with the No challenge was excluded at the help of expert workshops. workshops, but one was added: the participants agreed to include Global System Collapse as a category. Relevant literature Identification of credible sources: search relevant literature in academic literature included in World of Knowledge and Google Scholar. 1 Estimations of impact Only literature where there is some estimation of impact that indicates the possibility of an infinite impact is included. 2 Leading organisations’ priorities In order to increase the probability of covering all relevant risks an overview of leading organisations' work was conducted. This list was then compared with the initial list and subjected to the same filter regarding the possibility to affect the probability or impact. 3 Possibility of addressing the risk From the risks gathered from literature and organisations, only those where the probability or impact can be affected by human actions are included. 4 Expert review Qualitative assessment: Expert review in order to increase the probability of covering all relevant global risks. 5 List of risks Result: List of risks with potentially infinite impacts. 6 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 12

15 Executive Summary Quick overview of each risk Current Current Current risk risk risk Global System Major Asteroid Global Artificial Future Bad Extreme Collapse Impact Pandemic Intelligence Global Governance Climate Change Global System Global System Major Asteroid Major Asteroid Global Global Artificial Artificial Future Bad Future Bad Extreme Extreme Ecological Synthetic Super-volcano Unknown Nanotechnology Nuclear War Collapse Collapse Impact Impact Pandemic Pandemic Intelligence Intelligence Global Governance Global Governance Climate Change Climate Change Catastrophe Biology Consequences Current Current Exogenic risk risk risk Ecological Ecological Unknown Super-volcano Synthetic Unknown Synthetic Super-volcano Nanotechnology Nuclear War Nanotechnology Nuclear War Catastrophe Catastrophe Consequences Biology Consequences Biology Global System Major Asteroid Global Artificial Future Bad Extreme Collapse Impact Pandemic Intelligence Global Governance Climate Change Global System Major Asteroid Global Artificial Future Bad Extreme Global System Major Asteroid Global Artificial Future Bad Extreme Ecological Synthetic Super-volcano Unknown Nanotechnology Nuclear War Collapse Impact Pandemic Intelligence Global Governance Climate Change Collapse Impact Pandemic Intelligence Global Governance Climate Change Catastrophe Biology Consequences Emerging Exogenic Emerging risk risk risk Ecological Super-volcano Synthetic Unknown Nuclear War Nanotechnology Ecological Synthetic Super-volcano Unknown Nuclear War Nanotechnology Catastrophe Biology Consequences Catastrophe Biology Consequences Global System Global System Major Asteroid Major Asteroid Global Global Artificial Artificial Future Bad Future Bad Extreme Extreme Global System Major Asteroid Global Artificial Future Bad Extreme Collapse Collapse Impact Impact Pandemic Pandemic Intelligence Intelligence Global Governance Global Governance Climate Change Climate Change Collapse Impact Pandemic Intelligence Global Governance Climate Change Ecological Ecological Super-volcano Synthetic Synthetic Super-volcano Unknown Unknown Nuclear War Nanotechnology Nanotechnology Nuclear War Ecological Synthetic Super-volcano Unknown Nuclear War Nanotechnology Catastrophe Catastrophe Biology Biology Consequences Consequences Catastrophe Consequences Biology Emerging Global Policy Emerging risk risk risk Global System Major Asteroid Global Artificial Future Bad Extreme Collapse Impact Pandemic Intelligence Global Governance Climate Change Ecological Unknown Super-volcano Synthetic Nanotechnology Nuclear War Global System Major Asteroid Global Artificial Future Bad Extreme Global System Major Asteroid Global Artificial Future Bad Extreme Catastrophe Consequences Biology Collapse Impact Pandemic Intelligence Global Governance Climate Change Collapse Impact Pandemic Intelligence Global Governance Climate Change Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 13 Ecological Super-volcano Synthetic Unknown Nuclear War Nanotechnology Ecological Super-volcano Synthetic Unknown Nanotechnology Nuclear War Catastrophe Biology Consequences Catastrophe Biology Consequences

16 Executive Summary Current risk Extreme 5 key Climate Change factors: Extreme Mass deaths and famines, social As for all risks there are uncertainties The uncertainties in 1 Climate Change in the estimates, and warming could collapse and mass migration are climate sensitivity models, including the tail. be much more extreme than the certainly possible in this scenario. middle estimates suggest. Feedback Combined with shocks to the The likelihood - or not - of 2 loops could mean global average agriculture and biosphere-dependent global coordination on temperatures increase by 4°C or industries of the more developed controlling emissions. countries, this could lead to global even 6°C over pre-industrial levels. The future uptake of 3 Feedbacks could be the release conflict and possibly civilisation low carbon economies, collapse. Further evidence of the of methane from permafrost or the including energy, mobility dieback of the Amazon rainforest. risk comes from signs that past and food systems. The impact of global warming would civilisation collapses have been Whether technological 4 be strongest in poorer countries, driven by climate change. innovations will improve which could become completely or worsen the situation, uninhabitable for the highest range and by how much. of warming. The long-term climate 5 impact caused by global warming. Current risk 5 key Nuclear War factors: Nuclear War 1 The likelihood of a full-scale nuclear How relations between plunge temperatures below freezing war between the USA and Russia around the globe and possibly also current and future has probably decreased. Still, the nuclear powers develop. destroy most of the ozone layer. potential for deliberate or accidental The detonations would need to The probability of 2 nuclear conflict has not been start firestorms in the targeted accidental war. removed, with some estimates cities, which could lift the soot up Whether disarmament 3 putting the risk in the next century into the stratosphere. The risks are efforts will succeed in or so at around 10%. A larger impact severe and recent models have reducing the number of would depend on whether or not the confirmed the earlier analysis. The nuclear warheads. war triggered what is often called a disintegration of the global food The likelihood of a 4 nuclear winter or something similar – supply would make mass starvation nuclear winter. the creation of a pall of smoke high and state collapse likely. The long-term effects 5 in the stratosphere that would of a nuclear war on climate, infrastructure and technology. A new category of global risk. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 14

17 Executive Summary Current risk Global 5 key Pandemic factors: Global What the true probability (and influenza has demonstrated An epidemic of infectious disease 1 Pandemic that has spread through human distribution for pandemics antigenic shift, the ability to combine is, especially at the tail. features from different viruses), its populations across a large region or death toll would be extreme. even worldwide. There are grounds The capacity of international 2 The world has changed considerably, for suspecting that such a high- health systems to deal making comparisons with the past impact epidemic is more probable with an extreme pandemic. problematic.Today it has better than usually assumed. All the How fast medical research 3 features of an extremely devastating sanitation and medical research, as can proceed in an disease already exist in nature: well as national and supra-national emergency. institutions dedicated to combating essentially incurable (Ebola), nearly How mobility of goods and 4 diseases. But modern transport always fatal (rabies), extremely people, as well as population infectious (common cold), and and dense human population allow density, will affect pandemic long incubation periods (HIV). If infections to spread much more transmission. a pathogen were to emerge that rapidly, and slums can be breeding Whether humans can 5 somehow combined these features grounds for disease. develop novel and effective anti-pandemic solutions. Current risk Ecological 5 key Collapse factors: Ecological 1 This is where an ecosystem suffers The extent to which the damage and (unlike previous, Collapse a drastic, possibly permanent, humans are dependent on localised collapses) the whole world is potentially at risk. reduction in carrying capacity for the ecosystem. all organisms, often resulting in It seems plausible that some human Whether there will be 2 lifestyles could be sustained in a mass extinction. Humans are part effective political measures relatively ecosystem independent of the global ecosystem and so taken to protect the fundamentally depend on it. Species way, at relatively low costs. Whether ecosystem on a large scale. this can be achieved on a large extinction is now far faster than the 3 The likelihood of the historic rate, and attempts to scale in practice, especially during emergence of sustainable a collapse, will be a technological quantify a safe ecological operating economies. space place humanity well outside it. challenge and whether it is something The positive and negative 4 we want is an ethical question. Many of the problems of ecological impacts on the ecosystems degradation interact to multiply of both wealth and poverty. The long-term effects of 5 an ecological collapse on ecosystems. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 15

18 Executive Summary Current risk Global System 5 key Collapse factors: Global System An economic or societal collapse Whether global system structure of the network – even if 1 Collapse on the global scale. The term has collapse will trigger each component of the network is reliable. This gives rise to systemic subsequent collapses or been used to describe a broad risk, when parts that individually may range of conditions. Often economic fragility in other areas. collapse is accompanied by social function well become vulnerable What the true trade-off is 2 when connected as a system to a chaos, civil unrest and sometimes a between efficiency self-reinforcing joint risk that can breakdown of law and order. Societal and resilience. spread from part to part, potentially collapse usually refers to the fall or Whether effective 3 affecting the entire system and disintegration of human societies, regulation and resilience possibly spilling over to related often along with their life support can be developed. systems. The world economic and outside systems. Such effects have Whether an external 4 political system is made up of been observed in ecology, finance disruption will trigger and critical infrastructure such many actors with many objectives a collapse. and many links between them. as power grids. The possibility of Whether an internal event 5 Such intricate, interconnected collapse becomes more acute when will trigger a collapse. several independent networks systems are subject to unexpected system-wide failures caused by the depend on each other. Exogenic risk Major Asteroid 5 key Impact factors: Major Asteroid Large asteroid collisions – with There has been some discussion Whether detection and 1 Impact about possible methods for tracking of asteroids and objects 5 km or more in size – deflecting asteroids found on a happen about once every twenty other dangerous space objects is sufficiently collision course with the planet. million years and would have an exhaustive. Should an impact occur the main energy a hundred thousand times greater than the largest bomb ever destruction will not be from the How feasible it is to deflect 2 detonated. A land impact would initial impact, but from the clouds an asteroid. destroy an area the size of a nation of dust projected into the upper Whether measures such as 3 atmosphere. The damage from such like Holland. Larger asteroids could evacuation could reduce be extinction-level events. Asteroid an “impact winter” could affect the damage of an impact. impacts are probably one of the best the climate, damage the biosphere, The short- and long-term 4 affect food supplies, and create understood of all risks in this report. climate consequences of a political instability. collision. Whether our current 5 civilisation could adapt to a post-impact world. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 16

19 Executive Summary Exogenic risk 5 key Super-volcano factors: Super-volcano Whether countries will while the Toba eruption around Any volcano capable of producing 1 coordinate globally 70,000 years ago is thought by some an eruption with an ejecta volume 3 . This is against super-volcano risk to have cooled global temperatures greater than 1,000 km for over two centuries. thousands of times larger than and damage. The effect of these eruptions could normal eruptions. The danger from The predictability of super- 2 be best compared with that of a super-volcanoes is the amount of volcanic eruptions. aerosols and dust projected into the nuclear war. The eruption would How directly destructive 3 upper atmosphere. This dust would be more violent than the nuclear an eruption would be. explosions, but would be less likely absorb the Sun’s rays and cause The effectiveness of 4 to ignite firestorms and other a global volcanic winter. The Mt general mitigation efforts. secondary effects. Pinatubo eruption of 1991 caused How severe the long-term 5 an average global cooling of surface climate effects would be. temperatures by 0.5°C over three years, Emerging risk Synthetic 5 key Biology factors: Synthetic The true destructive potential 1 The design and construction of This could emerge through military Biology of synthetic biology, especially biological devices and systems or commercial bio-warfare, bio- the tail risk. terrorism (possibly using dual-use for useful purposes, but adding human intentionality to traditional products developed by legitimate 2 Whether the field will be pandemic risks. Attempts at researchers, and currently successfully regulated, or unprotected by international legal regulation or self-regulation are successfully manage to regimes), or dangerous pathogens currently in their infancy, and may regulate itself. leaked from a lab. Of relevance is not develop as fast as research 3 Whether the field will usher whether synthetic biology products does. One of the most damaging in a new era of bio-warfare. impacts from synthetic biology become integrated into the global Whether the tools of synthetic 4 would come from an engineered economy or biosphere. This could biology can be used pathogen targeting humans or a lead to additional vulnerabilities (a defensively to create effective benign but widespread synthetic crucial component of the ecosystem. counter measures. biology product could be specifically The dangers of relying 5 targeted as an entry point through on synthetic biologists to which to cause damage). estimate the danger of synthetic biology. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 17

20 Executive Summary Emerging risk 5 key Nanotechnology factors: Nanotechnology Of particular relevance is whether The timeline for nanotech Atomically precise manufacturing, 1 nanotechnology allows the development. the creation of effective, high- construction of nuclear bombs. throughput manufacturing processes Which aspects of 2 But many of the world’s current that operate at the atomic or nanotech research will problems may be solvable with the molecular level. It could create progress in what order. manufacturing possibilities that new products – such as smart or Whether small groups can 3 nanotechnology would offer, such extremely resilient materials – and assemble a weapons as depletion of natural resources, would allow many different groups arsenal quickly. pollution, climate change, clean or even individuals to manufacture a Whether nanotech tools 4 water and even poverty. Some have wide range of things. This could lead can be used defensively conjectured special self-replicating to the easy construction of large or for surveillance. nanomachines which would be arsenals of conventional or more Whether nanotech tools or engineered to consume the entire 5 novel weapons made possible by weaponry are made to be environment. The misuse of medical atomically precise manufacturing. outside human control. nanotechnology is another risk scenario. Emerging risk And if these motivations do not Artificial detail the survival and value of 5 key Intelligence humanity, the intelligence will be driven to construct a world without factors: humans. This makes extremely Artificial intelligent AIs a unique risk, in that The reliability of AI 1 AI is the intelligence exhibited by Intelligence predictions. extinction is more likely than lesser machines or software, and the impacts. On a more positive note, branch of computer science that 2 Whether there will be a an intelligence of such power could single dominant AI or a develops machines and software plethora of entities. easily combat most other risks with human-level intelligence. in this report, making extremely The field is often defined as “the 3 How intelligent AIs will intelligent AI into a tool of great become. study and design of intelligent potential. There is also the possibility agents”, systems that perceive their 4 Whether extremely of AI-enabled warfare and all the environment and act to maximise intelligent AIs can be risks of the technologies that AIs controlled, and if so, how. their chances of success. Such would make possible. An interesting extreme intelligences could not 5 Whether whole brain version of this scenario is the easily be controlled (either by the emulations (human minds possible creation of “whole brain in computer form) will groups creating them, or by some arrive before true AIs. emulations”: human brains scanned international regulatory regime), and physically represented in a and would probably act to boost machine. This would make the AIs their own intelligence and acquire into properly human minds, possibly maximal resources for almost all alleviating a lot of problems. initial AI motivations. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 18

21 Executive Summary Emerging risk Unknown 5 key Consequences factors: Unknown generic probability of intelligent life These represent the unknown Whether there will be 1 Consequences (self-)destruction, which includes unknowns in the family of global extensive research into uncertain risks. Anthropic reasoning catastrophic challenges. They unknown risks and their constitute an amalgamation of all the can also bound the total risk of human probabilities. extinction, and hence estimate the risks that can appear extremely The capacity to develop 2 unknown component. Non risk- unlikely in isolation, but can combine methods for limiting specific resilience and post-disaster to represent a not insignificant the combined probability proportion of the risk exposure. One rebuilding efforts will also reduce of all uncertain risks. resolution to the Fermi paradox – the damage from uncertain risks, The capacity for estimating 3 as would appropriate national and the apparent absence of alien life “out of-model” risks. in the galaxy – is that intelligent life international regulatory regimes. The culture of risk. 4 Most of these methods would also destroys itself before beginning to assessment in potentially help with the more conventional, expand into the galaxy. Results that risky areas. increase or decrease the probability known risks, which badly need Whether general, non- 5 more investment. of this explanation modify the risk-specific mitigation or resilience measures are implemented. Global Policy risk Future Bad 5 key Global Governance factors: Future Bad How the severity of non- 1 Two issues with governance There are two main divisions in Global Governance deadly policy failures disasters are first, the difficulty governance disasters: failing to can be compared with of estimating their probability, solve major solvable problems, and potential casualties. actively causing worse outcomes. and second, the dependence of the impact of these disasters on An example of the first would be Whether poor governance 2 failing to alleviate absolute poverty; subjective comparative evaluations: will result in a collapse of the second, constructing a it is not impartially obvious how to of the world system. global totalitarian state. Technology, rank continued poverty and global How mass surveillance 3 totalitarianism against billions of political and social change may and other technological enable the construction of new casualties or civilisation collapse. innovations will affect forms of governance, which may be governance. either much better or much worse. Whether there will be new 4 systems of governance in the future. Whether a world 5 dictatorship may end up being constructed. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 19

22 Executive Summary Two things make the understanding of the relation Relations between between the global risks particularly important. global risks The risks are 2. Specific measures to address a 1. Impacts: interconnected in different ways. risk: Global risks often require Often the situation resembles a significant changes, which will set of dominoes: if one falls, many result in situations where measures to reduce the risk in one area affect follow. Even small impacts can the probability and/or the impact in start a process where different risks interact. other areas, for better or worse. ALL RISKS first risk worsens second risk solving first risk improves second risk both of the above Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 20

23 Executive Summary In order to better understand the relations between The technical different global risks, work could start to analyse similarities and differences. difficulty of reducing the risk Below is an example of an overview of how different global risks can be and the difficulty plotted depending on the technical difficulty of reducing the risk and the of collaboration difficulty of collaborating to reduce it. technical difficulty of reducing risk collaboration difficulty of reducing risk Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 21

24 Executive Summary As the different challenges are very different and the Uncertainties status of probability estimates varies significantly, the initial probability numbers are provided together with estimates regarding: 1. Understanding 2. Data 3. Existing probability 1. Understanding 2. Data 1. Understanding 1. Understanding 2. Data 2. Data 3. Existing probability of sequence availability estimation of sequence availability of sequence of sequence availability availability estimation all parts all data all data all parts all parts all data all parts all data all data all parts calculations with small uncertainty small uncertainty calculations with small uncertainty calculations with most data most parts most parts most data most parts most data most parts most data most parts most data calculations with large uncertainty large uncertainty calculations with calculations with large uncertainty some data some parts some parts some data best guesses by experts some parts some data best guesses by experts some parts some parts some data some data best guesses by experts no estimates no data none at all no data none at all amount of data to make probability kind of estimation and uncertainty degree of events from today’s actions amount of data to make probability degree of events from today’s actions no estimates none at all no data assessment on all relevant steps to infinite impact assessment on all relevant steps to infinite impact of the sequence of the sequence no estimates none at all no data none at all no data amount of data to make probability degree of events from today’s actions kind of estimation and uncertainty assessment on all relevant steps to infinite impact of the sequence amount of data to make probability amount of data to make probability kind of estimation and uncertainty degree of events from today’s actions degree of events from today’s actions assessment on all relevant steps assessment on all relevant steps to infinite impact to infinite impact of the sequence of the sequence Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 22 n / a n / a 0.00003% 0.01% 0-10% 0.005% 0.00013% 0.0001% 0.1% n / a infinite impact % n / a 0-10% 5% 1% 0.5% 0.01% 0.002% n / a 0.8% infinite treshold % 0.0000001% 0.00001% 100% 0.0001% 0.001% 0.01% 0.1% 1% 10% one in one one one one in one in one one one hundred in ten in one in a hundred ten in a in a in ten million million million thousand thousand thousand hundred

25 Executive Summary These estimates are an attempt to assemble existing Probability estimates in order to encourage efforts to improve the numbers. They express estimates of probabilities over 100 years, except in the case of extreme climate change, where the time frame is 200 years. Global challenges need to be seen in Population growth – the UN’s estimates range from 6.8 billion people by 2100 the light of trends which help to shape the wider society. These include: to a high-variant projection of 16.6 bn (which would require the resources of Poverty 10 Earth-like planets to provide everyone – although it has fallen, with a modern Western lifestyle). it could increase again. This is Other trends include technological especially relevant to climate change development and demographic changes. and pandemics. n / a n / a 0.00003% 0.005% 0.0001% 0.00013% 0.01% 0-10% 0.1% n / a Probability of infinite impact (%) n / a 0.01% 0.002% 0.5% 5% 0-10% 1% 0.8% n / a Probability of reaching or exceeding the infinite threshold (%) 10% 0.1% 0.00001% 1% 0.0000001% 0.0001% 0.01% 100% 0.001% one one one in a one in a one one in one one one in a in ten hundred ten in one in a in a hundred in ten thousand million hundred million million thousand thousand Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 23 0.01% 0.01% 0.001% 0.1% one in ten one in hundred one in thousand thousand thousand 0.0001% 1% n /a n /a 0.8% one in a one in million hundred 0.00001% 10% one in ten one in million ten infinite impact % 0.0000001% 100% infinite threshold % one in hundred one in one 0.01% million 0.01% 0.01% 0.00013% 0.002% 0.1% 0.01% 0.0001% 0.00003% 5% 5% 5% 0.5% 0.01% 0.005% n /a n /a n /a 5% 1%

26 Executive Summary There are ten areas that could help mitigate immediate Possible threats while also contributing to a future global governance system capable of addressing global ways forward risks with a potential infinite impact: Global challenges leadership networks 1. Better quality risk assessment for 2. global challenges Development of early warning systems 3. Encouraging visualisation of 4. complex systems Highlighting early movers 5. Including the whole 6. probability distribution Increasing the focus on 7. the probability of extreme events Encouraging 8. appropriate language to describe extreme risks Establishing 9. a Global Risk and Opportunity Indicator to guide governance Explore the possibility of establishing a 10. Global Risk Organisation (GRO) Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 24

27 Executive Summary Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 25

28 Preface Preface Over the last century the world has changed in ways that humanity has never experienced within our history. The changes are being caused by the extremely rapid development of science and technology, by the population explosion that has quadrupled the number of people on Earth, and by a greatly improved but very resource-demanding standard of living in developed countries. The consequences of these changes are very diverse: – Less poverty, better health and longer life in many countries. – Globalisation, the most important effect of which is the emergence of a shattered global community where all people’s behaviour affects each other’s vital interests. – New global risks of previously unseen scope. This means that we are now forced to live with the risk of various kinds of extreme disaster with the potential of severely affecting billions of people. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 26

29 Preface In this Yearbook from the Global of global catastrophes are we decision-making system be Challenges Foundation, “risk” is created - with or without a global prepared to accept? This question defined as the potential damage legal system? has not yet appeared on the that can be caused by an extreme political agenda. The reason is that disaster multiplied by the probability both scientific reports and the media We are also convinced that choose to focus on the most likely knowledge of these risks is not only that it will occur. outcome of these risks. a prerequisite for reducing them, but also a responsibility which we owe For the risk of exceptional damage, In the absence of risk analysis both the probability of occurrence to our children, grandchildren and to is usually small, or very small, all future generations. It is up to us decision-makers and the public remain blissfully unaware that the probabilities compared with other risks in society, to decide whether these threats can possibly be reduced or not! These but the effects can be absolutely of certain global catastrophes are significantly higher than we would efforts do not only demand sacrifices dire, meaning they must be taken on our part. They also create accept in our everyday lives, where very seriously. opportunities for everyone to make a incomparably smaller values are at stake. significant contribution to improving Another, very important reason for We do not know what the exact nature of what these risks are or how not acting against acknowledged the future of humanity: they may strike. Some are obvious, global risks is that they require global responses and therefore global – For world leaders this means others may sound like pure science fiction, but they have led many assuming their responsibility and decisions. starting to work towards common, scientists to regard them as real threats - and therefore it is best to Regrettably there is no global global decision-making. – Scientists need to focus their decision-making body capable of include them in the calculations. research on areas that will help us take that, no globally functioning legal With few exceptions, humans have system, and so there is a lack of effective measures against the risks. – Companies should make created these risks. There are only effective tools for dealing with these sustainability a business model. a few risks where we are not the challenges. The result: the risks are increased in the absence of effective cause, for example natural disasters – And there is a special opportunity measures to counter them. such as an asteroid impact. for all of us - that when choosing our politicians and suppliers (of goods and services), we should We could eliminate some of these This report wants, on a strictly consider their ambition to eliminate scientific basis, to identify and risks (e.g. nuclear war). In other or at least minimise global risks cases, all we can do is minimise the describe the global risks of extreme disasters, and also to report the latest likelihood of damage, since we have and to create an efficient decision- already crossed the threshold that developments affecting these risks making system that can manage and measures to face up to them. can lead to serious consequences these risks. (with climate change, for example, The Global Challenges Foundation’s Finally, I would on behalf of the where we have already emitted such goal in this report is to accelerate Global Challenges Foundation extend high levels of greenhouse gases that there are small but not insignificant my sincere gratitude to both Dennis effective counter-actions against likelihoods of significant damage). Pamlin, editor of the report, and to all global events with the potential for For other risks we cannot affect large-scale unwanted effects by the scientists and other experts who have contributed their research and / deepening both decision makers’ and the likelihood of them occurring, or valuable comments. only minimise damage (with super- the public’s insights into the risks, and volcanic eruptions, for instance). also to inspire both debate and well- However, here we can build social judged decisions on these questions: and ecological resilience so as to reduce the damage. – What probabilities of extreme disasters are acceptable? Laszlo Szombatfalvy For decisions concerning – Which are the optimal countermeasures the first important Founder and Chairman, countermeasures? question is: What level of probability – How can an effective global The Global Challenges Foundation Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 27

30 1. Twelve risks threaten human civilisation 1. Twelve risks that threaten human civilisation “Tell me, and I’ll forget. Show me, and I may remember. Involve me, and I’ll understand.” Xunzi Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 28

31 1. Twelve risks threaten human civilisation Current Current Current risk risk risk Global System Major Asteroid Global Artificial Future Bad Extreme Collapse Impact Pandemic Intelligence Global Governance Climate Change Global System Global System Major Asteroid Major Asteroid Global Global Artificial Artificial Future Bad Future Bad Extreme Extreme Ecological Super-volcano Unknown Synthetic Nuclear War Nanotechnology Collapse Collapse Impact Impact Pandemic Pandemic Intelligence Intelligence Global Governance Global Governance Climate Change Climate Change Catastrophe Biology Consequences Current Current Exogenic risk risk risk Ecological Ecological Super-volcano Unknown Synthetic Unknown Super-volcano Synthetic Nanotechnology Nuclear War Nuclear War Nanotechnology Catastrophe Catastrophe Consequences Biology Consequences Biology Global System Major Asteroid Global Artificial Future Bad Extreme Collapse Impact Pandemic Intelligence Global Governance Climate Change Global System Major Asteroid Global Artificial Future Bad Extreme Global System Major Asteroid Global Artificial Future Bad Extreme Ecological Super-volcano Synthetic Unknown Nanotechnology Nuclear War Collapse Impact Pandemic Intelligence Global Governance Climate Change Collapse Impact Pandemic Intelligence Global Governance Climate Change Catastrophe Biology Consequences Emerging Exogenic Emerging risk risk risk Ecological Super-volcano Synthetic Unknown Nuclear War Nanotechnology Ecological Super-volcano Synthetic Unknown Nanotechnology Nuclear War Catastrophe Biology Consequences Catastrophe Biology Consequences Global System Global System Major Asteroid Major Asteroid Global Global Artificial Artificial Future Bad Future Bad Extreme Extreme Global System Major Asteroid Global Artificial Future Bad Extreme Collapse Collapse Impact Impact Pandemic Pandemic Intelligence Intelligence Global Governance Global Governance Climate Change Climate Change Collapse Impact Pandemic Intelligence Global Governance Climate Change Ecological Ecological Super-volcano Unknown Super-volcano Synthetic Unknown Synthetic Nuclear War Nanotechnology Nanotechnology Nuclear War Ecological Unknown Synthetic Super-volcano Nanotechnology Nuclear War Catastrophe Catastrophe Biology Consequences Biology Consequences Catastrophe Consequences Biology Emerging Emerging Global Policy risk risk risk Global System Major Asteroid Global Artificial Future Bad Extreme Collapse Impact Pandemic Intelligence Global Governance Climate Change Ecological Unknown Super-volcano Synthetic Nanotechnology Nuclear War Global System Major Asteroid Global Artificial Future Bad Extreme Global System Major Asteroid Global Artificial Future Bad Extreme Catastrophe Consequences Biology Collapse Impact Pandemic Intelligence Global Governance Climate Change Collapse Impact Pandemic Intelligence Global Governance Climate Change Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 29 Ecological Super-volcano Synthetic Unknown Nuclear War Nanotechnology Ecological Synthetic Super-volcano Unknown Nanotechnology Nuclear War Catastrophe Biology Consequences Catastrophe Biology Consequences

32 2. Risks with infinite impact: A new category of risks 2. Risks with infinite impact: A new category of risks “Most risk management is really just advanced contingency planning and disciplining yourself to realise that, given enough time, very low probability events not only can happen, but they absolutely will happen.” Lloyd Blankfein, Goldman Sachs CEO, 1 July 2013 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 30

33 2. Risks with infinite impact: A new category of risks What is risk? A new group of global risks = × Risk is the potential of losing something This is a report about a limited Risk Impact Probability of value, weighed against the potential number of global risks – that can be identified through a scientific to gain something of value. Every Impacts where civilisation and transparent process – with day we make different kinds of risk collapses to a state of great impacts of a magnitude that pose assessments, in more or less rational suffering and do not recover, ways, when we weigh different options a threat to human civilisation, or or a situation where all human even possibly to all human life. against each other. life end, are defined as infinite as the result is irreversible and The basic idea of risk is that With such a focus it may surprise lasts forever. some readers to find that the report’s an uncertainty exists regarding essential aim is to inspire action and the outcome and that we must dialogue as well as an increased find a way to take the best The slightly tilted bell curve is a use of the methodologies used for possible decision based on our common probability distribution, 3 understanding of this uncertainty. risk assessment. but the shape differs and in reality is seldom as smooth as the example. To calculate risk the probability of an The real focus is not on the almost outcome is often multiplied by the unimaginable impacts of the risks The total area under the curve impact. The impact is in most cases the report outlines. Its fundamental always represents 100 percent, i.e. measured in economic terms, but it purpose is to encourage global all the possible outcomes fit under can also be measured in anything collaboration and to use this new the curve. In this case (A) represents we want to avoid, such as suffering. category of risk as a driver the most probable impact. With a for innovation. much lower probability it will be a At the heart of a risk assessment close to zero impact, illustrated by is a probability distribution, often The idea that we face a number (B). In the same way as in case B described by a probability density of global challenges threatening there is also a low probability that 4 function ; see figure X for a the very basis of our civilisation at the situation will be very significant, graphic illustration. the beginning of the 21st century illustrated by (C). is well accepted in the scientific community, and is studied at a 2 number of leading universities. But there is still no coordinated approach to address this group of challenges and turn them into opportunities for a new generation of global cooperation and the creation of a global governance system capable of addressing the greatest challenges of our time. A This report has, to the best of our knowledge, created the first science- based list of global risks with a potentially infinite impact and has made the first attempt to provide an initial overview of the uncertainties C related to these risks as well D as rough quantifications for the B probability probabilities of these impacts. impact 0 Figure 1:Probability density function Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 31

34 2. Risks with infinite impact: A new category of risks On a probability curve the impacts in Not only could a better The impacts (A), (B) and (C) all belong to the same category, normal understanding of the unique this report are usually at the very far impacts: the impacts may be more magnitude of these risks help right with a relatively low probability address the risks we face, it could compared with other impacts, or less serious, but they can be dealt illustrated by (D) in Figure 2. also help to create a path towards with within the current system. more sustainable development. Often they are so far out on the tail The impacts in this report are of the curve that they are not even The group of global risks discussed however of a special kind. These included in studies. in this report are so different from are impacts where everything will be lost and the situation will not most of the challenges we face that be reversible, i.e challenges with they are hard to comprehend. For each risk in this report the potentially infinite impact. probability of an infinite impact is very low compared to the most But that is also why they can help us In insurance and finance this kind of to build the collaboration we need likely outcome. Some studies even indicate that not all risks in this report risk is called “risk of ruin”, an impact and drive the development of further 5 where all capital is lost. can result in an infinite impact. But a This impact solutions that benefit both people is however only infinite for the significant number of peer-reviewed and the planet. reports indicate that those impacts company that is losing the money. not only can happen, but that their As noted above, none of the risks in From society’s perspective, that is this report is likely to result directly not a special category of risk. probability is increasing due to in an infinite impact, and some are unsustainable trends. In this report the focus is on the “risk probably even physically incapable of ruin” on a global scale and on a of doing so. But all are so significant The assumption for this report is that by creating a better understanding human level, in the worst case this that they could reach a threshold is when we risk the extinction of our impact able to create social and of our scientific knowledge regarding own species. risks with a potentially infinite impact, ecological instability that could we can inspire initiatives that can turn trigger a process which could lead to these risks into drivers for innovation. an infinite impact. For several reasons the potentially infinite impacts of the risks in this Figure 2: Probability density function with tail highlighted report are not as well known as they should be. One reason is the way that extreme impacts are often masked by most of the theories and models used by governments and business today. For example, the probability of extreme impacts is often below what A is included in studies and strategies. The tendency to exclude impacts below a probability of five percent is one reason for the relative “invisibility” of infinite impacts. The almost standard use of a 95% C confidence interval is one reason D why low-probability high-impact B 6 probability events are often ignored. impact 0 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 32

35 2. Risks with infinite impact: A new category of risks Climate change is a good example, It should also be stressed that There is also the fact that the where almost all of the focus is on measures required to significantly uncertainty is not a weakness in the most likely scenarios and there reduce the probability of infinite science; it always exists in scientific are few studies that include the low- impacts will be radical compared to work. It is a systematic way of probability high-impact scenarios. In understanding the limitations of the a business-as-usual scenario with a 9 most reports about climate impacts, focus on incremental changes. methodology, data, etc. Uncertainty the impacts caused by warming is not a reason to wait to take action beyond five or six degrees Celsius if the impacts are serious. Increased The exact probability of a specific are even omitted from tables and impact is difficult or impossible to uncertainty is something that risk 8 graphs even though the IPCC’s However, the important estimate. experts, e.g. insurance experts and own research indicates that the security policy experts, interpret as a thing is to establish the current probability of these impacts are magnitude of the probabilities and signal for action. often between one and five percent, compare them with the probabilities 7 and sometimes even higher. A contrasting challenge is that our for such impacts we cannot accept. cultural references to the threat of A failure to provide any estimate for Other aspects that contribute to this infinite impacts have been dominated these riks often results in strategies and priorities defined as though the relative invisibility include the fact throughout history by religious groups probability of a totally unacceptable that extreme impacts are difficult to seeking to scare society without any translate into monetary terms, they scientific backing, often as a way outcome is zero. An approximate have a global scope, and they often to discipline people and implement number for a best estimate also unpopular measures. It should not require a time-horizon of a century makes it easier to understand that or more. They cannot be understood have to be said, but this report is a great uncertainty means the obviously fundamentally different as actual probability can be both simply by linear extrapolation it focuses on scientific evidence from much higher and much lower than of current trends, and they lack peer-reviewed sources. the best estimate. historical precedents. Infinite impact The concept infinite impact refers to two aspects in particular; the terminology is not meant to imply a literally infinite impact (with all the mathematical subtleties that would imply) but to serve as a reminder that these risks are of a different nature. Ethical These are impacts that threaten the very survival of humanity and life on Earth – and therefore can be seen as being infinitely negative from an ethical perspective. No positive gain can outweigh even a small probability for an infinite negative impact. Such risks require society to ensure that we eliminate these risks by reducing the impact below an infinite impact as a top priority, or at least do everything we can to reduce the probability of these risks. As some of these risks are impossible to eliminate today it is also important to discuss what probability can right now be accepted for risks with a possible infinite impact. Economic Infinite impacts are beyond what most traditional economic models today are able to cope with. The impacts are irreversible in the most fundamental way, so tools like cost-benefit assessment seldom make sense. To use discounting that makes infinite impacts (which could take place 100 years or more from now and affect all future generations) close to invisible in economic assessments, is another example of a challenge with current tools. So while tools like cost-benefit models and discounting can help us in some areas, they are seldom applicable in the context of infinite impacts. New tools are needed to guide the global economy in an age of potential infinite impacts. See chapter 2.2.2 for a more detailed iscussion. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 33

36 2. Risks with infinite impact: A new category of risks An additional challenge in This report emphasises the need for in most cases, but it is unscientific acknowledging the risks outlined an open and democratic process in and dangerous if different levels of in this report is that many of the addressing global challenges with probability are presented as equal. traditional risks including wars and potentially infinite impact. Hence, violence have decreased, even though this is a scientifically based invitation The tendency to compare the it might not always looks that way in to discuss how we as a global most probable climate impact 10 media. So a significant number of community can address what could with the possibility of a low or no be considered the greatest challenges experts today spend a substantial impact also results in a situation amount of time trying to explain of our time. where low-probability high-impact that much of what is discussed as outcomes are often totally ignored. The difficulty for individual scientists dangerous trends might not be as An honest and scientific approach to communicate a scientific risk dangerous as we think. For policy is to, whenever possible, present approach should however not be makers listening only to experts in the whole probability distribution underestimated. Scientists who traditional risk areas it is therefore and pay special attention to today talk about low-probability easy to get the impression that global unacceptable outcomes. impacts, that are serious but still risks are becoming less of a problem. far from infinite, are often accused The fact that we have challenges of pessimism and scaremongering, The chain of events that could that with some probability might be even if they do nothing but highlight result in infinite impacts in this infinite and therefore fundamentally 11 scientific findings. report also differ from most of the To highlight infinite irreversible is difficult to comprehend, traditional risks, as most of them impacts with even lower probability and physiologically they are are not triggered by wilful acts, can therefore be something that a something our brains are poorly but accidents/mistakes. Even the scientist who cares about his/her equipped to respond to, according to 12 probabilities related to nuclear war reputation would want to avoid. evolutionary psychologists. in this report are to a large degree It is hard for us as individuals to related to inadvertent escalation. In the media it is still common grasp that humanity for the first As many of the tools to analyse and time in its history now has the to contrast the most probable address risks have been developed climate impact with the probability capacity to create such catastrophic to protect nations and states from outcomes. Professor Marianne that nothing, or almost nothing, attacks, risks involving accidents will happen. The fact that almost Frankenhaeuser, former head of tend to get less attention. nothing could happen is not wrong the psychology division, Karolinska Roulette and Russian roulette When probability and normal risks are discussed the example of a casino and roulette is often used. You bet something, then spin the wheel and with a certain probability you win or lose. You can use different odds to discuss different kinds of risk taking. These kinds of thought experiment can be very useful, but when it comes to infinite risks these gaming analogies become problematic. For infinite impact a more appropriate analogy is probably Russian roulette. But instead of “normal” Russian roulette where you only bet your own life you are now also betting everyone you know and everyone you don’t know. Everyone alive will die if you lose. There will be no second chance for anyone as there will be no future generations; humanity will end with your loss. What probability would you accept for different sums of money if you played this version of Russian roulette? Most people would say that it is stupid and – no matter how low the probability is and no matter how big the potential win is – this kind of game should not be played, as it is unethical. Many would also say that no person should be allowed to make such a judgment, as those who are affected do not have a say. You could add that most of those who will lose from it cannot say anything as they are not born and will never exist if you lose. The difference between ordinary roulette and “allhumanity Russian roulette” is one way of illustrating the difference in nature between a “normal” risk that is reversible, and a risk with an infinite impact. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 34

37 2. Risks with infinite impact: A new category of risks Institute, Stockholm, put it this way: The LA-602 document can be seen curve exists. With climate change as the first scientific global risk “Part of the answer is to be found in this includes discussions about how psychological defence mechanisms. report addressing a category of risks sensitive the climate is, how much The nuclear threat is collectively where the worst possible impact in greenhouse gas will be emitted, and 19 all practical senses is infinite. Since denied, because to face it would what impacts that different warmings force us to face some aspects of the the atomic bomb more challenges will result in. world’s situation which we do not have emerged with potentially 13 infinite impact. Allmost all of these want to recognise.” Just as it is important not to ignore new challenges are linked to the challenges with potentially infinite increased knowledge, economic This psychological denial may be impacts, it is also important not to and technical development that one reason why there is a tendency use them to scare people. Dramatic has brought so many benefits. For among some stakeholders to images and strong language are best example, climate change is the confuse “being optimistic” with avoided whenever possible, as this denying what science is telling us, result of the industrial revolution and group of risks require sophisticated development that was, and still is, and ignoring parts of the probability strategies that benefit from rational 14 Ignoring the fact that there is curve. based heavily on fossil fuel. arguments. Throughout history strong scientific evidence for serious we have seen too many examples The increased potential for global impacts in different areas, and when threats of danger have been pandemics is the result of an focusing only on selected sources damagingly used to undermine integrated global economy where which suggest that the problem may important values. goods and services move quickly not be so serious, is not optimistic. It 15 around the world, combined is both unscientific and dangerous. with rapid urbanisation and high The history of infinite impacts: population density. A scientific approach requires us The LA-602 document to base our decisions on the whole The understanding of infinite impacts In parallel with the increased number probability distribution. Whether it is is very recent compared with most of risks with possible infinite impact, possible to address the challenge or of our institutions and laws. It is only our capacity to analyse and solve not is the area where optimism and 70 years ago that Edward Teller, them has greatly increased too. pessimism can make people look at one of the greatest physicists of his Science and technology today the same set of data and come to time, with his back-of-the-envelope provides us with knowledge and different conclusions. calculations, produced results that tools that can radically reduce the differed drastically from all that risks that historically have been Two things are important to keep had gone before. His calculations behind major extinctions, such as in mind: first, that there is always a indicated that the explosion of a pandemics and asteroids. probability distribution when it comes nuclear bomb – a creation of some to risk; second, that there are two of the brightest minds on the planet, Recent challenges like climate different kinds of impacts that are of including Teller himself – could result change, and emerging challenges interest for this report. The probability in a chain reaction so powerful that it like synthetic biology and distribution can have different shapes would ignite the world’s atmosphere, 16 nanotechnology, can to a large but in simplified cases the shape thereby ending human life on Earth. degree be addressed by smart use tends to look like a slightly modified of new technologies, new lifestyles Robert Oppenheimer, who led the clock (remember figure 1). and institutional structures. It will be Manhattan Project to develop the hard as it will require collaboration nuclear bomb, halted the project to In the media it can sound as though of a kind that we have not seen see whether Teller’s calculations were experts argue whether an impact, 17 before. It will also require us to The resulting document, LA- correct. for example a climate impact or a create systems that can deal with 602: Ignition of the Atmosphere with pandemic, will be dangerous or not. the problems before they occur. The Nuclear Bombs, concluded that Teller But what serious experts discuss is fact that the same knowledge and was wrong, But the sheer complexity the probability of different oucomes. tools can be both a problem and a drove them to end their assessment They can disagree on the shape of solution is important to understand by writing that “further work on the the curve or what curves should be 18 in order to avoid polarisation. subject [is] highly desirable”. studied, but not that a probability Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 35

38 2. Risks with infinite impact: A new category of risks The point of departure of this report Within a few decades, or even sooner, Creating innovative and resilient is the fact that we now have the many of the tools that can help us systems rather than simply knowledge, economic resources and solve the global challenges of today managing risk would let us focus technological ability to reduce most will come from fields likely to provide more on opportunities. But the of the greatest risks of our time. us with the most powerful instruments resilience needed require moving we have ever had – resulting in their away from legacy systems is likely own sets of challenges. Conversely, the infinite impacts to be disruptive, so an open and x = Impact Probability Risk we face are almost all unintended transparent discussion is needed results of human ingenuity. The Synthetic biology, nanotechnology regarding the transformative and artificial intelligence (AI) are all reason we are in this situation is solutions required. rapidly evolving fields with great that we have made progress in many areas without addressing potential. They may help solve many of today’s main challenges or, if not unintended low-probability high- guided in a benign direction, may impact consequences. result in catastrophic outcomes. Figure 3: Probability density function with tail and threshold highlighted New Category Normal Risks Requires new measures and tools Traditional measures and tools applicable Threshold probability impact 0 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 36

39 2.1 Report structure The third part of the report is the first “Current challenges” 2.1 Report structure relationship category and includes the risks that discusses the between the different risks. Action currently threaten humanity due to The first part of the report is an our economic and technological to reduce one risk can increase development - extreme climate another, unless their possible links introduction where the global risks change, for example, which depends are understood. Many solutions with potential infinite impact are introduced and defined. This part on how much greenhouse gas we emit. are also able to address multiple risks, so there are significant also includes the methodology for selecting these risks, and presents benefits from understanding how “Exogenic challenges” includes one relates to others. Investigating risks where the basic probability the twelve risks that meet this definition. Four goals of the report these correlations could be a start, of an event is beyond human control, but where the probability are also presented, under the but correlation is a linear measure and magnitude of the impact can headings “acknowledge”, “inspire”, and non-linear techniques may “connect” and “deliver”. be influenced - asteroid impacts, be more helpful for assessing the aggregate risk. for example, where the asteroids’ paths are beyond human control The second part is an overview of overview but an impact can be moderated by , the The fourth part is an the twelve global risks and key of the events that illustrate some of the either changing the direction of the first ever to our knowledge, uncertainties and probabilities of asteroid or preparing for an impact. work around the world to address global risks with potentially infinite them. For each challenge five important factors that influence the . The numbers are only “Emerging challenges” includes impacts rough estimates and are meant to areas where technological probability or impact are also listed. development and scientific be a first step in a dialogue where methodologies are developed and The risks are divided into four assessment indicate that they could both be a very important estimates refined. different categories depending on their characteristics. contribution to human welfare and The fifth part presents some of help reduce the risks associated with current challenges, but could underlying the most important 20 trends that influence the global also result in new infinite impacts. AI, nanotechnology and synthetic challenges , which often build up biology are examples. slowly until they reach a threshold and very rapid changes ensue. is “Global policy challenge” a different kind of risk. It is a The sixth and final part presents an probable threat arising from future . overview of possible ways forward global governance as it resorts to destructive policies, possibly in response to the other challenges listed above. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 37

40 2.2 Goals But we now face the possibility that 2.2 Goals Goal 1: Acknowledge even tools created with the best of intentions can have a darker side That key stakeholders, Establish a category of risks with too, a side that may threaten human influencing global challenges, potentially infinite impact. civilisation, and conceivably the acknowledge the existence of Before anything significant can continuation of human life. happen regarding global risks with the category of risks that could potentially infinite impacts, their result in infinite impact. They existence must be acknowledged. should also recognice that the This is what all decision-makers need list of risks that belong to this to recognise. Rather than succumbing category should be revised as Rapid technological development to terror, we need to acknowledge that new technologies are developed and economic growth have delivered we can let the prospect inspire and unprecedented material welfare to and our knowledge increases. drive us forward. Regardless of the risks included, billions of people in a veritable tide 21 the category should be given of utopias. special attention in all processes and decisions of relevance. The report also seeks to demonstrate to all key stakeholders that we have the capacity to reduce, or even eliminate, most of the risks in this category. Show concrete action that is It does so by combining information Goal 2: Inspire taking place today. about the risks with information about This report seeks to show that it is individuals and groups who has That policy makers inspire not only possible to contribute to made a significant contribution by action by explaining how the reducing these risks, but that it is turning challenges into opportunities. probabilities and impacts perhaps the most important thing can be reduced and turned anyone can spend their time on. By highlighting concrete examples into opportunities. Concrete the report hopes to inspire a new examples of initiatives should generation of leaders. be communicated in different networks in order to create ripple effects, with the long-term goal that all key stakeholders should be inspired to turn these risks into opportunities for positive action. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 38

41 2.2 Goals Support new meetings between Even with those risks where many interested stakeholders. Goal 3: Connect groups are involved, such as climate The nature of these risks spans change and pandemics, very few countries and continents; they That leaders in different sectors today address the possibility of require action by governments and connect with each other to infinite impact aspects. politicians, but also by companies, encourage collaboration. A academics, NGOs, and many other specific focus on financial and Even fewer groups address the links groups. The magnitude of the security policy where significant between the different risks. possible impacts requires not only risks combine to demand leaders to act but above all new action beyond the incremental There is also a need to connect models for global cooperation and is required. different levels of work, so that local, decision-making to ensure delivery. regional, national and international The need for political leadership is efforts can support each other when therefore crucial. it comes to risks with potentially infinite impacts. Identify and implement strategies In order to deliver results it is and initiatives. important to remember that global Goal 4: Deliver Reports can acknowledge, inspire governance to tackle these risks is and connect, but only people can the way we organise society in order That concrete strategies are deliver actual results. The main to address our greatest challenges. developed that allow key focus of the report is to show that It is not a question of establishing a stakeholders to identify, quantify actual initiatives need to be taken “world government”, it is about the and address global challenges that deliver actual results. way we organise ourselves on all as well as gather support for levels, from the local to the global. concrete steps towards a well- Only when the probability of an functioning global governance infinite impact becomes acceptably The report is a first step and should system. This would include low, very close to zero, and/or when be seen as an invitation to all tools and initiatives that can the maximum impact is significantly responsible parties that can affect help identify, quantify and reduced, should we talk about the probability and impact of risks reduce risks with potentially real progress. with potentially infinite impacts. infinite impacts. But its success will ultimately be measured only on how it contributes to concrete results. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 39

42 2.3 Global challenges and infinite impact Over the last few years a greater can be affected, for example with 2.3 Global challenges understanding of low probability nuclear war, where the number/size of and infinite impact weapons influences the impact and or unknown probability events has helped more people to understand tensions between countries affects the probability. This chapter first introduces the the importance of looking beyond the concept of infinite impact. It then most probable scenarios. Concepts like “black swans” and “perfect Other risks, such as a supervolcano, describes the methodology used to identify challenges with an infinite are included as it is possible to affect storms” are now part of mainstream 27 policy and business language. the impact through various mitigation impact. It then presents risks with potentially infinite impact that the methods, even if we currently cannot affect the probability. Risks that are Greater understanding of the methodology results in. susceptible to human influence are technology and science of complex indirectly linked, because efforts to systems has also resulted in a new 2.3.1 Definition of understanding of potentially disruptive address one of them may increase or infinite impact events. Humans now have such an decrease the likelihood of another. impact on the planet that the term “the anthropocene” is being used, The specific criterion for including a 2.3.2 Why use “infinite even by mainstream media like The risk in this report is that well-sourced impact” as a concept? 28 The term was introduced science shows the challenge can have Economist. 22 in the 90s by the Nobel Prize winner the following consequences: Paul Crutzen to describe how humans The concept of infinity was chosen 1. Infinite impact : When civilisation as it reflects many of the challenges, are now the dominant force changing 29 collapses to a state of great especially in economic theory, to the Earth’s ecosystems. suffering and does not recover, or a addressing these risks as well as the The idea to establish a well defined situation where all human life ends. need to question much of our current The existence of such threats is category of risks that focus on risks way of thinking. 23 well attested by science. with a potentially infinite impact – an 2. Infinite impact threshold that can be used as a practical tool The concept of a category of risks impact that can trigger a chain of by policy makers is partly inspired based on their extreme impact is events that could result first in a by Nick Bostrom’s philosophical meant to provide a tool to distinguish work and his introduction of a risk civilisation collapse, and then later one particular kind of risk from others. taxonomy that includes an academic result in an infinite impact. Such The benefit of this new concept 30 thresholds are especially important category called “existential risks”. should be assessed based on two to recognise in a complex and things. First, does the category exist, Introducing a category with risks interconnected society where and second, is the concept helpful in 24 that have a potentially infinite resilience is decreasing. addressing these risks? impact is not meant to be a mathematical definition; infinity is a A collapse of civilisation is defined as a The report has found ample evidence thorny mathematical concept and drastic decrease in human population that there are risks with an impact 31 It size and political/economic/social nothing in reality can be infinite. that can end human civilisation and complexity, globally for an extended is meant to illustrate a singularity, even all human life. The report further 25 time. The above definition means the when humanity is threatened, when concludes that a new category of list of challenges is not static. When many of the tools used to approach risk is not only meaningful but also new challenges emerge, or current most challenges today become timely. We live in a society where problematic, meaningless, or even ones fade away, the list will change. global risks with potentially infinite counterproductive. impacts increase in both number An additional criterion for including and probability according to multiple risks in this report is “human The concept of an infinite impact studies. Looking ahead, many influence”. Only risks where humans highlights a unique situation where emerging technologies which will can influence either the probability, humanity itself is threatened and the certainly provide beneficial results, the impact, or both, are included. For very idea of value and price collapses might also result in an increased 26 most risks both impact and probability from a human perspective, as the probability of infinite impacts. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 40

43 2.3 Global challenges and infinite impact Ethical aspects of infinite impact represents 1/4200, or 0.024%, of our price of the last humans also can be seen to be infinite. This is not to say potential history. So our generation The basic ethical aspect of infinite has the option of risking everything that those traditional tools cannot still impact is this: a very small group and annulling 99.976% of our alive today can take decisions be useful, but with infinite impacts potential history. Comparing 0.024% we need to add an additional set of that will fundamentally affect all analytical tools. with the days of a person living to 100 future generations. years from the day of conception, this would equal less than nine Some of the risks, including nuclear “All future generations” is not a concept that is often discussed, and days and is the first stage of human war, climate change and pandemics, 35 are often included in current risk for good reason. All through human embryogenesis, the germinal stage. history we have had no tools with a Two additional arguments to treat overviews, but in many cases their possible infinite impacts are measurable global impact for more potentially infinite impacts as a 36 excluded. The impacts which are separate category are: than a few generations. Only in the included are in most cases still very last few decades has our potential serious, but only the more probable impact reached a level where all 1. An approach to infinite impacts cannot be one of trial-and-error, future generations can be affected, parts of the probability distributions because there is no opportunity are included, and the last part of the for the simple reason that we now long tail – where the infinite impact have the technological capacity to The reactive to learn from errors. 32 approach – see what happens, is found – is excluded. end human civilisation. limit damage, and learn from Most risk reports do not differentiate experience – is unworkable. Instead If we count human history from the between challenges with a limited time when we began to practice society must be proactive. This impact and those with a potential for settled agriculture, that gives us requires foresight to foresee new 33 infinite impact. This is dangerous, as it If we make a types of threat and willingness to about 12,000 years. can mean resources are spent in ways moderate assumption that humanity take decisive preventative action that increase the probability of an will live for at least 50 million more and to bear the costs (moral and 34 years our 12,000-year history so far infinite impact. economic) of such actions. Life Value The following estimates have been applied to the value of life in the US. The estimates are either for one year of additional life or for the statistical value of a single life. – $50,000 per year of quality life (international standard most private and government-run health insurance plans worldwide use to determine whether to cover a new medical procedure) – $129,000 per year of quality life (based on analysis of kidney dialysis procedures by Stefanos Zenios and colleagues at Stanford Graduate School of Business) – $7.4 million (Environmental Protection Agency) – $7.9 million (Food and Drug Administration) – $6 million (Transportation Department) – $28 million (Richard Posner based on the willingness to pay for avoiding a plane crash) Source: Wikipedia: Value of life http://en.wikipedia.org/wiki/Value_of_life US EPA: Frequently Asked Questions on Mortality Risk Valuation http://yosemite.epa.gov/EE%5Cepa%5Ceed.nsf/webpages/MortalityRiskValuation.html Posner, Richard A. Catastrophe: risk and response. Oxford University Press, 2004 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 41

44 2.3 Global challenges and infinite impact 2. We cannot necessarily rely on In cases that do not include infinite economic perspective it makes no the institutions, morality, social impacts, discounting “reflects the sense. Money helps us to prioritise, attitudes or national security but with no humans there would be no fact that there are many high-yield policies that developed from our investments that would improve the economy and no need for priorities. experience of other sorts of risk. quality of life for future generations. Infinite impacts are in a different Ignoring, or discounting, future The discount rate should be set so category. Institutions and individuals that our investable funds are devoted generations is actually the only way 44 may find it hard to take these risks When to avoid astronomical numbers to the most productive uses.” seriously simply because they lie for impacts that may seriously there is a potentially infinite impact, the outside our experience. Our collective affect every generation to come. In focus is no longer on what investments fear-response will probably be ill- Catastrophe: Risk and Response, have the best rate of return, it is about calibrated to the magnitude of threat. Richard Posner provides a cost avoiding the ultimate end. estimate, based on the assumption Economic aspects of infinite While many economists shy away that a human life is worth $50,000, impact and discounting from infinite impacts, those exploring resulting in a $300 tn cost for the In today’s society a monetary value whole of humanity, assuming a the potentially extreme impacts of is sometimes ascribed to human life. global challenges often assume population of six billion. He then Some experts use this method to doubles the population number infinite numbers to make their point. estimate risk by assigning a monetary Nordhaus for example writes that “the to include the value of all future 37 value to human extinction. generations, ending up with $600 tn, sum of undiscounted anxieties would be infinite (i.e. equal to 1 + 1 +1 + ... = while acknowledging that “without We have to remember that the ∞). In this situation, most of us would discounting, the present value of the monetary values placed on a human dissolve in a sea of anxiety about all benefits of risk-avoidance measures life in most cases are not meant the things that could go wrong for would often approach infinity for the to suggest that we have actually type of catastrophic risk with which distant generations from asteroids, 40 assigned a specific value to a life. this book is concerned.” wars, out-of-control robots, fat tails, 45 Assigning a value to a human life is smart dust and other disasters.” a tool used in a society with a limited Discounting for risks that include supply of resources or infrastructure the possibility of an infinite impact It is interesting that Nordhaus (ambulances, perhaps) or skills. In himself provides very good graphs differs from risk discounting for less such a society it is impossible to save serious impacts. For example the that show why the most important 41 every life, so some trade-off must Stern Review prompted a discussion factor when determining actions 38 be made. The US Environmental is a possible threshold (see below between its chief author, Nicholas 42 Protection Agency explains its use like Stern, and William Nordhaus, Figure 4 and 5). Nordhaus was each this: “The EPA does not place a dollar of whom argued for different discount discussing climate change, but the value on individual lives. Rather, when role of thresholds is similar for most levels using different arguments. But conducting a benefit-cost analysis neither discussed a possible infinite infinite impacts. The first figure of new environmental policies, the climate impact. An overview of the is based on traditional economic Agency uses estimates of how much discussion by David Evans of Oxford approaches which assume that people are willing to pay for small Nature has no thresholds; the second Brookes University highlighted some 43 reductions in their risks of dying from graph illustrates what happens with of the differing assumptions. adverse health conditions that may be the curve when a threshold exists. 39 caused by environmental pollution.” As Nordhaus also notes, it is hard Two things make infinite impacts special to establish thresholds, but if they from a discounting perspective. First, The fact that monetary values for there is no way that future generations are significant all other assumptions human lives can help to define become secondary. The challenge can compensate for the impact, as they priorities when it comes to smaller that Nordhaus does not address, and will not exist. Second, the impact is risks does not mean that they are something that is beyond an individual which is important especially with suitable for quite different uses. climate change, is that thresholds preference, as society will no longer exist. Applying a monetary value to the become invisible in economic whole human race makes little Discounting is undertaken to allocate calculations if they occur far into the sense to most people, and from an resources in the most productive way. future, even if it is current actions that Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 42

45 2.3 Global challenges and infinite impact unbalance the system and eventually total discounted 46 push it over the threshold. discounted damages 6 abatement (limited part.) Note that these dramatic illustrations rest on assumptions 5 that the thresholds are still relatively benign, not moving us beyond tipping points which result in an 4 accelerated release of methane that could result in a temperature increase of more than 8 °C, possibly 3 47 producing infinite impacts. Calculating illustrative numbers 2 By including the welfare of future generations, something that is important when their very 1 Share of global income (%) existence is threatened, economic discounting becomes difficult. In this chapter, some illustrative 0 1 2 3 4 5 6 numbers are provided to indicate the order of magnitude of the Global temperature limit (°C) values that calculations provide Figure 4: : Total cost of different targets assuming limited participation and Nordhaus, The Climate Casino when traditional calculations also discounting of future incomes. include future generations. These illustrative calculations are only illustrative as the timespans that must be used make all traditional total discounted assumptions questionable to say damages with ripping costs 20 the least. Still, as an indicator for abatement (limited part.) why infinite impact might be a good approximation they might help. 16 As a species that can manipulate our environment it could be argued that the time the human 12 race will be around, if we do not kill ourselves, can be estimated to be between 1-10 million years 8 – the typical time period for the biological evolution of a successful 48 – and one billion years, species 49 the inhabitable time of Earth. 4 Share of global income (%) 1 5 3 0 2 4 6 Global temperature limit (°C) Nordhaus, The Climate Casino Figure 5: : Climate policy with a sharp tipping point at 3.5°C. This shows that the optimal temperature increase is very close to the threshold. It is constrained on the low side by abatement costs and on the high side by the sharp increase in damages. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 43

46 2.3 Global challenges and infinite impact For those that are reluctant or unable If we assume These numbers can be multiplied many times if a more philosophical to use infinity in calculations and – 50 million years for the future of and technology-optimistic scenario is are in need of a number for their formulas, $86 sextillion could be a assumed for how many lives we should humanity as our reference, – an average life expectancy of 100 include in future generations. The good initial start for the cost of infinite 50 years following quote is from an article by impacts. But it is important to note , and – a global population of 6 billion Nick Bostrom in Global Policy Journal: that this number might be orders of 51 people magnitude smaller than an estimate – all conservative estimate – , we “However, the relevant figure is not which actually took into account a have half a million generations how many people could live on more correct estimation of the number ahead of us with a total of 3 of people that should be included in Earth but how many descendants we could have in total. One lower future generations as well as the price quadrillion individuals. that should be assigned to the loss of bound of the number of biological Assuming a value of $50,000 per life, the last humans. human life-years in the future accessible universe (based on the cost of losing them would then be 20 current cosmological estimates) $1.5 ×10 , or $150 quintillion. 2.3.3 Infinite impact 34 years. Another estimate, is 10 threshold (IIT) This is a very low estimate, and which assumes that future minds Posner suggests that maybe the will be mainly implemented in computational hardware instead As we address very complex systems, cost of a life should be “written up 52 . of biological neuronal wetware, such as human civilisation and global $28 million” for catastrophic risks 54 Posner’s calculations where only one ecosystems, a concept as important produces a lower bound of 10 future generation is included result in a human-brain-emulation subjective as infinite impact in this report is that 54 life-years.” cost of $336 quadrillion. If we include of infinity impact threshold. This is the all future generations with the same impact level that can trigger a chain value, $28 million, the result is a total Likewise the value of a life, $28 of events that results in the end of 21 cost of $86 sextillion, or $86 × 10 million, a value that is based on . human civilisation. an assessment of how individuals This $86 sextillion is obviously a chose when it comes to flying, can The infinite impact threshold (IIT) very rough number (using one billion be seen as much too small. This concept represents the idea that years instead of 50 million would value is based on how much we long before an actual infinite impact value our own lives on the margin, for example require us to multiply is reached there is a tipping point the results by 20), but again it is the and it is reasonable to assume that where it (with some probability) is no magnitude that is interesting. As a the value would be higher than only longer possible to reverse events. 11 to reference there are about 10 a multiplication of our own value So instead of focusing only on the 12 if we also considered the risk of stars in our galaxy, and perhaps 10 ultimate impact it is important to losing our family, everyone we know, something like the same number of estimate what level of impact the as well as everyone else on the galaxies. With this simple calculation infinity threshold entails. 22 24 planet. In the same way as the cost , or 10 to 1,000 you get 10 to 10 sextillion, stars in the universe to increases when a certain product is The IIT is defined as an impact that in short supply, the cost of the last put the cost of infinite impacts can trigger a chain of events that when including future generations in humans could be assumed to be could result first in a civilisation 53 perspective. very high, if not infinite. collapse, and then later result in an infinite impact. Such thresholds are Obviously, the very idea to put a especially important to recognise in a price on the survival of humanity complex and interconnected society can be questioned for good where resilience is decreasing. reasons, but if we still want to use a number, $28 million per life should at least be considered as a significant underestimation. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 44

47 2.3 Global challenges and infinite impact The exact level for an infinite impact Orrell and McSharry also noted Social and ecological systems are threshold should not be the focus, but that “in orthodox economics, the complex, and in most complex systems there are thresholds where reductionist approach means that rather the fact that such thresholds the economy is seen as consisting positive feedback loops become exists and that an order of magnitude 57 of individual, independent agents self-reinforcing. In a system where should be estimated. During the resilience is too low, feedback loops who act to maximise their own process of writing the report, experts suggested that a relatively quick death utility. It assumes that prices are can result in a total system collapse. These thresholds are very difficult of two billion people could be used as driven to a state of near-equilibrium a tentative number until more research to estimate and in most cases it is by the ‘invisible hand’ of the 58 is available. economy. Deviations from this state With current trends possible only to estimate their order undermining ecological and social are assumed to be random and of magnitude. resilience it should be noted that the independent, so the price fluctuations are often modelled using the normal threshold level is likely to become As David Orrell and Patrick McSharry wrote in A Systems Approach to distribution or other distributions with lower as time progress. thin tails and finite variance.” Forecasting: “Complex systems have emergent properties, qualities that cannot be predicted in advance from The drawbacks of an approach using 2.3.4 Global F-N curves knowledge of systems components the normal distribution, or other and ALARP alone”. According to complexity distributions with thin tails and finite variance, become obvious when the In the context of global risks with scientist Stephen Wolfram’s principle of computational irreducibility, the only potentially infinite impact, the unexpected happens as in the recent credit crunch, when existing models way to predict the evolution of such possibility of establishing global F-N a system is to run the system itself: totally failed to capture the true risks curves is worth exploring. One of the economy. As an employee of of the most common and flexible “There is no simple set of equations 55 frameworks used for risk criteria that can look into its future.” Lehman Brothers put it on August 11, 59 2007: “Events that models predicted divides risks into three bands: would happen only once in 10,000 years 56 an unacceptable/ 1. Upper: happened every day for three days.” intolerable region, where risks are intolerable except in extraordinary circumstances and risk reduction Figure 6: Normal risks and risks with potentially infinite impact. measures are essential. an ALARP (“as low as 2. Middle: Threshold reasonably practicable”) region, where risk reduction measures are desirable but may not be implemented if their cost is disproportionate to the benefit achieved. a broadly acceptable/ 3. Lower: Risks with negligible region, where no further Normal risks infinite impact risk reduction measures are needed. Situation that requires Traditional measures new measures and tools The bands are expressed by F-N and tools applicable curves. When the frequency of events which cause at least N fatalities is plotted against the number N on log–log scales, the result is called an 60 probability F-N curve. If the frequency scale is replaced by annual probability, then impact 0 the resultant curve is called an f-N curve. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 45

48 2.3 Global challenges and infinite impact 1.00E-01 Figure 7: Example of F-n curve showing different 61 levels of risk 1.00E-02 Intollerable 1.00E-03 1.00E-04 ALARP 1.00E-05 1.00E-06 1.00E-07 Negligible 1.00E-08 Frequency (Events > N per Yr) 1 10000 100 10 1000 Fatalities (N) Detailed Analysis Intolerable Negligible The concept for the middle band when using F-N curves is ALARP. It Pros Cons is a term often used in the area of safety-critical and safety-involved 62 – Cumulative expression makes – Clearly shows relationship The ALARP principle is that systems. it difficult to interpret, especially the residual risk should be as low as between frequency and size reasonably practicable. by non-risk specialists of accident The upper band, the unacceptable/ – Allows judgement on relative – Can be awkard to derive importance of different sizes intolerable region, is usually the area of accident above the ALARP area (see figure 8) – Slope steeper than -1 By using F-N curves it is also – May be difficult to use if criterion provides explicit consideration possible to establish absolute impact is exceeded in one area but levels that are never acceptable, otherwise is well below of multiple fatality aversion regardless of probability (Figure 7. and favours concepts with lower potential for large Based on an actual F-n Curve fatality events showing an absolute impact level that is defined as unacceptable). This has – Allows company to manage – Much debate about criterion lines been done in some cases for local overall risk exposure from projects. The infinite threshold could portfolio of all existing and be used to create an impact limit on global F-N curves used for global future facilities challenges in the future. Such an approach would help governments, companies and researchers when they develop new technical solutions and when investing in resilience. Instead of reducing risk, such an approach encourages the building of systems which cannot have negative impacts above a certain level. 63 Figure 9: Pros and cons of F-N curves Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 46

49 2.3 Global challenges and infinite impact -2 10 macroeconomics and its challenges in discussed. In one way the name is relation to the kind of impacts that the not very important so long as people -3 10 risks in this report focus on. understand the impacts and risks Unacceptable Further, the name clearly highlights associated with it. Still, a name is -4 10 the unique nature without any symbolic and can either help or make it more difficult to get support to normative judgements. -5 establish the new category. 10 Still, infinity is an abstract concept ALARP region The work to establish a list of risks and it might not be best communicate -6 10 with infinite impact evolved from the unique group of risks that it covers to all stakeholders. In the same way “existential risk”, the philosophical -7 10 concept that inspired much of the as it can be hard to use singularity to work to establish a clearly defined describe a black hole, it can be difficult -8 group of risks. The reason for not to use infinity to describe a certain risk. 10 using the concept “existential risk If people can accept that it is only from Acceptable and impact” for this category, beside a specific perspective that the infinity -9 10 Frequency (F) of Accidents with N or More Fatalities (Per Year) concept is relevant it could be used the fact that existential impact is 10 10000 1000 100 beyond the areas of macroeconomics. also used in academic contexts to Number of Fatalities (N) refer to a personal impact, is that the Figure 8: Example of F-n curve showing an absolute Two other concepts that also have infinite category is a smaller subset impact level that is defined as unacceptable/ been considered during the process of “existential risk” and this new infinite. i.e no level of probability is acceptable above a certain level of impact, in this case of writing this report are “xrisks” and category is meant to be used as a 64 1000 dead tool, not a scientific concept. Not only “human risk of ruin”. Xrisk has the should the impacts in the category advantage, and disadvantage, of not potentially result in the end of all really saying anything at all about the 2.3.5 A name for a clearly risk. The positive aspect is that the human life, it should be possible to name can be associated with the affect the probability and/or impact defined group of risks general concept of extinction and the of that risk. There must also exist an that can provide philosophical concept of existential risk agreed methodology, such as the one practical guidance suggested in this report, that decides as both have the letter x in them. The disadvantage is the x often represents what risks belong and not belong on Today no established methodology the unknown and can therefore relate the list. exists that provides a constantly to any risk. There is nothing in the updated list of risks that threaten human name that directly relates to the kind of Another concept that the category civilisation, or even all human life. Given relates to is “global catastrophic risk” as it impacts that the category covers, so that such a category can help society to is one of the most used concepts among it is easy to interpret the term as just better understand and act to avoid such academics interested in infinite impacts. unknown risks. risks, and better understand the relation However it is vague enough to be used between these risks, it can be argued Human risk of ruin has the advantage to refer to impacts from a few thousand that a name for this category would of having a direct link to a concept, deaths to the end of human civilisation. 65 be helpful. risk of ruin, that relates to a very Already in use but not clearly defined, it includes both the academic concept specific state where all is lost. Risk of To name something that refers to the existential risks and the category of risks ruin is a concept in use in gambling, end of humanity is in itself a challenge, with infinite impacts. insurance, and finance that can all as the very idea is so far from our give very important contributions to usual references and to many the the work with this new category of intuitive feeling will be to dismiss any risk. The resemblance to an existing such thing. concept that is well established could be both a strength and a liability. The concept used in this report is “infinity”. The reson for this is that Below is an overview of the many of the challenges relate to process when different names were Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 47

50 2.3 Global challenges and infinite impact 9. Unacceptable risks in different 1. Risk of ruin – is a concept in The three concepts are very different. combinations, e.g. unacceptable Global catastrophic risk is possibly gambling, insurance and finance global risks – This is probably the most used concept in contexts relating to the likelihood of losing not appropriate for two main where infinite impacts are included, all one’s capital or affecting one’s reasons. First, it is a normative bankroll beyond the point of but it is without any clear definition. statement and the category aims recovery. It is used to describe Existential risk is an academic to be scientific; whether these concept used by a much smaller individual companies rather 66 risks are unacceptable or not is than systems. group and with particular focus on up to the citizens of the world – is used in 2. Extinction risk future technologies. The category in to decide. Second, the idea of biology for any species that this report is a tool to help decision risk is that it is a combination of is threatened. The concept is makers develop strategies that help probability times impact. If a risk reduce the probability that humanity also used in memory/cognition is unacceptable is therefore also will end when it can be avoided. The research. It is a very dramatic term, usually related to how easy it is to be used with care. These factors relation between the three concepts to avoid. Even if a risk is small, can be illustrated with three circles. make it probably unsuitable for use due to relatively low probability The large circle (1) represents global by stakeholders accustomed to and relatively low impact, but catastrophic risks, the middle one (2) traditional risk assessment. is very easy to address, it can 3. Astronomical risk existential risks and the small circle – is seldom be seen as unacceptable, in the used scientifically, but when it is (3) the list of twelve risks in this same way a large risk can be seen used it is often used for asteroids report, i.e. risks where there are peer as acceptable if it would require and is probably best reserved reviewed academic studies that 67 significant resources to reduce. for them. estimate the probability of an infinite 4. Apocalyptic risk – could have impact and where there are known There will not be a perfect concept been suitable, as the original ways to reduce the risk. A list that and the question is what concept meaning is apocálypsis, from the could be called infinite risks, xrisks, or can find the best balance between Greek ἀπό and καλύπτω meaning human risk of ruin. being easy to understand, acceptable ‘un-covering’. It is sometime used, where policy decisions needs to be but in a more general sense, to 68 made and also acceptable for all key mean significant risks. But through groups that are relevant for work in history and today it is mainly used these area. During the process to find for a religious end of time scenario. a name for this category inspiration Its strong links to unscientific has been found in the process when doom-mongers make it probably new concepts have been introduced; unsuitable for a scientific concept. from irrational numbers and genocide - of the - - belongs to world risk 5. End - to sustainable development and 3. 2. 1. the irrational doomsday narratives the Human Development Index. So and so is probably unsuitable for far “infinite risk” can be seen as the scientific risk assessments. least bad concept in some areas and 6. Extreme risk – is vague enough “xrisks” and “human risk of ruin” the to describe anything beyond the least bad in others. normal, so it is probably unsuitable for risk assessments of this The purpose of this report is to magnitude. establish a methodology to identify – is even vaguer, 7. Unique risk a very specific group of risks as as every risk is unique in some Other concepts that are related to well as continue to a process where way. Probably best avoided in infinite impacts that could potentially these risks will be addressed in a risk assessments. be used to describe the same systematic and appropriate way. – is based on 8. Collapse risk category if the above suggestions 69 The issue of naming this group Jared Diamond’s thinking. are not seen as acceptable concepts of risks will be left to others. The There are many different kinds of are presented below, together with important is that the category gets collapse and only a few result in the main reason why these concepts the attention it deserves. infinite impact. were not chosen for this report. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 48

51 2.4 Methodology 70 2.4 Methodology This chapter presents the methodology used to identify global risks with potentially infinite impact. Methodology overview In order to establish a list of global risks with potentially infinite impact a methodological triangulation was used, consisting of: A quantitative assessment of relevant literature. – – A strategic selection of relevant organisations and their priorities. – A qualitative assessment with the help of expert workshops. Relevant literature Identification of credible sources: search relevant literature in academic literature included in World of Knowledge and Google Scholar. 1 Estimations of impact Only literature where there is some estimation of impact that indicates the possibility of an infinite impact is included. 2 Leading organisations’ priorities In order to increase the probability of covering all relevant risks an overview of leading organisations' work was conducted. This list was then compared with the initial list and subjected to the same filter regarding the possibility to affect the probability or impact. 3 Possibility of addressing the risk Possibility of addressing the risk: From the risks gathered from literature and organisations, only those where the probability or impact can be affected by human actions are included. 4 Expert review Qualitative assessment: Expert review in order to increase the probability of covering all relevant global risks. 5 List of risks Result: List of risks with potentially infinite impacts. 6 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 49

52 2.4 Methodology A list of 178 relevant books and reports – Covers multiple global 2.4.1 A scientific . was established based on what other challenges (at least two) 71 review of key literature Publications that discuss a studies have referred to, and/or which variety of global challenges are of are seen as landmark studies by The scientific review of literature was groups interviewed during the process. particular importance because they led by Seth Baum, Executive Director aid in identifying and comparing the They were selected for a closer of the Global Catastrophic Risk examination regarding the challenges various challenges. This process 72 76 Institute and research scientist at the is essential for research on global they include. Center for Research on Environmental risks to identify boundaries and 73 Decisions, Columbia University. research priorities. The full bibliography, even with its focus on publications of general interest, is The methodology for including global still rather long. So it is helpful to have a In order to identify which global risks with a potentially infinite impact shorter list focused on the highlights; the challenges are most commonly is based on a scientific review of key most important publications based on discussed, key surveys were identified literature, with focus on peer-reviewed and coded. First, a list of publications how often they are quoted, how well- academic journals, using keyword that survey at least three global spread the content (methodology, lists, 74 search of both World of Knowledge etc.) is and how often key organisations challenges was compiled, and they 75 combined with and Google Scholar were then scanned to find which use them. The publications included must existing literature overviews in the meet at least one of the following criteria: challenges they discussed. area of global challenges. This also included a snowball methodology The publications that survey many – Historical significance . This where references in the leading studies global challenges were identified from includes being the first publication and books were used to identify other the full bibliography. Publications to introduce certain key concepts, scientific studies and books. or other early discussions of global from both the academic and popular challenges. Publications of historical literature were considered. Emphasis In order to select words for a literature significance are important for showing was placed on publications of repute 78 search to identify infinite impacts, a To qualify the intellectual history of global or other significance. process was established to identify as a survey of global challenges, challenges. Understanding how the words in the scientific literature the publication had to provide an state of the art research got to where it connected to global challenges with explicit list of challenges or to be of is today can also help us understand potentially infinite impacts. Some sufficient length and breadth for it where it might go in the future. words generate a lot of misses, i.e. to discuss a variety of challenges. publications that use the term but Many of the publications are books – . Influential in developing the field are not the focus of this report. For or book-length collections of articles This includes publications that are 77 example “existential risk” is used in published in book form or as special and those that have highly cited business; “human extinction” is used motivated significant additional issues of scholarly journals. Some in memory/cognition. Some search individual articles were also included research. They are not necessarily terms produced relatively few hits. because they discussed a significant the first publications to introduce breadth of challenges. the concepts they discuss, but for For example “global catastrophic whatever reason they will have proved risk” is not used much. Other words A total of 40 global challenge survey important in advancing research. are only used by people within a publications were identified. For specific research community: few use – State of the art . This includes authors with multiple entries (Bostrom “existential risk” in our sense unless publications developing new concepts with three and WEF with ten) each they are using Nick Bostrom’s work. challenge was counted only once to at the forefront of global challenges The term “global catastrophe” was avoid bias. research as well as those providing identified as a phrase that referred the best discussions of important almost exclusively to extremely established concepts. Reading these negative impacts on humans, by publications would bring a researcher a diversity of researchers, not just up to speed with current research people in one research community. on global challenges. So they are important for the quality of their ideas. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 50

53 2.4 Methodology The coding presented here erred on the was necessary because many of the In terms of authorship and audience, side of greater inclusivity: if a portion publications did not contain explicit there are 17 academic publications, of text was in the vicinity of a global lists of global challenges, and the ones 9 popular publications, 1 government challenge, then it was coded as one. report, 3 publications written by that did often mentioned additional academics for popular audiences. For example, some publications challenges separately from their lists. In terms of format, there are 15 So it was not required that a global discussed risks associated with books, 5 edited collections, 7 articles, challenge be mentioned in a list nuclear weapons in a general sense 3 of miscellaneous format. Of the for it to be counted – it only had to without specifically mentioning the 40 publications identified, 22 were possibility of large-scale nuclear war. be mentioned somewhere in the These discussions were coded as publication as a challenge. available at the time of coding. In mentions of nuclear war, even though addition, 10 Global Risks Reports they could also refer to single usages Assessing whether a particular from the World Economic Forum were of nuclear weapons that would not portion of text counts as a global coded and then gathered under one challenge and which category it heading: “WEF Global Risk Report rate as a global challenge. fits in sometimes requires some 2005-2014”. This more inclusive approach is interpretation. This is inevitable warranted because many of the for most types of textual analysis, A list of 34 global challenges was publications were not focused developed based on the challenges or, more generally, for the coding of qualitative data. The need for mentioned in the publications. A exclusively on global challenges. If they were focused on them, it is likely spreadsheet containing the challenges interpretation in this coding was that they would have included these and the publications was created to heightened by the fact that the publications often were not written risks in their global challenge form record mentions of specific challenges (e.g., nuclear war), given that they in each publication to be coded. with the purpose of surveying the were already discussing something breadth of global challenges, and even related (e.g., nuclear weapons). Then each publication was scanned the publications that were intended in its entirety for mentions of global as surveys did not use consistent Below are the results from the overview of the surveys. challenges. Scanning by this method definitions of global challenges. 25 21 20 18 17 15 15 14 14 13 13 13 13 11 11 11 10 8 8 8 8 7 7 5 5 5 5 5 4 4 4 4 3 3 2 1 1 1 1 0 Computer Failure Climate Change Pandemic Biodiversity loss Nanotech Asteroid / Comet / Meteor Volcano Genetic Engineering Unknown Simulation Phase Transition Atmosphere Aerosols Interstellar Cloud Ocean Acidification Dysgenics Ozone Depletion New Technology EM Pulse Nuclear War Reject Procreation Extraterrestrial Chemical Weapons Biological Weapons System Failure Poverty LULCC Astronomic Explosion Government Failure Biogeochem Ecological Catastrophe Chemical Pollution Artificial Intelligence Resource Depletion High Energy Physics Figure 9: Number of times global challenges are included in surveys of global challenges Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 51

54 2.4 Methodology 25 21 20 18 17 15 15 14 14 13 13 13 13 11 11 11 10 5 0 Genetic Engineering Resource Depletion Chemical Pollution Artificial Intelligence Ecological Catastrophe Climate Change Pandemic Biodiversity loss Asteroid / Comet / Meteor High Energy Physics Nanotech Volcano Nuclear War Figure 10: The global challenges included ten times or more in surveys of global challenges 2.4.2 A review of dramatic rhetoric to illustrate how It should be noted that the literature organisations working that includes multiple global little research is being done on them 81 challenges with potentially infinite compared with other areas. on global challenges impact is very small, given the fact However, it is important to note that many The list of risks found in the scientific that it is about the survival of the more studies exist that focus on individual human race. literature was checked against global risks, but often without including a review of what challenges key 80 low-probability high-impact outcomes. Experts in the field of global organisations working on global challenges, like Nick Bostrom, have challenges include in their material urged policymakers and donors to How much work actually exists on and on their webpages. This was human extinction infinite impact is focus more on the global challenges done to ensure that no important risk therefore difficult to assess. with infinite impacts and have used was excluded from the list. The coding of key organisations paralleled the coding of key survey publications. Organisations were 1000 identified via the global catastrophic risk organisation directory published by 800 82 the Global Catastrophic Risk Institute. They were selected from the directory 600 if they worked on a variety of global challenges – at least three, and ideally 400 more. The reason for focusing on those that work on multiple challenges 200 is to understand which challenges they consider important and why. In 0 contrast, organisations that focus on star trek dung beetle zinc oxalate human extinction only one or two challenges may not Figure 11: Number of academic papers on various topics (listed in Scopus, August 2012) 79 From the paper “Existential Risk Prevention as Global Priority” Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 52

55 2.4 Methodology be able to adjust their focus according Organisations working with global Then those working with multiple challenges were initially selected on to which challenges they consider the challenges were selected, resulting in 83 most important. the basis of the literature overview. A a list of 19 organisations. snowball sampling was conducted The organisation coding used the Below is the overview of the based on the list of organisations same coding scheme developed results from the overview of key identified, according to whether they for coding survey publications. claimed to work on global challenges organisations working with multiple global challenges. and/or their web page contained References to specific global challenges were obtained from information about “existential risk”, The organisations working on global organisations’ websites. Many have “global catastrophic risk”,“human challenges vary widely in: web pages which list the topics they extinction” or “greatest global work on. Where possible, references challenges”. Cross-references between organisations and input to global challenges were pulled from as a global 1. What they count these pages. Additional references to during the workshops were also challenge these challenges were identified by used to identify organisations. 2. How systematically they identify global challenges; and browsing other web pages, including on the most An initial list of 180 organisations 3. Their emphasis recent publications. While it is possible important global challenges which work with global challenges that some of these organisations have worked on global challenges was established. Based on the production of relevant literature, not mentioned on the web pages For most organisations working that were examined, overall the main which other organisations referred with global challenges there are no to the organisation, and/or are seen explanations for the methodology used challenges that they have worked on as influential by groups interviewed to select the challenges. Only a few have probably been identified and during the process, a short-list of thought leaders, like Tower Watson and coded. So the results should give a organisations were selected for a closer their Extreme Risk Report 2013, have reasonably accurate picture of what global challenges these organisations examination regarding the challenges a framework for the challenges and are working on. estimates of possible impacts. they work with. 14 13 13 12 12 10 9 8 8 7 7 7 6 6 5 4 4 4 2 2 2 2 2 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 Biodiversity loss System Failure Nuclear War High Energy Physics Chemical Weapons Nanotech Government Failure Computer Failure Biological Weapons Resource Depletion Pandemic Genetic Engineering Astronomic Explosion Asteroid / Comet / Meteor Volcano Poverty Artificial Intelligence Chemical Pollution Extraterrestrial Ozone Depletion Atmospheric Aerosols Dysgenics EM Pulse Interstellar Cloud LULCC New Technology Ocean Acidification Phase Transition Reject Procreation Simulation Unknown Biogeochem Ecological Failure Climate Change Figure 12: Global challenges that key organisations work with Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 53

56 2.4 Methodology In most cases there is neither The respondents are then asked to On the other hand, challenges which provide a number on two scales from have very big impacts but lower a definition of the impact, nor a probability, like extreme climate definition of the probability. The report 1-4, one for impact and another for 87 that focuses on global risk which is likelihood (within 10 years). change, nanotechnology, major volcanoes, AI, and asteroids, tend to probably best known is the WEF Global It is then up to the respondent to Risk Report. The WEF’s risk work, with get less, or no, attention. many other groups’, is probably best define what 1-4 means, so the major described as belonging to the category value of the report is to track the An important question to explore is whether a focus on the smaller but still of risk perception rather than risk changes in perception over the years. assessment, where experts are asked Such perception approaches are serious impacts of global challenges obviously very interesting and, as can result in an increased probability to estimate risks, but without any clear the WEF states, can influence actual of infinite impacts. For example, there definition of probability or impact. The probability as the readers’ decisions are reasons to believe that a focus more serious organisations, like the on incremental adaptation instead WEF, also clearly define what they do will be influenced by how different challenges are perceived. Still, it is as discussing perception of risk, not a of significant mitigation could be a important to remember that the report scientific assessment of the actual risk. problem for climate change as it could 88 does not provide an assessment of result in high-carbon lock-in. the actual probability (0-100%) or The WEF describes its perception an assessment of the impact (and Other research indicates that focus methodology as follows: “This not the impact on human suffering, on commercially relevant smaller approach can highlight areas that as many respondents likely define are of most concern to different pandemics could result in actions that stakeholders, and potentially galvanise make a major pandemic more likely. It risk in monetary terms for their own 85 is argued that this could happen, for company or country). shared efforts to address them.” example, by encouraging increased The question which people are asked trade of goods while investing in An overview of WEF reports from to answer is: “What occurrence the last ten years indicates that equipment that scans for the type of causes significant negative impact for the challenges that likely could pandemics that are known. Such a 86 several countries and industries?” system can reduce the probability for happen when applying a five year horizon, like the first signs of climate known pandemics while at the same change, governmental failure and time resulting in an increased probability 89 traditional pandemic, are identified. for new and more serious pandemics. 14 13 13 12 12 10 9 8 8 7 7 7 6 6 5 4 4 4 2 0 Resource Depletion Chemical Weapons Computer Failure Nuclear War Government Failure Artificial Intelligence System Failure Genetic Engeneering Atmospheric Aerosols Biological Weapons Nanotech Pandemic Climate Change Figure 13: The top 12 global challenges that key organisations work with Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 54

57 2.4 Methodology / 2.5 The rseulting list of global risks using this methodology The technologies included in emerging 2.5 The list of 2.4.3 Workshops challenges, including synthetic global risks Two workshops were arranged biology, nanotechnology and artificial where the selection of challenges intelligence (AI), will be critical to was discussed, one with risk experts finding solutions to infinite impacts. Based on the risks identified in the literature review and in the review of in Oxford at the Future of Humanity Including these technologies should organisations and applying the criteria Institute and the other in London not be seen as an attempt to arrest them. If anything, the development with experts from the financial for potentially infinite impact, these risks were identified: of sustainable solutions should be sector. See Appendix 2 for agenda accelerated. But it is equally important and participants. to create guidelines and frameworks 1. Extreme Climate Change to avoid their misuse, whether 2. Nuclear War In both workshops the list of global 3. Global Pandemic intentional or accidental. challenges was discussed to see if any additional challenges should be 4. Ecological Catastrophe 5. Global System Collapse future global included, or if there were reasons to The fourth category, exclude some from the list. policy challenges 6. Major Asteroid Impact , is of a different kind. It includes challenges related to the 7. Supervolcano 8. Synthetic Biology consequences of an inferior or destructive No challenge was excluded at the 9. Nanotechnology workshops, but one was added. global governance system. This is 10. Artificial Intelligence (AI) Although little research exists yet that especially important as well-intended is able to verify the potential impacts, actions to reduce global challenges 11. Unknown Consequences the participants agreed to include could lead to future global governance 12. Future Bad Global Governance systems with destructive impact. Global System Collapse as a risk This is an initial list. Additional risks with possible infinite impact. There will be added as new scientific studies current The first category, was agreement that further research challenges become available, and some will be is needed to clarify exactly what , includes: parts of the economic and political removed if steps are taken to reduce 90 and/or impact so their probability 1. Extreme Climate Change system could collapse and result in that they no longer meet the criteria. 2. Nuclear War a potentially infinite outcome. The conclusion was that enough research 3. Global Pandemic Four categories of global challenges 4. Ecological Catastrophe exists to include such a collapse on The challenges included in this report 5. Global System Collapse the list. belong to four categories. The first, , includes those current challenges exogenous The second category, where decisions today can result challenges , covers: directly in infinite impacts. They are included even if the time between 6. Major Asteroid Impact action and impact might be decades, 7. Supervolcano as with climate change. emerging Those in the third category, exogenous The second category is challenges , are: challenges , those where decisions do not – currently – influence probability, 8. Synthetic Biology but can influence impact. 9. Nanotechnology 10. Artificial Intelligence (AI) The third category is emerging 11. Unknown Consequences challenges , those where technology and science are not advanced enough The fourth category, global policy to pose a severe threat today, but challenges , is: where the challenges will probably soon be able to have an infinite impact. 12. Future Bad Global Governance Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 55

58 2.5 The rseulting list of global risks using this methodology 3. Chemical pollution. The challenges mentioned in the 2.5.1 Risks Increasingly, there is particular reviewed literature and organisations not included which are not included in this report concern about three types of often refer to economic damage such chemicals: those that persist in as “fiscal crises” or “unemployment”. Many risks could severely damage the environment and accumulate in the bodies of wildlife and While such impacts could have humanity but have not been included far-reaching consequences they are people, endocrine disruptors that in this report. They were excluded for obviously of another magnitude than one or more of three reasons: can interfere with hormones, and those included here. chemicals that cause cancer or 1. Limited impact. damage DNA. Many challenges Some of the risks that were suggested can have significant local negative Not included due to: ͢ effects, without approaching the “2 and/or which exist in books and Limited impact billion negatively affected” criterion reports about global risks were - tsunamis, for example, and rejected according to the criteria 91 above. They include: 4. Dangerous physics experiments chemical pollution. creating black holes/strangelets including high energy physics. 1. Astronomical explosion/nearby 2. No effective countermeasures. 94 92 The report focuses on promoting These risks are of low probability gamma-ray burst or supernova. effective interventions and so These seem to be events of and have been subsumed under ignores challenges where nothing extremely low probability and which “Uncertain Risks”. ͢ Not included due to: useful can be done to prevent or are unlikely to be survivable. Milder Included in other challenges mitigate the impact, as with nearby versions of them (where the source is sufficiently far away) may be gamma-ray bursts. considered in a subsequent report. Not included due to: 5. Destructive solar flares. ͢ 3. Included in other challenges. No effective countermeasures Though solar flares or coronal Many challenges are already mass ejections could cause covered by others, or have a great economic damage to our damage profile so similar that 95 they 2. False vacuum collapse. technological civilisation, there seemed no need to have would not lead directly to mass If our universe is in a false a separate category. Population casualties unless the system lacks vacuum and it collapses at any growth, for one, is an underlying point, the collapse would expand basic resilience. They have been driver significant for climate change at the speed of light destroying subsumed in the Global System and eco-system catastrophe, but all organised structures in the Collapse category. without direct large-scale impacts. 93 Not included due to: This would not ͢ universe. Limited impact/included in be survivable. Not included due to: other challenges ͢ No effective countermeasures Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 56

59 2.5 The rseulting list of global risks using this methodology 8. New technological 11. Computer failure/Cyber- 6. Moral collapse of humanity. experimental risks. warfare. Though an area of great Humanity may develop along It is possible and plausible that interest and research, cyber- a path that we would currently new unexpected technological warfare has never caused mass find morally repellent. The risks will emerge due to casualties and would be unlikely consequences of this are not experiments. However, until we to do so directly. It may be the clear-cut, and depend on value know what such risks may be, they subject of a future report, but in judgements that would be 96 are subsumed in the “Uncertain this report it is considered to be Some contentious and unshared. Risks” category. a subset of warfare and general of these risks (such as global Not included due to: ͢ destabilising risks. totalitarianism or enduring poverty) included in other challenges Not included due to: ͢ were included in the Governance Limited impact/Submersed in Disasters category. other challenges ͢ Not included due to: 9. Genocides. included in other challenges Though immense tragedies within 12. Underlying trends, e.g. specific areas, past genocides overpopulation. Though increased have remained contained in space population will put strains on 7. Resource depletion/LULCC/ resources and can contribute to and time and haven’t spread Biodiversity loss. 98 across the globe. increased probability for other It has often been argued that challenges included in this report Not included due to: ͢ declining resources will cause 97 (such as climate change and Limited impact Nevertheless increased conflict. ecosystem catastrophe), plausible such conflicts would not be population levels will not cause sufficient in themselves to threaten 101 any direct harm to humanity. 10. Natural disasters. humanity on a large scale, Most natural disasters, like Population growth is however an without a “ System Collapse” or important trend that is significantly tsunamis and hurricanes, have “Governance Disasters”. affecting several risks. no likelihood of causing the ͢ Not included due to: 100 Not included due to: needed extent of casualties ͢ included in other challenges for consideration on this list, as Limited impact/Submersed in they are geographically limited other challenges and follow relatively mild impact probability curves. Note: Important underlying trends are ͢ Not included due to: discussed in chapter 5. Limited impact Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 57

60 2.6 Relations between impact levels beyond the infinite threshold What makes things difficult is that None of the risks in this report is likely 2.6 Relationship between once a system is unstable, a small to result directly in an infinite impact, impact levels beyond and some cannot do so physically. disaster can have knock-on effects – the infinite threshold All the risks however are big enough the death of one Austrian nobleman to reach a threshold where the social can result in an ultimatum which draws in neighbours until Australians and ecological systems become so Complex systems are often stable unstable that an infinite impact could end up fighting Turks and the First only within certain boundaries. Outside these boundaries the World War is well under way, to be ensue, as the graph below shows. system can collapse and rapidly followed by communism, the Second change to a new stable state, or it World War and the Cold War. This graph and its accompanying text explain, how an event that reaches can trigger a process where change a threshold level could cascade into The challenge of understanding continues for a long time until a new even worse situations, via civilisation stable state is found. complex systems includes the fact 105 collapse to human extinction. that many of them have multiple Sometimes it can take a very long attractors, including what are called 103 Changes are time for a system to stabilise again. The graph also seeks to illustrate the “strange attractors”. close to linear as long as the system Looking at all the biotic crises over importance of ensuring ecological does not change very much, but once the past 530 million years, a research and social resilience, the two major it is pushed out of balance it will get team from Berkeley found an insurance policies we have against a negative spiral after a major impact that closer to other attractors, and when average of 10 million years between an extinction and a subsequent takes us beyond the infinite threshold. those become strong enough the 102 system will tend to move towards flourishing of life. chaos until a new balance is achieved 104 around the new attractor. Pre-risk rebuilding enablers (tech stores...) Post-collapse external Long term impact threats and risks Social and ecosystem resilience Extinction Post-risk Post-risk Post-collapse collapse politics politics countermeasures General Total Maintaining Civilisation mitigation short term technology base collapse and resilience casualties Long-term General pre-risk reconstruction Anthropic effect collapse countermeasures probability Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 58

61 2.6 Relations between impact levels beyond the infinite threshold 10. Post-collapse external threats 1. Social and ecosystem resilience. Some risks 5. Long-term impact. and risks. Simply because a (such as climate change) have Resilient systems are naturally risk has triggered the collapse of strong long-term impacts after years resistant to collapse, though human civilisation, that does not this often comes at the cost of or even decades. Others (such 106 mean that other risks are no longer as pandemics) are more likely to efficiency. The more resilient present. Humanity will have much have only a short-term impact. This the system, the more likely it is less resilience to deal with further to be able to adapt to even large category includes only direct long- damage, so the probability of these disasters. Improving resilience term impacts. risks is important to determine the ahead of time can improve ultimate fate of humanity. outcomes, even if the nature of the The political 6. Post-risk politics. disaster isn’t known. structures of the post-risk world 11. Anthropic effects. We cannot (governmental systems, conflicts observe a world incapable of 2. General pre-risk collapse between and within political supporting life, because we This category groupings, economic and political countermeasures. could not be alive to observe it. links between groups) will be consists of all those measures put When estimating the likelihood into place ahead of time to prevent important in determining if a large of disasters and recovery it is impact leads ultimately to civilisation civilisation collapse. It could include, very important to take this effect collapse or if recovery is possible. for instance, measures to ensure into consideration and to adjust continuity of government or prevent 110 probability estimates accordingly. breakup of countries (or to allow 7. Post-risk collapse these breakups to happen with the countermeasures. These are the 12. Long-term reconstruction countermeasures that the post- minimum of disruption). At the same probability. A post-collapse world risk political structures are likely to time it should be noted that these will differ significantly from a pre- implement to prevent a complete kinds of measures could also trigger industrial revolution world. Easy the breakdown. civilisation collapse. access to coal and oil will no longer be possible. In contrast, much 3. General mitigation and resilience. 8. Maintaining a technology base. usable aluminium will have been Current society is complex, This category consists of all extracted and processed and will with part of the world’s excess measures that can reduce the be left lying on the surface for easy impact of risks and prevent them production diverted into maintaining use. Thus it will be important to a population of scientists, getting out of hand (excluding social establish how technically possible it engineers and other experts, and ecosystem measures, which may be to have a second industrial capable of preserving knowledge are important and general enough to revolution and further reconstruction of technological innovations and deserve their own category). up to current capabilities without developing new ones. In the simpler creating the problems that the first post-collapse societies, with 4. Pre-risk rebuilding enablers. industrial revolution resulted in. On top of attempting to prevent possibly much lower populations, it will be a challenge to maintain collapses, measures can also current technology and prevent be taken to enable rebuilding 107 109 This could after a collapse. crucial skills from being lost. involve building stores of food, of technology, or crucial reconstruction Just as 9. Post-collapse politics. 108 Alternatively, it could post-risk politics are important for tools. preventing a collapse, post-collapse involve training of key individuals or institutions (such as the crews of politics will be important in allowing a recovery. The ultimate fate of nuclear submarines) to give them humanity may be tied up with the useful post-collapse skills. preservation of such concepts as human rights, the scientific method and technological progress. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 59

62 3. Twelve Global Challenges 3. Twelve Global Challenges “You may choose to look the other way but you can never say again that you did not know.” William Wilberforce Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 60

63 3. Twelve Global Challengesof risks The list of events was then ranked Four categories were used to classify For the selection of events based on their risk relevance, i.e. their information from specialised the different events: effect on the probability and/or the bodies and scientific journals in the 111 1. Policy: impact of the challenge. area of global risk was gathered. Global or national policy Using keywords related to the initiatives that affect probability and/or impact various risks, a global selection To finalise the list, a group of experts of events was sought, along with 2. Event: The challenge is made was consulted by email and a draft original sourcing in academic or real in some way that is relevant for overview of the challenges was official sources. probability and/or impact presented at a workshop at the Future 3. Research: New knowledge about of Humanity Institute (FHI) in Oxford, probability and/or impact where additional input was provided A stakeholder/group 4. Initiative: on selection and content. Issue experts were then consulted before the final list addressing the challenge in concrete 112 of events was established. ways to reduce probability and impact Global System Major Asteroid Global System Global Artificial Future Bad Extreme Major Asteroid Global Artificial Future Bad Extreme Collapse Impact Collapse Pandemic Intelligence Global Governance Climate Change Impact Pandemic Intelligence Global Governance Climate Change Global System Global System Major Asteroid Global Future Bad Artificial Extreme Major Asteroid Global Artificial Future Bad Extreme Ecological Global System Ecological Major Asteroid Global System Unknown Synthetic Super-volcano Nanotechnology Nuclear War Global Artificial Future Bad Extreme Major Asteroid Unknown Synthetic Super-volcano Nuclear War Nanotechnology Global Artificial Future Bad Extreme Collapse Collapse Impact Pandemic Global Governance Intelligence Climate Change Impact Pandemic Intelligence Global Governance Climate Change Catastrophe Collapse Catastrophe Impact Collapse Consequences Biology Pandemic Intelligence Global Governance Climate Change Impact Consequences Biology Pandemic Intelligence Global Governance Climate Change Global System Global System Major Asteroid Global Major Asteroid Artificial Future Bad Extreme Global Artificial Future Bad Extreme Ecological Ecological Global System Global System Super-volcano Synthetic Nuclear War Nanotechnology Unknown Unknown Synthetic Super-volcano Nuclear War Nanotechnology Major Asteroid Major Asteroid Global Global Artificial Artificial Future Bad Future Bad Extreme Extreme Ecological Ecological Unknown Synthetic Super-volcano Nuclear War Nanotechnology Unknown Synthetic Super-volcano Nuclear War Nanotechnology Collapse Collapse Impact Pandemic Impact Intelligence Global Governance Climate Change Pandemic Intelligence Global Governance Climate Change Catastrophe Catastrophe Collapse Collapse Biology Consequences Consequences Biology Impact Impact Pandemic Pandemic Intelligence Intelligence Global Governance Global Governance Climate Change Climate Change Catastrophe Catastrophe Consequences Biology Consequences Biology Ecological Ecological Global System Global System Unknown Synthetic Super-volcano Nanotechnology Nuclear War Major Asteroid Synthetic Super-volcano Unknown Nuclear War Nanotechnology Global Future Bad Artificial Extreme Major Asteroid Global Artificial Future Bad Extreme Ecological Ecological Unknown Synthetic Super-volcano Unknown Super-volcano Synthetic Nuclear War Nanotechnology Nuclear War Nanotechnology Catastrophe Catastrophe Collapse Collapse Consequences Biology Impact Biology Consequences Pandemic Global Governance Intelligence Climate Change Impact Pandemic Intelligence Global Governance Climate Change Catastrophe Catastrophe Consequences Biology Consequences Biology Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 61 Ecological Ecological Synthetic Super-volcano Unknown Nanotechnology Nuclear War Unknown Super-volcano Synthetic Nuclear War Nanotechnology Catastrophe Catastrophe Biology Consequences Consequences Biology

64 3.1 Current risks Current risks 3.1 3.1.1 Extreme Climate Change Global System Major Asteroid Global Artificial Future Bad Extreme Collapse Impact Pandemic Intelligence Global Governance Climate Change Climate change is a significant and lasting change in the statistical distribution of weather patterns over periods ranging from decades to millions of years. Ecological Synthetic Unknown Super-volcano Nanotechnology Nuclear War It may be a change in average weather conditions, Catastrophe Consequences Biology or in the distribution of weather around the average conditions (i.e., more or fewer extreme weather events). Extreme climate change is used to distinguish from the impacts beyond the dangerous climate 113 that a 2° C temperature rise is expected to result in. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 62

65 3.1 Current risks If for any reason it stopped (such as a Combined with shocks to the 3.1.1.1 Expected impact civilisation collapse), warming would agriculture and biosphere-dependent industries of the more developed resume at a significantly higher pace, countries, this could lead to global Many of the expected impacts of reaching the point where it would climate change are well known, have been without geo-engineering. conflict and possibly civilisation The speed of this rebound would including a warming climate, more collapse – to the extent that many severe storms and droughts, rising put extra pressure on the ecosystem experts see climate change as a 126 sea levels, ocean acidification, and national security risk and the world’s political system. So . Further 114 evidence of the risk comes from damage to vulnerable ecosystems. the biggest challenge is that geo- engineering may backfire and simply indications that past civilisation As for all risks there are uncertainties 134 make matters worse. in the estimates, and warming could collapses have been driven by 127 climate change. be much more extreme than the middle estimates suggest. Models 115 3.1.1.2 Probability Extinction risk could develop from this tend to underestimate uncertainty (especially where impact on humanity if the remaining human groups were disaggregation 116 is concerned, where the effect also vulnerable to other shocks, such as depends on modellers’ choices such pandemics, possibly exacerbated 117 128 Five important factors in estimating by the changed climate. There is ), so there is as the discount rate 118 the probabilities and impacts of the some evidence of 6°C climate change that humanity could a probability 129 120 119 or even 6°C be looking at a 4°C challenge: causing mass extinction in the past, warming in the coming decades. This but a technological species such as could arise from positive feedback ourselves might be more resilient to in climate s 1. The uncertaintie such a shock. loops, such as the release of methane sensitivity models, including the tail. 121 or the dieback from permafrost - or not - of 2. The likelihood 122 A unique feature of the climate that of the Amazon rainforests, global coordination on controlling change challenge is what is called strengthen the warming effect. So far, emissions. 130 efforts at curbing emissions have been Though this could geo-engineering. 3. The future uptake of low-carbon only moderately successful and are - if it works - reduce many impacts , including energy, economies 123 at a relatively low cost, it would still very far from what is needed. mobility and food systems. not do so evenly. Geo-engineering technological 4. Whether The impact of global warming, would possibly reduce the impacts innovations will improve or worsen of climate change in some countries, whether mild or severe, would the situation, and by how much. benefitting them while leaving others be felt most strongly in poorer 5. The long-term climate impact 131 to suffer. This could lead to greater countries. Adaptation that can caused by global warming. address significant warming is political instability. One of the most 124 and many popular geo-engineering ideas – often very expensive, of the poorest countries are in the stratospheric sulphate aerosols – suffers from the weakness that it tropics and sub-tropics that would 132 be hardest hit (they could become must be continuous. completely uninhabitable for the 125 highest range of warming ). Mass deaths and famines, social collapse and mass migration are certainly possible in this scenario. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 63

66 3.1 Current risks CLIMATE CHANGE GOVERNANCE DISASTERS Easily visible Research in Economic Technological effects of emmision-reducing Climate warfare Climate research innovations transformations climate change technologies Deliberate Global Collapse of New, polluting, Research in Technological attempts to Global Low-carbon Global instability Global povety coordination geoengineering uses for carbon mitigation and innovations construct world Geoengineering economies coordination projects products adaptation dictatorship Meta-uncertainty on tradeoffs New system between e.g. Smart sensors of governance poverty, survival, freedom Climate change Extreme Moderate Carbon mitigation and Feedback loops Global poverty climate change climate change emissions adaptation Making Improvements to Failing to solve things worse global governance important problems Ecosystem Political Increased storms, Disruption to damage Agriculture instability in flooding and world politics Direct casualties (e.g. ocean disruption and economy vulnerable nations natural disaters acidification) Not achieving Lack of human Global important Enduring poverty Climate change flourishing pollution ethical goals Long-term climate effects Undesirable world system (e.g. global dictatorship) Pre-warming Disruption to Collapse of Long-term Post-disaster Post warming Forced migration collapse world politics world system negative effects politics politics countermeasures and economy Pre-warming Civilization Total short-term casualties Extinction mitigation efforts collapse General mitigation Civilisation Total short-term Extinction effort collapse casualties Meta-uncertainty on the true uncertainty in climate Meta-uncertainty on how to predict the international change models political process Key Current Key Indirect impacts Direct impacts Risk events Meta-uncertainties Uncertain events Severe impacts Accidents Bad decisions intervention areas Current Risk events Direct impacts Indirect impacts Uncertain events Meta-uncertainties Severe impacts Bad decisions Accidents intervention areas Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 64

67 3.1 Current risks 1. Research which further refines our determining whether carbon emissions temperatures and climates. The indirect are self-damping or self-forcing (i.e. understanding of climate change and effect can however be significant, e.g. geo-engineering ideas will be essential in whether an extra ton of CO2 emissions migration, starvation, extreme weather. is likely to result in more or less than a predicting change, preparing for it, and ton in the atmosphere). potentially reversing it. On the negative 11. Climate change is likely to cause extensive ecosystem damage, such side, climate science research may allow the possibility of climate change tools 21. Transitioning to low carbon as ocean acidification and pressure economies will be crucial for reducing on many sensitive species that cannot being used for warfare. easily adapt to temperature changes. emissions without disrupting the world’s 2. Global poverty will affect both the political or economic systems. 12. Agriculture will be disrupted by vulnerability of many nations to the increased temperature. effects of climate change, and the 22. Geo-engineering offers the possibility likelihood of achieving global co- of decreasing carbon concentration in the atmosphere alongside, or instead of, 13. The direct and indirect effects of ordination earlier rather than later. emission reductions. But it may make climate change will have a great impact 3. Pre-extreme warming mitigation climate warfare a possibility. on the world’s political and economic efforts will affect the level of impact from systems, which will in turn determine the severity of the changes. 23. If geo-engineering projects collapse climate change. in the middle of implementation, this 4. Pre-warming collapse could lead to strong warming over a 14. Many nations will be made dangerously short period of time. countermeasures will affect the politically vulnerable to the direct and likelihood of civilisation collapse. indirect impacts of climate change, putting great pressure on their political 24. Technological innovations will be systems and institutions. 5. Research into mitigation and crucial for transitioning to low carbon economies or allowing geo-engineering. adaptation is necessary for effective But they may also result in new, 15. Climate change will cause an implementation of either approach. carbon-intensive innovations, which, increase in storms, floods, and other 6. Research into emission-reducing natural disasters. If political stability is if sufficiently profitable, could push technologies (such as alternative emissions up. maintained, most of the casualties are likely to result from these factors. energies) will be important for 25. Some level of changes to the transitioning to a low carbon economy. 16. Forced migration from unstable or standard economic system may 7. Global coordination and cooperation be needed to transition to low disrupted areas will put further pressure will be key to funding mitigation/ on more stable areas. carbon economies. adaptation research and development, 17. The long-term impact of 26. Easily visible impacts of climate and for the global control of carbon change may be instrumental in pushing emissions or transitioning to a global low climate change (including further carbon economy. better global coordination on the issue. carbon emissions and warming) will be important for determining 8. Climate warfare is possible if geo- the risk of collapse and 27. The political systems in place as subsequent rebuilding possibilities. engineering and climate modification warming increases will determine how well the world copes with a hotter planet. methods can be harnessed by nations to harm others. 18. Attempts to mitigate and adapt 28. Climate models are extremely to climate change will be important 9. New, more polluting uses of carbon detailed and inevitably uncertain. for reducing the severity of climate But the real level of uncertainty change’s impact. would, if they had a strong economic includes uncertainties about the rationale, put upwards pressure on 19. The level of carbon emissions is the models themselves. carbon emissions. driver of climate change, and will be 10. The direct casualties of limited 29. The course of international crucial in determining its ultimate impact. global warming are likely to be few, as politics is extremely hard to predict, 135 20. Feedback loops will be important in humans can adapt to many different even for political scientists. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 65

68 3.1 Current risks 21-May-13: China agrees Prior to the Industrial Revolution, 3.1.1.3 Main events 141 to impose carbon targets by 2016 natural climate variations caused during 2013 – Policy atmospheric CO2 to vary between about 200 ppm during ice ages Since China is the world’s greatest 19-Apr-13: Launch of the report and 300 ppm during the warmer 142 any reduction emitter of CO2, inter-glacial periods. The last time “Unburnable Carbon 2013: Wasted 136 steps it takes can have a substantial concentrations were as high as capital and stranded assets” impact. It has announced a “National they are now seems to have been – Research 143 Low Carbon Day“, a “series of during the Mid-Pliocene, about 3 major promotional events to improve To constrain the rise in global average million years before the present when awareness and get the whole society temperature to less than 2°C above temperatures were 2-3°C warmer, to address climate change.” More and in which geological evidence and pre-industrial levels, a maximum of practically, the Chinese government around 565 – 886 billion tonnes (Gt) isotopes agree that sea level was at has agreed to impose carbon targets least 15 to 25 m above today’s levels of carbon dioxide could be emitted 137 by 2016 - a ceiling on greenhouse with correspondingly smaller ice The world’s proven before 2050. 144 140 gas emissions. fossil fuel reserves amount to 2,860 Gt sheets and lower continental aridity. of CO2, however, and are viewed as assets by companies and countries. Since it is likely that these assets Figure 14-15, Source: Scripps Institution of Oceanography, via http://blogs.scientificamerican.com/ cannot be realised, these entities observations/2013/05/09/400-ppm-carbon-dioxide-in-the-atmosphere-reaches-prehistoric-levels are over-valued at current prices – arguably, a “carbon bubble.” The report provides evidence that serious risks are growing for high- carbon assets, and aims to help investors and regulators manage these risks more effectively and prepare for a global agreement on emissions reductions. It indirectly highlights part of the challenge of emissions reductions: they will mean the loss of highly valuable assets to corporations and governments. 02-May-13: CO2 at 400 PPM 138 for the first time in > 800,000 years – Event The Mauna Loa carbon dioxide record, also known as the “Keeling Curve,” is the world’s longest unbroken record of atmospheric CO2 concentrations. It recently reached 400 ppm (parts per million) of CO2. Such concentrations have not been 139 reached for at least 800,000 years, placing humanity in a historically unprecedented situation. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 66

69 3.1 Current risks 22-May-13: Private Sector Initiative 27-Sep-13: IPCC report: “Climate Change – The rate of sea level rise since 147 145 2013: The Physical Science Basis” - database of actions on adaptation the mid-19th century has been – Research – Initiative larger than the mean rate during the previous two millennia (high 146 The 5th IPCC report “considers new – a Global warming is an externality confidence). Over the period 1901 to evidence of climate change based consequence of business decisions 2010, global mean sea level rose by on many independent scientific made by entities that do not bear the 0.19 [0.17 to 0.21] m. analyses from observations of the full cost of what they decide – so the drive to mitigate its effects is more climate system, palaeoclimate – The atmospheric concentrations likely to come from governmental or archives, theoretical studies of climate of carbon dioxide, methane, and supra-governmental organisations. processes and simulations using nitrous oxide have increased to climate models.” It concludes that: Nevertheless, the private sector has levels unprecedented in at least the been involved in mitigation attempts for last 800,000 years. Carbon dioxide a variety of reasons, from investment – Warming of the climate system is concentrations have increased by opportunities to public relations. The unequivocal, and since the 1950s 40% since pre-industrial times, many of the observed changes United Nations Framework Convention primarily from fossil fuel emissions on Climate Change (UNFCCC) are unprecedented over decades and secondarily from net land use maintains a database of some of these to millennia. The atmosphere and change emissions. oceans have warmed, the amounts attempts, ranging from Ericsson’s enabling access to climate services in of snow and ice have diminished, The report further predicted, amongst sea level has risen, and the Uganda, through BASF’s development other points, that: concentrations of greenhouse gases of new technologies for food security, Allianz insurers rewarding sustainable have increased. – Continued emissions of greenhouse business practices, all the way to gases will cause further warming Chiles de Nicaragua’s attempts to – Human influence on the climate and changes in all components of enable small agro-exporters to adapt to system is clear. This is evident from the climate system. Limiting climate the increasing greenhouse gas climate change – and many more. change will require substantial and concentrations in the atmosphere, sustained reductions of greenhouse positive radiative forcing, observed The potential opportunities for private gas emissions. companies are listed as: warming, and understanding of the climate system. It is extremely likely – The oceans will continue to warm – New market opportunities and that human influence has been the during the 21st century. Heat will expansion; dominant cause of the observed penetrate from the surface to the deep warming since the mid-20th century. – Development of climate-friendly ocean and affect ocean circulation. goods and services; Further uptake of carbon by the oceans – Potential cost savings; – Each of the last three decades has will increase ocean acidification. – Risk reduction measures, including been successively warmer at the Global mean sea level will continue physical operations; Earth’s surface than any preceding to rise during the 21st century. – Climate proofing the supply chain; decade since 1850. – Enhanced corporate social – It is very likely that Arctic sea ice responsibility. – Over the last two decades, the cover will continue to shrink and Greenland and Antarctic ice sheets become thinner. Global glacier have been losing mass, glaciers volume will further decrease. have continued to shrink almost worldwide, and Arctic sea ice and – Most aspects of climate change Northern Hemisphere spring snow will persist for many centuries even cover have continued to decrease if emissions of CO2 are stopped. in extent. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 67

70 3.1 Current risks 27-Sep-13: Launch of the Global Risk 03-Dec-13 Abrupt Impacts of Climate 23-Nov-13: Limited progress at Warsaw 149 148 155 and Opportunity Indicator (GROI) COP 19 climate negotiations Change: Anticipating Surprises – Policy – Research – Research Launched by the Global Challenge The global environment can be Climate change has been developing gradually, at least on the human considered a global public good (i.e. Foundation, this Indicator is a web tool 150 156 non-excludable and non-rivalrous). for illustrating quantified risks, with scale (though very rapidly on a 157 the objective of increasing awareness Economic theory claims that such geological timescale ). This may goods will be undersupplied by the not continue, however: this paper about global risks and opportunities 151 market. Hence the importance of looks at the potential for abrupt and helping guide the changes required in the global governance system. The changes in physical, biological, and trans-national negotiations to address climate change. site is still under construction; the human systems, in response to steady climate change. It highlights Foundation’s aims are to achieve, by two abrupt changes that are already the end of 2014: Despite the importance of the subject, under way: the rapid decline in sea the main achievement of the Warsaw 158 and the extinction pressure on negotiations was to keep talks on ice 1. An interactive Global Risk & 159 152 species. track for more negotiations in 2015. On the other hand, some Opportunity Indicator that allows widely discussed abrupt changes – Though there was general agreement users to calculate the probability on the necessity of cutting carbon the rapid shutdown of the Atlantic for any global warming, between 160 emissions, the dispute was over Meridional Overturning Circulation one and ten degrees Celsius, at different greenhouse gas how to share the burden of doing and the rapid release of methane 161 concentrations. The indicator so. In this instance, the debate was or from either thawing permafrost 162 will then be further developed to between more- and less-developed methane hydrates – are shown to countries, with the latter demanding illustrate interdependencies with be unlikely to occur this century. compensation from the former to other global risks and highlight help them cope with the burden of opportunities for minimising the The report argues that large reducing emissions. That particular risks. Subsequent development uncertainties about the likelihood of 153 163 dispute was papered over, will allow users to change different some potential abrupt changes but underlying assumptions and see highlight the need for expanded similar ones will be likely in future due the corresponding change in risk. research and monitoring, and propose to the range of different actors and 154 their divergent agendas. an abrupt change early warning 2. Methodology and data to system. The aim would be to foresee estimate probabilities for a number abrupt change before it occurs, and of climate impacts at different reduce the potential consequences. temperature levels, e.g., sea level rise, droughts, flooding and heat waves, as well as to explore the risk of runaway global warming. 3. Methodology and data to estimate the probability of existential climate threats, i.e., to estimate the risk that climate change impacts pose a significant threat to human civilisation – defined as a serious negative impact on at least two billion people. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 68

71 3.1 Current risks Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 69

72 Global System Major Asteroid Global Artificial Future Bad Extreme Collapse Impact Pandemic Intelligence Global Governance Climate Change 3.1 Current risks 3.1 Current risks 3.1.2 Nuclear War Ecological Synthetic Unknown Super-volcano Nuclear War Nanotechnology Catastrophe Biology Consequences After their use in Hiroshima and Nagasaki nuclear weapons have never been used in a conflict, but because they are extremely powerful and could cause destruction throughout the world, the possibility of nuclear war has had a great effect on 164 international politics. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 70

73 3.1 Current risks society to the point where recovery There are some uncertainties about 3.1.2.1 Expected impact 178 becomes impossible both the climate models and the before 179 174 to other risks, likelihood of devastating firestorms, humanity succumbs 180 but the risks are severe and recent such as pandemics. The likelihood of a full-scale nuclear 176 175 models have confirmed the earlier war between the USA and Russia has analysis. Even a smaller nuclear probably decreased in recent decades 3.1.2.2 Probability conflict (between India and Pakistan, due to some improvements in relations disaggregation for instance) could trigger a smaller between these two countries and reductions in the size of their arsenals. nuclear winter which would place 177 Five important factors in Still, the potential for deliberate or billions in danger. 165 accidental nuclear conflict has not estimating the probabilities and impacts of the challenge: The disintegration of the global been removed, with some estimates putting the risk of nuclear war in the food supply would make mass 166 next century or so at around 10% between current 1. How relations starvation and state collapse likely. – it As the world balance of power may have been mostly down to luck and future nuclear powers develop . that such a war did not happen in the . 2. The probability of accidental war would be dramatically shifted and 167 . last half century previous ideological positions 3. Whether disarmament efforts will in reducing the number of succeed called into question, large-scale war would be likely. This could lead to a A nuclear war could have a range of nuclear warheads. civilisation collapse. . 4. The likelihood of a nuclear winter different impacts. At the lowest end 5. The long-term effects of a nuclear is the most obvious and immediate war impact: destruction and death in Extinction risk is only possible if on climate, infrastructure the aftermath of the nuclear war major cities across the world, due to and technology. fragments and diminishes human the explosions themselves and the radioactive fallout. But even if the entire populations of Europe, Russia and the USA were directly wiped out in a nuclear war – an outcome that some studies have shown to 168 , given be physically impossible population dispersal and the number 169 – that of missiles in existence would not raise the war to the first level of impact, which requires > 2 170 billion affected. A larger impact would depend on whether or not the war triggered what is often called a nuclear winter or 171 The term refers something similar. to the creation of a pall of smoke high in the stratosphere that would plunge temperatures below freezing around the globe and possibly also destroy 172 most of the ozone layer. The detonations would need to start firestorms in the targeted cities, which could lift the soot up 173 into the stratosphere. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 71

74 3.1 Current risks NUCLEAR WAR GOVERNANCE DISASTERS US-Russia relations Number of Relations between Full-scale Disarmament Nuclear attack Deliberate future major future major Nuclear War efforts Global Technological attempts to nuclear powers nuclear powers Global povety Global instability coordination innovations construct world dictatorship Meta-uncertainty Number of Proliferation: on tradeoffs Nuclear security Nuclear terrorism future small desire for New system between e.g. Smart sensors nuclear powers nuclear weapons of governance poverty, survival, freedom Nuclear Proliferation: Relations accidents or building nuclear between future misunderstandings weapons nuclear powers Making Improvements to Failing to solve things worse global governance important problems Relations Meta-certainty Meta-certainty Small-scale between current of changes in the of political Nuclear War predictions nuclear powers military technology Disruption to Not achieving Lack of human Global world politics Nuclear attack Firestorm risks Firestorm risks important Climate change Enduring poverty flourishing pollution and economy ethical goals Undesirable world system (e.g. global dictatorship) Small War casualties War casualties Post-war politics Nuclear Winter Nuclear Winter Disruption to Collapse of Long-term Post-disaster world politics world system negative effects politics and economy Total Long-term short term impact casualties General mitigation Civilisation Total short-term Extinction effort collapse casualties Pre-war casualty Civisation countermeasures Extinction collapse (bunkers, food...) Key Current Key Uncertain events Meta-uncertainties Risk events Direct impacts Indirect impacts Accidents Severe impacts Bad decisions intervention areas Current Risk events Direct impacts Uncertain events Indirect impacts Meta-uncertainties Accidents Severe impacts Bad decisions intervention areas Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 72

75 3.1 Current risks 1. The success or failure of 21. Since a major nuclear power 11. A small-scale nuclear war could start with an attack by one or more disarmament will determine the must be one of the parties to a major nuclear powers. number of nuclear warheads available nuclear war, the number of the former for a future nuclear conflict. affects the probability of the latter. 12. A full-scale nuclear war could start with an attack by one or more major 2. The first step of proliferation 22. Post-war politics will be nuclear powers. is countries desiring to possess determined by the war, the disruption it caused, and the number of nuclear weapons. Various political casualties it inflicted. 13. Aside from attacks, the other interventions may reduce or increase this desire. way a nuclear war could start would be through accidental firings or 23. Unlike other risks, nuclear misinterpretations of other incidents. weapons are targeted by humans, 3. The second step of proliferation is so may take out important parts countries building nuclear weapons. of the world’s infrastructure (and Various mechanisms, agreements and 14. Firestorms caused by burning cities are one of the main ways conventional weapons used in a inspections may be relevant a nuclear conflict could cause conflict may have the same effect). major climate disruption, and 4. Nuclear terrorism may be the trigger hence high casualties. 24. Unlike other risks, nuclear of a larger nuclear conflict, especially weapons are targeted by humans, so if the detonation is misinterpreted as a may take out important parts of the traditional attack. 15. The direct war casualties from world’s technology and research base a nuclear conflict are likely to be 5. The security of nuclear weapons small compared with the potential (and conventional weapons used in a and materials affects both the climate effects. conflict may have the same effect). probability of nuclear terrorism and the 25. Maintaining a technology base 16. A nuclear winter is the way control likelihood of nuclear accidents. will be complicated by the possible in which a nuclear conflict could targeting of infrastructure and the 6. The relations between future have the most damaging effects on the world. technology base during a conflict. nuclear powers will be the major determinant of whether a nuclear war 17. Even a smaller nuclear conflict breaks out. 26. The further development of military could trigger a smaller nuclear technology is hard to predict. The winter that could have major 7. The relations between current current balance of power under MAD (mutually assured destruction) is disruptive effects on agriculture nuclear powers will be a major and hence human survival. based on certain assumptions about determinant of the relations between the effectiveness of nuclear weapons, future nuclear powers. such as second strike capability. If this 18. Any war will have a disruptive were removed (such as by effective impact on the world’s politics and 8. The relations between future major economy. A nuclear conflict – possibly nuclear powers will be the major submarine detection, or anti-ballistic component of determining whether a accompanied by a nuclear winter – missile shields), the effect on the balance of power is hard to predict. major nuclear war breaks out. even more so. 27. The course of international politics 19. The long term impact of nuclear 9. Relations between the USA and is extremely hard to predict, even for winter, infrastructure disruption, and Russia (the only current major nuclear 181 political scientists. powers) will be a major determinant possibly radiation, will determine the of the relations between future major likelihood of collapse and rebuilding. nuclear powers. 20. Since a nuclear power must be one of the parties to a nuclear war, 10. Pre-war countermeasures (such as nuclear bunkers and food stores) the number of the former affects the can help mitigate the casualties of a probability of the latter. smaller nuclear conflict. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 73

76 3.1 Current risks Though the situation remains a – It is unlikely that any state or 3.1.2.3 Main events potential flashpoint for conventional international body could address the during 2013 and nuclear conflict, and its collapse immediate humanitarian emergency 191 could have disastrous consequences caused by a nuclear weapon 12-Feb-13: North Korea carries out (including the possibility of “loose detonation in an adequate manner 182 nukes” becoming available to various third, largest nuclear test and provide sufficient assistance to – Event groups), it should be noted that the those affected. Moreover, it might “North Korean problem” has existed not be possible to establish such in one form or another since the end On 12 February 2013, North Korea carried capacities at all. out its third nuclear test. The test was of the Korean War in 1953, without – The historical experience from the 183 192 condemned across the world, erupting into open conflict. use and testing of nuclear weapons 184 and led to increased sanctions has demonstrated their devastating 185 against the already isolated nation. immediate and long-term effects. North Korea is the only nation to have 04-Mar-13: Conference: Humanitarian While political circumstances have 193 withdrawn from the Nuclear Non- Impact of Nuclear Weapons changed, the destructive potential of 186 and is the only – Policy Proliferation Treaty, nuclear weapons remains. country to have conducted nuclear – The effects of a nuclear weapon On 4 and 5 March 2013, the tests in the 21st century, starting detonation, irrespective of cause, 187 Norwegian Minister of Foreign in 2006, as well as developing a will not be limited by national 188 It has also Affairs, Espen Barth Eide, hosted ballistic missile capability. borders, and will affect states and been involved in the export of weapons an international conference on the people to significant degrees, 189 technology, undermining the Treaty. humanitarian impact of nuclear regionally as well as globally. Diplomatic attempts to deal with North weapons. The conference heard Korea (especially on the part of the presentations on the effects of A number of states wished to explore United States) have generally been nuclear weapons detonations. these issues further, and Mexico said it 190 194 inconsistent and unsuccessful. Three key points emerged: would host a follow-up conference. Worldwide nuclear testing, 1945-2013 File:Worldwide_nuclear_testing.svg CC-BY-SA license. Figure 16, Source: Wikimedia Commons, http://en.wikipedia.org/wiki/ Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 74

77 3.1 Current risks 16-May-13: Revealed: The USSR 24-Jun-13: Report: “Analysing and Then it combined the fault tree-based Reducing the Risks of Inadvertent and US Came Closer to Nuclear War risk models with parameter estimates 195 Than Was Thought Nuclear War Between the United sourced from the academic literature, 204 – Research States and Russia” characterising uncertainties in the – Research form of probability distributions, Documents recently released under a with propagation of uncertainties Though the end of the Cold War has FOIA (US Freedom Of Information Act) in the fault tree using Monte Carlo request show that the risk of nuclear reduced the likelihood of deliberate simulation methods. Finally, it also nuclear war, its impact on the risk conflict between the superpowers performed sensitivity analyses to was higher than realised at the time. of accidental nuclear war is much identify dominant risks under various smaller. The arsenals remain on The large-scale 1983 NATO nuclear assumptions. This kind of highly 205 meaning that exercises Able Archer 83” spurred “a “launch on warning”, disaggregated analysis is most likely there is a possibility for a “retaliatory” high level of Soviet military activity, to elicit the best performance and 209 strike before an attack is confirmed. with new deployments of weapons estimates from experts. and strike forces.” This unprecedented The most likely cause of such an Soviet reaction in turn created a accident is either a false warning Their conclusion was that (under the (of which there have been many, series of introspective US intelligence more pessimistic assumption), there analyses and counter-analyses, with causes ranging from weather was a mean 2% risk of accidental debating whether US intelligence phenomena to a faulty computer nuclear war a year (a high risk when chip, wild animal activity, and control- had actually understood Soviet compounded over several decades), actions, perceptions, and fears – and room training tapes loaded at the with the risk from false alarm being 206 acknowledging the danger of nuclear wrong time) or a misinterpreted orders of magnitude higher than that 207 196 “miscalculation” if it had not. terrorist attack. from terrorist attacks. The analysis suggests that the most important The report attempted a rigorous This is but one of the many nuclear inadvertent nuclear war risk factor is the 197 210 accidents and incidents that estimate of the numerical probability inherent short launch decision times, peppered the Cold War and its of nuclear war. Such numerical rigour in the “launch on warning” posture. is rare, with the exception of Hellman’s aftermath, and which have been Some ways of improving this were 208 This report applied risk estimates. revealed only subsequently. We know suggested, for instance by moving each analysis methods using fault trees and now that there were at least three country’s strategic submarines away occasions – the Cuban missile crisis in mathematical modelling to assess the from the other’s coasts. 198 199 relative risks of multiple inadvertent the Petrov incident in 1983 1962, and the Norwegian rocket incident in nuclear war scenarios previously 200 – where a full-scale nuclear 1995 identified in the literature. 201 war was only narrowly averted. Further information on these incidents, and on how they were interpreted 202 and misinterpreted by the great powers, will be important to estimate the probability of nuclear conflict in the coming decades. On a more positive note, efforts are being made to reduce the probability of inadvertent 203 or accidental nuclear conflicts. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 75

78 3.1 Current risks 03-Sep-13: Report of the UN 15-Nov-13: International Physicians 24-Nov-13: Nuclear deal with Iran 224 for the Prevention of Nuclear War General Assembly working group on may reduce risk of proliferation “Taking Forward Multilateral Nuclear report: “Nuclear Famine: Two Billion – Policy 218 211 People at Risk?” Disarmament Negotiations” – Research In November, Iran struck a deal – Policy with the so called “P5+1” (the five This report is one of a series of The working group had extensive permanent members of the security reports and publications in recent council, plus Germany). The deal, if exchanges of view from different it holds, would allow Iran to continue years about the potential impacts of participants, and reviewed existing 219 some uranium enrichment, but it It looked at the disarmament commitments and nuclear conflicts. likely consequences of a “limited” proposals, including international law. would have to submit to inspections The issues surrounding disarmament nuclear war, such as between India to ensure it wasn’t developing a nuclear weapons programme (the deal and treaties were analysed in depth, and Pakistan. and several proposals were put would also result in eased sanctions While previous papers had estimated forward, with an eye to the complete in return). There have been long- elimination of nuclear weapons. running fears than Iran may have been that up to a billion people might 220 this attempting to construct a nuclear be at risk in such a conflict, 225 , resulting in sanctions report increased the estimate to weapon A key recognition was, however, that 226 “participants recognised the absence two billion. The main source of this being imposed on it. increase is decreased agricultural of concrete outcomes of multilateral 221 nuclear disarmament negotiations production in the United States This event illustrates the surprising 222 within the United Nations framework and in China. A key component success of the Non-Proliferation 227 for more than a decade”. Indeed, Treaty, which came into force in of these estimates was the severe though the Nuclear Non-Proliferation agricultural impact of the relatively 1970. At the time it was proposed 212 Treaty mild temperature reduction in 1816, (NPT) is a multilateral treaty there were fears of very rapid 223 228 proliferation of nuclear weapons. closely connected with the United , the “year without a summer” And though 40 countries or more Nations, and though it committed due mainly to the “volcanic winter” currently have the knowhow to the nuclear powers to reduce their caused by the eruption of Mount 229 build nuclear weapons, arsenals, all the major nuclear arms Tambora. The report highlights some only nine reduction deals have been bilateral countries are currently known to significant areas of uncertainty, treaties between the US and the possess them: the five security council such as whether a small nuclear USSR/Russia. These include the INF conflict and its consequences would members, India, Pakistan, and North 213 230 214 215 Korea, plus Israel. , and New lead to further conflicts across the treaty , SORT , START I 216 world, and doubts whether markets, , which have significantly START reduced the world’s stock of nuclear governments and other organisations could mitigate the negative impacts. weapons. It has also been argued The report is a reminder that even that the NPT has been undermined by a number of bilateral deals made small-scale nuclear conflict could have severe consequences. by NPT signatories, most notably 217 This further the United States. serves to emphasise the weakness of international institutions where nuclear arms control is concerned. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 76

79 3.1 Current risks Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 77

80 Global System Major Asteroid Global Artificial Future Bad Extreme Collapse Impact Pandemic Intelligence Global Governance Climate Change 3.1 Current risks 3.1 Current risks 3.1.3 Ecological Catastrophe Ecological Synthetic Super-volcano Unknown Nuclear War Nanotechnology Catastrophe Consequences Biology Ecological collapse refers to a situation where an ecosystem suffers a drastic, possibly permanent, reduction in carrying capacity for all organisms, often resulting in mass extinction. Usually an ecological collapse is precipitated by a disastrous event 231 occurring on a short time scale. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 78

81 3.1 Current risks There is currently more than enough 3.1.3.2 Probability 3.1.3.1 Expected impact food for everyone on the planet to disaggregation 240 but ensure the nutrition needed, its distribution is extremely uneven Humans are part of the global ecosystem and so fundamentally Five important factors in and malnutrition persists. Thus ecological collapse need not have depend on it for our welfare. estimating the probabilities and a strong absolute effect in order to impacts of the challenge: Species extinction is proceeding at result in strong localised, or global, effects. Even a partial collapse could humans are 1. The extent to which a greatly increased rate compared 232 , and attempts to dependent on the ecosystem with historic data . lead to wars, mass migrations, and effective Whether there will be 2. social instability. It is conceivable that quantify a safe ecological operating political measures space place humanity well outside such a scenario, if drawn out and taken to protect 233 Furthermore, there may be signs exacerbated by poor decision-making, it. the ecosystem on a large scale. of a “sudden” biosphere collapse, could eventually lead to mass deaths 3. The likelihood of the emergence 234 of and even the collapse of civilisation. sustainable economies . possibly within a few generations. 4. The positive and negative Many of the problems of ecological of degradation interact to multiply on the eco systems impacts Extinction risk is possible only if the both wealth and poverty . aftermath of collapse fragments the damage and (unlike previous, 5. The long-term effects localised collapses) the whole world and diminishes human society of an 235 with severe so far that recovery becomes ecological collapse on ecosystems. is potentially at risk, 241 before humanity challenges to countering this risk impossible 236 through global policy. succumbs to other risks (such as climate change or pandemics). If animals are seen to have intrinsic 237 or if human quality of After a post-civilisation collapse, value, life is dependent on a functioning human society could still be suffering 238 the current situation from the effects of ecological ecosystem, already represents a large loss. collapse, and depending on what form it took, this could make the recovery of human civilisation more Whether such a loss will extend to challenging than in some of the other human lives depends on technological scenarios presented here. and political factors - technological, because it seems plausible that some human lifestyles could be sustained in a relatively ecosystem-independent 239 way, at relatively low costs. Whether this can be implemented on a large scale in practice, especially during a collapse, will be a political challenge and whether it is something we want is an ethical question. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 79

82 3.1 Current risks ECOLOGICAL CATASTROPHE GOVERNANCE DISASTERS Global Sustainability Post-eco-collapse Global poverty coordination research climate change Long-term ecological effects Deliberate Global Technological attempts to Global povety Global instability coordination innovations construct world dictatorship Quality of life Ecological Preservation Meta-uncertainty loss from collapse efforts on tradeoffs ecosystem loss New system between e.g. Smart sensors of governance poverty, survival, freedom Threat to Moral tragedy from Pollution food supply ecosystem loss Making Improvements to Failing to solve things worse global governance important problems Loss of Pre-eco-collapse Economic costs biodiversity climate change Disruption to Post-eco-collapse Rebuilding politics and politics the ecosystem economy Not achieving Lack of human Global important Climate change Enduring poverty flourishing pollution ethical goals New, Vulnerabilities Sustainable or environmentally to flood and non-sustainable damaging industries economies other disasters Undesirable world system (e.g. global dictatorship) Technological Pre-eco-collapse innovations mitigation efforts Disruption to Collapse of Long-term Post-disaster world politics world system negative effects politics and economy Meta-uncertainty on the true dependence of Human survivability humanity on the ecosystem in “closed” systems General mitigation Civilisation Total short-term Extinction effort collapse casualties Total short-term Civilisation Extinction casualties collapse Key Current Uncertain events Meta-uncertainties Direct impacts Risk events Indirect impacts Bad decisions Accidents Severe impacts intervention areas Key Current Direct impacts Risk events Uncertain events Indirect impacts Meta-uncertainties Accidents Bad decisions Severe impacts intervention areas Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 80

83 3.1 Current risks 1. Global coordination and 19. Technological innovations 9. New, profitable, but environmentally may result in more sustainable damaging industries could put extra cooperation will be important to any strain on the ecosystem. attempt to control ecological damage economies, or in more on a large scale and prevent “races to environmentally damaging products. the bottom”. 10. According to some systems of 20. It may be possible to value, the loss of certain animals and ecosystems constitutes a moral 2. Poverty is often seen as ensure human survival in semi- exacerbating ecological damage tragedy in and of itself. “closed” systems (solar power, hydroponic food, distilled water), through unsustainable practices, 11. Humans derive much pleasure and with minimal dependency on the while richer countries introduce environmental regulations – but richer external ecosystem. many benefits from various parts of nations exploit many resources (such the ecosystem, and losing this would result in a loss to human quality of life. 21. Over the long term, it may become as fossil fuels) in non-sustainable and damaging ways. possible and necessary to go about 12. Ongoing and continuous rebuilding the ecosystem and healing its damage. 3. Transitioning to sustainable biodiversity loss is a clear economies, or sustainable consequence of ecological collapse. economic trajectories, could control 22. Political decisions will be the most 13. Ecological damage can put ecological damage. likely factors to exacerbate or mitigate an ecological disaster. the human food system in danger, triggering famines. 4. Research into sustainability could allow the construction of sustainable 23. It is unclear how dependent 14. Ecological damage increases economies or environments at costs humans truly are on the vulnerability to floods and other that people are willing to bear. ecosystem, and how much damage they could inflict without natural disasters. 5. Climate change exacerbates the threatening their own survival. 15. Disruptions to the world’s political pressure on the ecological system and economic systems could trigger by changing weather patterns and further conflicts or instabilities, increasing natural disasters in ways ecosystems find hard to adapt to. causing more casualties and impairing effective response. 6. Global pollution is a visible source of ecological damage, one that global 16. Since a lot of the world’s carbon is locked up in trees, ecological collapse agreements have had moderate success at tackling. could exacerbate climate change. 7. Truly global preservation efforts 17. The ecosystem is of great may be needed for some threatened economic benefit to humanity, so its loss would have large ecosystems that stretch beyond natural boundaries (e.g. in the seas economic costs. and oceans). 18. Ecological damage is likely to be long-term: the effects will last for 8. Beyond general all-purpose mitigation efforts, addressing many generations. this threat could include the preservation of ecosystems, species or genetic codes, to allow a subsequent rebuilding. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 81

84 3.1 Current risks 05-Apr-13: Ocean data added to 30-May-13: Improvement in 3.1.3.3 Main events 252 Microsoft Eye on Earth project managed fisheries in Europe during 2013 – Initiative – Research Human action has been shown to In order to safeguard ecological 22-Jan-13: Current extinctions be able to mitigate some ecosystem resources, it is important to probably the result of past actions; 242 track and quantify them. This damage. Overfishing is expected by many future extinctions to come has traditionally been the role of standard economic theory: the sea’s – Research resources are a (global) common, governments or non-governmental 248 where the rational behaviour of organisations. Recently, An estimated 40% of world trade however, private organisations individual fishermen must lead to is based on biological products 253 dilapidation of the resource. Unlike have started developing tools to or processes such as agriculture, enable companies and individuals on land, where nature reserves or forestry, fisheries and plant-derived parks can be established, there to track ecological damage and pharmaceuticals, and biodiversity are no easy ways of establishing make decisions in consequence. comprises an invaluable pool 254 243 (thus property rights in the sea One such tool was Eye on Earth, And yet this for innovations. privatising that “common”). A developed by Microsoft in alliance biodiversity is being lost at an typical example of this behaviour with the European Environment alarming rate – the rate of extinctions 249 Agency and Esri. It was launched is the collapse of the Grand Banks for plants and animals is 100 to 1,000 fisheries off Canada’s Atlantic coast with three services – WaterWatch, times higher than their pre-human 244 in the 1990s, where cod biomass fell AirWatch and NoiseWatch – keeping A variety of methods have levels. by over 95% from its peak and has track of the levels of different been suggested to halt or slow 255 currently not recovered. pollutants, using official sources this loss, ranging from putting an 250 245 and inputs from citizens. This was on biodiversity and explicit value It is therefore significant that the subsequently expanded to include ecosystem services (human benefits European Union has been partly other environmentally sensitive from a multitude of resources and pieces of information, such as the successful in its attempts to control processes that are supplied by 246 states of coral reefs and invasive over-fishing through legislation. ecosystems), to performing triage 247 For instance, despite the fact that alien species. This on the most valuable species. North Sea cod remains vulnerable, research paper suggests, however, there has been a recent increase It was primarily land-based, so that there is a lag of several decades in stock size and a decrease in the oceans were missing from between human pressure on the this visualisation tool. This lack fish mortality. This may point to ecosystem and ultimate species has been partially overcome the potential for further ecological extinction. This suggests that many with the inclusion of data from improvements through well-chosen extinctions will continue in decades 251 policy interventions. the MyOcean 2 project to come, irrespective of current (partly funded by the European conservation efforts. Commission). The data cover sea surface temperature, salinity and currents for the Mediterranean Sea and the Black Sea. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 82

85 3.1 Current risks 02-Jul-13: About 21,000 Species Face Extinction, says International Union 256 for Conservation of Nature (IUCN) – Event In 2013 the IUCN added an additional 4,807 species to its Red List of Threatened Species. This brings the total to about 21,000. Some have argued that we are entering a new geological era in Earth’s 257 history: the Anthropocene , when human actions are one of the major impactors on the planet’s biosphere. The graph shows a fairly steady growth in the (estimated) number of threatened species. This steadiness may be illusory, as the biosphere shows signs that it may be approaching a planetary-scale tipping Figure 17: Collapse of Atlantic cod stocks (East Coast of Newfoundland), 1992 Source: http://en.wikipedia.org/wiki/File:Surexploitation_morue_surp%C3%AAcheEn.jpg) point, where it may shift abruptly and irreversibly from one state to another. As a result, the biological resources humans presently take for granted may be subject to rapid and unpredictable transformations 258 within a few human generations. This could be seen as a great tragedy beyond purely human concerns, if animals (and animal welfare) are seen 259 to have intrinsic value. Figure 18: Increase in the number of species assessed for the IUCN Red List of Threatened SpeciesTM (2000–2013.2). Source: http://www.iucnredlist.org/about/summary-statistics Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 83

86 3.1 Current risks 3.1 Current risks 3.1.4 Global Pandemic Global System Major Asteroid Global Artificial Future Bad Extreme Collapse Impact Pandemic Intelligence Global Governance Climate Change A pandemic (from Greek π ν, pan, ᾶ μος demos, “people”) ῆ “all”, and δ is an epidemic of infectious disease that has spread through human populations Ecological Synthetic Super-volcano Unknown Nuclear War Nanotechnology across a large region; for instance Catastrophe Consequences Biology several continents, or even worldwide. Here only worldwide events are included. A widespread endemic disease that is stable in terms of how many people 260 become sick from it is not a pandemic. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 84

87 3.1 Current risks Extinction risk is only possible if the Many relevant features of the world 3.1.4.1 Expected impact have changed considerably, making aftermath of the epidemic fragments past comparisons problematic. and diminishes human society to Infectious diseases have been one the extent that recovery becomes The modern world has better 277 of the greatest causes of mortality impossible before humanity sanitation and medical research, as well as national and supra-national succumbs to other risks (such as in history. Unlike many other institutions dedicated to combating global challenges pandemics have climate change or further pandemics). diseases. Private insurers are also happened recently, as we can see interested in modelling pandemic where reasonably good data exist. 3.1.4.2 Probability 273 risks. Set against this is the fact Plotting historic epidemic fatalities disaggregation on a log scale reveals that these that modern transport and dense tend to follow a power law with a human population allow infections 274 Five important factors in small exponent: many plagues have , to spread much more rapidly and there is the potential for urban been found to follow a power law estimating the probabilities and 261 slums to serve as breeding grounds impacts of the challenge: with exponent 0.26. 275 for disease. 1. What the true probability These kinds of power laws are 262 , Unlike events such as nuclear wars, distribution for pandemics is heavy-tailed to a significant 263 In consequence most of especially at the tail. pandemics would not damage the degree. world’s infrastructure, and initial the fatalities are accounted for by the 2. The capacity of modern 264 If this law holds for top few events. survivors would likely be resistant international health systems to 265 future pandemics as well, then the with an extreme pandemic. deal to the infection. And there would majority of people who will die from probably be survivors, if only in 3. How fast medical research can epidemics will likely die from the in an emergency. proceed isolated locations. Hence the risk single largest pandemic. of a civilisation collapse would 4. How mobility of goods and population , as well as people come from the ripple effect of the Most epidemic fatalities follow a density, fatalities and the policy responses. will affect pandemic power law, with some extreme . These would include political and transmission events – such as the Black Death agricultural disruption as well as 5. Whether humans can develop and Spanish Flu – being even anti-pandemic novel and effective economic dislocation and damage 267 more deadly. solutions. to the world’s trade network (including the food trade). There are other grounds for suspecting that such a high- impact epidemic will have a greater probability than usually assumed. All the features of an extremely devastating disease already exist in nature: essentially incurable 268 (Ebola ), nearly always fatal 269 ), extremely infectious (rabies 270 ), and long (common cold 271 ). If a incubation periods (HIV pathogen were to emerge that somehow combined these features (and influenza has demonstrated antigenic shift, the ability to combine 272 features from different viruses ), its death toll would be extreme. 266 Influenza subtypes Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 85

88 GLOBAL PANDEMIC 3.1 Current risks GOVERNANCE DISASTERS Contact with Small pandemic Global poverty scares reservoir species Deliberate Impact of Global Technological attempts to Density of Global increased Global povety Global instability Medical research Bio-terrorism coordination innovations construct world population coordination movement of dictatorship goods and people Meta-uncertainty on tradeoffs New system between e.g. Smart sensors of governance poverty, survival, Pandemic Impact of Healthcare in freedom Accidental release Antibiotics combining sanitation or individual from lab resistance different deadly countries lack thereof features Making Improvements to Pandemic Impact of Failing to solve Post-pandemic Deadly things worse global governance Smart sensors monoculture leaping the important problems politics pandemic food supply species barrier Impact on meat Disruption to Effectiveness of Pandemic Direct casualties world politics production and countermeasures transmission food supply and economy Not achieving Lack of human Global important Enduring poverty Climate change flourishing pollution ethical goals Undesirable world system Long-term fate (e.g. global of pandemic virus/ dictatorship) bacteria/parasite Disruption to Collapse of Long-term Post-disaster world politics world system negative effects politics and economy Pre-pandemic Total short-term medical casualties contingency plans Civilisation Meta-uncertainty of how the changed world Extinction collapse has affected pandemic probabilities General mitigation Civilisation Total short-term Extinction effort collapse casualties Meta-uncertainty of what probability distributions pandemics follow Key Current Uncertain events Indirect impacts Direct impacts Meta-uncertainties Risk events Bad decisions Accidents Severe impacts intervention areas Key Current Direct impacts Risk events Meta-uncertainties Indirect impacts Uncertain events Severe impacts Accidents Bad decisions intervention areas Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 86

89 3.1 Current risks 1. Extensive medical research 20. The improvements to surveillance 9. Mass casualties and finger-pointing and sensing technologies (including will be key to preventing and could destabilise the world political combatting large scale pandemics. indirect detection via web queries or and economic systems. social media) open the possibility of The drawbacks are the possibility 10. If the pathogen is transmissible of accidental release of dangerous smarter interventions (such as micro- to farm animals, this could affect the pathogens from laboratories and quarantines) and faster understanding world food supply. of the pathogen’s transmissibility. of bioterrorism. 21. Post-pandemic politics will be 2. As so much is known about 11. It is unlikely the pathogen would pandemic risks compared with important for preventing a civilisation be a recurrent, long-term risk, but collapse or enabling reconstruction. variants of it could continue to affect other risks, there are more possibilities for specific pre- people and animals for many years, 22. Many pathogens incubate in dependent on its transmissibility and pandemic contingency plans. species close to humans, before life cycle. leaping the species barrier. 3. The effectiveness of healthcare systems will be important, especially 12. Small pandemic scares could 23. Monoculture food systems make in less developed nations where improve global coordination on the issue. the pandemic may overwhelm the it easier to transmit any pathogen system, and then transmit from there infecting human food animals. to other nations. 13. Increased population density 24. The mode of transmission of the causes increased transmissibility 4. Global coordination in detection, pathogen will be critical to its ultimate of the pathogen, especially in reach and impact. analysis and treatment are vital urban slums. for stopping a pandemic in its 14. Some pathogens, such as bird flu, early stages, and for implementing 25. Various countermeasures are depend on regular contact between measures such as quarantines and available in terms of detection, virus analysis, treatment, and humans and “reservoir species” more advanced countermeasures. in order to evolve into periodically quarantining. Future research, dangerous strains. 5. Poverty will affect the quality technological and political of national healthcare systems, developments may open up new methods of fighting population density and sanitation 15. If antibiotic resistance develops, quality, the movement of local goods humanity could see the resurgence of the pathogen. and people, and the effectiveness of bacteria-based pandemics. 26. Many of the current factors the political response. 16. The increased movement of determining pathogen transmission 6. Bioterrorists may unleash a are unprecedented, such as people and products increases the speed and spread of pathogen held in storage, such movements of goods and people, the pandemic transmission. as smallpox. quality of healthcare systems, and the existence of a centralised political 7. Laboratory security at the top labs response. This means that data from 17. Sanitation or its lack will strongly past pandemics will not be as reliable is insufficient for the danger at hand, affect the spread of certain pathogens for computing probability distributions. and accidental release is a non- in key areas. negligible possibility. 27. The pandemic risk lies in the 18. The efficiency of global reaction 8. Pandemics are one of the risks to a new pandemic will be strongly “tails” – the extreme events – and determined by the speed of research these tails must be estimated from where there is a possibility for a very large number of direct on the pathogen during the pandemic. few data points, making them tricky and uncertain. casualties, depending on the severity of the pathogen. 19. A great risk will arise if a pathogen combines the different dangerous features of current viruses or bacteria. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 87

90 3.1 Current risks 09-Aug-13: Epihack: Digital disease The guidance paper indicates 3.1.4.3 Main events 292 surveillance hack-a-thon simultaneously the weaknesses during 2013 – Initiative of pandemic preparations, the improvements in these Beyond the formal, top-down preparations, and the continued 10-Jun-13: Pandemic Influenza initiatives to deal with pandemics, Risk Management: WHO Interim role of the WHO as global directing 278 there are openings for bottom-up, Guidance and coordinating authority. innovative ideas. Epihack attempted – Policy to generate just such ideas, through three days of designing and hacking This is an updated document 24-Jul-13: Bacteria become in Cambodia. Descriptions of the resistant to some of the last that replaces the 2009 Pandemic 285 winning projects were given: Influenza Preparedness and remaining antibiotics Response: a WHO guidance – Event 279 – CoPanFlu: This project included document. It updates its home visits to collect blood recommendations based on lessons Bacterial infections, such as the Black 286 287 samples from 807 homes and syphilis, Death, from the influenza A(H1N1) 2009 and tuberculosis, 288 280 weekly follow-up phone calls have been responsible for millions pandemic (swine flu), the adoption to document the occurrence of of deaths, over the thousands of years by the Sixty-fourth World Health infectious respiratory symptoms. Assembly of the Pandemic Influenza they have co-existed with humanity. 281 These visits and phone calls (for Though these diseases have not Preparedness Framework caused disturbance to the the sharing of influenza viruses been eradicated – overall, a third of participants. The new system uses and access to vaccines and other the world is currently infected with the 289 SMS for users to report symptoms. – they have benefits), and the States Parties’ tuberculosis bacillus Chart and map visualisation of been controlled since the introduction obligations on capacity strengthening the data (with full case details) of antibiotics, and prognostics have contained in the International Health 282 and a fieldwork tracking tool were Regulations of 2005. improved tremendously. But recently developed to help the research a rising number of bacteria have team analyse and monitor data. Of significance was the Report developed antibiotic resistance, due 290 mainly to antibiotic over-prescription of the Review Committee on the 291 – DoctorMe: In addition to all of the Functioning of the International This and use in livestock feed. popular features of DoctorMe (free Nature report highlights the worrying Health Regulations (2005) on the 283 health information for the general which way in which Enterobacteriaceae A(H1N1) 2009 pandemic, public), the tool now features a (bacteria with a 50% mortality concluded: “We were lucky this time, weekly survey for users. The survey rate) have become resistant to but as the report concludes, the world will ask participants to select is ill-prepared to respond to a severe carbapenems, one of the last whether they are experiencing any influenza pandemic or to any similarly remaining antibiotics that had been symptoms from a list. global, sustained and threatening effective against them. public-health emergency.” This is reinforced by the fact that the 2009 pandemic is alleged to have infected 284 24% of the population. The main lesson the WHO drew from that epidemic was that member states generally had communication issues (between ministries of health and decision,makers, and with the public), and were prepared for a pandemic of high severity and appeared unable to adapt their national and subnational responses adequately to a more moderate event. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 88

91 3.1 Current risks 22-Sep-13: Research hints at This report uses an agent-based model – ILI Surveillance, Bureau of 293 possibility for universal flu vaccine Epidemiology Thailand: The old to analyse whether the accidental – Research system was web-based and had laboratory release of pandemic flu no visual element. The new mobile viruses could be contained, and The Spanish flu outbreak was the application and website provides a concludes that controllability of escape deadliest short pandemic in history, events is not guaranteed. map visualisation for the reported infecting about a third of the world cases of influenza-like illness (ILI) in population (≈ 500 million people) Thailand. The map shows hospital 294 and killing 50-100 million people. 3-Dec-13: Global pandemic tops poll ILI cases with colour-coded pins to 306 There have been numerous flu of insurance industry risks indicate the level of ILI and allows pandemics in the last few centuries, for simple analysis of the situation. – Initiative with three others having around a 307 million casualties (the 1889-1890 – Mae Tao Clinic: The electronic Academics and governmental / 295 308 the 1957-1958 Russian Flu, supra-governmental organisations records for this healthcare clinic were Asian Flu, and the 1968-1969 very basic. During EpiHack, the data have long worried about the risks of 296 outbreaks). The Hong Kong Flu pandemics. But such organisations was moved to the cloud and is now most recent pandemic was that in open-source. A data visualisation attract certain types of people with 2009, which killed 150,000-500,000 dashboard was created to allow for specific outlooks, who can be subject 297 people. Thus any move towards map visualisation of diagnoses. The to further biases because of their a universal flu vaccine would be staff at Mae Tao Clinic can now easily profession and the social milieu 309 of great importance to combating Insurers come from surrounding it. view and analyse the data to spot such recurring pandemics. This a different background, focusing on trends and send alerts. They plan to paper, analysing the role of T cells in practical profitability in the business pilot this programme at their clinic combating influenza, suggests a way world. It is therefore instructive that and, if successful, to replicate it with that such a vaccine could be feasible. they too see pandemics as among other clinics. the major threats in the world today. This also implies that combating – Verboice: The technology platform of 28-Nov-13: Difficulties in containing pandemics is of use not only from Verboice is so user-friendly the accidental laboratory escape of a humanitarian but also from an it doesn’t require technical 298 potential pandemic influenza viruses economic standpoint. developers to develop the systems. – Research At EpiHack, project managers were able to design and create systems Biosafety laboratories experiment to address needs in their work with some of the deadliest of the completely on their own. In just world’s pathogens, and occasionally eight hours, four project managers 299 Their number create new ones. each completed their own voice- is increasing globally, and their based participatory surveillance safety record is far from perfect, systems to monitor One Health with several pathogen leaks in Kenya and Tanzania; early 300 301 and others suspected reported warning generation in South Sudan; (the last smallpox fatality was due animal health in Laos; unexploded 302 , to a virus that escaped a lab ordnance in Laos; child trafficking after eradication of the virus in the in Cambodia. The project owners wild). The rate of pathogen escape of these new systems will now has been estimated at 0.3% per take them back to their countries 303 – a very high laboratory, per year and develop implementation and 304 probability, given the 44 BSL-4 sustainability plans. labs and several thousands of BSL-3 labs. There have already been three known escapes from BSL-4 labs 305 since 1990. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 89

92 3.1 Current risks 3.1 Current risks 3.1.5 Global System Collapse Global System Major Asteroid Global Artificial Future Bad Extreme Collapse Impact Pandemic Intelligence Global Governance Climate Change Global system collapse is defined here as either an economic or societal collapse on the global scale. There is no precise definition of a system collapse. Ecological Unknown Synthetic Super-volcano Nanotechnology Nuclear War The term has been used to describe a broad range Catastrophe Biology Consequences of bad economic conditions, ranging from a severe, prolonged depression with high bankruptcy rates and high unemployment, to a breakdown in normal commerce caused by hyperinflation, or even an economically-caused sharp increase in the death rate 310 and perhaps even a decline in population. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 90

93 3.1 Current risks 327 reasons. This trend is likely to be intensified Often economic collapse is And institutional collapses 320 by continuing globalisation, accompanied by social chaos, civil while can create knock-on effects, such as the descent of formerly prosperous unrest and sometimes a breakdown of global governance and regulatory law and order. Societal collapse usually mechanisms seem inadequate to states to much more impoverished 321 328 refers to the fall or disintegration of Such and destabilising entities. This is possibly address the issue. processes could trigger damage on because the tension between human societies, often along with their 322 resilience and efficiency a large scale if they weaken global life support systems. It broadly includes can even 323 political and economic systems to such both quite abrupt societal failures typified exacerbate the problem. an extent that secondary effects (such by collapses, and more extended as conflict or starvation) could cause gradual declines of superpowers. Here Many triggers could start such a failure only the former is included. great death and suffering. cascade, such as the infrastructure damage wrought by a coronal mass 324 ejection, an ongoing cyber conflict, 3.1.5.2 Probability 3.1.5.1 Expected impact or a milder form of some of the risks disaggregation presented in the rest of the paper. Indeed the main risk factor with global The world economic and political systems collapse is as something system is made up of many actors Five important factors in estimating with many objectives and many which may exacerbate some of the probabilities of various impacts: the other risks in this paper, or as a links between them. Such intricate, interconnected systems are subject trigger. But a simple global systems 1. Whether global system collapse to unexpected system-wide failures will trigger subsequent collapses collapse still poses risks on its own. 311 The productivity of modern societies or fragility in other areas. due to the structure of the network – even if each component of the is largely dependent on the careful 2. is the true trade-off What 325 network is reliable. This gives rise to between efficiency and resilienc matching of different types of capital e. (social, technological, natural...) systemic risk: systemic risk occurs 3. Whether effective regulation and when parts that individually may with each other. If this matching is . resilience can be developed function well become vulnerable disrupted, this could trigger a “social 4. Whether an external disruption when connected as a system to a collapse” far out of proportion to . will trigger a collapse 326 self-reinforcing joint risk that can the initial disruption. States and 5. Whether an internal event will spread from part to part (contagion), institutions have collapsed in the past trigger a collapse . for seemingly minor systemic potentially affecting the entire system and possibly spilling over to related 312 Such effects outside systems. have been observed in such diverse 314 313 areas as ecology, finance and 315 (such as critical infrastructure power grids). They are characterised by the possibility that a small internal or external disruption could cause a 316 including highly non-linear effect, a cascading failure that infects the 317 as in the 2008-2009 whole system, financial crisis. The possibility of collapse becomes more acute when several independent networks depend on each other, as is increasingly the case (water supply, transport, fuel and power stations 318 are strongly coupled, for instance). This dependence links social and 319 technological systems as well. Example of an interconnected network: the Internet. Each line is drawn between two nodes, representing two IP addresses. This is a small look at the backbone of the Internet. Source: http://en.wikipedia.org/wiki/File:Internet_map_1024.jpg Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 91

94 GOVERNANCE DISASTERS 3.1 Current risks GOVERNANCE DISASTERS Deliberate Global Technological attempts to Global instability Global povety coordination innovations construct world dictatorship Deliberate Global Technological attempts to Global povety Global instability coordination innovations construct world dictatorship Meta-uncertainty on tradeoffs New system between e.g. Smart sensors of governance poverty, survival, Meta-uncertainty freedom on tradeoffs New system between e.g. Smart sensors of governance poverty, survival, freedom Improvements to Making Failing to solve things worse global governance important problems Making Improvements to Failing to solve things worse global governance important problems Not achieving Lack of human Global important Climate change Enduring poverty flourishing pollution ethical goals Not achieving Lack of human Global important Enduring poverty Climate change flourishing pollution ethical goals Undesirable world system (e.g. global dictatorship) Undesirable world system (e.g. global dictatorship) Disruption to Collapse of Long-term Post-disaster world politics world system negative effects politics and economy Disruption to Collapse of Long-term Post-disaster world politics world system negative effects politics and economy General mitigation Civilisation Total short-term Extinction effort collapse casualties General mitigation Civilisation Total short-term Extinction effort collapse casualties Key Current Meta-uncertainties Uncertain events Risk events Direct impacts Indirect impacts Bad decisions Accidents Severe impacts intervention areas Key Current Uncertain events Indirect impacts Meta-uncertainties Direct impacts Risk events Severe impacts Accidents Bad decisions intervention areas Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 92

95 3.1 Current risks 1. Increased global coordination 18. The world’s political and legal 8. The system collapse may lead to increased fragility in areas that it does and cooperation may allow effective systems are becoming more closely not directly damage, making them regulatory responses, but it also integrated as well. Any risk has not vulnerable to subsequent shocks. causes the integration of many been extensively researched yet, and different aspects of today’s world, there remain strong obstacles (mainly likely increasing systemic risk. 9. A collapse that spread to at the nation state level) slowing down this form of integration. government institutions would 2. Systemic risk is only gradually undermine the possibilities of 19. The politics of the post-system becoming understood, and further combating the collapse. collapse world will be important in research is needed, especially formulating an effective response when it comes to actually reducing 10. A natural ecosystem collapse systemic risk. could be a cause or consequence of a instead of an indifferent or counter- productive one. collapse in humanity’s institutions. 3. Since systemic risk is risk in the 11. Economic collapse is an obvious 20. System collapses can be triggered entire system, rather than in any individual component of it, only and visible way in which system internally by very small events, without institutions with overall views and an apparent cause. collapse could cause a lot of damage. effects can tackle it. But regulating 21. External disruptions can trigger the 12. In order to cause mass casualties, systemic risk is a new and a system collapse would need to uncertain task. collapse of an already fragile system. cause major disruptions to the world’s political and economic system. 4. Building resilience – the ability of 22. The trade-off between efficiency and resilience is a key source of system components to survive shocks fragility in a world economy built – should reduce systemic risk. 13. If the current world system collapses, there is a risk of casualties around maximising efficiency. 5. Fragile systems are often built through loss of trade, poverty, wars and increased fragility. 23. Climate change, mass because they are more efficient than robust systems, and hence movements of animals and agricultural mono-cultures are 14. It is not obvious that the world’s more profitable. institutions and systems can be put interlinking ecosystems with each together again after a collapse; they may other and with human institutions. 6. General mitigation efforts should be stuck in a suboptimal equilibrium. involve features that are disconnected 24. There is a lot of uncertainty from the standard system, and thus should remain able to continue being about systemic risk, especially in 15. Power grids are often analysed of use if the main system collapses as possible candidates for system the interactions between different fragilities that would not be sufficient collapse, and they are becoming 7. A system collapse could spread more integrated. to cause a collapse on their own. to other areas, infecting previously 16. The world’s financial systems untouched systems (as the sub- have already caused a system prime mortgage crisis affected the collapse, and they are still growing world financial system, economy, and more integrated. ultimately its political system). 17. The world’s economies are also getting integrated, spreading recessions across national boundaries. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 93

96 3.1 Current risks 14-Mar-13: Systemic sovereign credit It will bring together experts from 3.1.5.3 Main events risk has “deep roots in the flows and finance, economics, computer during 2013 330 liquidity of financial markets.” science, political science, law and the – Research natural and mathematical sciences. 16-Jan-13: Systemic Risk Centre This will allow researchers affiliated to 329 It is important to estimate the source the Centre to investigate how risk is founded at the LSE of systemic risk. Different mitigation created through feedback loops within – Event policies should be implemented and between the financial, economic, if sovereign systemic risks spring Effective interventions into systemic legal and political systems. Political from financial markets rather than risks depend on high quality research, decisions, for example, can directly macroeconomic fundamentals. This which may be why the London affect people’s behaviour in the paper argues that systemic sovereign financial markets, which in turn affects School of Economics (LSE) founded risks spring from financial markets political decision-making and so on – a £5 million research centre to study (through capital flows, funding systemic financial risk. A press with the outcomes being unexpected availability, risk premiums, and release said: and complex.” 331 liquidity shocks ) rather than from 332 It further estimates fundamentals. Besides the research results produced “The Centre will undertake an that systemic risks are three times by the centre, its very existence shows economic analysis of the fundamental larger in eurozone countries than in that systemic risk is being taken risks to the financial system, based on US states. seriously in academic quarters. an interdisciplinary approach. Figure 19: Network Diagram of connections between, banks, brokers/dealers, insurers and hedge funds. Jan 1994-Dec 1996 Source: https://app.box.com/shared/oesro8zzco0mtvuymh3f Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 94

97 3.1 Current risks 335 17-Jul-13 IMF launches “Systemic The paper concludes, however, that rely – Fundamentals-based models Risk Monitoring (“SysMo”) Toolkit—A the systemic risk monitoring toolkit on macroeconomic or balance sheet 333 User Guide” is incomplete and that “tools exist data to help assess macro-financial – Policy to assess most sectors and levels of linkages (e.g., macro stress testing aggregation, but they provide only or network models). By providing In order to mitigate or prevent partial coverage of potential risks and vulnerability measures based on systemic risk, it needs to be actual interconnectedness and only tentative signals on the likelihood monitored. In this paper, the exposures, these models may help and impact of systemic risk events. As authors set out to clarify the nature build a realistic “story”. However, such, they may not provide sufficient and use of the systemic risk they often require long-term data comfort to policymakers.” monitoring tools that are currently series, assume that parameters available, providing guidance on and relationships are stable under how to select the best set of tools stressed conditions, and produce 23-Dec-13: Citigroup analysis reports depending on the circumstances. only low-frequency risk estimates. reduced systemic political and 338 The paper breaks down the tools financial risks in 2013 and 2014 336 into four categories, each with their – Market-based models. These – Initiative strengths and weaknesses: models uncover information Tracking the ebb and flow of about risks from high-frequency 334 – Single risk/soundness indicators. the likelihood of various risks is market data and are thus suitable Indicators based on balance sheet for tracking rapidly-changing important for estimating where best data, such as financial soundness conditions of a firm or sector. to direct energy and resources. Even indicators (FSIs), are widely These approaches are more approximate, order of magnitude available and cover many risk estimates are sufficient if they dynamic, but their capacity to dimensions. However, they tend to reliably predict financial stress has establish that some risks are much be backward-looking and do not more dangerous than others (order yet to be firmly established. account for probabilities of default of magnitude estimates correspond 339 337 or correlation structures. Moreover, These to the “Class 5 cost estimate”, – Hybrid, structural models. only some of these indicators models estimate the impact undertaken at the very beginning of can be used as early-warning of shocks on key financial and the project, between 0% and 2% of tools (e.g., indicators of funding its completion). In 2013, Citigroup real variables (e.g., default structures). Market data can be probabilities, or credit growth) by analysts predicted that (with caveats) used to construct complementary integrating balance sheet data and systemic risks would recede in indicators for higher-frequency market prices. Examples include Europe during the year, a prediction risk monitoring. the CCA and distance-to-default which seems to have been vindicated measures, which compare the by events. As for the future, Tina market value of an entity‘s assets Fordham, chief global political analyst at Citigroup Global Markets, predicted to its debt obligations. that “systemic political risks will decline in 2014, but country-level and geopolitical risks remain significant.” It seems positive both that market analysts are tracking systemic risks and that they see them as decreasing (though their focus is mainly on political and financial systemic risks). Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 95

98 3.2 Exogenic risks 3.2 Exogenic risks 3.2.1 Major Asteroid Impact Global System Major Asteroid Global Artificial Future Bad Extreme Collapse Impact Pandemic Intelligence Global Governance Climate Change When large objects strike terrestrial planets like the Earth, there can be significant physical and biospheric consequences, though Ecological Synthetic Super-volcano Unknown Nanotechnology Nuclear War atmospheres mitigate many surface impacts Catastrophe Consequences Biology by slowing an object’s entry. Impact structures are dominant landforms on many of the solar system’s solid objects and present the strongest empirical evidence 340 for their frequency and scale. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 96

99 3.2 Exogenic risks Should an impact occur, though, 3.2.1.2 Probability 3.2.1.1 Expected impact asteroid impact risks are similar to disaggregation those of super-volcanoes, in that the main destruction will not be wrought Asteroids have caused significant extinction events throughout the Five important factors in by the initial impact, but by the clouds estimating the probabilities and of dust projected into the upper Earth’s history. The most famous is the impacts of the challenge: Chicxulub impactor, which probably atmosphere. The damage from such an “impact winter” could affect the helped cause the extinction of the climate, damage the biosphere, affect 1. Whether detection and tracking of non-avian dinosaurs and more than 341 Large asteroid 75% of all species. food supplies, and create political asteroids and other dangerous space objects instability. Though humanity currently is sufficiently collisions – objects 5 km or more in . produces enough food to feed all exhaustive size – happen approximately once 352 2. How feasible it is to deflect humans, this supply is distributed every twenty million years and would extremely unevenly, and starvation still have an energy a hundred thousand an asteroid . 342 times greater exists. Therefore a disruption that is than the largest bomb 3. Whether measures such as 343 evacuation could reduce the A land impact ever detonated. small in an absolute sense could still of an impact. would destroy an area the size of a cause mass starvation in the future. damage 344 The short- and long-term climate nation like Holland. 4. Mass starvation, mass migration, Larger asteroids could be extinction level events. political instability and wars could consequences of a collision. civilisation 5. Whether our current be triggered, possibly leading to a could adapt to a post-impact world Asteroid impacts are probably one . civilisation collapse. Unless the impact of the best understood of all risks is at the extreme end of the damage scale and makes the planet unviable, in this report. Their mechanisms human extinction is possible only as and frequencies are reasonably well 345 Recent ground- and a consequence of civilisation collapse estimated. 353 346 space-based tracking projects and subsequent shocks. have been cataloguing and tracking 347 the largest asteroids, and have discovered that the risks were lower 348 than was previously feared. The projects are now cataloguing asteroids of smaller size and damage potential. There has been some speculation about possible methods for deflecting 350 , should they be found on a asteroids collision course with the planet. Such means remain speculative, currently, but may become more feasible given technological progress and potentially 351 more affordable access to space. 349 Figure 20: How the Spaceguard Survey has reduced the short-term risk of impacts from near-Earth objects Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 97

100 3.2 Exogenic risks MAJOR ASTEROID IMPACT GOVERNANCE DISASTERS Meta-uncertainty of Easier access Private space anthropic effects to space programs upon the true risk probability National space programs Deliberate Global Technological attempts to Global povety Global instability coordination innovations construct world dictatorship Making the risk Asteroid deflection Meta-uncertainty technologies a priority on tradeoffs New system between e.g. Smart sensors of governance poverty, survival, freedom Asteroid dection Small asteroid Asteroid impact and tracking impacts Making Improvements to Failing to solve things worse global governance Disruption to important problems Global Preemptive Long-term world politics Climate impacts coordination evacuation climate impact and economy Post-impact politics Not achieving Lack of human Global important Climate change Enduring poverty flourishing pollution ethical goals Civilisation Total short-term Direct destruction Extinction collapse casuaities Undesirable world system (e.g. global dictatorship) Disruption to Collapse of Long-term Post-disaster General mitigation world politics world system negative effects politics efforts and economy General mitigation Civilisation Total short-term Extinction effort collapse casualties Key Key Current Meta-uncertainties Direct impacts Indirect impacts Uncertain events Risk events Bad decisions Severe impacts Accidents intervention areas Current Meta-uncertainties Risk events Indirect impacts Direct impacts Uncertain events Accidents Severe impacts Bad decisions intervention areas Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 98

101 3.2 Exogenic risks 1. Competition between private 12. There are currently no asteroid 6. General mitigation efforts may deflection abilities, but there are space programmes could be a help reduce the direct and indirect many plans that could conceivably be determining factor in reducing the negative outcomes of an impact by, implemented in due course. for instance, equipping people to deal cost of space flight. with the changed climate. 2. National space programmes have 13. Small asteroid impacts could motivate 7. Unlike many risks, there is no upper always provided the impetus for space increased anti-asteroid precautions. flight projects, especially the more bound on how destructive an asteroid 14. With enough warning, it could be impact could be, though the largest speculative and cutting-edge ones. impacts are the rarest. possible to preemptively evacuate the impact area. 3. Protecting against asteroid impacts 8. The aftermath of an impact could is already accepted as a project worth funding, but increased focus on the greatly disrupt the world economic 15. Post-impact politics will be problem could increase the ability to important for reconstruction, and political system. predict and prevent such impacts. adapting to the changed climate, and 9. Climate changes would be the most prevention of further harm. destructive consequences of medium- 4. Asteroid detection and tracking 16. Estimating the likelihood of scale meteor impacts, with the world continues to progress well currently, plunged into an “impact winter”. asteroid impacts suffers from and is key to preventing such 355 we “anthropic shadow” effects: collisions in future. may be underestimating the danger 10. The effects of an impact winter because if there had been many more 5. Better global coordination is not could last for a long time. impacts in recent times, humans strongly needed to track or deflect would not currently be around to 11. Easier access to space would be asteroids, but would be important if a observe their effects and take them large-scale evacuation was needed. important for any plans to actually into account. deflect an asteroid. Source: http://en.wikipedia.org/wiki/File:Chelyabinsk_meteor_trace_15-02-2013.jpg Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 99

102 3.2 Exogenic risks 14-Nov-13: Risk of medium asteroid 17-Oct-13: The probability for 3.2.1.3 Main events strike may be ten times larger than “Asteroid 2013 TV135” to impact during 2013 367 363 previously thought Earth in 2032 is one in 63,000 – Research – Event 15-Feb-13: Chelyabinsk meteor 356 This paper analyses in detail the NASA reports that a 400-metre causes large fireball Chelyabinsk impact, estimated to have – Event asteroid has one chance in 63,000 had an energy of 500 kilotonnes of of impacting the Earth. An asteroid TNT. It demonstrates problems with this size would produce ocean- The Chelyabinsk meteor was a the standard methods for estimating near-Earth asteroid that entered wide tsunamis or destroy land areas the energy of collisions – derived from Earth’s atmosphere over Russia, with the size of a small state (Delaware, 368 364 – and nuclear weapons results an estimated speed of 18.6 km/s, Estonia). For comparison, the odds from that deduces that the number of of dying from lightning strike are 1 almost 60 times the speed of sound. impactors with diameters of tens of in 83,930, of a snake, bee or other It exploded into a very visible air burst metres may be an order of magnitude venomous bite or sting is 1 in 100,000, over Chelyabinsk Oblast, which was higher than estimated. It argues that this of an earthquake 1 in 131,890, and of recorded by numerous video sources. 365 demonstrates a deviation from a simple a dog attack 1 in 147,717. So the The meteor was undetected before it power law, and thus that there is a non- risk of asteroid death, though low, is entered the Earth’s atmosphere, and equilibrium in the near-Earth asteroid caused numerous injuries, extensive comparable to more common risks. population for objects 10 to 50 metres damage, but no deaths. It was the in diameter. This shifts more of the 28-Oct-13: United Nations to Adopt largest to crash to Earth since 1908, 357 366 impact risk to asteroids of these sizes. when an object hit Tunguska in Asteroid Defence Plan 358 – Policy The meteor seemed ideal Siberia. 3-Dec-13: SpaceX launches into from the risk reduction perspective: 369 geostationary orbit a large, visible impact that attracted The UN plans to set up an – Initiative International Asteroid Warning great attention, and a renewed 359 Group for member nations to commitment to asteroid precautions, Easy access to space is important for all share information about potentially but no actual fatalities. 370 asteroid deflection proposals. Since hazardous space rocks. If 371 America retired the Space Shuttle, astronomers detect an asteroid that 19-Jun-13: Space Research it has been putting its hope in private Institute of Russian Academy of poses a threat to Earth, the UN’s 372 space companies. The success Science presents a strategy to Committee on the Peaceful Uses of of SpaceX opens the possibility of use small asteroids to deflect Outer Space will help coordinate a eventual cheaper access to space. hazardous objects from the mission to launch a spacecraft to slam 360 trajectory of collision with Earth into the object and deflect it from its – Research collision course. This marks the first time an Though the analysis and tracking of 361 international body has assigned asteroids has progressed rapidly, responsibility for tracking and methods for deflecting a dangerous intercepting dangerous asteroids. asteroid, should it be detected, remain 362 speculative. The Space Research Institute of the Russian Academy of Science introduces another approach: selecting small (10-15m) near-Earth asteroids and causing them to strike a larger dangerous one, altering its trajectory. The more suggestions and ideas there are for such deflections, the more likely it is that one of them will yield an implementable approach. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 100

103 3.2 Exogenic risks Yield NEO Crater Average Consequences diameter Megatonnes diameter interval b/w (MT*) (km) impact (years) <10 Upper atmosphere detonation of “stones” (stony asteroids) and comets; only “irons” (iron asteroids) <3% penetrate to surface. 10 to 100 75m 1.5 1,000 Irons make craters (Barringer Crater); Stones produce air-bursts (Tunguska). Land impacts could destroy areas the size of a city (Washington, London, Moscow). 160m 100 to 1,000 3 4,000 Irons and stones produce ground-bursts; comets - produce air-bursts. Ocean impacts produce signifi cant tsunamis. Land impacts destroy areas the size of a large urban area (New York, Tokyo). 1,000 350m 6 16,000 Impacts on land produce craters; ocean-wide tsunamis to are produced by ocean impacts. Land impacts destroy 10,000 areas the size of a small state (Delaware, Estonia). 10,000 700m 12 63,000 Tsunamis reach hemispheric scales, exceed damage to from land impacts. Land impacts destroy areas the 100,000 size of a moderate state (Virginia, Taiwan). 1.7km 100,000 30 250,000 Both land and ocean impacts raise enough dust to to affect climate, freeze crops. Ocean impacts generate 1 million global scale tsunamis. Global destruction of ozone. Land impacts destroy areas the size of a large state (California, France, Japan). A 30 kilometre crater pen- etrates through all but the deepest ocean depths. 3km 1 million 60 1 million Both land and ocean impacts raise dust, change to climate. Impact ejecta are global, triggering wide- 10 million spread fires. Land impacts destroy areas the size of a large nation (Mexico, India). 7km 10 million 125 10 million Prolonged climate effects, global conflagration, to probable mass extinction. Direct destruction ap- 100 million proaches continental scale (Australia, Europe, Usa). 100 million 16km 250 100 million Large mass extinction (for example K/T or Creta- to ceous-Tertiary geological boundary). 1 billion >1 billion Threatens survival of all advanced life forms. 354 Figure 15: Impact effects by size of Near Earth Object Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 101

104 Global System Major Asteroid Global Artificial Future Bad Extreme Collapse Impact Pandemic Intelligence Global Governance Climate Change 3.2 Exogenic risks 3.2 Exogenic risks 3.2.2 Super-volcano Ecological Unknown Synthetic Super-volcano Nanotechnology Nuclear War Catastrophe Consequences Biology A super-volcano is any volcano capable of producing an eruption with an ejecta volume 3 . This is thousands of times greater than 1,000 km larger than normal eruptions. Super-volcanoes can occur when magma in the mantle rises into the crust from a hotspot but is unable to break through it, so that pressure builds in a large and growing magma pool until the crust 373 is unable to contain the pressure. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 102

105 3.2 Exogenic risks Unless the eruption is at the extreme end The extent of the impact would 3.2.2.1 Expected impact of the damage scale and makes the planet thus depend on the severity of the unviable, human extinction is possible only eruption - which might or might not be foreseen, depending on improvements as a consequence of civilisation collapse The eruption which formed the 384 387 Siberian Traps was one of the largest and subsequent shocks. - and the in volcanic predictions subsequent policy response. Another in history. It was immediately followed Prof. Michael Rampino, New York by the most severe wave of extinction Siberian Trap-like eruption is extremely 374 in the planet’s history, University, has estimated that a large the Permian– unlikely on human timescales, but the 375 where Triassic extinction event, (1,000 cubic kilometres of magma) damage from even a smaller eruption 96% of all marine species and 70% of super-eruption would have global could affect the climate, damage the biosphere, affect food supplies and effects comparable to an object 1.5km terrestrial vertebrate species died out. 388 in diameter striking the Earth. create political instability. Recent research has provided evidence of a causal link: that the eruption 376 A report by a Geological Society There caused the mass extinction. 3.2.2.2 Probability of London working group notes: have been many other super-volcanic disaggregation 377 “Although at present there is no The eruptions throughout history. return period for the largest super- technical fix for averting super- Five important factors in volcanoes (those with a Volcanic eruptions, improved monitoring, 378 awareness-raising and research-based of 8 or above) has Explosivity Index estimating the probabilities and 379 impacts of the challenge: planning would reduce the suffering of been estimated from 30,000 years 385 at the low end, to 45,000 or even many millions of people.” 380 at the high end. 700,000 years 1. Whether countries will coordinate Though humanity currently produces against super-volcano risk globally 386 enough food to feed everyone, Many aspects of super-volcanic activity and damage. are not well understood as there have this supply is distributed extremely 2. The predictability of super- unevenly, and starvation still exists. been no historical precedents, and volcanic eruptions. Therefore a disruption that is small in such eruptions must be reconstructed 3. How directly destructive an 381 an absolute sense could still cause from their deposits. eruption would be . mass starvation. Mass starvation, mass 4. The effectiveness of general The danger from super-volcanoes migration, political instability and wars mitigation efforts . could be triggered, possibly leading to is the amount of aerosols and dust How severe the long-term 5. projected into the upper atmosphere. a civilisation collapse. would be. climate effects This dust would absorb the Sun’s rays and cause a global volcanic winter. The Mt Pinatubo eruption of 1991 caused an average global cooling of surface temperatures by 0.5°C over three years, while the Toba eruption around 70,000 years ago is thought by some to have cooled global 382 temperatures for over two centuries. The effect of these eruptions could be best compared with that of a nuclear war. The eruption would be more 383 violent than the nuclear explosions, but would be less likely to ignite firestorms and other secondary effects. Unlike nuclear weapons, a super-volcano would not be targeted, leaving most of the world’s infrastructure intact. Satellite image of Lake Toba Source: NASA, http://en.wikipedia.org/wiki/File:Toba_overview.jpg Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 103

106 SUPER VOLCANO 3.2 Exogenic risks GOVERNANCE DISASTERS Making the risk Medium volcanic eruptions a priority Supervolcano Predictability research of eruptions Deliberate Global Technological attempts to Global instability Global povety coordination innovations construct world dictatorship Meta-uncertainty on tradeoffs New system between e.g. Smart sensors Better monitoring Supervolcano of governance poverty, survival, and prediction eruption freedom Preemptive Global Long-term Direct destruction Climate impacts evacuation coordination climate impact Making Improvements to Failing to solve things worse global governance important problems Disruption to world politics and economy Not achieving Lack of human Global Meta-uncertainty important Climate change Enduring poverty flourishing pollution Post-eruption of anthropic effects ethical goals politics upon the true risk probability Undesirable world system (e.g. global dictatorship) Total short-term Civilisation General Disruption to Extinction Collapse of Long-term Post-disaster collapse casualties mitigation efforts world politics world system negative effects politics and economy General mitigation Civilisation Total short-term Extinction effort collapse casualties Key Current Uncertain events Indirect impacts Risk events Direct impacts Meta-uncertainties Bad decisions Accidents Severe impacts intervention areas Key Current Meta-uncertainties Indirect impacts Risk events Direct impacts Uncertain events Accidents Bad decisions Severe impacts intervention areas Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 104

107 3.2 Exogenic risks 1. Whether super-volcano risk is made 10. Better volcano monitoring 6. A super-volcano’s main destructive impact is through its and prediction (if possible) will a priority will have a large impact on research and volcano monitoring. allow such interventions as pre- effect on the climate, akin to a emptive evacuations. nuclear winter cooling effect. This will strongly affect all impact levels, 2. Further super-volcano research will and the disruption to the world’s 11. Evacuations are likely to be the be important in any mitigation and only effective response to an imminent monitoring efforts. political and economic system. eruption, as super-volcanoes are 3. Global coordination and 7. The level of this disruption will unlikely to be controllable or divertible. determine how well countries cope cooperation between nations will with the aftermath of the eruption and determine research levels, the 12. Post-eruption politics will be a subsequent climate changes, and consequence of the number of short- chances of evacuations, and post- eruption disruption to the world whether subsequent conflicts or trade term casualties, and the disruption to the world system. political and economic system. wars will occur, adding to the damage. 4. General mitigation efforts may help 8. The long-term climate impact will 13. Medium scale volcanic eruptions may persuade leaders to make the determine in what state the post- reduce the direct and indirect negative impact of an eruption, by, for instance, eruption world will find itself, relevant risk more of a priority. both for reconstruction after a collapse equipping people to deal with the 14. Estimating the likelihood of and for preventing such a collapse. changed climate. super-volcanic eruptions suffers from 390 we 9. Whether eruptions are “anthropic shadow” effects: 5. The direct destructive effect may be underestimating the danger of a super-volcano can be fundamentally predictable or because if there had been many more extensive, especially in the area not, and how far in advance, eruptions in recent times, humans will be very important for many around the eruption. would not currently be around to mitigation strategies. observe their effects and take them into account. Classification VEI Ejecta volume Description Frequency Plume 3 Hawaiian < 10,000 m 0 Effusive constant < 100 m 3 Hawaiian / Strombolian 1 > 10,000 m Gentle 100-1000 m daily 3 Strombolian / Vulcanian > 1,000,000 m 2 1-5 km Explosive weekly 3 Vulcanian / Peléan > 10,000,000 m 3 few months Catastrophic 3-15 km 3 Peléan / Plinian > 0.1 km 4 ≥ 1 yr Cataclysmic 10-25 km 3 Plinian > 1 km 5 Paroxysmic 20-35 km ≥ 10 yrs 3 Plinian / Ultra-Plinian > 10 km 6 ≥ 100 yrs Colossal > 30 km 3 Ultra-Plinian > 100 km 7 ≥ 1,000 yrs Mega-colossal > 40 km 3 Supervolcanic > 1,000 km 8 > 50 km ≥ 10,000 yrs Apocalyptic 389 Figure 21: Volcanic Explosivity Index Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 105

108 3.2 Exogenic risks This is due to the normalcy bias, They put the risk of super-volcanic 3.2.2.3 Main events “the tendency to minimise the eruptions in the context of standard during 2013 volcanic eruptions, just on a larger probability of potential threats or 396 scale (noting that super-volcanic their dangerous implications,”. eruptions have affected Japan in 15-Mar-13: Climate impact of the past). Japan has been a very super-volcanoes may be less than 391 27-Oct-13: Yellowstone super- previously thought seismically active country for its entire 394 history, – Research so it might be hoped that volcano larger than previously 397 adequate volcanic mitigation measures thought would have been implemented. The Toba eruption around 70,000 – Research years ago was one of the world’s largest super-volcanic eruptions. In But the report notes that “remarkably Another continuing development in the science of super-volcanoes, this few [of Japan’s local governments] contrast with some theories that claim paper demonstrates that the crustal it caused a volcanic winter that may have drafted volcanic disaster 392 395 magma reservoir under Yellowstone countermeasure[s]”, adding that have lasted over two centuries, this paper claims that analysis of ash was 50% larger than was previously “Local governments that have actually experienced a volcanic disaster focus thought. However, despite this from the Toba super-eruption in Lake Malawi shows no evidence of volcanic attention on volcanic disaster-related increase, integrated probabilistic winter in East Africa. This further discussion, but most have not drafted hazard assessment shows that the illustrates the difficulty of establishing biggest Yellowstone Plateau threat specific procedures for volcanic is from large M7+ earthquakes - the exact impact of large-scale disasters and seem to think that the 398 , but very general disaster countermeasure disasters when the evidence record is significantly damaging unlikely to threaten billions - not from volume is adequate.” This provokes poor. volcanic or super-volcanic eruptions. some pessimism about the likelihood of effective planetary super-volcano mitigation measures being implemented, 17-Jul-13: The Volcanological Society of Japan looks at volcano especially in those areas with no 393 and super-volcano mitigation direct experience of volcanic risk. – Policy Prevention of super-volcano eruptions Figure 22: Location of Yellowstone hotspot over time (numbers indicate millions of years before the present). is impossible with current technology, Source: http://en.wikipedia.org/wiki/File:HotspotsSRP.jpg but there may be some possibility of mitigating their effects. The Volcanological Society of Japan is one of the few organisations that have looked at such potential mitigation. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 106

109 3.2 Exogenic risks 15-Nov-13: Insurance executives 20-Dec-13: Super-volcano confirmed as responsible for one of the largest rank super-volcanoes low on the list 399 401 of extreme risks extinctions in history – Research – Initiative Academics have long worried about The maximal destructive potential of super-volcanoes is uncertain. the probability of super-volcanic eruptions. But academia attracts There have been large super- volcanic eruptions throughout certain types of people with specific 402 and many extinction history, outlooks, who can be subject to events, but uncertainties in the further biases because of their geological record mean that it was profession and the social milieu 400 hard to establish whether they surrounding it. Insurers come from a different background, focusing on were causally linked. One eloquent practical profitability in the business example was the eruption which 403 (one world and using a relatively short formed the Siberian Traps of the largest in history), and the time horizon. So it is instructive that 404 they do not see super-volcanoes as Permian–Triassic extinction, where 96% of all marine species a major threat in the world today: “Of interest to us is the very low ranking and 70% of terrestrial vertebrates of the user-submitted idea of super- died out. The two events were close on the geological timeline, volcanoes in the US”. and this paper, using recent dating techniques, confirmed that the super-volcano erupted shortly before the extinction, making it the likely culprit. The risk of large impacts from super-volcanoes has thus gained in plausibility. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 107

110 Global System Major Asteroid Global Artificial Future Bad Extreme Collapse Impact Pandemic Intelligence Global Governance Climate Change 3.3 Emerging risks 3.3 Emerging risks 3.3.1 Synthetic Biology Ecological Synthetic Super-volcano Unknown Nuclear War Nanotechnology Catastrophe Consequences Biology Synthetic biology is the design and construction of biological devices and systems for useful purposes. It is an area of biological research and technology that combines biology and engineering, and so often overlaps with bioengineering and biomedical engineering. It encompasses a variety of different approaches, methodologies, and disciplines with a focus 405 on engineering biology and biotechnology. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 108

111 3.3 Emerging risks 423 but Extinction risk is unlikely, This could emerge through military 3.3.1.1 Expected impact 415 possible if the aftermath of the commercial bio-warfare, bio-warfare, 417 416 bio-terrorism (possibly using epidemic fragments and diminishes 418 Pandemics are one of the human society to the extent that developed by dual-use products 424 recovery becomes impossible before worst killers in human history. legitimate researchers, and currently 425 unprotected by international legal humanity succumbs to other risks. Synthetic biology is the design 419 regimes ), or dangerous pathogens and construction of biological 420 leaked from a lab . Of relevance is devices and systems to accomplish the specific goal of whether synthetic biology products 3.3.1.2 Probability 406 adding the synthetic biologist, become integrated into the global disaggregation economy or biosphere. This could human intentionality to traditional pandemic risks. lead to additional vulnerabilities (a benign but widespread synthetic Five important factors in estimating the probabilities and biology product could be specifically The positive and negative potentials 407 targeted as an entry point through impacts of the challenge: of synthetic biology are unclear – much of the information currently which to cause damage). But such a 408 comes from synthetic biologists, 1. The true destructive potential development would lead to greater who may not be able to provide an , especially of synthetic biology industry and academic research, which could allow the creation of impartial overview (the problem is the tail risk. 421 reactive or pre-emptive cures. field will be Whether the exacerbated by the decentralised 2. 409 ). Attempts at , or successfully regulated nature of the field 411 410 are or self-regulation regulation The impact is very similar to that of successfully manage to pandemics: mass casualties and currently in their infancy, and may not regulate itself. 412 develop as fast as research does. subsequent economic and political 3. Whether the field will usher in a instabilities leading to possible new era of bio-warfare . civilisation collapse. A bio-war would One of the most damaging impacts 4. Whether the tools of synthetic from synthetic biology would come contribute greatly to the resulting biology can be used defensively to 413 instability. Even for the most perfectly from an engineered pathogen, . create effective counter measures targeting humans or a crucial engineered pathogen, survivors are The dangers of relying on 5. component of the ecosystem (such likely, if only in isolated or mainly synthetic biologists to estimate the 422 isolated locations. as rice, which accounts for 20% of all danger of synthetic biology. 414 calories consumed by humans). Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 109

112 SYNTHETIC BIOLOGY 3.3 Emerging risks GOVERNANCE DISASTERS Global Knowledge Military research Smart sensors coordination control Deliberate Global Technological attempts to Global instability Global povety coordination innovations construct world dictatorship Meta-uncertainty Continuing Dual use Effective regulatory Knowledge of reying on synthetic biology research framework leak synthetic biologists research for risk estimates Meta-uncertainty on tradeoffs New system between e.g. Smart sensors of governance poverty, survival, freedom Commercially Meta-uncertainty Pathogen Small of true potential targeted damaging Bio-terrorism research security scares of synthetic biology synthetic biology Making Improvements to Failing to solve things worse global governance important problems Engineered Long-term ecosystem- Bio-warfare ecosystem impact affecting pathogen Not achieving Lack of human Global important Climate change Enduring poverty flourishing Prevalence of Engineered pollution Accidental New ethical goals synthetic biology human-affecting vulnerabilities release products pathogen Undesirable world system (e.g. global dictatorship) Disruption to Pathogen Post-pathogen Agriculture Direct world politics transmission Extinction impact casualties politics and economy chains Disruption to Collapse of Long-term Post-disaster world politics world system negative effects politics and economy Pre-release Civilsation Total short-term medical collapse casualties contingency plans General mitigation Civilisation Total short-term Extinction effort collapse casualties Key Current Indirect impacts Direct impacts Risk events Meta-uncertainties Uncertain events Severe impacts Accidents Bad decisions intervention areas Key Current Risk events Uncertain events Indirect impacts Direct impacts Meta-uncertainties Severe impacts Accidents Bad decisions intervention areas Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 110

113 3.3 Emerging risks 1. Global coordination and 20. It may be possible to control direct 10. Bio-terrorism has the potential to be the most destructive form of cooperation will be important to pathogen research through regulations terrorism of all, with a small group preventing biowarfare and creating an – certainly more so than dual use causing billions of casualties. effective regulatory framework. products. This kind of research is the most likely to lead to bio-weapons, or 2. Military research in synthetic biology 11. The various products produced by to accidental release. would be a direct risk for creating synthetic biology research could be deadly if accidentally released. 21. If synthetic biology products dangerous bio-weapons. are prevalent, this may introduce new vulnerabilities. 12. It is hard to estimate ahead of 3. Effective and intelligent regulatory frameworks are the great challenge for time, but the direct casualties of an engineered pathogen could potentially controlling synthetic biology risks. The 22. Post-pathogen politics will be important for preventing civilisation include everyone infected, which field is currently self-regulated, and it collapses, and regulating further isn’t clear whether this is sufficient. could include almost everyone alive. synthetic biology experiments 13. The most devastating pathogen and developments. 4. Synthetic biology is novel enough affecting the ecosystem would be one for some parts of the field potentially to be shut down if they are seen to be targeting food production in one form 23. The pathogen transmission or another. chains are important in determining too dangerous: continuing synthetic biology research is not a given. the transmissibility of the pathogens in the human population, and 14. The widespread use of synthetic 5. Of all technological fields, synthetic biology products could introduce new whether quarantine or similar biology could be one requiring vulnerabilities, if these products are methods will be effective. knowledge control: where dangerous specifically targeted. 24. Synthetic biology research knowledge (such as how to synthesise certain pathogens) is kept out of the may enable the construction of 15. Human- or ecosystem-targeting effective preventative measures or public domain. Other dangerous pathogens on a large scale could disrupt the world’s political and countermeasures to an outbreak technologies (e.g. nuclear weapons) (both for a designed pathogen and a require a large project or rare economic system, especially if one party is blamed for their release. materials, and could be regulated at natural one). that level instead. 25. The pathogen transmission chains 16. Natural pathogens are unlikely to have a long-term devastating effect, 6. Mass surveillance and smart are important in determining the but human-designed ones could transmissibility of the pathogens in the sensors may be needed to ensure ecosystem, and the effectiveness of dangerous synthetic biology projects – or they could be upgraded and changed regularly. various countermeasures. are not carried out. 7. Most of the pre-release mitigation 26. Of all risks, those of synthetic 17. Small security scares could biology are the most uncertain: they efforts are similar to those for fighting provide impetus to the development a conventional pandemic. could turn out to be very high, or very of effective regulations. low; it is currently not known. 8. Biowarfare is one major scenario 18. Knowledge leaks (such as in which synthesised biological 27. Active synthetic biologists are genomes published online) could the major source of information on enable bioterrorism if the cost of agents are targeted at humans or at the ecosystem. synthetic biology risks, which calls producing pathogens is low. the impartiality of their estimates 9. Commercial enterprises, especially 19. Much legitimate synthetic biology into question. those exploiting natural resources, research could have dual use for terrorists or as weapons. may be tempted to target their rival’s products with pathogens that may get out of control. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 111

114 3.3 Emerging risks 23-Jan-13: Work resumes on lethal 28-Feb-13: WHO report: Many 3.3.1.3 Main events 430 flu strains countries and institutions lack during 2013 – Event oversight of “dual use” biological research, and there is a lack of global 436 In 2011, scientists working in frameworks on the issue 15-Jan-13: Improved bio-safety in 426 avian flu research performed two – Policy iGEM synthetic biology competition 431 showing how the flu experiments – Initiative virus could be made transmissible to Dual use biological research concerns life sciences research ferrets (and, by extension, humans). A significant part of synthetic intended for benefit, but with results This generated protests and calls for biology is developed by “bio- 427 which might easily be misapplied the papers to remain fully or partially hackers“, small-scale operations 432 because of the unpublished, to produce harm when used by with a hobbyist or competitive potential for misuse by bio-terrorists bio-terrorists or in bio-weapon hacker ethos. This ethos would be 428 or bio-weapons programmes. In research. Examples of these included more attracted to self-regulation the experiments making avian ‘flu response, researchers in the field rather than outside governmental 437 declared a voluntary moratorium in But there transmissible to humans. regulation. But industry self- 433 438 were other examples too, including: A year later, they January 2012. regulations often fail in their decided to lift the moratorium. goals (especially without explicit 429 – Accidentally increasing the virulence sanctions for malfeasance), so it of mousepox as part of an One cannot expect workers in a is currently unclear whether it can experiment to control mice as field to be unbiased about their be relied upon to reduce risk. The 434 439 own research, pests in Australia. so it is significant International Genetically Engineered that this decision was condemned – Variola virus immune Machines (iGEM) competition is one 440 by many scientists, including other evasion design. of the largest in synthetic biology, 435 virologists. This provides strong – Chemical synthesis of and has attempted to promote 441 evidence that ending the moratorium poliovirus cDNA. bio-safety in its participants. It is – Reconstruction of the 1918 was a dangerous decision. significant for the potential of self- 442 flu virus. regulation that such attempts have – Creating and synthesising been partially successful. 443 a minimal organism. As life science techniques develop, there is the potential for more such potentially dual use research in future. Yet, despite these dangers, the WHO reports that many countries and institutions lack oversight of such research, and that there is a lack of global frameworks on the issue. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 112

115 3.3 Emerging risks 01-Jun-13: Scientists create hybrid 444 airborne H5N1 flu – Research The H in H5N1 stands for “hemagglutinin”, as depicted in this molecular model. Permission is granted to copy, distribute and/ or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled GNU Free Documentation Research continues into gain of function (GOF) for different influenza viruses. This report detailed methods for airborne mammal-to-mammal Hemagglutinin molecule Source: http://en.wikipedia.org/wiki/ transmission of the H5N1 flu virus, File:Hemagglutinin_molecule.png when hybridised with a highly transmissible (and human-infective) H1N1 virus. There is a possibility that such viruses (or those created in similar GOF experiments) could become transmissible to humans, and potentially cause a pandemic if they escaped from the lab. A report from the Center for Arms 445 Control and Non-Proliferation applied likelihood-weighted- consequence analysis to estimate the probability and impact of such escapes. It estimated that the risk was considerable: even if a rapid quarantine was instituted, each lab-year of such research carried an expected casualty rate of 180 to 1,100 fatalities, and $2.3 million to $390 million in economic damage. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 113

116 Global System Major Asteroid Global Artificial Future Bad Extreme Collapse Impact Pandemic Intelligence Global Governance Climate Change 3.3 Emerging risks 3.3 Emerging risks 3.3.2 Nanotechnology Ecological Synthetic Unknown Super-volcano Nanotechnology Nuclear War Catastrophe Biology Consequences 446 Nanotechnology, here defined as atomically 447 is the creation of effective, precise manufacturing, high-throughput manufacturing processes that operate at the atomic or molecular level. 448 There are many suggested designs, but there are no immediately available 449 methods to construct them. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 114

117 3.3 Emerging risks Extinction risk is only likely as Some nanotechnology 3.3.2.1 Expected impact pathways could mitigate a long-term consequence of these developments, however. civilisation collapse, if the survivors 456 are unable to rebuild and succumb Cheap mass surveillance, It is currently unclear whether 464 nanotechnology would be a for instance, could catch such to other threats. The possibility re-armament efforts (though revolution in manufacturing, or merely of nanomachines or nanoweapons surveillance could have its own remaining active after a civilisation a continuation of current trends. collapse may make the rebuilding detrimental effects). Many of the Industry represents 30% of world 450 GDP, world’s current problems may be a declining fraction, so from more difficult, however, while the availability of atomically precise solvable with the manufacturing a narrow economic perspective it manufacturing systems, by contrast, could be argued that the impact of possibilities that nanotechnology could aid rebuilding. nanotechnology would be relatively would make possible, such as small. However, nanotechnology could depletion of natural resources, 3.3.2.2 Probability pollution, climate change, clean create new products – such as smart 457 451 water, and even poverty. There – and or extremely resilient materials disaggregation would allow many different groups are currently few applicable international legal regimes or even individuals to manufacture a 458 wide range of things. governing nanotechnology. Five important factors in In the media the label “grey estimating the probabilities and This could lead to the easy 459 is sometimes applied to goo” impacts of the challenge: construction of large arsenals of 452 These nanotechnology. This is meant to weapons by small groups. might be masses of conventional describe a hypothetical situation 1. The timeline for nanotech where special self-replicating weapons (such as drones or cruise development. missiles), or more novel weapons nanomachines would be engineered 2. Which aspects of nanotech to consume the entire environment. made possible by atomically precise research will progress in what order. It is unclear how effective they could manufacturing. If this is combined 3. Whether it will be possible with a possible collapse in world trade be, and they play no role in atomically assemble a for small groups to 453 460 – since manufacturing precise manufacturing. networks Mass self- in a short period weapons arsenal replication would be detectable, and could now be entirely local – there of time. vulnerable to human-directed counter- would be a likely increase in the 4. Whether nanotech tools can be 461 measures. However, it is possible number of conflicts throughout effectively used defensively or for that such replicating machines the world. Of particular relevance surveillance . could endure and thrive in particular is whether nanotechnology allows 5. Whether nanotech tools rapid uranium extraction and isotope ecological niches, where the cost or weaponry are made to be 454 462 The of removing them is too high. separation and the construction of independent of human control . 463 misuse of medical nanotechnology nuclear bombs, which would increase is another risk scenario. the severity of the consequent conflicts. Unlike the strategic stalemate of nuclear weapons, nanotechnology arms races could involve constantly evolving arsenals 455 These and become very unstable. conflicts could lead to mass casualties and potentially to civilisation collapse if the world's political and social systems were too damaged. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 115

118 NANOTECHNOLOGY 3.3 Emerging risks GOVERNANCE DISASTERS Meta-uncertainty: Minor negative Effective Continuing Military nanotech revolution impacts of early regulatory nanotech research nanotech research or evolution, speed framework nanotechnology of implementation? Deliberate Prevalence of Global Technological Meta-uncertainty attempts to Ultra- Global Global povety Global instability Smart sensors nanotechnology coordination innovations of political construct world manufacturing coordination products predictions dictatorship Meta-uncertainty on tradeoffs New system between e.g. Smart sensors Breakdown of governance New vulnerabilities New vulnerabilities poverty, survival, of trade freedom International relations Making Improvements to Failing to solve things worse global governance important problems Nano-terrorism AI Nano-warfare Loss of control over aggressive Not achieving Lack of human Global nanotechnology important Climate change Enduring poverty flourishing pollution ethical goals Undesirable Large-scale Long-term Uncontrolled Post-nanotech world system nano-war nano-ecology ecosystem impact politics (e.g. global dictatorship) Disruption to Collapse of Long-term Post-disaster world politics world system negative effects politics and economy Disruption to Damage to the Direct Effectiveness of Direct world politics world’s ecosystem casualties countermeasures casualties and economy and agriculture General Total short- Civilsation General mitigation Civilisation Total short-term Extinction Extinction mitigation effort term casualties collapse effort collapse casualties Key Current Meta-uncertainties Uncertain events Indirect impacts Direct impacts Risk events Bad decisions Accidents Severe impacts intervention areas Key Current Indirect impacts Meta-uncertainties Uncertain events Direct impacts Risk events Severe impacts Accidents Bad decisions intervention areas Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 116

119 3.3 Emerging risks 1. An effective regulatory framework 17. A nano-ecology becomes 9. The direct casualties of an uncontrolled nanotechnology are hard could control the potential dangers considerably more dangerous if to estimate, as they depend critically of nanotechnology, though this there is an intelligence controlling it on the nature of the nanotechnology, depends very much on the nature of (or pieces of it). Successful artificial the countermeasures used, and the the problems and the design of the intelligence could allow this to happen. general technological abilities of the regulations. Regulating the potentially human race after nanotechnology 18. It is in its potential for extreme pollutant aspects of nanotechnology – such as micro-particles – would manufacturing that the promise and development. The casualties from nanowarfare are similarly hard to be more feasible under traditional perils of nanotechnology lie. determine, as it is unclear what would frameworks, but somewhat tangential to the main issues. 19. Nanoweapon proliferation could be the most effective military use of completely destabilise international nanoweapons, and whether this would involve high or low casualties (contrast relations and arms control treaties, 2. Continuing research – into the by allowing small groups to rapidly mass nuclear weapons with targeted transformative aspects, not just shutdown of information networks). standard materials science – is required construct large arsenals. for nanotechnology to become a viable option for manufacturing. 20. One of the greatest threats of 10. Disruption of the world nanotechnology is the possibility political and economic system that it could result in a breakdown 3. Military nanotechnology (exacerbated by the collapse of of trade between currently research increases the chance that trade routes or nanowarfare) could interdependent nations. lead to further casualties. nanotechnology will be used for effective weapons production, and 11. A nano-ecology could disrupt and 21. International relations could break may lead to an arms race. undermine the standard biological down if trade does, leading to much potential for conflict. ecology, including food production. 4. Global coordination allows for regulatory responses, and may mitigate the effect of possible 22. The effectiveness of 12. The widespread use of countermeasures is extremely collapse of trade routes. nanotechnology could generate new vulnerabilities (just as modern cities are hard to judge, as is the balance vulnerable to EMP (electro-magnetic 5. The general mitigation efforts of between “defensive” and “offensive” pulse) weapons that would have had most relevance to nanotechnology are nanoweaponry. Nanotechnology could no effects in previous eras). allow novel approaches to controlling probably in surveillance and improved the problem, such as extremely international relations. effective sensors. 13. Over the long term, a nano-ecology could spread and develop in ways that 6. Nanoterrorism is one way in 23. Post-nanotech politics will are hard to predict or control (especially which humanity could lose control of if there are new vulnerabilities to it). determine the risk of collapse and the aggressive nanotechnology. potential for reconstruction. 7. Nanotechnology-empowered 14. Any problems with early nanotechnology could provide impetus warfare could spiral out of control, 24. Much of the analysis of the impact or could lead to the deployment of nanotechnology proceeds by for a regulatory or political response. of uncontrolled aggressive analogy with previous discoveries 15. The prevalence of or economic changes. It is unclear nanotechnology. The risk would be whether evolution or revolution is the nanotechnology products could acute if small groups were capable of effective nanowarfare on their own. introduce new vulnerabilities. better analogy, and what the speed of implementation of nanotechnological discoveries will be. 16. Smart sensors of all kinds would 8. Uncontrolled aggressive nanotechnology is a scenario in which be very important to either controlling a nano-ecology or preventing small 25. The course of international politics humanity unleashes weapons that it is extremely hard to predict, even for cannot subsequently bring under control, groups from rapidly constructing 465 arsenals of weapons. which go on to have independent political scientists. negative impacts on the world. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 117

120 3.3 Emerging risks 07-May-13: Publication of Eric range of products without requiring 3.3.2.3 Main events Drexler’s book “Radical Abundance: large amounts of capital or long during 2013 How a Revolution in Nanotechnology supply chains. The risks of social 473 Will Change Civilization” and political disruption are then – Research examined. The disruptions that 11-Jan-13: Artificial molecular 466 assembly device created can be anticipated include “falling Eric Drexler is one of the pioneers of – Research demand for conventional labor, nanotechnology, and introduced the resources, and capital in physical concepts to the general public with A functional and practical design production, with the potential 474 his book “Engines of Creation”. for cascading disruptive effects for assembling molecules is an Twenty seven years later, he presents throughout the global economy”, essential feature for successful a history, progress report, and as well as disruptions in supply nanotechnology. There have been 467 updated version of his vision, the and chains, trade, dependence, and many designs proposed, central theme of which is to “imagine the revaluation of assets (mineral some constructed, but not yet a a world where the gadgets and goods resources and large industrial fully functional molecular assembly 468 that run our society are produced not This design, based on device. facilities, for example, will lose much in far-flung supply chains of industrial principles from biology (it uses of their value). facilities, but in compact, even messenger RNA as its input code, desktop-scale, machines.” and synthesises peptides) represents This would go together with an another step towards that important goal. increase in surveillance capability The revolution in manufacturing and a potential nanotechnology would produce the “radical arms race. The book recommends abundance” of the title, with small taking pre-emptive action at the 06-May-13: First weapon made with 469 groups and individuals capable of 3D printer international level to prepare for producing an extraordinarily wide – Event these disruptions. It is the ability to make weapons en masse that represents one of the 470 dangers of nanotechnology. 3D 471 printing (or additive manufacturing) is not nanotechnology, but can be considered a precursor, as it similarly allows small groups to design and manufacture their desired products themselves. That one of the early designs has been a functioning weapon, and that such weapon design was justified on moral 472 grounds, indicates a very high probability that nanotechnology will be used for weapon production. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 118

121 3.3 Emerging risks 01-Jun-13: Nanostart AG: Venture 16-Dec-13: Nanotechnology: A Policy 475 478 Capital Investments in Nanotech Primer, CRS report of Congress – Policy – Initiative Governmental and supra- A key sign of a developing technology governmental policies will be is interest from investment companies. key to dealing with the dangers Nanostart AG is an example of such a company, with extensive investments and destabilising influences of nanotechnology, through regulation, in various nanotechnology projects. treaties, redistributive efforts or Interestingly, their interests are simply through preparing their not limited to more conventional populations for the change. nanotech projects, but extend to And institutions such as the US such speculative endeavours as 476 Congress are keeping an eye This serves as space elevators. on nanotechnology, in this case a reminder of the potentially large profits available in nanotechnology. through the Congressional Research Service. This report, however, Thus it seems likely that when the does not delve into the major risks technology matures sufficiently to cause increased risks, there will be of nanotechnology, but restricts many commercial entities heavily itself to minor subjects such as the safety of nanomaterials and investing in the technology, which will make the process of regulation US competitiveness in that field. more contentious, possibly leading War, trade disruption and potential 477 to “regulatory capture” development and misuse of nano- by 479 replicators these entities, with their interests are not discussed. This seems to reflect a certain lack represented rather than those of the of prioritisation and perhaps even broader community. a misplaced focus on the less important risks. The World's First 3D-Printed Gun Source: http://upload.wikimedia.org/wikipedia/commons/thumb/b/be/XP002.jpg/1279px-XP002.jpg Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 119

122 3.3 Emerging risks 3.3 Emerging risks 3.3.3 Artificial Intelligence Global System Major Asteroid Global Artificial Future Bad Extreme Collapse Impact Pandemic Intelligence Global Governance Climate Change Ecological Synthetic Unknown Super-volcano Nanotechnology Nuclear War Catastrophe Consequences Biology Artificial intelligence (AI) is the intelligence exhibited by machines or software, and the branch of computer science that develops machines and software with human-level intelligence. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 120

123 3.3 Emerging risks Such extreme intelligences could Major AI researchers and textbooks define the field as “the study and not easily be controlled (either by the design of intelligent agents”, where groups creating them, or by some 485 an intelligent agent is a system that international regulatory regime), and would probably act in a way perceives its environment and takes actions that maximise its chances to boost their own intelligence and 480 acquire maximal resources for of success. 486 almost all initial AI motivations. And 487 if these motivations do not detail the survival and value of humanity 3.3.3.1 Expected impact in exhaustive detail, the intelligence will be driven to construct a world Artificial Intelligence (AI) is one of the without humans or without meaningful least understood global challenges. features of human existence. There is considerable uncertainty on This makes extremely intelligent AIs what timescales an AI could be built, Source: http://commons.wikimedia.org/wiki/ 488 if at all, with expert opinion shown to in that extinction is a unique risk, File:Artificial.intelligence.jpg 481 more likely than lesser impacts. An be very unreliable in this domain. AI would only turn on humans if it This uncertainty is bi-directional: AIs foresaw a likely chance of winning; could be developed much sooner or otherwise it would remain fully much later than expected. An interesting version of this scenario integrated into society. And if an is the possible creation of “whole brain AI had been able to successfully Despite the uncertainty of when emulations,” human brains scanned and how AI could be developed, engineer a civilisation collapse, for and physically instantiated - physically there are reasons to suspect that instance, then it could certainly drive represented - in a machine. This an AI with human-comparable the remaining humans to extinction. would make the AIs into what could skills would be a major risk factor. be called properly human minds, AIs would immediately benefit On a more positive note, an possibly alleviating a lot of problems. intelligence of such power could easily from improvements to computer speed and any computer research. combat most other risks in this report, They could be trained in specific making extremely intelligent AI into 3.3.3.2 Probability professions and copied at will, thus a tool of great positive potential as 489 Whether such an intelligence replacing most human capital in well. disaggregation is developed safely depends on how the world, causing potentially great much effort is invested in AI safety economic disruption. Through 490 (“Friendly AI”) as opposed to simply Five important factors in their advantages in speed and 491 estimating the probabilities and building an AI. performance, and through their impacts of the challenge: better integration with standard If the returns from increased computer software, they could intelligence are low, intelligence 1. The reliability . AI predictions of quickly become extremely 2. explosions and extreme intelligence single Whether there will be a intelligent in one or more domains may not be possible. In that case, of entities. dominant AI or a plethora (research, planning, social skills...). intelligent AIs will become How 3. there would probably be an ecology . If they became skilled at computer extremely intelligent AIs Whether 4. of AIs of different levels of intelligence, research, the recursive self- can be controlled performing different tasks. In this , and how. improvement could generate what scenario, apart from the economic 5. Whether whole brain emulations is sometime called a “singularity”, 482 dislocation already noted, there is also but is perhaps better described (human minds in computer form) 483 before true AIs as an “intelligence explosion”, the possibility of AI-enabled warfare . will arrive and all the risks of the technologies with the AI’s intelligence increasing 484 that AIs would make possible. very rapidly. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 121

124 AI 3.3 Emerging risks GOVERNANCE DISASTERS AI AI arms race Extinction amalgamation Effective Global Long-term Friendly regulatory coordination impact AI projects Deliberate framework Global Technological attempts to Global povety Global instability coordination innovations construct world dictatorship Military Extreme AI-warfare Meta-uncertainty intelligence AI AI research on tradeoffs New system between e.g. Smart sensors of governance poverty, survival, freedom Meta-uncertainty Diminishing return AI attack Reduced impact of what an AI is contained to intelligence AI/Oracle AI likely to be Making Improvements to Failing to solve things worse global governance Meta-uncertainty important problems Partial Uploads Plethora of AIs on the reliability of friendliness AI predictions Copyable Uncontrolled human capital AI research Not achieving Lack of human Global important Enduring poverty Climate change flourishing pollution ethical goals Suffering of artifical agents Undesirable world system (e.g. global dictatorship) Human Economic collapse redundancy Disruption to Collapse of Long-term Post-disaster world politics world system negative effects politics and economy Civilisation Total short-term General mitigation casualties Collapse effort General mitigation Civilisation Total short-term AI Post-eruption Extinction effort collapse casualties politics amalgamation Key Current Risk events Direct impacts Indirect impacts Uncertain events Meta-uncertainties Bad decisions Accidents Severe impacts intervention areas Key Current Meta-uncertainties Indirect impacts Uncertain events Risk events Direct impacts Severe impacts Accidents Bad decisions intervention areas Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 122

125 3.3 Emerging risks 1. The advantages of global 15. Disparate AIs may amalgamate 7. General mitigation methods will be coordination and cooperation are of little use against intelligent AIs, but by sharing their code or negotiating to clear if there are diminishing returns may help in the aftermath of conflict. share a common goal to pursue their to intelligence and a plethora of AIs, objectives more effectively. but less clear if there is a strong first 8. Copyable human capital – 16. There may be diminishing returns software with the capability to mover advantage to the first group to perform tasks with human-like skills produce AI: then the decisions of that to intelligence, limiting the power of any one AI, and leading to the – would revolutionise the economic first group are more relevant than the 496 existence of many different AIs. and social systems. general international environment. 17. Partial “friendliness” may 9. Economic collapse may follow from 2. Military AI research will result in mass unemployment as humans are AIs built for military purposes, but be sufficient to control AIs in replaced by copyable human capital. possibly with more safeguards than certain circumstances. other designs. 10. Many economic and social 18. Containing an AI attack may be 3. Effective regulatory frameworks set-ups could inflict great suffering possible, if the AIs are of reduced would be very difficult without on artificial agents, a great moral intelligence or are forced to attack before being ready. negative if they are capable of feeling knowledge of what forms AIs will 494 such suffering. ultimately take. 19. New political systems may emerge 11. Human redundancy may follow the in the wake of AI creation, or after an 4. Uncontrolled AI research (or creation of copyable human capital, as AI attack, and will profoundly influence research by teams unconcerned software replaces human jobs. with security) increases the risk of the shape of future society. potentially dangerous AI development. 12. Once invented, AIs will 20. AI is the domain with the largest be integrated into the world’s uncertainties; it isn’t clear what an AI 5. “Friendly AI” projects aim to directly economic and social system, is likely to be like. produce AIs with goals compatible barring massive resistance. with human survival. 21. Predictions concerning 13. An AI arms race could result in AI are very unreliable and 6. Reduced impact and Oracle AI AIs being constructed with pernicious underestimate uncertainties. are examples of projects that aim goals or lack of safety precautions. to produce AIs whose abilities and goals are restricted in some sense, to 14. Uploads – human brains prevent them having a strong negative 493 instantiated in software – are one impact on humanity. route to AIs. These AIs would have safer goals, lower likelihood of extreme intelligence, and would be 495 more likely to be able to suffer. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 123

126 3.3 Emerging risks development. 3.3.3.3 Main events 13-Sep-13: Publication: “Responses to 25-Apr-13: Kurzweil plans to help during 2013 507 501 Catastrophic AGI Risk: A Survey” Google make an AI brain – Research – Initiative 19-Mar-13: DARPA Sets Out Since the recognition of the potential The idea of creating a fully general to Make Computers That Can 497 risk with AGI (Artificial General Teach Themselves AI, an AI that is capable of all tasks 508 various proposals Intelligence), requiring intelligence, went into – Policy 502 have been put forward to deal with abeyance during the AI winter, the problem. After arguing that a period of reduced interest and The amount of information stored 509 uncertainty about a timeline to AI in a human brain is extremely funding in AI. The term AI itself fell into 503 does not translate into a certainty But recent AI successes large. Similarly, the amount of disfavour. that AIs will take a long time, the information needed to perform such as Watson’s triumph on 504 paper analyses why AIs could be an adequately at complex human tasks (demonstrating a certain “Jeopardy!” existential risk. It argues that a trend level of natural language recognition is considerable – far more than is toward automatisation would give and processing) and Google’s self- easily programmable by hand (the 505 498 AIs increased influence in society, (demonstrating spatial driving car for instance, started Cyc project, as such systems would be easier in 1984, aiming to rapidly formally awareness and movement) have to control, and there could be a codify all human common sense – revived interest in constructing a discontinuity in which they gained human-like mind in digital form. and is still running). Thus the interest 510 power rapidly. This could pose a in the field of machine learning, Kurzweil, hired by Google at the end great risk to humanity if the AIs did not of 2012, reveals in this interview his and in algorithms that can teach share human values (intelligence and interest in doing just that. A notable themselves skills and knowledge values are argued to be independent feature of Kurzweil is his optimism from raw data. With the rise of 511 499 for an AI), a task which seems vast databases and about the consequences of creating “Big Data”, 506 difficult to achieve if human values are increased computer power, there has which could affect the level of AIs, 512 complex and fragile, and therefore precautions his team would include in been a flowering of applications of 500 problematic to specify. This has caught computer learning. its design. the eye of the Defense Advanced The authors then turned to analysing Research Projects Agency (DARPA), the AI safety proposals, dividing them a research arm of the US defense into proposals for societal action, department responsible for the external constraints, and internal development of new technologies. constraints. They found that many In this project, DARPA aims both proposals seemed to suffer from to “enable new applications that serious problems, or to be of limited are impossible to conceive of using effectiveness. They concluded by today’s technology” and to simplify reviewing the proposals they thought machines so that non-experts most worthy of further study, including can effectively use them and build AI confinement, Oracle AI, and applications for them. This most motivational weaknesses. For the long recent project confirms the interest term, they thought the most promising of the military in artificial intelligence approaches were value learning (with human-like architecture as a less reliable but possibly easier alternative). Formal verification was valued, whenever it could be implemented. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 124

127 3.3 Emerging risks even of their own. for the human race. 01-Oct-13: Publication of “Our Final 15-Oct-13: “Racing to the precipice: 24-Oct-13: Growing researcher Invention: Artificial Intelligence awareness of the threat of artificial a model of artificial intelligence 525 development” lays out the dangers intelligence and the End of the Human Era” 522 – Research of AI arms races by James Barrat, warning of the 519 dangers of AI – Research Much more effort is devoted to – Research, creating AI than to ensuring that AIs may be developed by different 526 In this book, James Barrat argues Those it is developed safely. groups, each desiring to be the for the possibility of human-level AI first to produce an artificial mind. working in developing AI could be The competitive pressure will be motivated to minimise the extent being developed within a decade, stronger the more powerful AIs are their creation represented a potential based on the current progress in 527 It is therefore significant computer intelligence and the large danger. believed to be, thus maximising sums invested by governments and when a researcher focused on the the danger in those situations. danger of AI is invited to speak at corporations into AI research. Once This paper considers an AI arms 523 race, where different teams have this is achieved, the AI would soon a mainstream AI conference, as the option of reducing their safety Dr. Anders Sandberg of the Future surpass human intelligence, and precautions in order to perfect of Humanity Institute was, at the would develop survival drives similar 23rd International Joint Conference their device first – but running the to humans (a point also made in 520 on Artificial Intelligence in Beijing. Omohundro’s “AI drives” thesis). risk of creating a dangerous and The book then imagines the uncontrollable AI. In the absence of He took part in a panel discussion competition between humanity and entitled “The Future of AI: What if enforceable agreements between the a cunning, powerful rival, in the form we succeed?”, along with Joanna teams, this dynamic pushes each to of the AI – a rival, moreover, that may Bryson, Henry Kautz and Sebastian take on more risk than they would not be “evil” but simply harmful to want (similarly to the “prisoner’s Thrun. He argued that though 524 potentially causing an humanity as a side effect of its goals, dilemma”), current AI research does not appear or simply through monopolising extremely damaging outcome. to directly lead to dangerous AIs, scarce resources. the time to design and implement The situation is improved if risk- safety measures is now. This is taking makes little difference to Along with many interviews of both because of the time needed researchers working in the forefront speed of development, if the teams to develop such safety measures, have reduced enmity between of current AI development, the which could necessitate solving 528 and them, or if there are fewer teams book further claims that without hard philosophical problems, 521 extraordinarily careful planning, because of the potential for sudden involved (those last two factors also help with reaching agreements). powerful “thinking” machines present increases in AI skill and intelligence. potentially catastrophic consequences Further, security precautions would Somewhat surprisingly, information has a negative impact: the outcome be easier to implement if they is safer if the teams are ignorant of were integrated into the design by the designers themselves (or by each other’s rate of progress, and researchers intimately aware of the properties of the design). Further evidence of the increased awareness 529 of risks was Stuart Russell’s joining of the board of the Cambridge Centre for the Study of Existential 530 Risks (CSER). Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 125

128 Global System Major Asteroid Global Artificial Future Bad Extreme Collapse Impact Pandemic Intelligence Global Governance Climate Change 3.3 Emerging risks 3.3 Emerging risks 3.3.4 Unknown Consequences Ecological Synthetic Super-volcano Unknown Nuclear War Nanotechnology Catastrophe Consequences Biology Uncertain risks represent the unknown unknowns in the family of global catastrophic challenges. They constitute an amalgamation of all the risks that can appear extremely unlikely in isolation, but can combine to represent a not insignificant proportion 531 of the risk exposure. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 126

129 3.3 Emerging risks But in these situations, the main 3.3.4.1 Expected impact 3.3.4.2 Probability source of probability of the risk is disaggregation not the quoted number, but the There are many different possible risks much greater probability that the Five important factors in experts’ models and world views that seem individually very unlikely 537 If marginal scientific and speculative. Could someone estimating the probabilities and are wrong. impacts of the challenge: develop a super-pollutant that renders theories predict large risks, the the human race sterile? Could the probability is concentrated in the 1. LHC have created a black hole likelihood that the theory might extensive Whether there will be 532 538 be correct. that swallowed the Earth? Might and research into unknown risks Conversely, if many computer games become so addictive independent models, theories, and their probabilities. that large populations will die rather The capacity to develop arguments all point in the direction 2. 533 than ceasing to indulge in them? of safety, then the conclusion is methods for limiting the of all combined probability Could experiments on animals more reliable. uncertain risks. lift them to a level of intelligence 534 3. estimating Might There are methods to estimate The capacity for comparable with humans? uncertain risks without needing to be . “out-of-model” risks some of the people sending signals to extra-terrestrial intelligences attract culture of risk assessment The 4. explicit about them. One resolution 535 deadly alien attention? What are to the Fermi paradox – the apparent in potentially risky areas. non-risk- 5. Whether general, the risks out there that we can’t yet absence of alien life in the galaxy – is specific mitigation or resilience that intelligent life destroys itself before conceive of? beginning to expand into the galaxy. measures are implemented . 539 or decrease Results that increase These risks sound unlikely and for the probability of this explanation many possibly ridiculous. But many modify the generic probability of of today’s risks would have sounded intelligent life (self-)destruction, which ridiculous to people from the past. If includes uncertain risks. Anthropic this trend is extrapolated, there will be 540 reasoning can also bound the total risks in the future that sound ridiculous risk of human extinction, and hence today, which means that absurdity is estimate the unknown component. not a useful guide to risk intensity. Non-risk-specific resilience and Expert opinion provides some 541 post-disaster rebuilding efforts information on specific speculative will also reduce the damage from risks. But it will tend to give them uncertain risks, as would appropriate extremely low probabilities – after all, national and international regulatory the risks are highly speculative, which 542 Most of these methods regimes. also means the expert’s judgement is 536 would also help with the more less reliable. conventional, known risks, and badly need more investment. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 127

130 3.3 Emerging risks ECOLOGICAL CATASTROPHE GOVERNANCE DISASTERS Global Sustainability Post-eco-collapse Global poverty coordination research climate change Long-term ecological effects Deliberate Global Technological attempts to Global povety Global instability coordination innovations construct world dictatorship Quality of life Preservation Ecological loss from collapse efforts ecosystem loss Meta-uncertainty on tradeoffs New system between e.g. Smart sensors of governance poverty, survival, freedom Moral tragedy from Threat to Pollution food supply ecosystem loss Loss of Pre-eco-collapse Making Improvements to Economic costs Failing to solve biodiversity climate change things worse global governance important problems Disruption to Post-eco-collapse Rebuilding politics and politics the ecosystem economy Not achieving Lack of human Global important Climate change Enduring poverty flourishing pollution New, ethical goals Vulnerabilities Sustainable or environmentally to flood and non-sustainable damaging industries economies other disasters Undesirable world system (e.g. global Technological Pre-eco-collapse dictatorship) innovations mitigation efforts Disruption to Collapse of Long-term Post-disaster world politics world system negative effects politics and economy Meta-uncertainty on the true dependence of Human survivability humanity on the ecosystem in “closed” systems Civilisation Total short-term General mitigation Civilisation Total short-term Extinction Extinction collapse casualties effort collapse casualties Key Current Uncertain events Direct impacts Risk events Indirect impacts Meta-uncertainties Severe impacts Bad decisions Accidents intervention areas Key Current Indirect impacts Meta-uncertainties Risk events Direct impacts Uncertain events Accidents Bad decisions Severe impacts intervention areas Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 128

131 3.3 Emerging risks 1. Smart sensors and surveillance 12. The world’s political structure, 6. Some institutions may after an unknown risk is triggered, will could detect many uncertain risks deliberately pursue dangerous in the early stages, and allow determine whether humanity improves technologies or experiments, or or worsens the situation. may convince themselves that their researchers to grasp what is going on. research is not dangerous. 2. Proper risk assessment in 13. Some methods (such as domains where uncertain risks are 7. Unforeseen accidents could be the considering the Fermi paradox) may bound the total probability of trigger for many uncertain risks. possible could cut down on the risk considerably. destructive uncertain risks, but these 8. The amount of direct casualties are very speculative. varies wildly depending on the 3. Global coordination would aid risk 14. Trying to estimate unknown or risk involved. assessment and mitigation. out of model risks is by definition very difficult and uncertain. 9. The disruptions to the world’s 4. Specific research into uncertain and economic and political system vary unknown risks would increase our wildly depending on the risk involved. understanding of the risks involved. 10. The uncertain risk may have other 5. General mitigation efforts are mostly general resilience building. disruptive effects (such as loss of trust in certain technologies). 11. The long-term impact varies wildly depending on the risk involved. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 129

132 3.3 Emerging risks 01-Aug-13: The Fermi paradox Such an explanation gives a bound 3.3.4.3 Main events provides an estimate of total to existential risk from all sources, during 2013 existential risk (including uncertain including uncertain risks. 548 risks) – Research This paper demonstrates the 28-Mar-13: Paper on Adaptation relative ease with which a space- to and recovery from global 543 The Fermi paradox is the seeming catastrophes in general faring civilisation could cross contradiction between the apparent – Research between galaxies. ease with which intelligent life could arise in the galaxy, and the lack of One approach to dealing with Combined with recent evidence evidence of any such life. that the majority of Earth-like uncertain risks is to build general 551 planets formed before the Earth, adaptation and recovery methods that Many explanations have been this makes the absence of visible would be relevant to a wide class of 549 proposed to resolve the paradox, intelligent life more inexplicable, potential disasters. This paper notes one of which is relevant to the absence of published research and worsens the Fermi paradox, 544 existential risks: the “Late Great in this area, and seeks to begin to increasing the probability of a Late 550 Filter” explanation. This posits Great Filter and thus of existential fill the gap. It identifies methods for that intelligent life is inevitably increasing survivor resilience and risk from all sources. destroyed before it can expand promoting successful adaptation through the galaxy. and recovery, even for isolated communities. It recognises that the process is highly complex, and needs further research. Figure 23: Number of galaxies that can reach us with speeds of 50%c, 80%c, 99%c and c, from different 552 starting moments 28-Mar-13: Paper Evaluating Methods 545 for Estimating Existential Risks – Research It would be advantageous to have a rigorous approach for estimating severe risks, including uncertain and unknown ones. This paper reviews and assesses various methods for estimating existential risks, such as simple elicitation; whole evidence Bayesian; evidential reasoning using imprecise probabilities; Bayesian networks; influence modelling based on environmental scans; simple elicitation using extinction scenarios as anchors; and computationally 546 intensive possible-worlds modelling. These methods can be applied rigorously to uncertain risks, assessing them in the same way as more standard risks. Influence modelling 547 based on environmental scans can even suggest some new as yet unknown risks. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 130

133 3.3 Emerging risks 01-Nov-13: Top conglomerates spearhead creation of private sector 553 disaster response body – Initiative One of the most effective tools against uncertain risks is to adopt general disaster recovery measures. Anything that enables the preservation of resources or knowledge and the rapid reconstruction of key infrastructure will be of use against a wide variety of risks. Though governments and supra-governmental organisations play a vital role in this, it would be beneficial to get the private sector, with its funds and its expertise, involved too. The private sector has played a key role in recovery from many disasters (such as the Japanese 2011 earthquake/ 554 tsunami). This news report shows that the private sector aims to take on a larger role in disaster relief in the Philippines. More importantly, the key players aim for the creation of a private sector disaster response body, so as to have a better organised private sector response during disasters. This is significant as it disperses disaster recovery expertise to a wider group of individuals, and suggests that private companies may be alternate entities capable of providing relief after a major disaster. Thus preparations for post-disaster recovery could include building up private sector capacity as well as other measures. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 131

134 3.4 Global Policy risk 3.4 Global Policy risk 3.4.1 Future Bad Global Governance Global System Major Asteroid Global Artificial Future Bad Extreme Collapse Impact Pandemic Intelligence Global Governance Climate Change “Global governance refers to the way in which global affairs are managed. As there is no global government, global Ecological Synthetic Super-volcano Unknown Nanotechnology Nuclear War governance typically involves a range of Catastrophe Consequences Biology actors including states, as well as regional and international organisations. However, a single organisation may nominally be given the lead 485 role on an issue.” Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 132

135 3.4 Global Policy risk An example of the second would Often global governance is confused 3.4.1.2 Probability be constructing a global totalitarian with global government, but they disaggregation 557 are two very different things. Global In general, technology, state. governance is just a term to describe political and social change may Five important factors in estimating enable the construction of new forms the way global affairs are managed, or not managed. Global government the probabilities of various impacts: of governance, which may be either much better or much worse. is the idea that the world should be 1. run like a country with a government. severity of non-deadly How the The global governance system policy failures can be compared These examples immediately will inevitably have pros and cons, illustrate two issues with governance with potential casualties. poor governance will result disasters. First, the task of estimating 2. Whether depending on the political decisions that are made. their probability is difficult. Long- in a collapse of the world system. mass surveillance and term political predictions are of How 3. other technological innovations questionable validity and subject to 558 especially where strong biases, will affect governance. 3.4.1.1 Expected impact 559 new systems Whether there will be 4. strongly-held values are concerned. in the future. of governance Second, the impact of these governance disasters depends to a This section looks at global a world dictatorship Whether 5. governance disasters. Though all the may end up being constructed. large extent on subjective comparative evaluations. It is not impartially risks in this report can be exacerbated obvious how to rank continued by poorly chosen policy decisions, this poverty and global totalitarianism classification contains those problems versus billions of casualties or that arise almost exclusively from bad 560 The long term policy choices. civilisation collapse. impact needs also to be considered: how will poverty and global There are two main divisions in governance change? If there are many governance disasters: failing to solve generations ahead of us, then the major solvable problems, and actively 561 long term state of humanity’s policy causing worse outcomes. An example becomes much more important than of the first would be failing to alleviate 556 the short term one. absolute poverty. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 133

136 GOVERNANCE DISASTERS 3.4 Global Policy risk GOVERNANCE DISASTERS Deliberate Global Technological attempts to Global instability Global povety coordination innovations construct world dictatorship Deliberate Global Technological attempts to Global povety Global instability coordination innovations construct world dictatorship Meta-uncertainty on tradeoffs New system between e.g. Smart sensors of governance poverty, survival, freedom Meta-uncertainty on tradeoffs New system between e.g. Smart sensors of governance poverty, survival, freedom Improvements to Making Failing to solve things worse global governance important problems Making Improvements to Failing to solve things worse global governance important problems Not achieving Lack of human Global important Climate change Enduring poverty flourishing pollution ethical goals Not achieving Lack of human Global important Enduring poverty Climate change flourishing pollution ethical goals Undesirable world system (e.g. global dictatorship) Undesirable world system (e.g. global dictatorship) Disruption to Collapse of Long-term Post-disaster world politics world system negative effects politics and economy Disruption to Collapse of Long-term Post-disaster world politics world system negative effects politics and economy General mitigation Civilisation Total short-term Extinction effort collapse casualties General mitigation Civilisation Total short-term Extinction effort collapse casualties Key Current Uncertain events Meta-uncertainties Risk events Direct impacts Indirect impacts Bad decisions Accidents Severe impacts intervention areas Key Current Uncertain events Indirect impacts Meta-uncertainties Direct impacts Risk events Severe impacts Accidents Bad decisions intervention areas Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 134

137 3.4 Global Policy risk 1. Global coordination between 14. Governance decisions taken at 6. Some groups may deliberately seek nations is essential for building a good to construct a world dictatorship, the global level have a high potential either through self-interest or because to cause disruptions to the world’s global governance system – but also they believe it would be the best essential for building a bad one. political and economic systems. design for global governance. 15. Bad governance at the global 2. Global poverty is one of the important problems that are being 7. Undesirable world systems (such as level may not be susceptible to only partially solved by current global dictatorships) could result from improvements and could cause problems for a considerable amount a worsening of global governance. policies. In turn, it can contribute to global instability, worsening likely of time. governance outcomes. 8. Many value systems do not distinguish between action and 16. Technological innovations could inaction, so a global system that 3. Smart sensors and mass allow completely new models of didn’t positively encourage human surveillance can contribute to new government, but could also facilitate surveillance dictatorships. systems of governance, but also to flourishing would be almost as pernicious as one that blocked it. large-scale dictatorships. 17. Global instability could result 9. Global pollution is a problem 4. The global system of governance in more pernicious systems of consists of the UN and a wide requiring solutions at the global governance, as well as an increased variety of bilateral or multilateral failure to solve important problems. governance level. agreements and norms, constructed 18. New systems of governance mainly according to national 10. Climate change is a problem requiring solutions at the global could be developed, using self-interests. Thus significant governance level. improvements to global governance modern communication technology for instance. are currently possible. 11. Various ethical systems have 19. The political landscape after desirable goals that could be achieved 5. General mitigation efforts against governance disasters are tricky – a disaster will be important in in theory, but would not be achieved determining whether governance under suboptimal governance. most mitigation efforts are the results disasters could cause civilisation of governance decisions! However, collapses or mass casualties. 12. It would be a tragedy if absolute some efforts can be made – for instance, an increase in recognised poverty were to endure over the human rights across the globe could generations to come, especially if this 20. How to compare enduring poverty, outcome were avoidable. militate against certain pernicious actual casualties, and repressive governance directions. These efforts governance is a question of values are of a very different nature to 13. A collapse of the world system, for and not just of direct comparison of any reason (including revolution) is the mitigating other risks. lives lost. most direct way a governance disaster could result in mass casualties. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 135

138 3.4 Global Policy risk This confirms other studies, by – Perhaps the most cost-effective 3.4.1.3 Main events 565 564 and others: the World Bank way to reduce existential risks during 2013 poverty reduction is possible, and today is to fund analysis of a has been successfully implemented wide range of existential risks and in many countries. potential mitigation strategies, with 15-Feb-13: Existential risk reduction 562 a long-term perspective. as a global priority – Research 05-Jun-13: Guardian leaks NSA If this paper is right, a general lack 566 of focus on existential risks by In this paper Nick Bostrom, the spying programme – Initiative director of the Future of Humanity governments and other agents can Institute, lays out the case for making be considered a governance disaster in itself. A significant event was the revelation existential risk reduction a global priority. Existential risks (Xrisks) are the by Edward Snowden of the extent of highest category of negative impact the NSA’s surveillance programme. in this report, those that threaten the 19-Apr-13: Multidimensional poverty This included the mass recording entire future of humanity. The policy index diminishes in 18 out of 22 and mining of data across the 563 implications of the paper are: United States and the interception of analysed countries – Event foreign politicians’ data. – Existential risk is a concept that can Of 22 countries for which the Oxford The revelations caused great focus long-term global efforts and 567 and raised questions sustainability concerns. Poverty and Human Development controversy about the NSA’s surveillance Initiative analysed changes in MPI – The biggest existential risks 568 oversight. The episode established (Multidimensional Poverty Index) are anthropogenic and related to that discrete mass surveillance – an poverty over time, 18 reduced potential future technologies. important component of potential poverty significantly. – A moral case can be made that existential risk reduction is strictly more important than any other global public good. – Sustainability should be rethought Less poverty of nations in dynamic terms, as aiming for a 100 sustainable trajectory rather than a Gini index* sustainable state. – Some small existential risks can 29.1 be mitigated today directly (e.g. 80 asteroids) or indirectly (by building resilience and reserves to increase China survivability in a range of extreme 35.5 60 scenarios) but it is more important 35.1 to build capacity to improve humanity’s ability to deal with the India larger existential risks that will arise 33.4 30.8 40 later in this century. This will require collective wisdom, technology foresight, and the ability when necessary to mobilise a strong 20 57.5 global coordinated response to 41.5 expected existential risks. Brazil 57.6 59.7 0 05 1981 95 90 81 2000 93 Figure 24: Less poverty of nations - Population living below $1.25 a day at 2005 Purchasing-pover parity, % * 0=absolute equality, 100=absolute inequality Source: Economist, originally World Bank, http://www.economist.com/node/14979330 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 136

139 3.4 Global Policy risk totalitarianism – was already possible using current technology and political organisation. 14-Jun-13: UNDP-UNEP Poverty- Environment Initiative Launches New Five-Year Phase to Meet Growing 569 Demand from Member States – Policy To reduce poverty in the future, it is important to maintain and extend past trends in poverty mitigation. The United Nations’ Poverty-Environment Initiative (PEI), launched in 2008, has had a number 570 of success stories from Uruguay 571 to Malawi. Due to increased demand from member states, the programme has been extended for another five years, 2013-2017, and may add countries such as Myanmar, Mongolia, Indonesia, Albania, Peru and Paraguay. Such programmes demonstrate that the bureaucratic/policy side of poverty reduction is supported by an international infrastructure with a strong emphasis on assessments. The effect of such approaches on overall poverty will depend on the interplay between these policies and the other side of poverty reduction: 572 573 and trade. economic growth Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 137

140 4. Relations between global risk 4. Relations between global risks “We have some idea what might happen if, in the face of other pressing global challenges, we divert our focus from making systemic improvements in public health and veterinary services — and that prospect is frightening.” 574 The World Bank Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 138

141 4. Relations between global risk / 4.1 General relations between global risks and their potential impacts 577 578 Two things make the understanding Nuclear war, asteroid impacts 4.1 General relations 579 and super-volcanoes have direct of the relation between the global between global risks impacts on the climate, and, through risks particularly important. 580 and their potential impacts that, on the ecosystem. 1. Impacts: The global risks are interconnected in different Relations between global risks is an The kinds of mitigation efforts capable area where surprisingly little work is of containing the damage from a ways. Often the situation can be being done. Most research focuses super-volcano would most likely be described as a set of dominoes: if one falls, many others follow. Even on individual or closely related effective against asteroid impact groups of challenges. Organisations damage, because of the similar small impacts can start a process nature of the impacts. The converse where different challenges interact. working on global challenges are almost always working on individual is not true, since one major method Higher temperatures due to global risks. The initial overview below is of reducing asteroid impact – space- warming can result in the spreading 581 based on individual studies where – would have no based deflection of pandemics which increase different relations are analysed, but impact on super-volcano risk. tensions between countries, and so on. no work has been identified where the relations between all twelve Solving climate change would help 582 challenges have been analysed. reduce current ecological pressure. 2. Specific measures to address Global risks often require a risk: International agreements to reduce ecological damage could be extended A risk that is natural to start with is significant changes in our current society, from how we build cities to combating climate change as future bad global governance, as all to how food is produced and well, by establishing structures other global challenges exacerbate 575 and all other governance disasters, provided. Such significant changes for international collaboration and global challenges can potentially be will result in situations where encouraging resource-efficient exacerbated by governance disasters. solutions. Climate change also creates measures to reduce the risk in A well functioning global governance one area affect the probability conditions more suitable for the 583 Measures to system is therefore a key factor to and/or the impact in other areas. spread of pandemics. address global catastrophic risks. combat global pandemics, such as Depending on the measure chosen strengthened outbreak coordination to reduce the risk, and other 584 Conversely, avoiding governance could be and statistical modelling, complementary measures, the disasters improves all risks, as used to combat synthetic pathogens effect can be positive or negative. better institutions are better able to as well. mitigate risks. Governance disasters If a safe artificial intelligence is directly increase the problems of climate change (through a lack of developed, this provides a great coordination between countries), resource for improving outcomes 585 the risk of nuclear war (by stoking and mitigating all types of risk. conflict between nuclear powers) and Artificial intelligence risks worsening global system collapse (by weakening nanotechnology risks, by allowing nanomachines and weapons to be global responses to systemic risks). All risks exacerbate global system designed with intelligence and without centralised control, overcoming the collapse, by putting extra stress on an 576 interconnected system. main potential weaknesses of these Conversely, 586 a resilient governance system is machines by putting planning abilities on the other side. better able to cope with all risks, and a collapsed global system is more vulnerable to all risks. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 139

142 4.1 General relations between global risks and their potential impacts Conversely, nanotechnology controlled and benevolent synthetic Nanotechnology and synthetic biology 590 abilities worsen artificial intelligence creations could act to improve and are sufficiently closely related (both heal current ecological damage. risks, by giving AI extra tools which dealing with properties on an atomic scale) for methods developed in it could use for developing its 587 power base. Nanotechnology one to be ported over to the other, There are many secondary effects and synthetic biology could allow potentially worsening the other risk. that are not covered here. Increasing the efficient creation of vaccines They are sufficiently distinct though nuclear power could for instance and other tools to combat global (a mainly technological versus a improve the outlook for climate 588 591 pandemics. mainly biological approach) for Nanotechnology’s while increasing the risk of change, 592 proliferation increased industrial capacity could and thus of nuclear war. countermeasures in one domain not necessarily to be of help in the other. There are many such effects between allow the creation of large amounts various strategies for addressing of efficient solar panels to combat different risks, but they are specific climate change, or even potentially Uncontrolled or malicious synthetic the efficient scrubbing of CO2 from pathogens could wreak great damage enough for there to be no simple 589 on the ecosystem; conversely, arguments of the type which says that the atmosphere. mitigating risk X worsens risk Y. ALL RISKS first risk worsens second risk solving first risk improves second risk both of the above Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 140

143 4.1 Specific relations between global risks An international initiative should start Below is an example of an overview 4.2 Specific to achieve better understanding of how different global challenges relations between of the relations between global can be plotted depending on the global risks challenges in order to ensure technical difficulty of reducing synergies and avoid strategies that the risk and the difficulty of In parallel with work to increase our will undermine other challenges. collaborating to reduce it. understanding about the general relations between global risks, work to identify more specific relations should people die / suffer also be initiated. This is an area where many pieces of research exist. But very little work has been done to combine them and assess different strategies to address specific global risks and understand how these strategies will affect other global risks. It is important support use people afraid of infections short term thinking to distinguish between two different kinds of specific relations. Extreme Climate Change Global Pandemic First, there are solution strategies for one global risk and the ways it affects other global risks. For example, using video conferences can reduce the probability of pandemics by reducing unnecessary travel. On the other hand, unsustainable use of bio-energy could accident from nature more renewable energy attack increase spillover opportunities when more video meetings a zoonosis (a disease transmitted consumption less meat from animals to humans) increases reduce risk the spread of pandemics due to an increased number of contacts between increase risk humans and infected animals in forests 593 around the world. Second, how society reacts to the very threat of different risks can affect other challenges. For example, if people are afraid of pandemics they might use more video meetings and in that way help reduce carbon emissions. Attempts to develop solutions for specific global challenges should assess their impacts, positive and negative, on other challenges. In order to better understand the relations between different global challenges, work could start to analyse similarities and differences. technical difficulty of reducing risk collaboration difficulty of reducing risk Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 141

144 5. Probabilities and uncertainties – an initial overview 5. Probabilities and uncertainties – an initial overview “Uncertainty is an uncomfortable position. But certainty is an absurd one.” Voltaire Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 142

145 5. Probabilities and uncertainties – an initial overview During the process of identifying These estimates are only an attempt almost all steps of the assessment. to assemble existing estimates in risks that could have an infinite Not only do great uncertainties exist impact it became evident that order to encourage a process to for all the risks, but the difficulties of improve these numbers. estimating probabilities are also very the most common question among people interested in different. At one end of the spectrum the global challenges is this: “How These estimates range from rigorous probability of a nuclear war can change probable is it that this impact dramatically from one day to another calculations based on large amounts of high-quality data (asteroids) to due to political decisions. Much of the will ever happen?” For those uncertainty is related to psychological with expert knowledge in one guesstimates by interested experts assumptions of how different individuals (AI). The result is that some have a area the first question is often: more rigorous methodology behind “How does the probability and will react under stress. magnitude of impact in this area them, and others should be taken compare with the probability and At the other end of the spectrum with a large grain of salt, but all are there is AI, where there is not even a still very rough estimates. As science magnitude of impact in other generally accepted understanding of progresses they will be updated. areas?” Finally, those who have tried to estimate probabilities for It is even possible that some will the possibility of the impacts capable global challenges ask: “What is the of creating the risks covered in this change by orders of magnitude. status of knowledge in other areas report. There are challenges with But instead of no estimate at all, we very much data, including asteroids, compared to mine?” now have an initial reference that we and other challenges with very little hope will trigger a discussion and relevant data, such as bad future collaboration that will help improve These are all very important questions, what we have already. global governance. and this chapter is not an attempt to answer them. But, as there is no As many of the challenges are long- Obviously the risks also share a organisation, process or report that number of characteristics: they all term and require early action to be has provided an overview of quantified avoided or mitigated, the probability have potentially extreme outcomes assessment for global challenges with potential infinite impact, the chapter is provided for the next 100 years, and have never been experienced does try to present the current state instead of the annual probability that before. The possibility of studying is often provided. The reason for this of knowledge in order to inspire series of data, exploring how the outcome will change with incremental is that a 100-year perspective helps further work. us understand that even relatively changes in input data, and testing small probabilities can become It is easy to argue that it is too difficult, conclusions on similar events are just a few examples of things that in most significant over a century. Say that or even impossible, to assess the it is a one in 100 probability (1%) for probabilities that are at all meaningful cases cannot be done. Estimating an impact to occur. Over a century for the risks in this report, and probabilities in traditional ways is 594 therefore very difficult. therefore to exclude them. There are there is a 63.4% probability of one 595 Further, or more such impacts. many good reasons for not trying, structures that need to change require However, as the current lack of including significant uncertainty in us to look beyond the immediate interest in global risks with potentially infinite impacts may in part be due to and incremental changes that most the lack of actual numbers, the best discussions focus on today. estimates that could be found are presented below with explanations. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 143

146 5. Probabilities and uncertainties – an initial overview Structure of the probability estimates One of the key guiding rules they 2. Data availability follow is to ensure a capital adequacy This is an estimate of the amount of As the different challenges are very different and the status of probability at a 1-in-200 level. data available to make probability assessments on all relevant steps estimates varies significantly, the initial of the sequence. In some areas a probability numbers are provided This rule, which is included in for 596 597 together with estimates regarding: example ICA and Solvency II, lot of hard-to-get data is needed provides an opportunity to discuss to make an assessment (e.g a risks with a possible infinite impact. global pandemic); in other areas 1. The understanding of sequence the data is related to secret and/ This is an estimation of how well the One contribution could be to discuss or psychological factors (e.g. sequence from today to a possible infinite impact is understood. At the pros and cons with different large-scale nuclear war). In others one extreme all the different paths relatively little data is needed definitions of the 1-in-200 level. For from today to an infinite impact are example, one definition is that “each (asteroids), or a lot has been done company holds enough capital to to gather data (e.g. climate change). understood. At the other extreme, there is only a theoretical idea that withstand the events of the next one is coherent and does not break any 3. Existing probability estimates year with a probability of 199 out of 598 natural laws. In the latter case there This would exclude many 200.” These form an estimate of the of the risks in this report and could would be no understanding of how kind of uncertainty that exists. it is possible to get from where we even result in the risks increasing, This obviously depends on an are today to an infinite impact. A as the time perspective is so short. understanding of sequence and data availability, but it also Investments could help reduce short- sequence is required to calculate depends on resources and interest an estimate instead of only having term risks at the same time as they in communicating with the rest of educated guesses. increase long-term risks. the world. Another definition is that “a company The estimates below are preliminary, should hold enough capital to be but a sound risk approach requires able to withstand a ‘reasonably 599 This stakeholders to begin to include them foreseeable’ adverse event”. highlights the challenge of determining in strategic assessments. what “reasonably foreseeable” is. Hopefully all the risks in this report One group in particular is of interest could be included on such a list. Then and that is actuaries, the professionals the questions would be regarding who deal with the financial impact of what we can do about it. risk and uncertainty. 0.01% 0.01% 0.001% 0.1% one in ten one in a one in a hundred thousand thousand thousand 0.0001% 1% n /a 0.8% n /a one in a one in a million hundred 0.00001% 10% one in ten one in million ten infinite impact % 0.0000001% 100% infinite threshold % one in a hundred one in one 0-10% 0.01% million 0-10% 0.00013% Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 144 0.1% 0.002% 0.0001% 0.01% 0.00003% 5% 5% 5% 0.5% 0.005% 0.01% n /a n /a 5% n /a 1%

147 5. Probabilities and uncertainties – an initial overview 1. Understanding 2. Data 3. Existing probability of sequence availability estimation all data all parts calculations with small uncertainty most data most parts calculations with large uncertainty some data some parts best guesses by experts no data no estimates none at all amount of data to make probability degree of events from today’s actions kind of estimation and uncertainty assessment on all relevant steps to infinite impact of the sequence Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 145

148 5. Probabilities and uncertainties – an initial overview Extreme Climate Change The IPCC process ensures that Extreme climate change is one of Most models indicate that it will be the risks where global research data is widely available and of decades before the Earth reaches collaboration has taken place on a good quality, thanks to intensive equilibrium, and some impacts, significant scale. peer review in the natural science like sea-level rise, will happen 0.01% 602 area. Estimates of political and This long interval over millennia. 0.01% 0.001% 0.1% one in ten The IPCC process is an technological development exist, but between action and impact means one in a one in a hundred thousand they are more rudimentary compared unprecedented scientific achievement the probability of climate change is thousand thousand 601 to the natural sciences. that has helped provide a unique level expressed on a 200-year timescale, of understanding for such a complex compared with 100 years for the 0.0001% 1% area. Even so, the understanding of With such a high-profile area there are other challenges. 0.8% n /a n /a one in a one in a also a number of quantified estimates tipping points and collapses is still million hundred 603 rudimentary. From a risk perspective it in key areas such as emissions Based on available assessments is important to know that many factors trajectories, climate sensitivity, the best current estimate for that could result in infinite impacts impacts and thresholds that allow for extreme climate change in the 0.00001% 10% approximations of probabilities. But are excluded from most studies, for next 200 years is: one in ten one in only a few estimates exist that provide example the rapid release of methane million ten 600 Similarly, significant clathrates. probability assessments, as there is a 5% for infinite threshold, infinite impact % uncertainty exists about political tendency to use scenarios instead. 0.01% for infinite impact decisions in many countries, about the implementation of new solutions and One aspect that makes climate 0.0000001% 100% infinite threshold % about what lifestyles will dominate. change different from all other risks one in a hundred one in one 0.01% 0-10% is that the time from initial action million to impact is very long. The great uncertainty is where the threshold 0-10% lies where the planet begins to emit greenhouse gases that start irreversible feedbacks. 0.00013% 0.1% 0.002% 0.0001% 0.01% 5% 5% 0.00003% 5% Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 146 0.5% 0.005% 0.01% n /a n /a 1% n /a 5%

149 5. Probabilities and uncertainties – an initial overview 1. Understanding 2. Data 3. Existing probability of sequence availability estimation all data all parts calculations with small uncertainty most data most parts calculations with large uncertainty some data some parts best guesses by experts no data no estimates none at all amount of data to make probability degree of events from today’s actions kind of estimation and uncertainty assessment on all relevant steps to infinite impact of the sequence Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 147

150 5. Probabilities and uncertainties – an initial overview 0.01% 0.01% 0.001% 0.1% one in ten one in a hundred one in a thousand thousand thousand 0.0001% 1% Nuclear War n /a 0.8% n /a one in a one in a million hundred 0.00001% 10% one in ten one in million ten infinite impact % The data availability is relatively Nuclear war is the risk that started There are some estimates of key the work with scientific assessments low as much of the probability is aspects, such as the probability of 604 decided by factors that are secret related to infinite impact. accidental initiation of a nuclear war, 0.0000001% 100% infinite threshold % (e.g. the targets for nuclear weapons). but few estimates of the probability of one in a hundred one in one 0.01% 0-10% It depends on knowledge that by The understanding of the sequence a full-scale nuclear war. million is relatively well known. Still, the definition is unavailable (no nuclear 607 explosions, for example, have taken fact is that the impact will depend Based on available assessments place in a modern city); on human significantly on how serious the the best current estimate for nuclear 0-10% factors (e.g. stress tolerance and nuclear winter will be as the result of war in the next 100 years is: a war (if there is any nuclear winter aggressive tendencies among those 0.00013% at all). The probability of a nuclear who will have to decide whether 5% for infinite threshold, winter will depend on when during the or not to launch nuclear weapons); 0.005% for infinite impact year the war happens, and what the and on the effectiveness of current policies (e.g. how efficient current weather is during this time. The result deterrence policies are). The fact that is that the probability of an infinite impact has an inherent uncertainty climate change research has provided and can be estimated only once a war better scientific understanding of the has already started. probability and nature of a nuclear winter, thanks to better climate modelling, is worth noting as it 0.1% 0.002% 0.0001% 0.01% shows how research programmes on different global challenges can be 605 mutually supportive. 0.00003% 5% 5% 5% 0.5% 0.005% 0.01% n /a n /a 1% 5% n /a Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 148

151 5. Probabilities and uncertainties – an initial overview 1. Understanding 2. Data 3. Existing probability of sequence availability estimation all data all parts calculations with small uncertainty most data most parts calculations with large uncertainty some data some parts best guesses by experts no estimates none at all no data amount of data to make probability kind of estimation and uncertainty degree of events from today’s actions assessment on all relevant steps to infinite impact of the sequence Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 149

152 5. Probabilities and uncertainties – an initial overview Global Pandemic When it comes to data availability for On a general level the sequence, or Most of the probability estimates rather sequences, for a pandemic pandemics the situation is different made for pandemics are for their more compared with nuclear war or are relatively well-known. The benign versions. For the possible climate change, where the impact challenge here is that there are pandemic that could kill two billion or 0.01% 0.01% depends on something that can so many different scenarios that more there are very few estimates. 0.01% 0.01% 0.1% 0.001% 0.1% 0.001% one in ten one in ten be removed (nuclear weapons and it is very difficult to calculate all 608 one in a hundred one in a one in a one in a hundred thousand thousand the different possibilities even greenhouse gases). It is not possible Based on available assessments thousand thousand thousand thousand to get rid of mutating viruses and if the sequence is well-known. of a global the best current estimate Compared with climate change other organisms, so the data needed pandemic in the next 100 years is: 0.0001% 0.0001% 1% 1% is of another kind and magnitude. where greenhouse gases are a n /a 0.8% n /a n /a 0.8% n /a one in a one in a one in a one in a small group, and nuclear war There will always be new diseases 5% for infinite threshold, million million hundred hundred emerging, because there is constant where the number of warheads 0.0001% for infinite impact is relatively limited, the number of evolution, resulting from microbes ways that a global pandemic can looking for ecological niches. 0.00001% 0.00001% 10% 10% start is almost unlimited. The reason for the big difference one in ten one in ten one in one in With many of the spillover effects between threshold and impact is million million ten ten Making things worse too is the fact occurring in remote areas, even mainly that a pandemic will not infinite impact % infinite impact % basic data is still very rudimentary. that a global pandemic that reached directly affect infrastructure or the Scientists who collect data relevant the infinite threshold would most rest of the ecosystem in the way that for pandemics are often working with certainly be very different from almost extreme climate change or nuclear 0.0000001% 0.0000001% 100% 100% infinite threshold % infinite threshold % all earlier pandemics, and maybe very small resources, and there is no war would. This means that resilience one in a hundred one in a hundred one in one one in one 0-10% 0.01% 0-10% 0.01% something totally new that has never systematic way of collecting data on will be relatively better after the infinite million million a global scale, although interesting happened before. Understanding threshold is crossed. 607 While more than the most basic sequence initiatives are under way. 0-10% 0-10% an early warning system would be therefore becomes a challenge. comparatively inexpensive, there are still no resources available. 0.00013% 0.00013% 0.1% 0.002% 0.01% 0.0001% 0.002% 0.1% 0.01% 0.0001% 5% 5% 0.00003% 5% 0.00003% 5% 5% 5% Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 150 0.5% 0.5% 0.005% 0.01% 0.005% 0.01% n /a n /a n /a n /a n /a 5% 1% n /a 5% 1%

153 5. Probabilities and uncertainties – an initial overview 1. Understanding 2. Data 3. Existing probability of sequence availability estimation all data all parts calculations with small uncertainty most data most parts calculations with large uncertainty some data some parts best guesses by experts no data no estimates none at all amount of data to make probability degree of events from today’s actions kind of estimation and uncertainty assessment on all relevant steps to infinite impact of the sequence Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 151

154 5. Probabilities and uncertainties – an initial overview 0.01% 0.01% 0.01% 0.01% 0.1% 0.001% one in ten 0.1% 0.001% one in ten one in a one in a hundred thousand one in a hundred one in a thousand thousand thousand thousand thousand 0.0001% 1% 0.0001% 1% Ecological n /a n /a 0.8% n /a 0.8% n /a one in a one in a one in a one in a million hundred million hundred 609 Catastrophe 0.00001% 10% 0.00001% 10% one in ten one in one in ten one in million ten million ten infinite impact % infinite impact % There are many studies about the This is one of the more complex Regarding probability estimates, it is stability and possible collapse of risks as it can be seen more as a only ecological collapse and global different ecosystems, but there are heading than a description of a system collapse of the current man- 0.0000001% 100% infinite threshold % 0.0000001% 100% few that look into the possibility specific challenge with a well-defined made global challenges that have no infinite threshold % one in a hundred one in one 0-10% 0.01% one in a hundred one in one 0-10% 0.01% sequence. In other words it is not for a full ecological collapse that estimates for infinite impact. million million one sequence, but very many still would result in at least two billion 611 people suffering. unknown sequences. The concept Based on available assessments of ecological collapse usually refers of an the best current estimate 0-10% 0-10% to a situation where some part of the Data availability is good in many ecological catastrophe in the next 100 areas, but the challenge is that ecological web becomes so weak that years is: 0.00013% 0.00013% it collapses. without an understanding of the system dynamics, and because of its 0.5% for infinite threshold, complexity, there are inherent limits to Not available for infinite impact how exact the knowledge is that can 610 be achieved. 0.1% 0.002% 0.01% 0.0001% 0.002% 0.1% 0.0001% 0.01% 0.00003% 5% 5% 5% 0.00003% 5% 5% 5% 0.5% 0.005% 0.01% 0.5% 0.005% 0.01% n /a n /a n /a n /a n /a 5% 1% 5% n /a 1% Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 152

155 5. Probabilities and uncertainties – an initial overview 1. Understanding 2. Data 3. Existing probability of sequence availability estimation all data all parts calculations with small uncertainty most data most parts calculations with large uncertainty some data some parts best guesses by experts no data no estimates none at all amount of data to make probability degree of events from today’s actions kind of estimation and uncertainty assessment on all relevant steps to infinite impact of the sequence Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 153

156 5. Probabilities and uncertainties – an initial overview Global System Collapse Second, it is a recent system that Since the financial crisis the The data availability of global system possibility of a global collapse of has been so interconnected for collapse is something of a paradox. only a few years, as it depends on the current political, economic and On the one hand the system is almost financial system has been discussed an infrastructure that did not exist nothing but information, but at the before the internet, so there is little intensively. A rapidly evolving and same time data about how the system experience of how it works. increasingly interconnected system, it itself operates, what algorithms are is subject to unexpected, system-wide used, and so on, are not well known. Third, the system is rapidly failures because of the structure of the network – it faces a systemic risk. changing and becoming No estimate of the probability of a even more complex as more global system collapse that would Possible sequences for a global connections are added and result in an impact beyond the infinite its speed increases. Better system collapse resulting in infinite threshold has been identified during impacts are very hard to establish, understanding of complex the project. systems with multiple for three reasons. First, it is a very attractors and bifurcation complicated system, with many Based on available assessments the behaviour will help improve the dynamic interactions, as there are best current estimate of a global possibility of understanding many people who, together with system collapse in the next 100 613 the possible sequences. machines, react to each other. The years is: current global system shows a lot An additional challenge for the of complex dynamic phenomena, Not available for infinite threshold, understanding of sequences that such as business cycles, financial Not available for infinite impact can result in impacts beyond the crises, irregular growth, and bullwhip 612 infinite threshold is that almost all Many nonlinear dynamic effects. models of economics and finance research being done in the area of global system collapse focuses on its present various complex dynamic behaviours such as chaos, fractals, economic or geopolitical implications, not on a full system collapse and not and bifurcation. on human suffering. 0.01% 0.01% 0.01% 0.01% 0.1% 0.001% one in ten 0.001% 0.1% one in ten one in a one in a hundred thousand one in a hundred one in a thousand thousand thousand thousand thousand 0.0001% 1% 0.0001% 1% 0.8% n /a n /a one in a one in a n /a n /a 0.8% one in a one in a million hundred million hundred 0.00001% 10% 0.00001% 10% one in ten one in one in ten one in Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks million ten 154 million ten infinite impact % infinite impact % 0.0000001% 100% infinite threshold % 0.0000001% 100% infinite threshold % one in a hundred one in one 0.01% 0-10% one in a hundred one in one 0.01% 0-10% million million 0-10% 0-10% 0.00013% 0.00013% 0.002% 0.1% 0.0001% 0.01% 0.002% 0.1% 0.0001% 0.01% 0.00003% 5% 5% 5% 0.00003% 5% 5% 5% 0.5% 0.01% 0.005% 0.5% 0.005% 0.01% n /a n /a 5% n /a 1% n /a n /a n /a 1% 5%

157 5. Probabilities and uncertainties – an initial overview 1. Understanding 2. Data 3. Existing probability of sequence availability estimation all data all parts calculations with small uncertainty most data most parts calculations with large uncertainty some data some parts best guesses by experts no data no estimates none at all amount of data to make probability degree of events from today’s actions kind of estimation and uncertainty assessment on all relevant steps to infinite impact of the sequence Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 155

158 5. Probabilities and uncertainties – an initial overview Major Asteroid Impact 618 The understanding of sequence Other initiatives, like the the Based on available assessments Sentinel Mission by the B612 when it comes to asteroid impacts the best current estimate of a major is relatively straightforward, and our foundation, are under way that will asteroid impact in the next 100 planet is constantly experiencing further improve availability. years is: asteroids, so assumptions can be tested. This, combined with the fact Compared with most other 0.01% for infinite threshold, global challenges there are many that there has been a number of 0.00013% for infinite impact probability estimates with transparent major impacts in the Earth’s history, makes the sequence reasonably methodology, and the degree of 614 well known. uncertainty is relatively low compared with other challenges. NASA even The data availability is still far from has an overview of the probability of 617 For individual objects hitting Earth. perfect, but it is rapidly improving. the most severe impacts, the size of Currently NASA has a table with the asteroid will make it visible years in potential future Earth impact advance, and this will only improve as events that the JPL Sentry System our capacity to scan the space around has detected, based on currently 615 us increases. available observations. 0.01% 0.01% 0.01% 0.01% 0.1% 0.001% one in ten 0.001% 0.1% one in ten one in a one in a hundred thousand one in a hundred one in a thousand thousand thousand thousand thousand 0.0001% 1% 0.0001% 1% 0.8% n /a n /a n /a n /a 0.8% one in a one in a one in a one in a million hundred million hundred 0.00001% 10% 0.00001% 10% one in ten one in one in ten one in million ten million ten infinite impact % infinite impact % 0.0000001% 100% infinite threshold % 0.0000001% 100% infinite threshold % one in a hundred one in one 0-10% 0.01% one in a hundred one in one 0.01% 0-10% million million 0-10% 0-10% 0.00013% 0.00013% Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 156 0.1% 0.0001% 0.01% 0.002% 0.002% 0.1% 0.01% 0.0001% 5% 5% 0.00003% 5% 0.00003% 5% 5% 5% 0.5% 0.005% 0.01% 0.5% 0.005% 0.01% n /a n /a 1% 5% n /a n /a n /a 5% n /a 1%

159 5. Probabilities and uncertainties – an initial overview 1. Understanding 2. Data 3. Existing probability of sequence availability estimation all data all parts calculations with small uncertainty most data most parts calculations with large uncertainty some data some parts best guesses by experts no data no estimates none at all amount of data to make probability degree of events from today’s actions kind of estimation and uncertainty assessment on all relevant steps to infinite impact of the sequence Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 157

160 5. Probabilities and uncertainties – an initial overview Super-volcano 619 There is data available for different The super-volcano risk has many Based on available assessments similarities with a major asteroid risk. impacts, and knowledge of where of a super- the best current estimate Both have happened a number of super-volcanoes might erupt is volcano in the next 100 years is: times through our planet’s history, and increasing, but due to the lack of 0.01% understanding when it comes to the both have had major consequences. 0.002% for infinite threshold, 0.01% 0.1% 0.001% one in ten sequence, the probability estimations 0.00003% for infinite impact one in a hundred one in a thousand are still very rudimentary. The understanding of the sequence is thousand thousand however a lot lower than for asteroids, A number of estimates exist where as the mechanisms behind volcano 0.0001% 1% eruptions are not very well known. The the probability is assessed, but they 0.8% n /a n /a one in a one in a possibility of foreseeing when a super- are quite rudimentary, based on the million hundred volcano will erupt and how big the historic frequency of earlier super- impact will be is therefore low. Compared volcano eruptions. As these are so with a major asteroid, there will therefore infrequent, the uncertainty becomes 0.00001% 10% very significant. be much less time to prepare. one in ten one in million ten infinite impact % 0.0000001% 100% infinite threshold % one in a hundred one in one 0.01% 0-10% million 0-10% 0.00013% 0.1% 0.01% 0.0001% 0.002% 0.00003% 5% 5% 5% Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 158 0.5% 0.005% 0.01% n /a n /a n /a 5% 1%

161 5. Probabilities and uncertainties – an initial overview 1. Understanding 2. Data 3. Existing probability of sequence availability estimation all data all parts calculations with small uncertainty most data most parts calculations with large uncertainty some data some parts best guesses by experts no data no estimates none at all amount of data to make probability degree of events from today’s actions kind of estimation and uncertainty assessment on all relevant steps to infinite impact of the sequence Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 159

162 5. Probabilities and uncertainties – an initial overview 0.01% 0.01% 0.01% 0.01% 0.001% 0.1% one in ten 0.001% 0.1% one in ten one in a hundred one in a thousand one in a one in a hundred thousand thousand thousand thousand thousand 0.0001% 1% 0.0001% 1% Synthetic 0.8% n /a n /a one in a one in a 0.8% n /a n /a one in a one in a million hundred million hundred Biology 0.00001% 10% 0.00001% 10% one in ten one in one in ten one in million ten million ten 622 infinite impact % Many experts see synthetic biology One of the challenges to Based on available assessments infinite impact % as the most serious future risk. The understanding the sequence is that of an the best current estimate ability already exists to develop very the spreading of synthetic biology impact from synthetic biology in the 0.0000001% 100% infinite threshold % deadly viruses, and as knowledge will come either from a wilful act next 100 years is: 0.0000001% 100% infinite threshold % one in a hundred one in one 0-10% 0.01% and technology develop further, more (e.g. terrorism) or an accident one in a hundred one in one 0.01% 0-10% million (e.g. unintentional release from a deadly pandemics can be developed 1% for infinite threshold, million by an increasing number of people. laboratory). This also makes data 0.01% for infinite impact hard to get. There are some numbers 0-10% 0-10% for accidents in labs, but they are The basic sequence is relatively The probability numbers for synthetic available in only a few countries and well-known, given that it would be a biology are very high and can 0.00013% there are probably many more than more deadly version of a current virus, hopefully be reduced once better 0.00013% 620 With terrorist acts those reported. but there is also the possibility that monitoring is in place, together with there are probability estimates that a new virus (or other organism) may increased global collaboration. can be used as a basis for the use of be found where the sequence will be 621 synthetic biology as well. unknown and therefore also much more dangerous. There are some existing estimates for synthetic biology, but these are based on possible use in war, where calculations depend on some specific 0.1% 0.002% 0.01% 0.0001% differences from existing pathogens 0.002% 0.1% 0.01% 0.0001% that are assumed to be necessary for a pandemic with an infinite impact. 5% 0.00003% 5% 5% 0.00003% 5% 5% 5% 0.5% 0.01% 0.005% 0.5% 0.005% 0.01% n /a n /a 1% n /a 5% n /a n /a n /a 1% 5% Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 160

163 5. Probabilities and uncertainties – an initial overview 1. Understanding 2. Data 3. Existing probability of sequence availability estimation all data all parts calculations with small uncertainty most data most parts calculations with large uncertainty some data some parts best guesses by experts no data no estimates none at all amount of data to make probability degree of events from today’s actions kind of estimation and uncertainty assessment on all relevant steps to infinite impact of the sequence Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 161

164 5. Probabilities and uncertainties – an initial overview Nanotechnology Compared with many other global Nanotechnology is best described Admiral David E. Jeremiah, as a general capacity, rather than challenges, nanotechnology could for example, said at the 1995 a specific tool. In relation to infinite result in many different risks and Foresight Conference on Molecular impacts this is a challenge, as there opportunities, from an accelerated Technology: “Military applications 623 ability to manufacture (new) weapons are many ways that nanotechnology of molecular manufacturing have can be used that could result in infinite to the creation of new materials and even greater potential than nuclear impacts, but also many others where substances. These are certainly orders weapons to radically change the 624 of magnitude more likely, far likelier it can help reduce infinite impacts. A systems- balance of power.” than any probability of the “grey forecasting approach could probably goo” that has resulted in significant Different possible sequences from provide better estimates and help misunderstanding. today’s situation to precise atomic develop complementary measures manufacturing are well documented, that would support the positive parts and the probability that none of The data availability is difficult to of nanotechnology while reducing the possible paths would deliver estimate as there are very different the negative. results is very small. What specific kinds of data, and also an obvious 625 lack of data, as nanotechnology is in sequence and with what results is Based on available assessments its very early days. however very uncertain. of an the best current estimate impact from nanotechnology in the There are some estimates from next 100 years is: experts, but the uncertainty is significant. A relative probability 0.8% for infinite threshold, estimate is a possible first step, 0.01% for infinite impact comparing nanotechnology solutions with existing systems where the probability is better known. 0.01% 0.01% 0.01% 0.01% 0.1% 0.001% one in ten 0.001% 0.1% one in ten one in a hundred one in a thousand one in a hundred one in a thousand thousand thousand thousand thousand 0.0001% 1% 0.0001% 1% n /a 0.8% n /a n /a n /a 0.8% one in a one in a one in a one in a million hundred million hundred 0.00001% 10% 0.00001% 10% one in ten one in one in ten one in Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks million 162 ten million ten infinite impact % infinite impact % 0.0000001% 100% infinite threshold % 0.0000001% 100% infinite threshold % one in a hundred one in one 0-10% 0.01% one in a hundred one in one 0.01% 0-10% million million 0-10% 0-10% 0.00013% 0.00013% 0.1% 0.002% 0.0001% 0.01% 0.002% 0.1% 0.0001% 0.01% 5% 0.00003% 5% 5% 0.00003% 5% 5% 5% 0.5% 0.005% 0.01% 0.5% 0.005% 0.01% n /a n /a 1% 5% n /a n /a n /a n /a 1% 5%

165 5. Probabilities and uncertainties – an initial overview 1. Understanding 2. Data 3. Existing probability of sequence availability estimation all data all parts calculations with small uncertainty most data most parts calculations with large uncertainty some data some parts best guesses by experts no data no estimates none at all amount of data to make probability degree of events from today’s actions kind of estimation and uncertainty assessment on all relevant steps to infinite impact of the sequence Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 163

166 5. Probabilities and uncertainties – an initial overview Artificial Intelligence 627 Artificial Intelligence is the global risk What is possible is to define a number Based on available assessments where least is known. Not even those of general factors determining of an the best current estimate who see the possibility of developing risk. These include Capability and impact from AI in the next 100 years 626 For global Compatible goals. an AI claim to be able to describe is: challenges in rapidly evolving areas what a working AI is in detail, let alone provide a description of the sequence where incremental development might 0-10% for infinite threshold, not happen and little is known about from where we are today to an AI that 0-10% for infinite impact could result in infinite impact. the sequence, the only way to reduce risks with possible infinite impacts The reason for 0-10% on both impact might be to ensure focus on these The assumptions for an AI are based levels is that most experts assume on the current rapid technological general factors. that the kind of AI capable of impacts development, but as it is not even beyond the infinite threshold is likely The only estimates of probabilities possible to simulate a simple version to be one that also can result in an of AI it is hard to get any data. that exist so far have been made infinite impact. If we succeed it will by a small group with a significant move beyond control very rapidly. proportion of people with a passion Due to the significant impact it would for AI. Compared with many other have if it worked, there is no difference challenges the possibility of an AI between the two impact levels. capable of infinite impact can almost be described as all or nothing. This 0.01% 0.01% is also why the estimates are the 0.01% 0.01% 0.1% 0.001% one in ten same for the infinite threshold and the 0.001% 0.1% one in ten one in a hundred one in a thousand one in a one in a hundred thousand infinite impact. thousand thousand thousand thousand 0.0001% 1% 0.0001% 1% n /a n /a 0.8% n /a n /a 0.8% one in a one in a one in a one in a million hundred million hundred 0.00001% 10% 0.00001% 10% one in ten one in one in ten one in million ten million ten infinite impact % infinite impact % 0.0000001% 100% infinite threshold % 0.0000001% 100% infinite threshold % one in a hundred one in one 0-10% 0.01% one in a hundred one in one 0.01% 0-10% million million 0-10% 0-10% 0.00013% 0.00013% Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 164 0.002% 0.01% 0.1% 0.0001% 0.002% 0.1% 0.0001% 0.01% 0.00003% 5% 5% 5% 0.00003% 5% 5% 5% 0.5% 0.005% 0.01% 0.5% 0.005% 0.01% n /a n /a 5% n /a 1% n /a n /a n /a 1% 5%

167 5. Probabilities and uncertainties – an initial overview 1. Understanding 2. Data 3. Existing probability of sequence availability estimation all data all parts calculations with small uncertainty most data most parts calculations with large uncertainty some data some parts best guesses by experts no data no estimates none at all amount of data to make probability degree of events from today’s actions kind of estimation and uncertainty assessment on all relevant steps to infinite impact of the sequence Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 165

168 5. Probabilities and uncertainties – an initial overview Unknown Consequences 628 Uncertain risks must be included But we know from history that these Based on available assessments in any project addressing low- kinds of events happen over and the best current estimate of an over again. With rapid technological probability high-impact events. The uncertain risk in the next 100 years is: development and increased tensions way to approach uncertain risks is, by 0.01% 0.01% definition, uncertain. over coming decades, the magnitude 5% for infinite threshold, 0.01% 0.01% 0.1% 0.001% 0.001% 0.1% one in ten one in ten of the impacts can be assumed to 0.1% for infinite impact one in a one in a hundred one in a hundred one in a thousand thousand The sequence can only be assessed increase. The probability estimates thousand thousand thousand thousand on the basis of experience of exist only as best guesses by experts, and while it is possible to make unexpected events, so actual data 0.0001% 0.0001% 1% 1% does not exist. the assessments more formal, it is n /a n /a n /a 0.8% 0.8% n /a one in a one in a one in a one in a currently the best existing estimates million million hundred hundred that at least provide a preliminary order of magnitude for these risks. 0.00001% 0.00001% 10% 10% one in ten one in ten one in one in million million ten ten infinite impact % infinite impact % 0.0000001% 0.0000001% 100% 100% infinite threshold % infinite threshold % one in a hundred one in a hundred one in one one in one 0.01% 0-10% 0.01% 0-10% million million 0-10% 0-10% 0.00013% 0.00013% 0.002% 0.1% 0.002% 0.1% 0.01% 0.01% 0.0001% 0.0001% 5% 0.00003% 0.00003% 5% 5% 5% 5% 5% Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 166 0.5% 0.5% 0.005% 0.005% 0.01% 0.01% n /a n /a n /a n /a n /a 1% 5% 5% 1% n /a

169 5. Probabilities and uncertainties – an initial overview 1. Understanding 2. Data 3. Existing probability of sequence availability estimation all data all parts calculations with small uncertainty most data most parts calculations with large uncertainty some data some parts best guesses by experts no data no estimates none at all amount of data to make probability degree of events from today’s actions kind of estimation and uncertainty assessment on all relevant steps to infinite impact of the sequence Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 167

170 5. Probabilities and uncertainties – an initial overview 0.01% 0.01% 0.01% 0.01% 0.001% 0.1% one in ten 0.001% 0.1% one in ten one in a hundred one in a thousand one in a one in a hundred thousand thousand thousand thousand thousand 0.0001% 1% 0.0001% 1% Future Bad n /a 0.8% n /a one in a one in a 0.8% n /a n /a one in a one in a million hundred million hundred Global Governance 0.00001% 10% 0.00001% 10% one in ten one in one in ten one in million ten million ten infinite impact % As there is no global government, The probability of a bad global the Based on available assessments infinite impact % global governance today typically governance system will increase the of a future bad best current estimate longer it takes to implement solutions involves a range of actors including global governance with potentially 0.0000001% 100% infinite threshold % to address global challenges. states, regional and international infinite impacts in the next 100 years is: 0.0000001% 100% infinite threshold % one in a hundred one in one 0.01% 0-10% When the world experiences organisations. A single organisation one in a hundred one in one 0-10% 0.01% million may nominally be given the lead role significant negative impacts, the Not available for infinite threshold, million 629 time for reflection will be shorter and on an issue. Not available for infinite impact polarisation is likely to increase. 0-10% 0-10% Future bad global governance is an important challenge, although To establish a failed governance 0.00013% system on a global level will require it is totally different from the other 0.00013% something extraordinary, as nations challenges. We should remember that at present about two billion people tend to protect their national live in poverty, and the risks from sovereignty at almost any cost. global challenges are increasing. No governance system is perfect, but it What can help in understanding is possible to improve significantly on possible sequences is an increasing the current system without increasing amount of data on how complex organisations work and the the risks. psychology of organisations that can 630 0.1% 0.002% 0.0001% 0.01% create destructive patterns. 0.1% 0.002% 0.01% 0.0001% 0.00003% 5% 5% 5% 0.00003% 5% 5% 5% 0.5% 0.005% 0.01% 0.5% 0.005% 0.01% n /a n /a 1% 5% n /a n /a n /a 5% 1% n /a Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 168

171 5. Probabilities and uncertainties – an initial overview 1. Understanding 2. Data 3. Existing probability of sequence availability estimation all data all parts calculations with small uncertainty most data most parts calculations with large uncertainty some data some parts best guesses by experts no data no estimates none at all amount of data to make probability degree of events from today’s actions kind of estimation and uncertainty assessment on all relevant steps to infinite impact of the sequence Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 169

172 6. Underlying trends of key importance 6. Underlying trends of key importance “Those who do not remember the past are condemned to repeat it.” George Santayana Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 170

173 6. Underlying trends of key importance / 6.1 Poverty There are reasons to celebrate this To address the global challenges 6.1 Poverty means recognising several development as more people than ever live a life where they do not have underlying trends. These will to constantly worry about their most influence the challenges by Global poverty has fallen dramatically affecting society as a whole as basic needs. But there are two things over the last two centuries, and the worth remembering: fall has intensified in recent decades, well as more directly affecting the raising hopes that poverty, defined challenges. Currently most risk 1. Poverty could increase again. by the World Bank as an income assessments and other studies 2. Defining poverty is difficult. related to global challenges below US $1.25 per day, may be neglect the fact that these trends eliminated within the next 50 years. Today very few people assume that The Economist even had a cover, in can have very different outcomes. June 2013, with the title “Towards the poverty could increase again. But 631 The World Bank everything from economic crisis and end of poverty”. The most common mistake is that the most likely development of these has set an interim target of reducing pandemics to climate change and global extreme poverty to 9% of the wars could change that. The situation underlying trends is taken for granted after the fall of the Soviet Union as the only possible outcome. world’s population by 2020, which, if 634 resulted in increased poverty. Most of the trends have probability achieved, would mark the first time 632 distributions where low-probability/ the rate has fallen to single digits. The milestone is based on a World Even conservative estimates show high-impact possibilities are often that the percentage of people in Bank economic analysis of global ignored. In this chapter some of poverty by 2030 could range from poverty trends aimed at the goal of the most important trends, where 636 the possible outcomes can differ almost zero to nearly 20%. ending extreme poverty by 2030. significantly, are described through a global risk perspective. Reaching 9% in 2020 would mean an estimated 690m people would still be living in extreme poverty by For each of the trends the simple rule based on a risk perspective is: then, 510m fewer in poverty than a decade earlier. That would be the “Aim for the best, but be prepared for the worst”. equivalent of half the population of Africa, or more than double the 633 population of Indonesia. 635 Figure 25: Russian Male Life Expectancy 637 Figure 26: Prospects for Ending Extreme Poverty by 2030. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 171

174 6.1 Poverty Even more complicated is the The conclusion is that climate Understanding the relationship strategies should prepare for definition of poverty. The level between poverty and global successful poverty reduction by of $1.25 per day is a very rough challenges requires us to develop setting targets and developing indicator and does not say very much strategies that help ensure poverty about the situation of the people reduction. Planning means it is solutions that work in a world with low living life at that level. How desperate poverty, and should not assume high important to assume different levels of poverty reduction - for example: levels of poverty. At the same time are they (important for war/terrorism)? For successful poverty reduction, What risks do they feel they must our strategies for pandemics must take (important for climate change low-carbon solutions are crucial, as assume that poverty reduction could fail and develop solutions accordingly. and ecological collapse as people it is rich people who are the main emitters on the planet because of will engage in illegal deforestation)? their unsustainable lifestyles. Poverty is also important in If poverty reduction is unsuccessful, pandemics, as reduced income can structures to address a higher result in increased migration and also likelihood of outbreaks that can turn increased hunting of wild animals. into pandemics are required as poor people tend to live in societies where they are more likely to get infected and where often even basic health service is lacking. 2030 Number of people in poverty, m 1990 2010 Sub-Saharan Africa India China $ 1.25 $ 1.25 $ 1.25 30 30 30 25 25 25 20 20 20 15 15 15 10 10 10 5 5 5 0 0 0 6 5 2 3 4 5 6 4 0 0 0 1 2 3 4 5 6 1 2 3 1 Mean daily consumption (PPP $) Mean daily consumption (PPP $) Mean daily consumption (PPP $) 638 Figure 27: Different kinds of poverty - Number of people in poverty Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 172

175 6.2 Population growth The fact that projections can change The high-variant projection depicted 6.2 Population is clearly demonstrated by the in the figure below assumes an extra growth difference between the current (2012) half a child per woman (on average) compared with the medium variant, Revision, and the 2010 Revision of World Population Prospects. The Population growth is a trend that implying a world population of 10.9 645 latter was published only two years bn in 2050 and 16.6 bn in 2100. has been discussed intensively That is equal to a 133% population from a sustainability and risk earlier and projected world population reaching 9.3 bn in 2050 and 10.1 bn increase in just 86 years . perspective since Malthus did his 647 639 This is famous projection. in 2100 (medium variant). The difference between projections almost a 10% difference in the space A “natural population increase” of two years. for 2100, from 10.9 bn people in the medium scenario, to 16.6 bn in occurs when the birth rate is the high scenario, equals the world higher than the death rate. While Current ways to provide the lifestyles enjoyed in countries like the UK population in 1995. There is also a a country’s population growth rate credible low scenario with 6.8 bn and US today would require 3.5-5 depends on this natural increase 646 A strategic approach and on migration, global population by 2100. planets, while the global population 648 Under the growth is determined exclusively by is about 7 bn people. must be based on all possible 640 outcomes. Planning as though the natural increase. high-variant projection, more than 10 world population will be only 6.8 bn planets would be needed. Around the world, death rates is not optimistic: it is unscientific gradually decreased in the late 19th and dangerous. Even to plan for a With other underlying trends, and the 20th centuries, with those technology breakthroughs and world with 10.9 bn is not strategic in developing countries plummeting as this would ignore the significant institutional changes can result after World War II thanks to the spread probability that the world’s population in very rapid changes. Global of modern medicine which allowed population growth cannot change would be much larger. There should 641 control of infectious diseases. as rapidly. And as it is related be a plan for a world with 16.6 bn people, combined with a long-term to many factors, including other According to the 2012 Revision of underlying trends such as income strategy to ensure a sustainable the official United Nations population population level. levels, education, access to health estimates and projections, the world services and cultural values that population of 7.2 billion in mid-2013 It is also important to ensure that are all assumed to be undergoing is projected to increase by almost more attention is paid to early significant changes over coming one billion people within twelve decades, population growth over warning systems that allow us to years, reaching 8.1 bn in 2025, and to long time periods is even more influence population development in further increase to 9.6 bn in 2050 and difficult to estimate. a sustainable direction. 642 10.9 bn by 2100. These results are based on the medium-variant projection, which assumes a decline of fertility for countries where large families are still prevalent as well as a slight increase of fertility in several countries with fewer than two children per woman 643 on average. The medium projection is still dramatic as it assumes another four bn people on the planet, more than a 50% increase in population, equal to the Earth’s entire population in 1975, 644 in just 86 years. Figure 28: Population of the world, 1950-2100, according to different projections and variants Source: http://esa.un.org/wpp/documentation/pdf/WPP2012_Volume-I_Comprehensive-Tables.pdf, p. xv Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 173

176 6.3 Technological development forever. Second, nature itself may On the other hand there are natural 6.3 Technological set limits. We may choose to take limits that could begin to constrain development technological development in more care of the planet, or limits to materials like rare earths may begin two ways. The technology itself 655 But regardless to slow technology. Technological development since the may hit a barrier. For example, at industrial revolution has been faster some stage a processor may not of ultimate limits, many exponential trends are likely to continue over the continue to become smaller and than most experts expected. For coming decades and will present us material welfare this has been very faster, as the speed of light and quantum mechanics will limit its with new opportunities as well as positive; average longevity and health 654 There might be other improvements in general have all risks in the 21st century, as these development. trends converge in a society with shown dramatic positive development. ways to overcome such boundaries, During the second half of the 20th 20th century institutions. but no exponential trend can last century global health improved more than in all previous human history. Average life expectancy at birth in low- and middle-income countries increased from 40 years in 1950 to 65 649 years in 1998. While weapons have become more deadly the death toll from wars has 650 How actually decreased over time. big a part technology has played by creating greater transparency, or increasing the fear of using weapons which have become too powerful (for example nuclear bombs), is disputed. But most experts agree that technology has played an important 651 This is not the same as saying role. 656 Figure 29: Moore’s forecast for PV 657 Figure 30: Global ICT development 2000-2013 that this development will continue. Estimating the future development of technology is very difficult. On 100 95.5 the one hand there is evidence that 90 mobile-cellular telephone subscriptions technology will continue to accelerate individuals usign the Internet at the pace it has achieved so far. fixed-telephone subscriptions 80 Researchers at MIT and the Santa Fe Institute have found that some widely 70 used formulas for predicting how 60 rapidly technology will advance — notably, Moore’s and Wright’s Laws 50 — offer superior approximations of 652 40.4 the pace of technological progress. 40 Experts like Ray Kurzweil, who was 30 recently hired by Google, is one of those who think that most people 20 15.8 do not understand the implications of exponential growth in the area 10 of technology and the results it Per 100 inhabitants 0 2010 2012 2014 2000 2002 2004 2006 2008 generates in price, capacity and 653 Global ITC developments, 2000-2014* overall transformation of society. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 174

177 6.3 Technological development / 6.4 Demographic changes How technological development can In 2000 there were an estimated 6.4 Demographic 180,000 centenarians globally. By be supported in order to increase changes 2050 they are projected to number opportunities and reduce risks will be 3.2 m, an increase of about eighteen increasingly important to discuss. 659 times. The world’s population is undergoing As technology in many areas is a massive demographic shift. Fertility Within the more developed regions, developing exponentially, it is rates have fallen and the number of Japan, in particular, will experience important to analyse its development children has stopped growing. It is a a remarkable increase in the number very carefully. The potential for historic shift. technology to help solve existing and of centenarians over the next half future global challenges is almost Those who are 80 or more now make century, from fewer than 13,000 in 2000 to almost 1 m in 2050. By limitless. And so unfortunately is its up only slightly more than 1% of the then Japan is expected to have by total human population. This proportion potential to accelerate existing risks and create new ones. Too many is projected to increase almost fourfold far the world’s largest number and proportion of centenarians, nearly initiatives today focus on only one over the next 50 years, to reach 4.1% 660 1% of its population. side of technology, either the positive in 2050. Currently, only one country, or the negative. Acknowledging both Sweden, has more than 5% in this age group. By 2050 the over-80s are The stagnating and ageing sides is necessary in order to ensure a strategic response. population in many OECD countries projected to number almost 379 m and China will put pressure on people globally, about 5.5 times as many as in 2000 (69 m). In 1950, the current systems, which were not over-80s numbered under 14 m. designed to deal with a situation of ageing and often shrinking Although the proportion of people who populations in many parts of the live beyond 100 is still very small, their world, while the populations in other parts of the world are rapidly growing. number is growing rapidly. 2050 379.0 2025 153.4 2000 69.2 1975 31.4 1950 13.8 millions population aged 80 or over: World, 1950-2050 661 Figure 24: Population aged 80 or over Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 175

178 7. Possible ways forward 7. Possible ways forward “Our very survival depends on our ability to stay awake, to adjust to new ideas, to remain vigilant and to face the challenge of change.” Martin Luther King Jr. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 176

179 7. Possible ways forward To better address global challenges Global challenges with a potentially infinite impact, both immediate action and long- leadership networks 1. term work are needed. Below are ten areas that could help mitigate Better quality risk immediate threats while also contributing to a future global assessment for 2. governance system capable of global challenges addressing global challenges with infinite impacts. For all these areas more research is needed. Development of early warning systems 3. Encouraging visualisation of 4. complex systems Highlighting early movers 5. Including the whole 6. probability distribution Increasing the focus on 7. the probability of extreme events Encouraging 8. appropriate language to describe extreme risks Establishing 9. a Global Risk and Opportunity Indicator to guide governance Explore the possibility of establishing a 10. Global Risk Organisation (GRO) Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 177

180 7. Possible ways forward Global challenges leadership networks 1. Better quality risk Four groups are of particular infinite impacts, and could work The long-term goal needs to be assessment for 2. importance: experts in finance, the establishment of a transparent on a roadmap for a future global global challenges experts in security policy, lawyers and democratic global governance governance system that can address existing and new global challenges. with knowledge of global risks and system that can address global international law, and finally a group challenges with infinite impacts. To Development of The networks should be as support this leadership, networks can consisting of clusters of stakeholders early warning systems 3. transparent and inclusive as possible, be established involving interested with solutions that can reduce the especially as global collaboration is risks. Leadership networks that governments, major companies, Encouraging include participants from all four needed. The use of new collaboration NGOs, researchers and other relevant visualisation of 4. groups are of particular interest. tools and principles, such as wiki- stakeholders. These networks could complex systems processes and creative commons, develop strategies to address multiple Global challenges should be encouraged. challenges with potentially leadership networks 1. Highlighting early movers 5. Better quality risk assessment for 2. Including the whole global challenges 6. probability distribution Development of Increasing Institutions and universities methodology development could There is currently no global early warning systems 3. the focus on 7. be accelerated and improved, as engaged in developing new coordination when it comes the probability to risk assessments of global the possibility of learning from methodologies to assess global Encouraging risks have a particular responsibility different areas would increase. Such challenges. Different experts use of extreme events visualisation of 4. a process could also encourage different methodologies, data for developing and refining risk Global challenges assessments for global challenges. and ways to present their results, increased investments in methodology complex systems Encouraging leadership networks 1. development based on the latest making it very difficult to compare 8. appropriate language innovations, such as systems- the risk assessments that exist. to describe extreme risks Highlighting forecasting approaches. By establishing a process that Better quality risk early movers 5. coordinates and encourages risk assessment for 2. Establishing assessments of global challenges, global challenges 9. a Global Risk and Including the whole 6. Opportunity Indicator probability distribution Development of to guide governance early warning systems 3. Increasing the focus on 7. Explore the possibility Encouraging 662 the probability These opportunities “big data”. be assumed to increase or decrease The rapid technological development of establishing a 10. visualisation of 4. has many benefits, but also include both new ways of collecting global risks. Such a system would of extreme events Global Risk Organisation (GRO) complex systems allow more time to develop strategies large amounts of high-quality data, challenges, as risks can rapidly become very serious and reach infinite and new ways to analyse them. to mitigate risks and turn global Encouraging thresholds. To develop early warning challenges into opportunities for Highlighting 8. appropriate language innovation and collaboration. systems that can gather and process Early warning systems should be early movers 5. to describe extreme risks built to ensure that data is collected data transparently is therefore of the and analysed in ways that can be The warning system would require utmost importance. Technological Including the whole Establishing useful for multiple global challenges. progress, from smart phones and significant research into infinitey 6. probability distribution 9. a Global Risk and The warnings should not only include sensors to significant processing thresholds. Both traditional as well as Opportunity Indicator changes in the physical world, power and networks, allows for totally more recent methodologies based on Increasing to guide governance understanding of complex systems new ways of establishing early warning but also indicate when decisions, the focus on 7. should be encouraged. investments and legal changes can systems based on so-called the probability Explore the possibility of extreme events of establishing a 10. Global Risk Organisation (GRO) Encouraging Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 178 8. appropriate language to describe extreme risks Establishing 9. a Global Risk and Opportunity Indicator to guide governance Explore the possibility of establishing a 10. Global Risk Organisation (GRO)

181 Global challenges leadership networks 1. Better quality risk assessment for 2. global challenges 7. Possible ways forward Development of early warning systems 3. Encouraging visualisation of 4. complex systems Global challenges leadership networks 1. Highlighting The global challenges depend on New visualisation tools could early movers 5. a very complex ecosystem and help make complex systems easier Better quality risk social system. to understand and also help the assessment for 2. communication of challenges Including the whole 663 global challenges With a global economic and and opportunities. 6. probability distribution technological system, which both helps and creates risks that are Visualisation tools are needed both Development of Increasing for decision makers to highlight the increasingly interconnected and early warning systems 3. the focus on 7. consequences of different strategies difficult to understand, there is a the probability challenge to understand and for citizens to increase their basic Encouraging of extreme events the challenges. understanding of infinite impacts. visualisation of 4. Global challenges complex systems Encouraging leadership networks 1. 8. appropriate language to describe extreme risks Highlighting Better quality risk early movers 5. assessment for 2. Establishing global challenges 9. a Global Risk and Including the whole Governments, companies, In particular, leadership with a focus Opportunity Indicator 6. probability distribution organisations and networks working on multiple global challenges and the Development of to guide governance on global challenges should relationship between them should be early warning systems 3. Increasing increase their efforts to reward highlighted, as very little is being done the focus on 7. leadership when they find it. Major in this area. Explore the possibility Encouraging the probability news outlets can also report when of establishing a 10. visualisation of 4. of extreme events significant positive steps are taken Global Risk Organisation (GRO) complex systems to reduce global risks with potential infinite impacts. Encouraging 8. Highlighting appropriate language early movers 5. to describe extreme risks Including the whole Establishing 6. probability distribution 9. a Global Risk and Opportunity Indicator to guide governance Increasing Current lack of data and of scientific Tables, graphs and key conclusions Governments, major companies, the focus on 7. in reports related to global challenges studies regarding low-probability NGOs, researchers and other the probability should, when possible, include the high-impact outcomes in many areas relevant stakeholders should address Explore the possibility 664 of extreme events whole probability distribution. the whole probability distribution, should not be used as an excuse to of establishing a 10. including low-probability high- ignore the probability distribution. This Global Risk Organisation (GRO) is especially important when many of impact scenarios. This would ensure Encouraging that serious risks are not disregarded the global challenges have a very long 8. appropriate language and fat “tail”. or obscured. to describe extreme risks Establishing 9. a Global Risk and Opportunity Indicator to guide governance Explore the possibility of establishing a 10. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 179 Global Risk Organisation (GRO)

182 Global challenges leadership networks 1. Better quality risk assessment for 2. global challenges Development of early warning systems 3. Encouraging visualisation of 4. complex systems Global challenges leadership networks 1. Highlighting early movers 5. 7. Possible ways forward Better quality risk assessment for 2. Including the whole global challenges 6. probability distribution Increasing Development of the focus on early warning systems 3. 7. the probability of extreme events Encouraging visualisation of 4. Encouraging complex systems When the impact is infinite it is not The use of methodologies and Stakeholders should include the most 8. appropriate language enough only to reveal the whole approaches from security policy extreme impacts in all relevant work. to describe extreme risks If the probability of infinite impacts and the financial sector that focus probability distribution. It is important Highlighting increases instead of decreasing on extreme events could be used to also to avoid confusing uncertain risk early movers 5. develop strategies for rapid action because of new scientific findings or with low risk. Establishing lack of action, strategies should be beyond the incremental approaches 9. a Global Risk and Including the whole prepared to allow more decisive action. that dominate today. Infinite impacts render many of the Opportunity Indicator 6. probability distribution traditional models for risk management to guide governance almost meaningless. Monetary Increasing calculations are often useless, and the focus on Explore the possibility 7. discounting is not always advisable. the probability of establishing a 10. of extreme events Global Risk Organisation (GRO) Encouraging 8. appropriate language to describe extreme risks 2-9 dead and/or 10-49 Average: The IPCC uses specific and defined Establishing Often words like “unlikely”, “negligible” seriously injured language in its reports to describe 9. a Global Risk and and “insignificant” are used to describe different probabilities and thus ensure Opportunity Indicator a risk when the probability is considered 10-49 dead and/or 50-100 Large: clarity, but taken out of context and to guide governance low. What is low is however relative; seriously injured without supporting definitions this a low probability in one area can be language can be misleading. >50 dead and/or >100 Very large: extremely high in another. If I attend one Explore the possibility seriously injured of ten lectures - 10% - people might say For example, the term “very unlikely” of establishing a 10. there is a low probability that I will be is used by the IPCC to describe a Global Risk Organisation (GRO) 665 The use of terms that can be there. But if someone says that a new but probability of between 0-10%, aircraft crashes once in every ten flights, interpreted as having normative out of context its use could easily be most people will say that is an extremely values to explain probability is understood as a normative judgement high probability and will be likely to problematic and in future all bodies, suggesting that we do not need to including the IPCC, should explore assume it is an early prototype that is engage with the risk. the possibility of using only numbers nowhere close to commercial success. in external communications, at least The language of the IPCC can be in the summary for policy makers, to A major problem is that probabilities compared with that used in the that ought to be seen as very high for help everyone understand the reality Swedish National Risk Assessment 666 risks with potentially infinite impact are of the situation. The scale of impact is (SNRA). described in a way that makes them not defined for the IPCC, but for the sound less urgent than they are - by the Stakeholders should explore Swedish Assessment it is: ways to use language that better media, business, politicians and even by scientists. communicates how serious extreme Very small: no deaths or risks are in the case of climate serious injuries change, and where possible compare One example is how probabilities are this with other risk areas to help described by the Intergovernmental One dead and/or 1-9 Small: Panel on Climate Change. illustrate the situation. seriously injured Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 180

183 7. Possible ways forward IPCC Term Likelihood of the SNRA Term Likelihood of the Outcome: SNRA Outcome: IPCC Very certainly > 20% [more than once 99-100% probability Very high every 5 years] Very likely 90-100% probability Likely 66-100% probability About as likely as not 33-66% probability Unlikely 20%-2% [Between once High every 5 years and once every 50 years] Very unlikely Average 2%-0.02% [Between once 0-10% probability every 50 years and once every 500 years] 0-1% probability Exceptionally unlikely Low 0.02%-0.002% [Between once every 500 years and once every 5000 years] < 0.002% [Less than once Very low every 5000 years] 667 668 and the Likelihood Scale used by the IPCC Figure 31: Comparing the probability scale in the Swedish National Risk Assessment Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 181

184 Global challenges leadership networks 1. Better quality risk assessment for 2. global challenges Development of early warning systems 3. Encouraging visualisation of 4. complex systems Highlighting Global challenges early movers 5. leadership networks 1. Including the whole Better quality risk 6. probability distribution assessment for 2. global challenges Increasing the focus on 7. Development of the probability early warning systems 3. of extreme events 7. Possible ways forward Encouraging Encouraging visualisation of 4. 8. appropriate language complex systems to describe extreme risks Highlighting Establishing early movers 5. 9. a Global Risk and Opportunity Indicator Including the whole to guide governance 6. probability distribution Explore the possibility No mechanisms currently exist to companies, influence the probability has responsibility for increasing and Increasing provide updated and comprehensive decreasing the risk; who will suffer its of different impacts and outcomes. of establishing a 10. the focus on 7. global risk assessments for consequences, and who will benefit. Global Risk Organisation (GRO) the probability phenomena capable of threatening Establishing a global risk indicator, of extreme events Stakeholders should explore the with sub-indicators for different areas, human civilisation. as part of the UN system would help establishment of a global risk Encouraging With many unsustainable trends indicator that will help guide priorities create a better understanding of 8. appropriate language and inform society about different converging, it is crucial that leaders extreme global risks individually and to describe extreme risks of their interconnection, and it should are able to act before it is too late risks, and about the relationship and to assess how actions, such as between them. track both. An important feature political decisions or investments by would be its ability to illustrate who Establishing 9. a Global Risk and Opportunity Indicator to guide governance Explore the possibility of establishing a 10. Global Risk Organisation (GRO) In addition, and probably equally There is currently no international or important, is the fact that a body set up global body that is coordinating work on global risks with a potentially infinite to deal with such challenges could also impact. The following areas would ensure that the links between them benefit from global coordination: could be better understood. – Probability estimations A first step could be to establish – Early warning systems, a centre for global risks and 669 focusing initially opportunities, – Global coordination of solutions only on knowledge-gathering and – Legal development development of proposals, and with no mandate to implement any solutions. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 182

185 7. Possible ways forward Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 183

186 Endnotes Endnotes 1 27 Ridley cherrypicks data and tends to avoid the E.g. HBR blog about “black swans”: http://blogs. http://www.cnbc.com/id/100915696 2 probability that we will see significant warming. hbr.org/2010/09/the-competing-black-swans-of-s/ http://en.wikipedia.org/wiki/Global_catastrophic_risks http://www.mattridley.co.uk/blog/the-probable-net- and The Guardian about “perfect storms” http://www. 3 http://www.merriam-webster.com/dictionary/risk benefits-of-climate-change-till-2080.aspxhttp://www. theguardian.com/environment/earth-insight/2014/ 4 http://en.wikipedia.org/wiki/Probability_density_ rationaloptimist.com/ apr/03/ipcc-un-climate-change-perfect-storm-zombie- function 15 oil Climate Impacts: http://www.ipcc.ch/report/ar5/wg2/ 5 Dickson, David CM. Insurance risk and ruin. 28 http://www.economist.com/node/18744401 Pandemics: http://www.cmu.edu/dietrich/sds/docs/ Cambridge University Press 2005. 29 http://e360.yale.edu/feature/living_in_the_ fischhoff/AF-GPH.pdf http://books.google.se/ anthropocene_toward_a_new_global_ethos/2363/ Nuclear war: http://www.ippnw.org/nuclear-famine.html books?id=9zPOMMqJXcUC&printsec 30 16 http://www.nickbostrom.com/existential/risks.html http://news.stanford.edu/news/2003/september24/ =frontcover&source=gbs_ge_summary_r&redir_ 31 tellerobit-924.html https://www.youtube.com/watch?v=VjZyOTES6iQ esc=y#v=onepage&q&f=false 32 17 http://www.fas.org/sgp/othergov/doe/lanl/ http://en.wikipedia.org/wiki/Long_tail 6 http://www.itl.nist.gov/div898/handbook/eda/ 33 docs1/00329010.pdf http://en.wikipedia.org/wiki/Human section3/eda352.htm 18 34 While the LA-602 document is relatively well-known, 7 Mammals have an average species lifespan from A recent example is IPCC WGII Summary for policy its final paragraph is often forgotten. The scientists origin to extinction of about 1 million years, but that makers. In this report http://ipcc-wg2.gov/AR5/ IPCC say: “However, the complexity of the argument and the will not necessarily apply to humans today because of acknowledges the need to include low-probability high- absence of satisfactory experimental foundations makes factors like local climate change or new species in the impact scenarios: “assessment of the widest possible further work on the subject highly desirable.” same ecological niche. The dinosaurs were around for range of potential impacts, including low-probability 19 135 million years and if we are intelligent, there are good Since Teller’s calculations the science has developed outcomes with large consequences, is central to chances that we could live for much longer. and is now clear that the main nuclear threat that understanding the benefits and trade-offs of alternative 35 potentially could threaten human civilisation is a full risk management actions.” Yet nothing is included in the http://en.wikipedia.org/wiki/Human_embryogenesis nuclar war and the consequenses that this could result in 36 report about impacts above 4 degrees. These four points are slightly rewritten versions of a from effects like a nuclear winter. 8 list of Nick Bostrom’s in the text “Existential Risks: As an infinite impact by definition can’t have 20 For an overview of positive and negative aspects happened before, models are needed. Analyzing Human Extinction Scenarios and Related of nanotechnology see for example Drexler, Eric and 9 Making Sense of Uncertainty: Why uncertainty is part Hazards” http://www.nickbostrom.com/existential/risks. Pamlin, Dennis (2013): “Nano-solutions for the 21st http://www.lse.ac.uk/CATS/Media/SAS012- of science html. Note that these four points were originally century. Unleashing the fourth technological revolution” MakingSenseofUncertainty.pdf developed for “existential risks”, those that threaten the http://www.oxfordmartin.ox.ac.uk/downloads/ 10 Pinker, Steven. The better angels of our nature: The extinction of intelligent life originating on Earth or the academic/201310Nano_Solutions.pdf decline of violence in history and its causes. Penguin UK, permanent destruction of its potential for desirable future 21 This is true for utopias ranging from Plato’s Republic, 2011. development. via Thomas More’s book Utopia, to Edward Bellamy’s 11 37 Giddens, Anthony. “Risk and responsibility.” The Posner, Richard A. Catastrophe: Risk and Response. Looking Backward and William Morris’ News from Oxford University Press, 2004, modern law review 62.1 (1999): 1-10. Nowhere. 12 38 http://blogs.ei.columbia.edu/2012/01/09/evolutionary- http://en.wikipedia.org/wiki/Value_of_life 22 See next chapter for the methodology. 39 psychology-of-climate-change/ http://yosemite.epa.gov/EE%5Cepa%5Ceed.nsf/ 23 A list of organisations and studies that discuss webpages/MortalityRiskValuation.html http://www-ee.stanford.edu/~hellman/Breakthrough/ challenges that threaten human civilisation can be found 40 book/chapters/frankenhaeuser.html Posner, Richard A. Catastrophe: risk and response. here: http://en.wikipedia.org/wiki/Global_catastrophic_ Oxford University Press, 2004. Loc 2363 “The challenge now is to help people extend their risks 41 attachments, their loyalties, and their engagement, to http://webarchive.nationalarchives.gov.uk/+/http:/ 24 The number two billion was established during the include people outside their own narrow circle, their www.hm-treasury.gov.uk/sternreview_index.htm workshop in Oxford and is not an exact number. Further own country, their own imminent future. This global 42 William Nordhaus , The Stern Review on the research is needed to establish a better understanding reorientation is a prerequisite for changing the present Economics of Climate Change http://www.econ.yale. of thresholds that can result in an infinite impact, fatal course of development.” edu/~nordhaus/homepage/stern_050307.pdf depending on what challenge resulted in the two billion 13 http://www-ee.stanford.edu/~hellman/Breakthrough/ 43 http://www.res.org.uk/view/art3Apr08Features.html impact and how the estimate for an infinite impact was book/chapters/frankenhaeuser.html 44 assumed to be between 0.01% and 10%. Nordhaus, William D. The Climate Casino: Risk, 14 Two of the most famous “optimists”, who tend to 25 Uncertainty, and Economics for a Warming World. Yale The definition is based on the definition used by look at only the parts of the probability distribution University Press, 2013. Loc 2895 Jared Diamond: http://www.jareddiamond.org/Jared_ that support their opinion, are the Danish writer Bjorn 45 Diamond/Collapse.html Nordhaus, William D. The Climate Casino: Risk, Lomborg and the British journalist Matt Ridley. While 26 Uncertainty, and Economics for a Warming World. Yale For examples see the recently established Centre scientists in the areas he writes about constantly refute University Press, 2013. Loc 2895 for Study of Existential Risk at Cambridge. It is an Lomborg, his message of optimism is well received by 46 interdisciplinary research centre focused on the study Nordhaus, William D. The Climate Casino: Risk, many policy makers and business leaders. https://www. of human extinction-level risks that may emerge from Uncertainty, and Economics for a Warming World. Yale ma.utexas.edu/users/davis/375/reading/sciam.pdf technological advance University Press, 2013. Loc 3176 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 184

187 Endnotes 47 Timothy M. Lenton, Juan-Carlos Ciscar: Integrating criteria%20-%20when%20is%20low%20enough%20 http://gcrinstitute.org/research/ tipping points into climate impact assessments. Springer good%20enough%20-%20saudi.pdf Institute for Defence Studies and Analyses 64 Netherlands 2013-04-01 http://link.springer.com/ SO 16732-1, Fire Safety Engineering – Guidance http://www.idsa.in/topics article/10.1007%2Fs10584-012-0572-8 on Fire Risk Assessment, International Organization for International Risk Governance Council 48 Standardization, Geneva, Switzerland, 2012. Carl Sagan use 10 million years in: Sagan, Carl http://www.irgc.org/issues/ 65 (1983). “Nuclear war and climatic catastrophe: Some An example of a concept that helped increase the Lifeboat Foundation policy implications”. Foreign Affairs 62: 275, and The focus on an important category is Genocide, which was http://lifeboat.com/ex/programs Blackwell’s Concise Encyclopedia of Ecology and other introduced by Raphael Lemkin. http://en.wikipedia.org/ Nuclear Threat Initiative sources provide an estimate of about 1 million years wiki/Genocide for mammals. http://eu.wiley.com/WileyCDA/WileyTitle/ http://www.nti.org/ 66 E.g. Vinso, Joseph D. “A Determination of the Risk productCd-0632048727.html Saving Humanity from Homo Sapiens of Ruin.” Journal of Financial and Quantitative Analysis 49 One billion years is used by Bruce E. Tonn in the 14.01 (1979): 77-100. http://shfhs.org/whatarexrisks.html paper Obligations to future generations and acceptable 67 E.g. Reed, Christina. Earth Science: Decade by Skoll Global Threats Fund risks of human extinction: “ Futures 41.7 (2009): 427- Decade. Infobase Publishing, 2009. http://www.skollglobalthreats.org/ 435. 68 Culpitt, Ian. Social policy and risk. Sage, 1999. Stimson Center 50 In Japan in Japan the life expectancy at age zero, that 69 http://en.wikipedia.org/wiki/Jared_Diamond http://www.stimson.org/programs/ is, at birth (LEB), is 83 years. In 2010 the world LEB was 70 The methodology applied in this study has been Risk Response Network 67.2. developed by Dennis Pamlin from Global Challenges 51 http://forumblog.org/communities/risk-response- This is based on a low estimate for the planet’s Foundation and Stuart Armstrong, James Martin network/ population as far out as current projections are made, Research Fellow, Future of Humanity Institute at the World Economic Forum 2100. http://esa.un.org/wpp/ Looking further into the University of Oxford together with Seth Baum, Executive far future, beyond 500 years, it is likely that humanity http://www.weforum.org/reports/global-risks-2014- Director of the Global Catastrophic Risk Institute and will have expanded into space and have a much larger report affiliate researcher at the Center for Research on population. Tower Watson Environmental Decisions, Columbia University. 52 Posner, Richard A. Catastrophe: Risk and Response. 71 http://www.towerswatson.com/en/Insights/IC-Types/ The literature overview is included as Appendix 1. Oxford University Press, 2004, Loc 2378 Survey-Research-Results/2013/10/Extreme-risks-2013 72 http://gcrinstitute.org/people/ 53 84 http://www.esa.int/Our_Activities/Space_Science/ http://www.towerswatson.com/en/Insights/IC-Types/ 73 http://cred.columbia.edu/about-cred/people/ Herschel/How_many_stars_are_there_in_the_Universe? Survey-Research-Results/2013/10/Extreme-risks-2013 affiliated-researchers/sethbaum/ 54 85 http://www.existential-risk.org/concept.pdf Page 50 http://www3.weforum.org/docs/WEF_ 74 http://wokinfo.com/ 55 McSharry, Patrick E., and David Orrell (2009). A GlobalRisks_Report_2014.pdf 75 http://scholar.google.com 86 Systems Approach to Forecasting. FORESIGHT: Page 50 http://www3.weforum.org/docs/WEF_ 76 See Appendix 1 for the full overview. The International Journal of Applied Forecasting, GlobalRisks_Report_2014.pdf 77 http://highlycited.com/ referring to Wolfram, Stephen (2002):A new kind 87 http://www.weforum.org/reports/global-risks-2014- 78 of science, V Wolfram media. http://www.irgc. Specifically, publications selected met one or more report org/wp-content/uploads/2013/06/McSharry- of the following criteria, and were more likely to be 88 http://www.iea.org/newsroomandevents/ OrrellMcSharry2009foresight.pdf selected if they met multiple criteria: (1) published in a pressreleases/2011/november/name,20318,en.html 56 peer-reviewed journal or academic press; (2) authored From A Systems Approach to Forecasting” by http://www.ipcc.ch/pdf/assessment-report/ar4/wg3/ar4- by scholars of established distinction; (3) published in David Orrell and Patrick McSharry, http://www. wg3-chapter12.pdf an authoritative popular media outlet; (4) authored by irgc.org/wp-content/uploads/2013/06/McSharry- 89 Wolfe, Nathan. The viral storm: the dawn of a new journalists or other non-academic writers of established OrrellMcSharry2009foresight.pdf pandemic age. Macmillan, 2011. distinction; (5) highly cited by other scholars; (6) thorough referring to Wolfram, S. (2002). A New Kind of Science. 90 This includes any research that indicates the in presentation and analysis. https://www.wolframscience.com/ 79 probabilities are much lower than first believed. http://www.existential-risk.org/concept.pdf 57 A possible infinite threshold could be compared with 91 80 For a list of other overviews please see Annex 1 Climate change is a good example. Very little is done the two events that are often cited as the world’s worst 92 See Podsiadlowski, Ph, et al. “The rates of on the infinite impacts where global warming would wars and anthropogenic disasters: the second world hypernovae and gamma-ray bursts: implications for their result in an average temperature rise of 6°C or more. war, with 40-71 million dead (1.7-3.1% of the world’s 81 progenitors.” The Astrophysical Journal Letters 607.1 population) and the Mongol Conquests, 1206-1368, with The list of organisations is included as Appendix 2. (2004). 82 30 million dead (7.5% of the global total). See: http:// http://gcrinstitute.org/organization-directory/ 93 See Coleman, Sidney, and Frank De Luccia. en.wikipedia.org/wiki/List_of_wars_and_anthropogenic_ 83 The 19 organisations are: “Gravitational effects on and of vacuum decay.” Physical disasters_by_death_toll Brookings 58 Review D 21.12 (1980): 3305-3315. The number two billion was established during http://www.brookings.edu/research#topics/ 94 See Iliopoulos, John, et al. “Study of potentially workshops arranged during the process and is not an Bulletin of the Atomic Scientists dangerous events during heavy-ion collisions at the LHC: exact number. Further research is needed to establish a http://thebulletin.org/ report of the LHC safety study group.” Vol. 1. CERN better understanding of thresholds that can result in an CSER (2003). http://194.109.159.7/cern/20101116222315/ infinite impact. The number of people dead is only one cdsweb.cern.ch/record/613175/files/p1.pdf factor and different global risks are likley to have very http://cser.org/emerging-risks-from-technology/ 95 different thresholds. See for instance “Solar storm Risk to the North Center for International Security and Cooperation 59 American electric grid“, Lloyds (2013). HSE 2001. Reducing Risks, Protecting People – http://cisac.stanford.edu/ 96 HSE’s Decision-Making Process, 2001. See Bennett, Oliver. “Cultural pessimism: Narratives Club of Rome of decline in the postmodern world.” Edinburgh http://www.hse.gov.uk/risk/theory/r2p2.pdf http://www.clubofrome.org/ 60 University Press (2001). Bedford, Paul, Millard, Stephen and Jing Yang, Council on Foreign Relations 97 See Klare, Michael T. “Resource wars: The new Assessing operational risk in CHAPS Sterling: a http://www.cfr.org/issue/ landscape of global conflict.” Macmillan (2001). simulation approach (2004). Federation of American Scientists 98 61 See for instance the genocides listed in White, SO 16732-1, Fire Safety Engineering – Guidance http://www.fas.org/programs Matthew “Atrocitology: Humanity’s 100 Deadliest on Fire Risk Assessment, International Organization for Future of Humanity Institute Achievements.” Canongate Books (2011). Standardization, Geneva, Switzerland, 2012. 99 62 http://www.fhi.ox.ac.uk/research/research-areas/ See for instance the exponents on different power http://en.wikipedia.org/wiki/ALARP 63 laws of natural and other disasters in Hanson, Robin. Global Catastrophic Risk Institute http://www.risktec.co.uk/media/43520/risk%20 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 185

188 Endnotes event.” Trends in Ecology & Evolution 18.7 (2003): 358- “Catastrophe, social collapse, and human extinction.” Nick Bostrom & Milan Cirkovic Eds.(Oxford University 365. Global Catastrophic Risks 1 (2008). Press, 2008) Global Catastrophic Risks. 100 130 See Shepherd, J. G.: “Geoengineering the climate: See the damage and impacts listed in Carr, Jeffrey. Martin Rees, “Our final century”, Random House, 2003. “Inside cyber warfare: Mapping the cyber underworld.” science, governance and uncertainty.” Royal Society, Martin, James, 2007. The Meaning of the 21st Century. O’Reilly Media, Inc. (2011). http://www.amazon. (2009). New York: Riverhead Penguin. 131 co.uk/Inside-Cyber-Warfare-Mapping-Underworld/ See Robock, Alan, et al: “Benefits, risks, and costs of Posner, Richard, 2004. Catastrophe: Risk and Response. dp/1449310044 stratospheric geoengineering.” Geophysical Research Oxford: Oxford University Press. 101 112 Letters 36.19 (2009). See the medium estimate in United Nations. Dept. See appendix 2 for participants at workshops 132 of Economic and Social Affairs. “World Population 113 See Brovkin, Victor, et al.: “Geoengineering climate by http://en.wikipedia.org/wiki/Climate_change to 2300.” New York: United Nations, Department of stratospheric sulfur injections: Earth system vulnerability 114 See the IPCC’s Fifth Assessment Report. Economic and Social Affairs, (2004). https://www. to technological failure.” Climatic Change 92.3-4 (2009): 115 See Chateld, Chris. “Model uncertainty, data mining un.org/esa/population/publications/longrange2/ 243-259. and statistical inference.” JR Statist. Soc. A 158 (1995): WorldPop2300final.pdf 133 See Baum, Seth D., Timothy M. Maher Jr, and 419-466, and for climate change models specifically, 102 See the article “Delayed biological recovery from Jacob Haqq-Misra.: “Double catastrophe: intermittent see Pindyck, Robert S. “Climate Change Policy: What extinctions throughout the fossil record” by James W. stratospheric geoengineering induced by societal do the models tell us?“ No. w19244. National Bureau of Kirchner & Anne Weil in Nature: http://seismo.berkeley. collapse.” Environment Systems & Decisions (2013): Economic Research (2013). edu/~kirchner/reprints/2000_34_delayed_recovery.pdf 1-13. 116 See Stern, Nicholas. “The structure of economic 103 134 Attractor: a set of physical properties towards which a http://www.nature.com/ncomms/2014/140225/ modeling of the potential impacts of climate change: system tends to evolve. Strange attractor: a complicated ncomms4304/full/ncomms4304.html grafting gross underestimation of risk onto already set with a fractal structure. http://en.wikipedia.org/wiki/ http://www.geomar.de/en/news/article/climate- narrow science models.” Journal of Economic Literature Attractor engineering-geringes-potential-grosse-nebenwirkungen/ 51.3 (2013): 838-859. http://www.stsci.edu/~lbradley/seminar/attractors.html 135 117 See Tetlock, Philip E.: “Expert political judgment: How See Weitzman, Martin L. “Tail-Hedge Discounting 104 Folke, C., S. R. Carpenter, B. Walker, M. Scheffer, good is it? How can we know?” Princeton University and the Social Cost of Carbon.” Journal of Economic T. Chapin, and J. Rockström. 2010. Resilience Press (2005). Literature 51.3 (2013): 873-882. thinking: integrating resilience, adaptability and 136 118 See the Carbon Tracker Initiative report: “Unburnable See Bodman, Roger W., Peter J. Rayner, and David J. transformability. Ecology and Society 15(4): 20. [online] carbon 2013: Wasted capital and stranded assets.” Karoly. “Uncertainty in temperature projections reduced URL: http://www.ecologyandsociety.org/vol15/iss4/ 137 using carbon cycle and climate observations.” Nature These figures are broadly in line with those of art20/ Climate Change (2013). Meinshausen, Malte, et al.: “Greenhouse-gas emission 105 We define use Jared Diamond's definition of 119 targets for limiting global warming to 2°C.” Nature Some of the severe consequences of this are collapse for ”end of civilisation”, but focus only on 458.7242 (2009): 1158-1162. explored in the World Bank report “Turn Down The Heat: global impacts, so the definiton is ”A drastic decrease in 138 why a 4°C warmer world must be avoided“ (2012). See for instance the Scientific American report: “400 human population size and/or political/economic/social 120 PPM: Carbon Dioxide in the Atmosphere Reaches See Lynas, Mark. “Six degrees: Our future on a hotter complexity, globally for an extended time.” http://www. Prehistoric Levels.” planet.” HarperCollins UK (2008), also summarised by jareddiamond.org/Jared_Diamond/Collapse.html 139 its author in a Guardian article “Six steps to hell,” and See Lüthi, Dieter, et al.: “High-resolution carbon 106 See Korhonen, Jouni, and Thomas P. Seager. Schneider, Stephen. “The worst-case scenario.” Nature dioxide concentration record 650,000–800,000 years “Beyond eco‐efficiency: a resilience perspective.” 458.7242 (2009): 1104-1105. before present.” Nature 453.7193 (2008): 379-382. Business Strategy and the Environment 17.7 (2008): 411- 140 121 See Zimov, Sergey A., Edward AG Schuur, and F. See the IPCC’s Fourth Assessment Report. 419. 141 Stuart Chapin III. “Permafrost and the global carbon See the Independent report “China agrees to impose 107 Maher, Timothy M., and Seth D. Baum. “Adaptation budget.” Science 312.5780 (2006): 1612-1613. carbon targets by 2016.” to and recovery from global catastrophe.” Sustainability 122 142 See Malhi, Yadvinder, et al. “Exploring the likelihood See the UN Millennium Development Goal database 5.4 (2013): 1461-1479. and mechanism of a climate-change-induced dieback “Carbon dioxide emissions (CO2), thousand metric tons 108 Such as the Svalbard Global Seed Vault. of the Amazon rainforest.” Proceedings of the National of CO2 (CDIAC).” 109 The fall of the Roman Empire, for instance, resulted Academy of Sciences 106.49 (2009): 20610-20615. 143 See the announcement by the Chinese Bureau of in the loss of many technologies and techniques: 123 There are arguments that climate change is Meteorology: “National Low Carbon Day logo and Aiyar, Shekhar, Carl-Johan Dalgaard, and Omer Moav. structurally unsuited to treaty-based emissions curbs; slogan officially announced.” “Technological progress and regress in pre-industrial see Barrett, Scott. “Climate treaties and approaching 144 See the Warsaw Climate Change Conference Press times.” Journal of Economic Growth 13.2 (2008): 125- catastrophes.” Journal of Environmental Economics and Release “UN Climate Change Conference in Warsaw 144. Management (2013). keeps governments on a track towards 2015 climate 110 See Ćirković, Milan M., Anders Sandberg, and Nick 124 The World Bank has estimated that the cost of agreement.” Bostrom. “Anthropic shadow: Observation selection adapting to 2°C climate change would be of the order of 145 See the UNFCCC’s “Private Sector Initiative - effects and human extinction risks.” Risk analysis 30.10 $70-100 billion a year (source: Economics of Adaptation database of actions on adaptation.” (2010): 1495-1506. to Climate Change, World Bank). 146 See Buchanan, James M.: “Externality.” Economica 111 Organisations working with global challenges 125 See Lynas, Mark. “Six degrees: Our future on a hotter 29.116 (1962): 371-384. included in the process where challenges were selected. planet.” HarperCollins UK (2008), also summarised by its 147 See the IPCC’s Fifth Assessment Report. Below: example of organisations where information was author in a Guardian article “Six steps to hell.” 148 gathered from, in alphabetical order: See the Global Challenge Foundation’s explanation 126 See for instance the open letter from the Partnership “About the Global Risk and Opportunity Indicator.” http://cass.cssn.cn/ for a Secure America: “Thirty-Eight Leading U.S. National 149 See the Warsaw Climate Change Conference Press http://www.cfr.org Security Experts Urge Action on International Climate Release “UN Climate Change Conference in Warsaw http://cser.org Change Initiatives“ (2013). keeps governments on a track towards 2015 climate 127 http://www.thebulletin.org See Weiss, Harvey, and Raymond Bradley; “What agreement.” drives societal collapse?“ The Anthropology of Climate http://www.fhi.ox.ac.uk 150 http://en.wikipedia.org/wiki/Public_good Change: An Historical Reader (2013): 151-154. http://www.sipri.org 151 See Feldman, Allan M., and A. M. Feldman.: “Welfare 128 See Patz, Jonathan A., et al.; “Global climate change http://www.skollglobalthreats.org economics and social choice theory.” Boston: Martinus and emerging infectious diseases.” JAMA: the journal http://www.stockholmresilience.org Nijhoff (1980). of the American Medical Association 275.3 (1996): 217- http://www.un.org/en/sc/ 152 See the Warsaw Climate Change Conference Press 223. http://www.weforum.org/issues/global-risks Release “UN Climate Change Conference in Warsaw 129 See Benton, Michael J., and Richard J. Twitchett. keeps governments on a track towards 2015 climate Examples of relevant literature: “How to kill (almost) all life: the end-Permian extinction Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 186

189 Endnotes 105.14 (2008): 5307-5312. Congress, 2010. agreement.” 191 153 173 “The so-called “Warsaw International Mechanism See Bennett, Bruce W., and Jennifer Lind.: “The Though some have argued for a significant climate Collapse of North Korea: Military Missions and effect of the nuclear explosions themselves, see Fujii, for Loss and Damage” will from next year commit developed nations to provide expertise and potentially Yoshiaki.: “The role of atmospheric nuclear explosions Requirements.” International Security 36.2 (2011): 84- aid to countries hit by climate-related impacts. [...] on the stagnation of global warming in the mid 20th 119. 192 century.” Journal of Atmospheric and Solar-Terrestrial However, the vague wording fell short of the kind of Even the 1968 capture by North Korea of the USS Physics 73.5 (2011): 643-652. detailed commitments on additional funding and the Pueblo did not reignite conflict (see Armbrister, Trevor.: 174 commitment to compensation that many developing “A Matter of Accountability: The True Story of the Pueblo There is (fortunately) very little empirical evidence nations had been seeking.” (source: Business Green: Affair.” Globe Pequot, 2004). on the impact of nuclear bombs on cities. Hiroshima “COP 19: Warsaw climate deal finalised as deadlock 193 suffered a firestorm, while Nagasaki did not – and both See the Norwegian Ministry of Foreign Affairs: broken“). cities and nuclear weapons are very different now from “Conference: Humanitarian Impact of Nuclear weapons.” 154 what they were in 1945. See for instance Bulkeley, Harriet, and Peter Newell.: 194 This conference took place 13-14 February of 2014 175 “Governing climate change.” Routledge (2010). See Toon, Owen B., et al.: “Atmospheric effects and http://www.sre.gob.mx/en/index.php/humanimpact- 155 societal consequences of regional scale nuclear conflicts See the National Research Council: “Abrupt Impacts nayarit-2014 and acts of individual nuclear terrorism.” Atmospheric of Climate Change: Anticipating Surprises.” Washington, 195 See The 1983 War Scare: “The Last Paroxysm” of Chemistry and Physics 7.8 (2007): 1973-2002, and DC: The National Academies Press (2013). the Cold War Part I, The 1983 War Scare: “The Last 156 Robock, Alan, Luke Oman, and Georgiy L. Stenchikov.: See NASA’s “GISS Surface Temperature Analysis.” Paroxysm” of the Cold War Part II, and The 1983 War “Nuclear winter revisited with a modern climate 157 Scare: “The Last Paroxysm” of the Cold War Part III. See the IPCC’s Fourth Assessment Report. model and current nuclear arsenals: Still catastrophic 158 196 See the IPCC’s Fifth Assessment Report. See The 1983 War Scare: “The Last Paroxysm” of the consequences.” Journal of Geophysical Research: 159 Cold War Part III. See the IUCN’s Red List of Threatened Species. Atmospheres (1984–2012) 112.D13 (2007). 197 160 An extensive but certainly not exhaustive list can be See McManus, J. F., et al.: “Collapse and rapid 176 Arsenals have been reduced, but there remain found at http://en.wikipedia.org/wiki/List_of_military_ resumption of Atlantic meridional circulation linked to over 17,000 nuclear warheads in the world’s arsenals nuclear_accidents. deglacial climate changes.” Nature 428.6985 (2004): (source: SIPRI yearbook 2013), down from a peak of 198 834-837. See especially the role of Vasili Arkhipov in preventing some 68,000 in 1983, still more than enough to trigger a 161 the conflict from going nuclear – a role that was only See Schuur, Edward AG, et al.: “Vulnerability of nuclear winter. revealed in 2002 (source: Roberts, Priscilla, ed. “Cuban permafrost carbon to climate change: Implications for 177 See Toon, Owen B., et al.: “Atmospheric effects and Missile Crisis: The Essential Reference Guide.” ABC- the global carbon cycle.” BioScience 58.8 (2008): 701- societal consequences of regional scale nuclear conflicts CLIO, 2012). 714. and acts of individual nuclear terrorism.” Atmospheric 199 162 See Scott Shane, writing in the Baltimore Sun: “The See Archer, David.: “Methane hydrate stability Chemistry and Physics 7.8 (2007): 1973-2002, and Nuclear War that Almost Happened in 1983” (August 31, and anthropogenic climate change.” Biogeosciences Helfand, Ira.: “NUCLEAR FAMINE: TWO BILLION 2003). Discussions 4.2 (2007): 993-1057. PEOPLE AT RISK?.” International Physicians for the 200 163 See the PBS report on the incident. For example, as climate warms, the destabilization of Prevention of Nuclear War (2013). 201 the West Antarctic ice sheet could raise sea level rapidly, Errors of a different kind happened in the early days 178 Maher, Timothy M., and Seth D. Baum.: “Adaptation with serious consequences for coastal communities. of the Cold War, when simple game theory models were to and recovery from global catastrophe.” Sustainability 164 used to argue for a nuclear first strike (source: Kaku, http://simple.wikipedia.org/wiki/Nuclear_war 5.4 (2013): 1461-1479. Michio, and Daniel Axelrod. “To win a nuclear war: the 165 179 For an analysis of the possibility of accidental nuclear See the later part of the scenario in Tonn, Bruce, and Pentagon’s secret war plans.” Black Rose Books Ltd., war between Russia and the USA, see Barrett, Anthony Donald MacGregor.: “A singular chain of events.” Futures 1987). M., Seth D. Baum, and Kelly Hostetler.: “Analyzing and 41.10 (2009): 706-714. 202 Some have modelled the nuclear standoff as a Reducing the Risks of Inadvertent Nuclear War Between 180 A synthesis blogpost by Carl Shulman of the Future “perceptual dilemma,” where misperception of the the United States and Russia.” Science & Global of Humanity Institute puts the risk of extinction, given adversary is the main cause of arms races (source: Security 21.2 (2013): 106-133. civilisation collapse, at no more than 10%. Plous, Scott.: “The nuclear arms race: prisoner’s 166 Hellman, Martin E.: “How risky is nuclear optimism?.” 181 See Tetlock, Philip E.: “Expert political judgment: How dilemma or perceptual dilemma?.” Journal of Peace Bulletin of the Atomic Scientists 67.2 (2011): 47-56. good is it? How can we know?” Princeton University Research 30.2 (1993): 163-179). 167 See Lundgren, Carl.: “What are the odds? Assessing Press (2005). 203 See Barrett, Anthony M., Seth D. Baum, and Kelly the probability of a nuclear war.” The Nonproliferation 182 See the official Remarks by Ambassador Susan Hostetler.: “Analyzing and Reducing the Risks of Review 20.2 (2013): 361-374. E. Rice, U.S. Permanent Representative to the United Inadvertent Nuclear War Between the United States and 168 A 1979 study by the United States Office of Nations, at the Security Council Stakeout. Russia.” Science & Global Security 21.2 (2013): 106-133. Technology Assessment, “The Effects of Nuclear War,” 183 “North Korean nuclear test draws anger, including 204 See Barrett, Anthony M., Seth D. Baum, and Kelly estimated 20-160 million immediate casualties from a from China” (Reuters). Hostetler.: “Analyzing and Reducing the Risks of full-scale nuclear war. 184 See the Security Council Resolution 2087 (2013). Inadvertent Nuclear War Between the United States and 169 Currently estimated at around 17,000 (source: SIPRI 185 See the description of the sanctions in the Security Russia.” Science & Global Security 21.2 (2013): 106-133. yearbook 2013). Council Committee established pursuant to resolution 205 See Podvig, Pavel.: “Reducing the risk of an 170 Though it has been argued that scientists, under 1718. Accidental Launch.” Science and Global Security 14.2-3 pressure from governments and industry, have 186 See the States Parties of the Nuclear Non- (2006): 75-115. systematically underestimated the deleterious global Proliferation Treaty (NPT). 206 See Intriligator, Michael D., and Dagobert L. Brito.: impact of radiation. See Perrow, Charles.: “Nuclear 187 See North Korean security challenges: a net “Accidental nuclear war: a significant issue for arms denial: From Hiroshima to Fukushima.” Bulletin of the assessment. International Institute for Strategic Studies, control.” Current Research on Peace and Violence Atomic Scientists 69.5 (2013): 56-67. 2011, Chapter 5. 11.1/2 (1988): 14-23. 171 The seminal American paper was Turco, Richard 188 207 See North Korean security challenges: a net See Ayson, Robert.: “After a terrorist nuclear attack: P., et al.: “Nuclear winter: global consequences of assessment. International Institute for Strategic Studies, Envisaging catalytic effects.” Studies in Conflict & multiple nuclear explosions.” Science 222.4630 (1983): 2011, Chapter 6. Terrorism 33.7 (2010): 571-593. 1283-1292. The Soviet Union was working on similar 189 208 See North Korean security challenges: a net See Hellman, Martin E.: “How risky is nuclear simulations; see for instance Peterson, Jeannie.: “The assessment. International Institute for Strategic Studies, optimism?.” Bulletin of the Atomic Scientists 67.2 (2011): aftermath: The human and ecological consequences of 2011, Chapter 8. 47-56. nuclear war.” New York: Pantheon (1983). 190 209 See Chanlett-Avery, Emma, and Mi Ae Taylor.: “North 172 See Shanteau, James.: “Competence in experts: The See Mills, Michael J., et al.: “Massive global ozone Korea: US Relations, Nuclear Diplomacy, and Internal role of task characteristics.” Organizational behavior and loss predicted following regional nuclear conflict.” Situation.” Congressional Research Service, Library of human decision processes 53.2 (1992): 252-266. Proceedings of the National Academy of Sciences Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 187

190 Endnotes 210 254 (Nuclear Threat Initiative). See Schlager, Edella, and Elinor Ostrom. “Property- See also Mosher, David E., et al.: Beyond the Nuclear 230 Shadow. “A Phased Approach for Improving Nuclear rights regimes and natural resources: a conceptual http://en.wikipedia.org/wiki/List_of_states_with_ Safety and US Russian Relations.” No. RAND/MR-1666- analysis.” Land economics (1992): 249-262. nuclear_weapons 255 NSRD. RAND CORP SANTA MONICA CA, 2003. 231 See Frank, Kenneth T., et al. “Trophic cascades in a http://en.wikipedia.org/wiki/Ecological_collapse 211 formerly cod-dominated ecosystem.” Science 308.5728 See the Report of the Open-ended Working Group 232 See Pimm, Stuart L., et al.: “The future of (2005): 1621-1623. to develop proposals to take forward multilateral nuclear biodiversity.” Science 269 (1995): 347-347. 256 disarmament negotiations for the achievement and See the IUCN’s Red List of Threatened Species. 233 See Rockström, Johan, et al.: “Planetary boundaries: maintenance of a world without nuclear weapons. 257 See Zalasiewicz, Jan, et al. “Are we now living in the exploring the safe operating space for humanity.” 212 See the text of the Treaty on the Non-Proliferation of Anthropocene?” GSA Today 18.2 (2008): 4. Ecology and society 14.2 (2009). Nuclear Weapons. 258 234 See Barnosky, Anthony D., et al.: “Approaching a See Barnosky, Anthony D., et al.: “Approaching a 213 Treaty Between the United States of America and the state shift in Earth’s biosphere.” Nature 486.7401 (2012): state shift in Earth’s biosphere.” Nature 486.7401 (2012): Union of Soviet Socialist Republics on the Elimination of 52-58. 52-58. Their Intermediate-Range and Shorter-Range Missiles. 259 235 See Dol, Marcel, ed.: “Recognising the intrinsic See Ehrlich, Paul R., and Anne H. Ehrlich.: “Can a 214 Treaty Between the United States of America and the value of animals: beyond animal welfare.” Uitgeverij Van collapse of global civilization be avoided?.” Proceedings Union of Soviet Socialist Republics on the Reduction Gorcum (1999). of the Royal Society B: Biological Sciences 280.1754 and Limitation of Strategic Offensive Arms. 260 (2013). http://en.wikipedia.org/wiki/Pandemic 215 Treaty Between the United States of America and the 236 261 See Galaz, Victor, et al.: “Global environmental See Hanson, Robin.: “Catastrophe, social collapse, Russian Federation On Strategic Offensive Reductions. and human extinction.” Global Catastrophic Risks 1 governance and planetary boundaries: An introduction.” 216 The Treaty between the United States of America (2008): 357. Ecological Economics 81 (2012): 1-3. and the Russian Federation on Measures for the Further 237 262 See Dol, Marcel, ed.: “Recognizing the intrinsic See Asmussen, Søren. “Steady-State Properties of Reduction and Limitation of Strategic Offensive Arms value of animals: beyond animal welfare.” Uitgeverij Van of GI/G/1.” Applied Probability and Queues (2003): 266- also known as the New START Treaty. 301. Gorcum (1999). 217 See Sovacool, Benjamin K. “Contesting the future of 263 238 See Dasgupta, Partha.: “Human well-being and the Possessing neither variance nor mean. nuclear power: a critical global assessment of atomic 264 natural environment.” OUP Catalogue (2001). This explains why governments and international energy.” World Scientific, 2011. 239 organisations seem to over-react to pandemics (such See the author’s blog post: “Water, food or energy: 218 Helfand, Ira.: “Nuclear Famine: Two Billion People as the swine flu scare). Though any single pandemic is we won’t lack them.” summarising the current evidence. at Risk?” International Physicians for the Prevention of 240 unlikely to cause a large number of casualties, there is See “2013 World Hunger and Poverty Facts and Nuclear War (2013). always the small risk that it could get out of hand and kill Statistics.” World Hunger Education Service (2013). 219 See for instance Robock, Alan, Luke Oman, and millions. 241 Maher, Timothy M., and Seth D. Baum.: “Adaptation Georgiy L. Stenchikov.: “Nuclear winter revisited with a 265 There is always a probability that the power law will to and recovery from global catastrophe.” Sustainability modern climate model and current nuclear arsenals: Still not hold for future pandemics – but the majority of the 5.4 (2013): 1461-1479. catastrophic consequences.” Journal of Geophysical risk is concentrated in scenarios where it does. 242 See Dullinger, Stefan, et al.: “Europe’s other debt Research: Atmospheres (1984–2012) 112.D13 (2007), 266 See Hanson, Robin.: “Catastrophe, social collapse, crisis caused by the long legacy of future extinctions.” Toon, Owen B., et al. “Atmospheric effects and societal and human extinction.” Global Catastrophic Risks 1 Proceedings of the National Academy of Sciences consequences of regional scale nuclear conflicts and (2008): 357. 110.18 (2013): 7342-7347. acts of individual nuclear terrorism.” Atmospheric 267 243 Source: EM-DAT, The International Disaster Database. G8 2007 Environment Ministers Meeting. Chemistry and Physics 7.8 (2007): 1973-2002, and 268 244 See Jeffs, Benjamin.: “A clinical guide to viral See Pimm, Stuart L., et al. “The future of biodiversity.” Stenke, A., et al. “Climate and chemistry effects of a haemorrhagic fevers: Ebola, Marburg and Lassa.” Science 269 (1995): 347-347. regional scale nuclear conflict.” Atmospheric Chemistry Tropical Doctor 36.1 (2006): 1-4. 245 and Physics Discussions 13.5 (2013): 12089-12134. See Jones-Walters, Lawrence, and Ivo Mulder: 269 See the WHO Factsheet 99 on Rabies. 220 “Valuing nature: The economics of biodiversity.” Journal Helfand, Ira.: “An assessment of the extent of 270 for Nature Conservation 17.4 (2009): 245-247. See Arroll, Bruce.: “Common cold.” Clinical evidence projected global famine resulting from limited, regional 246 2011 (2011). nuclear war.” Royal Soc. Med, London (2007). See the UN-mandated “Millennium Ecosystem 271 221 Assessment, Ecosystems and Human Well-being: See “Stages of HIV.” U.S. Department of Health & See Özdoğan, Mutlu, Alan Robock, and Christopher Synthesis“ (2005). Human Services. J. Kucharik.: “Impacts of a nuclear war in South Asia on 272 247 soybean and maize production in the Midwest United See Bouvier, Nicole M., and Peter Palese.: “The See Bottrill, Madeleine C., et al.: “Is conservation States.” Climatic Change 116.2 (2013): 373-387. triage just smart decision making?“ Trends in Ecology & biology of influenza viruses.” Vaccine 26 (2008): Evolution 23.12 (2008): 649-654. D49-D53. Xia, Lili, and Alan Robock.: “Impacts of a nuclear war 273 248 in South Asia on rice production in Mainland China.” See Fox, Jonathan A., and Lloyd David Brown, eds.: See for instance Risk Management Solutions’ Climatic Change 116.2 (2013): 357-372. “The struggle for accountability: the World Bank, NGOs, pandemic model, overviewed in “The RMS® Influenza 223 Pandemic Risk Model.” and grassroots movements.” MIT Press (1998). See Stommel H, Stommel E.: “Volcano Weather: The 249 274 Story of 1816, The Year without a Summer.” Seven Seas SARS (Severe), for instance, started in China and http://www.esri.com/news/releases/11-4qtr/gis- Press: Newport (1983). cloud-featured-at-eye-on-earth-summit.html spread to 17 other countries, including Canada (source: 224 250 Lung Disease & Respiratory Health Center, SARS: See the European External Action Service Joint Plan See Microsoft’s press release “Eye on Earth Enables Timeline of an Outbreak). of Action (24 November 2013). Cloud-Based Environmental Data Sharing” (2011). 275 225 251 See Riley, Lee W., et al.: “Slum health: diseases of See the report of the Director General of the IAEA (8 “The main objective of the MyOcean2 project is to neglected populations.” BMC international health and November 2011). deliver and operate a rigorous, robust and sustainable human rights 7.1 (2007): 2. 226 Ocean Monitoring and Forecasting system of the GMES See Security Council Resolution 1737 (2006). 276 Marine Service (OMF/GMS) to users for all marine Brazil alone is estimated to have over 50 uncontacted 227 See the text of the Treaty on the Non-Proliferation of applications: maritime safety, marine resources, marine tribes (source: Survival International). Nuclear Weapons. 277 and coastal environment and climate, seasonal and Maher, Timothy M., and Seth D. Baum.: “Adaptation 228 See for instance President Kennedy’s quote weather forecasting.” (Source the MyOcean2 website). to and recovery from global catastrophe.” Sustainability during the third Nixon-Kennedy presidential debate, 252 5.4 (2013): 1461-1479. See the “Communication from the (European) October 13, 1960: “There are indications because of 278 commission to the council, concerning a consultation on See the WHO Interim Guidance “Pandemic Influenza new inventions, that 10, 15, or 20 nations will have a Fishing Opportunities for 2014.” Risk Management.” nuclear capacity, including Red China, by the end of the 253 279 The famous “tragedy of the commons.” See Garrett Presidential office in 1964.” See the WHO guidance document “Pandemic Hardin. “The Tragedy of the Commons” Science 229 influenza preparedness and response.” “More Than 40 Countries Could Have Nuclear 162.3859 (1968): 1243-1248. 280 Weapons Know-How, IAEA Chief ElBaradei Warns” See Trifonov, Vladimir, Hossein Khiabanian, and Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 188

191 Endnotes Release Reports in the United States—2004-2010. Appl Defect: How Globalization Creates Systemic Risks, and Raul Rabadan.: “Geographic dependence, surveillance, Biosaf 17 (2012):171-180.) What to Do about It. Princeton University Press (2014). and origins of the 2009 influenza A (H1N1) virus.” New 321 301 England Journal of Medicine 361.2 (2009): 115-119. It is suspected that the 1977-1978 Russian Flu See Goldin, Ian, and Tiffany Vogel.: Global 281 epidemic was caused by a laboratory incident in governance and systemic risk in the 21st century: See the WHO Sixty Fourth World Health Assembly Russia or northern China (source: NPR’s interview lessons from the financial crisis. Global Policy 1.1 (2010): “Resolutions and Decisions,” resolution WHA64.5. 282 4-15. with Professor John Oxford at St. Bart’s and the Royal See the WHO International Health Regulations 322 London Hospital). (2005) “Alert, response, and capacity building under the See Korhonen, Jouni, and Thomas P. Seager.: 302 International Health Regulations (IHR).” Beyond eco‐efficiency: a resilience perspective. See Pennington, Hugh.: Smallpox and bioterrorism. 283 Business Strategy and the Environment 17.7 (2008): 411- Bulletin of the World Health Organization 81.10 (2003): See the “Report of the Review Committee on the 419. 762-767. Functioning of the International Health Regulations 303 322 (2005) in Relation to Pandemic (H1N1) 2009.” Geneva, See Klotz, L. C., and E. J. Sylvester.: The See J. Doyne Farmer.: Toward better quantitative World Health Organization (2011). models of systemic risk. Presentation at Institute for New unacceptable risks of a man-made pandemic. Bulletin of 284 Economic Thinking (2010). the Atomic Scientists 7 (2012). See Kerkhove, Maria D., et al.: “Estimating age‐ 304 323 specific cumulative incidence for the 2009 influenza The highest level of biosecurity. See the Lloyds Insurance report “Solar storm risk to 305 pandemic: a meta‐analysis of A (H1N1) pdm09 the North American electric grid.” (2013). A Marburg virus laboratory-acquired infection at the 324 serological studies from 19 countries.” Influenza and Vector facility in the Soviet Union in 1990, a foot-and- See Barro, Robert J., and Xavier Sala-i-Martin.: Other Respiratory Viruses (2013). mouth disease virus escape from the Pirbright facility in Economic growth and convergence across the United 285 England, and a SARS virus laboratory-acquired infection States. No. w3419. National Bureau of Economic Report in Nature by Maryn McKenna, “Antibiotic from a BSL-4-rated biosafety cabinet in a Taiwan Research (1990). resistance: The last resort.” 325 286 laboratory. See Alchon, Suzanne Austin. “A Pest in the Land: See Hanson, Robin.: Catastrophe, social collapse, 306 and human extinction. Global Catastrophic Risks 1 New World Epidemics in a Global Perspective.” See the Towers Watson poll of global insurance (2008): 357. University of New Mexico Press (2003). executives on the most worrying extreme risks facing the 326 287 insurance industry. See Rothschild, Bruce M..: “History of syphilis.” See for instance the temporary collapse of the 307 Clinical infectious diseases: an official publication of the Albanian state in 1997 over poorly regulated investment See the CDC’s report Taubenberger, Jeffery K., and pyramid schemes: Jarvis, Christopher.: The rise Infectious Diseases Society of America 40.10 (2005): David M. Morens.: 1918 Influenza: the mother of all and fall of Albania’s pyramid schemes. Finance and 1454–63 pandemics. Rev Biomed 17 (2006): 69-79. 288 308 Development 37.1 (2000): 46-49. See the World Health Organization Global Influenza See Lawn, Stephen D., and Zumla, Alimuddin I.: 327 Programme: Pandemic influenza preparedness and “Tuberculosis,” The Lancet 378.9785 (2011): 57-72. See Acemoglu, Daron, James A. Robinson, and Dan 289 response: a WHO guidance document. World Health Woren.: Why nations fail: the origins of power, prosperity, See the World Health Organization Fact Sheet No 104 Organisation (2009). and poverty. Vol. 4. New York: Crown Business (2012). on Tuberculosis. 328 309 290 See the LSE press release Research centre to study See Merton, Robert K.: Bureaucratic structure and See Harrison, J. W., and T. A. Svec: “The beginning of personality. Social forces 18.4 (1940): 560-568, and risks to financial system launched at LSE (2013). the end of the antibiotic era? Part II. Proposed solutions 330 Turner, Marlene E., and Anthony R. Pratkanis: Twenty- to antibiotic abuse.” Quintessence international (Berlin, See Ang, Andrew, and Francis A. Longstaff.: Systemic five years of groupthink theory and research: Lessons Germany: 1985) 29.4 (1998): 223. sovereign credit risk: Lessons from the US and Europe. from the evaluation of a theory. Organizational Behavior 291 Journal of Monetary Economics (2013). See Mathew, Alan G., Robin Cissell, and S. and Human Decision Processes 73.2 (1998): 105-115. 331 Liamthong: “Antibiotic resistance in bacteria associated See for instance Brunnermeier, Markus K., and Lasse 310 http://en.wikipedia.org/wiki/Economic_collapse with food animals: a United States perspective of Heje Pedersen.: Market liquidity and funding liquidity. livestock production.” Foodborne pathogens and Review of Financial studies 22.6 (2009): 2201-2238. http://en.wikipedia.org/wiki/Societal_collapse 311 332 disease 4.2 (2007): 115-133. See for instance Calomiris, Charles W., and Joseph R. See Helbing, Dirk.: Globally networked risks and how 292 to respond. Nature 497.7447 (2013): 51-59. Mason.: Fundamentals, panics, and bank distress during See the final summary from the Skoll Global Threats 312 the depression. American Economic Review (2003): Fund “Wrap up from Epihack.” See Anders Sandberg, Nick Beckstead, Stuart 293 1615-1647. Armstrong.: “Defining systemic risk.” Report from See Sridhar, Saranya, et al.: “Cellular immune 333 the FHI-Amlin Systemic Risk of Risk Modelling correlates of protection against symptomatic pandemic See the IMF working paper Systemic Risk Monitoring Collaboration. Oxford University (2014). influenza.” Nature Medicine 19.10 (2013): 1305-1312. (“SysMo”) Toolkit—A User Guide (2013). 294 334 313 See for instance Moorhouse, Andrew.: An See Gunderson, Lance H.: “Ecological resilience--in See Taubenberger, Jeffery K., and David M. Morens.: theory and application.” Annual review of ecology and “1918 Influenza: the mother of all pandemics.” Rev introduction to financial soundness indicators. Monetary Biomed 17 (2006): 69-79. & Financial Statistics (2004). systematics (2000): 425-439, and Kambhu, John, Scott 295 335 Weidman, and Neel Krishnan. Part 3: Systemic risk in See for instance Baldacci, E., McHugh, J. and See Valleron, Alain-Jacques, et al.: “Transmissibility ecology and engineering. Economic Policy Review 13.2 Petrova, I.: Measuring Fiscal Vulnerabilities and Fiscal and geographic spread of the 1889 influenza pandemic.” (2007). Stress: A Proposed Set of Indicators, IMF Working Paper Proceedings of the National Academy of Sciences 314 11/94 (2011). 107.19 (2010): 8778-8781. See De Bandt, Olivier, and Philipp Hartmann.: 296 336 Systemic risk: A survey. ECB Working Paper No. 35 See for instance Hamilton, James D., and Raul See the WHO report Ten things you need to know (2000). about pandemic influenza. Susmel.: Autoregressive conditional heteroskedasticity 315 297 and changes in regime. Journal of Econometrics 64.1 See Rinaldi, Steven M., James P. Peerenboom, See Dawood, Fatimah S., et al.: “Estimated global (1994): 307-333. and Terrence K. Kelly.: Identifying, understanding, and mortality associated with the first 12 months of 2009 337 analyzing critical infrastructure interdependencies. pandemic influenza A H1N1 virus circulation: a modelling See for instance Gray, Dale F., and Andreas A. Jobst.: Control Systems, IEEE 21.6 (2001): 11-25. study.” The Lancet infectious diseases 12.9 (2012): 687- Modelling systemic financial sector and sovereign risk. 316 695. Sveriges Riksbank Economic Review (2011): 68-97. See Homer-Dixon, Thomas.: Complexity Science. 298 338 Oxford Leadership Journal 2.1 (2011): 15. See Merler, Stefano, et al.: “Containing the accidental See Citigroup’s paper 2013 Outlook and Strategies 317 laboratory escape of potential pandemic influenza (2013) and the Financial Post news report by David Pett: See Allen, Franklin, and Douglas Gale.: Financial Three geopolitical risks investors should keep an eye on viruses.” BMC medicine 11.1 (2013): 252. contagion. Journal of political economy 108.1 (2000): 299 in 2014 (2013). 1-33. Report in Nature by Declan Butler, “Work resumes on 339 318 lethal flu strains.” Used by such diverse bodies as the US Department See Buldyrev, Sergey V., et al.: Catastrophic cascade 300 of Energy (source: the US Department of Energy’s Cost of failures in interdependent networks. Nature 464.7291 From 2004 to 2010, 639 release reports were Estimating Guide (2011)) and AACE (Association for the (2010): 1025-1028. reported to the Centers for Disease Control (CDC), 319 Advancement of Cost Engineering) International (source: 11 of them reporting laboratory-acquired infections See Vespignani, Alessandro.: Predicting the behavior AACE Recommended Practice No. 56R-08 Cost (source: Henkel, Richard D., Thomas Miller, and Robbin of techno-social systems. Science 325.5939 (2009): 425. Estimate Classification System (2012)). 320 S. Weyant. Monitoring Select Agent Theft, Loss and See Goldin, Ian, and Mike Mariathasan.: The Butterfly Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 189

192 Endnotes 340 (2010): 1495-1506. http://en.wikipedia.org/wiki/Impact_event en.wikipedia.org/wiki/Timetable_of_major_worldwide_ 356 341 volcanic_eruptions. See Jablonski, David, and W. G. Chaloner.: See NASA’s press release on the meteor, Additional 378 Details on the Large Fireball Event over Russia on Feb. Extinctions in the Fossil Record [and Discussion]. See Newhall, Christopher G., and Stephen Self.: Philosophical Transactions of the Royal Society of 15, 2013. The Volcanic Explosivity Index (VEI): an estimate of 357 London. Series B: Biological Sciences 344.1307 (1994): explosive magnitude for historical volcanism. Journal See Borovička, Jiří, et al.: The trajectory, structure 11-17. of Geophysical Research: Oceans (1978–2012) 87.C2 and origin of the Chelyabinsk asteroidal impactor. Nature 342 (1982): 1231-1238. (2013). See Collins, Gareth S., H. Jay Melosh, and Robert 379 358 A. Marcus.: Earth Impact Effects Program: A Web‐ See Aspinall, W., et al.: Volcano hazard and See Chyba, Christopher F., Paul J. Thomas, and based computer program for calculating the regional exposure in GDRFF priority countries and risk mitigation Kevin J. Zahnle.: The 1908 Tunguska explosion: environmental consequences of a meteoroid impact on measures-GFDRR Volcano Risk Study. Bristol: Bristol atmospheric disruption of a stony asteroid. Nature Earth. Meteoritics & planetary science 40.6 (2005): 817- University Cabot Institute and NGI Norway for the World 361.6407 (1993): 40-44. 840. Input data: Projectile Diameter: 5000m, Projectile Bank: NGI Report 20100806.2011 (2011). 359 See for instance the Russian reaction: PM Medvedev 380 Density: 2600 kg/m3, Impact Velocity: 17.00 km/s, See Mason, Ben G., David M. Pyle, and Clive Says Russian Meteorite KEF-2013 Shows “Entire Planet” Impact Angle: 45 degrees. Oppenheimer.: The size and frequency of the largest Vulnerable. 343 explosive eruptions on Earth. Bulletin of Volcanology The “Tsar Bomba” at 2x1017 Joules (source: 360 See the Space Research Institute of Russian 66.8 (2004): 735-748. Adamsky, Viktor, and Yuri Smirnov.: Moscow’s Biggest Academy of Science report Deflecting Hazardous 381 Bomb: The 50-Megaton Test of October 1961. Cold War See Stephen Self and Stephen Blake, Asteroids from Collision with the Earth by Using Small International History Project Bulletin 4 (1994): 19-21.). “Consequences of Explosive Super-eruptions” Asteroids. 344 ELEMENTS , VOL . 4, 41–46: http://chuma.cas.usf. The effects of various sizes of impacts have been 361 See for instance Collins, Gareth S., H. Jay Melosh, edu/~juster/GLY2030-F09/supervolcanoes-self%20 estimated in Atkinson, Harry, Crispin Tickell, and and Robert A. Marcus.: Earth Impact Effects Program: and%20blake.pdf David Williams.: Report of the task force on potentially A Web‐based computer program for calculating the 382 hazardous near-Earth objects. (2000). See Zielinski, G. A., et al.: Potential atmospheric regional environmental consequences of a meteoroid 345 impact of the Toba Mega‐Eruption 71,000 years ago. See for instance Collins, Gareth S., H. Jay Melosh, impact on Earth. Meteoritics & planetary science 40.6 Geophysical Research Letters 23.8 (1996): 837-840. and Robert A. Marcus.: Earth Impact Effects Program: (2005): 817-840, and Jeffers, S. V., et al.: Near‐Earth 383 A Web‐based computer program for calculating the object velocity distributions and consequences for It is estimated that the largest volcanic eruptions have regional environmental consequences of a meteoroid the Chicxulub impactor. Monthly Notices of the Royal an energy greater than 1021 Joules (source: Mason, Ben impact on Earth. Meteoritics & planetary science Astronomical Society 327.1 (2001): 126-132. G., David M. Pyle, and Clive Oppenheimer.: The size and 40.6 (2005): 817-840, Jeffers, S. V., et al.: Near‐Earth 362 frequency of the largest explosive eruptions on Earth. See Hall, Christopher D., and I. Michael Ross.: object velocity distributions and consequences for Bulletin of Volcanology 66.8 (2004): 735-748.), while Dynamics and control problems in the deflection of the Chicxulub impactor. Monthly Notices of the Royal the largest nuclear device ever exploded had a yield of Near-Earth Objects. Adv. Astronaut. Sci 67.640 (1997): Astronomical Society 327.1 (2001): 126-132, and Board, around 2x1017 Joules (source: Adamsky, Viktor, and Yuri 1-18. Space Studies.: Defending Planet Earth: Near-Earth Smirnov.: Moscow’s Biggest Bomb: The 50-Megaton 363 See NASA’s Asteroid Watch, Asteroid 2013 TV135 - A Object Surveys and Hazard Mitigation Strategies. Test of October 1961. Cold War International History Reality Check. National Academies Press (2010). A summary of the last Project Bulletin 4 (1994): 19-21.). 364 The effects of various sizes of impacts have been paper can be found at The Danger from Asteroid impact. 384 See Scarpa, Roberto.: Predicting volcanic eruptions. estimated in Atkinson, Harry, Crispin Tickell, and 346 See the NASA press release NASA Spacecraft Science 293.5530 (2001): 615-616. David Williams.: Report of the task force on potentially Reactivated to Hunt for Asteroids. 385 Sparks, Stephen. Super-eruptions: Global effects and hazardous near-Earth objects. (2000). 347 See NASA’s Near Earth Objects Discovery Statistics. future threats, report of a Geological Society of London 365 See Live Science, The Odds of Dying. 348 working group. Geological Society, 2005. See Harris, Alan.: What Spaceguard did. Nature 366 See Scientific American, United Nations to Adopt 386 453.7199 (2008): 1178-1179. See 2013 World Hunger and Poverty Facts and Asteroid Defense Plan. 349 Statistics. World Hunger Education Service (2013). http://www.nature.com/nature/journal/v453/n7199/ 367 See Brown, P. G., et al.: A 500-kiloton airburst over 387 fig_tab/4531178a_F2.html Maher, Timothy M., and Seth D. Baum.: Adaptation to Chelyabinsk and an enhanced hazard from small 350 and recovery from global catastrophe. Sustainability 5.4 See Hall, Christopher D., and I. Michael Ross.: impactors. Nature (2013). (2013): 1461-1479. Dynamics and control problems in the deflection of 368 See Glasstone, Samuel.: The effects of nuclear 388 Near-Earth Objects. Adv. Astronaut. Sci 67.640 (1997): See Sparks, S. & Self. S. et al., 2005: Super-eruptions: weapons. US Department of Defense (1964). 1-18. global effects and future threats: Report of a Geological 369 See NBC’s report Third time’s the charm: SpaceX 351 Society of London working group. http://www.geo.mtu. Some are more optimistic than others: “An impact launches big commercial satellite. edu/~raman/VBigIdeas/Supereruptions_files/Super- can be predicted in advance in ways that remain 370 See Hall, Christopher D., and I. Michael Ross.: eruptionsGeolSocLon.pdf imperfect but are much more reliable than predictions Dynamics and control problems in the deflection of 389 of earthquakes or even storms, and the components http://en.wikipedia.org/wiki/Volcanic_Explosivity_ Near-Earth Objects. Adv. Astronaut. Sci 67.640 (1997): of technology exist - at affordable costs given the Index 1-18. consequences of an actual impact - to move any http://volcanoes.usgs.gov/images/pglossary/vei.php 371 See NASA News, NASA Announces New Homes for threatening object away and avoid the disaster 390 See Ćirković, Milan M., Anders Sandberg, and Nick Space Shuttle Orbiters After Retirement. altogether.” (source: Chapman, Clark R. “The hazard Bostrom.: Anthropic shadow: Observation selection 372 See NASA Report Launching into History. of near-Earth asteroid impacts on earth.” Earth and effects and human extinction risks. Risk analysis 30.10 373 Planetary Science Letters 222.1 (2004): 1-15.) See also http://en.wikipedia.org/wiki/Supervolcano (2010): 1495-1506. 374 NASA’s report to Congress “Near-Earth Object Survey See Reichow, Marc K., et al.: The timing and extent of 391 See Lane, Christine S., Ben T. Chorn, and Thomas and Deflection Analysis of Alternatives“ (2007). the eruption of the Siberian Traps large igneous province: C. Johnson.: Ash from the Toba supereruption in Lake 352 See 2013 World Hunger and Poverty Facts and Implications for the end-Permian environmental crisis. Malawi shows no volcanic winter in East Africa at 75 Statistics. World Hunger Education Service (2013). Earth and Planetary Science Letters 277.1 (2009): 9-20, ka. Proceedings of the National Academy of Sciences 353 and Benton, Michael J., and M. J. Benton.: When life See Maher, Timothy M., and Seth D. Baum.: 110.20 (2013): 8025-8029. nearly died: the greatest mass extinction of all time. Vol. Adaptation to and recovery from global catastrophe. 392 See Zielinski, G. A., et al.: Potential atmospheric 5. London: Thames & Hudson (2003). Sustainability 5.4 (2013): 1461-1479. impact of the Toba Mega‐Eruption ∼ 71,000 years ago. 375 354 See Shen, Shu-zhong, et al.: Calibrating the end- Report of the Task Force on potentially hazardous Geophysical Research Letters 23.8 (1996): 837-840. Permian mass extinction. Science 334.6061 (2011): Near-Earth Objects. http://spaceguardcentre.com/wp- 393 See Yoichi Nakamura, Shigeo Aramaki and Eisuke 1367-1372. content/uploads/2014/04/full_report.pdf Fujita, Japan’s Volcanic Disaster Mitigation Initiatives: 376 355 Kerr, Richard A.: Mega-Eruptions Drove the Mother of See Ćirković, Milan M., Anders Sandberg, and Nick Activities of the Commission on Mitigation of Volcanic Mass Extinctions. Science 342.6165 (2013): 1424-1424. Bostrom.: Anthropic shadow: Observation selection Disasters, the Volcanological Society of Japan, Technical 377 effects and human extinction risks. Risk analysis 30.10 A partial list can be found on Wikipedia: http:// Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 190

193 Endnotes and existential risks from emerging technologies through Note of the National Research Institute for Earth Science strains. Nature 493.7433 (2013): 459-459. 431 international law. Available at SSRN 2179094 (2012). and Disaster Prevention 380 (2013). See Imai, Masaki, et al.: Experimental adaptation 411 394 of an influenza H5 HA confers respiratory droplet See Kelle, Alexander.: Ensuring the security of See Barnes, Gina L.: Origins of the Japanese Islands: transmission to a reassortant H5 HA/H1N1 virus in The New “Big Picture”. Nichibunken Japan Review synthetic biology—towards a 5P governance strategy. ferrets. Nature 486.7403 (2012): 420-428, and Herfst, Systems and synthetic biology 3.1-4 (2009): 85-90, (2003): 3-50. 395 Sander, et al.: Airborne transmission of influenza A/H5N1 and Bernauer, Hubert, et al.: Technical solutions for See Nakamura, Y., Fukushima, K., Jin, X., Ukawa, M., virus between ferrets. Science 336.6088 (2012): 1534- biosecurity in synthetic biology. Industry Association Sato, T., and Hotta, Y.: Mitigation Systems by Hazard 1541. Synthetic Biology, Munich, Germany (2008). Maps, Mitigation Plans, and Risk Analyses Regarding 432 412 Volcanic Disasters in Japan. J. Disaster Research, 3-4, Though see some of the suggestions in Garfinkel, See Butler, Declan.: Fears grow over lab-bred flu. (2007) 297-304. Michele S., et al.: Synthetic genomics: options for Nature 480 (2011): 421-422. 396 433 governance. Industrial Biotechnology 3.4 (2007): 333- See Omer, Haim, and Nahman Alon.: The continuity See Fouchier, R. A., et al.: Pause on avian flu 365. principle: A unified approach to disaster and trauma. transmission research. Science 335.6067 (2012): 400- 413 American Journal of Community Psychology 22.2 401. See Dennis, Carina.: The bugs of war. Nature 434 (1994): 273-287. 411.6835 (2001): 232-235. See Sherrill, Michelle.: Self-Serving Bias. American 414 397 Psychologist 41: 954-969 (2008). See Smith, Robert B. et al.: Discoveries and See Smith, Bruce D., and Mark Nesbitt.: The 435 emergence of agriculture. New York: Scientific American breakthroughs from integrative analysis of Yellowstone See for instance the London Independent: Leading Library (1995). science revealing new tectonic-volcanic models and scientists condemn decision to continue controversial 415 a new assessment of earthquake-volcano hazards. research into deadly H5N1 bird-flu virus. See Tucker, Jonathan B.: The Current Bioweapons Geological Society of America Abstracts with Programs. 436 Threat. Biopreparedness and Public Health. Springer See the Report of the WHO Informal Consultation on Vol. 45. No. 7. 2013. Netherlands (2013) 7-16. Dual Use Research of Concern. 398 416 See Earthquake-hazards Scenario for a M7 437 With corporations targeting each other’s biological See Imai, Masaki, et al.: Experimental adaptation Earthquake on the Salt Lake City Segment of the products, for instance. of an influenza H5 HA confers respiratory droplet Wasatch Fault Zone, Utah. No. 111. Utah Geological 417 transmission to a reassortant H5 HA/H1N1 virus in A risk the US Government is already taking seriously; Survey (2004). ferrets. Nature 486.7403 (2012): 420-428, and Herfst, see for instance President George W. Bush’s April 2004 399 See the Towers Watson poll of global insurance Sander, et al.: Airborne transmission of influenza A/H5N1 Biodefense for the 21st Century initiative. The threat executives on the most worrying extreme risks facing virus between ferrets. Science 336.6088 (2012): 1534- of bioterrorism is also prominent in the 2007 report the insurance industry. 1541. Mapping the Global Future: Report of the National 400 438 Intelligence Council’s 2020 Project. See Merton, Robert K.: Bureaucratic structure and See the WHO report Responsible life sciences 418 personality. Social forces 18.4 (1940): 560-568, and research for global health security: a guidance The World Health Organization estimates that many Turner, Marlene E., and Anthony R. Pratkanis.: Twenty- document. (2010). countries and institutions lack oversight of such dual five years of groupthink theory and research: Lessons 439 use technologies (source: Report of the WHO Informal See Jackson, Ronald J., et al.: Expression of from the evaluation of a theory. Organizational Behavior Consultation on Dual Use Research of Concern, World mouse interleukin-4 by a recombinant ectromelia and Human Decision Processes 73.2 (1998): 105-115. Health Organization). virus suppresses cytolytic lymphocyte responses and 401 419 See Kerr, Richard A.: Mega-Eruptions Drove the overcomes genetic resistance to mousepox. Journal of See Baum, Seth, and Grant Wilson.: The Ethics of Mother of Mass Extinctions. Science 342.6165 (2013): virology 75.3 (2001): 1205-1210. Global Catastrophic Risk from Dual-Use Bioengineering. 1424-1424. 440 Ethics in Biology, Engineering and Medicine 4 (2013). See Rosengard, Ariella M., et al.: Variola virus immune 402 420 A partial list can be found on Wikipedia: http:// evasion design: expression of a highly efficient inhibitor Attempts at containing such a leak may not be en.wikipedia.org/wiki/Timetable_of_major_worldwide_ of human complement. Proceedings of the National successful; see Merler, Stefano, et al.: Containing the volcanic_eruptions. Academy of Sciences 99.13 (2002): 8808-8813. accidental laboratory escape of potential pandemic 403 441 influenza viruses. BMC medicine 11.1 (2013): 252. See Reichow, Marc K., et al.: The timing and See Cello, Jeronimo, Aniko V. Paul, and Eckard 421 extent of the eruption of the Siberian Traps large Wimmer.: Chemical synthesis of poliovirus cDNA: See for instance Hwang, In Young, et al.: igneous province: Implications for the end-Permian generation of infectious virus in the absence of natural Reprogramming Microbes to Be Pathogen-Seeking environmental crisis. Earth and Planetary Science template. Science 297.5583 (2002): 1016-1018. Killers. ACS synthetic biology (2013). Letters 277.1 (2009): 9-20, and Benton, Michael J., and 442 422 Brazil alone is estimated to have over 50 uncontacted See Tumpey, Terrence M., et al.: Characterization of M. J. Benton.: When life nearly died: the greatest mass the reconstructed 1918 Spanish influenza pandemic tribes (source: Survival International). extinction of all time. Vol. 5. London: Thames & Hudson 423 virus. Science 310.5745 (2005): 77-80. See Top U.S. Disease Fighters Warn of New (2003). 443 Engineered Pathogens but Call Bioweapons Doomsday See Gibson, Daniel G., et al.: Complete chemical 404 See Shen, Shu-zhong, et al.: Calibrating the end- Unlikely. Global Security Newswire (2005). synthesis, assembly, and cloning of a Mycoplasma Permian mass extinction. Science 334.6061 (2011): 424 genitalium genome. Science 319.5867 (2008): 1215- Maher, Timothy M., and Seth D. Baum.: Adaptation to 1367-1372. 1220. and recovery from global catastrophe. Sustainability 5.4 405 http://en.wikipedia.org/wiki/Synthetic_biology 444 (2013): 1461-1479. See Zhang, Ying, et al.: H5N1 hybrid viruses bearing 406 See Schmidt, Markus, ed.: Synthetic biology: 425 2009/H1N1 virus genes transmit in guinea pigs by See the later part of the scenario in Tonn, Bruce, and industrial and environmental applications. Wiley- respiratory droplet. Science 340.6139 (2013): 1459- Donald MacGregor.: A singular chain of events. Futures Blackwell (2012). 1463. 41.10 (2009): 706-714. 407 445 Making Synthetic biology into an “epistemic” risk: the 426 The Human Fatality and Economic Burden of a Man- See Guan, Zheng-Jun, et al.: Biosafety risk might be minute or huge, it’s unclear at present. See made Influenza Pandemic. Considerations of Synthetic Biology in the International Sahlin, Nils-Eric, and Johannes Persson.: Epistemic risk: 446 Genetically Engineered Machine (iGEM) Competition. See Drexler, K. Eric.: Engines of Creation: the Coming the significance of knowing what one does not know. BioScience 63.1 (2013): 25-34. Era of Nanotechnology. Fourth Estate (1990). Future risks and risk management. Springer Netherlands 427 447 See Wired Magazine’s feature article Genome at See Drexler, K. Eric.: Radical Abundance: How a (1994) 37-62. Revolution in Nanotechnology Will Change Civilization. Home: Biohackers Build Their Own Labs. 408 The decision to lift the moratorium on potentially 428 Public Affairs (2013). Such as Bernauer, Hubert, et al.: Technical solutions dangerous gain of function flu research was taken by 448 for biosecurity in synthetic biology. Industry Association See Freitas, Robert A., and Ralph Charles Merkle.: the researchers themselves (source: Declan Butler: Work Synthetic Biology, Munich, Germany (2008). Kinematic self-replicating machines. Georgetown, TX: resumes on lethal flu strains, Nature News). 429 Landes Bioscience/Eurekah. com (2004). See Lenox, Michael J., and Jennifer Nash.: Industry 409 See Schmidt, Markus.: Diffusion of synthetic biology: 449 self‐regulation and adverse selection: a comparison See Committee to Review the National a challenge to biosafety. Systems and synthetic biology across four trade association programs. Business Nanotechnology Initiative.: A matter of size: Triennial 2.1-2 (2008): 1-6. strategy and the environment 12.6 (2003): 343-356. review of the National Nanotechnology Initiative. (2006). 410 See Wilson, Grant.: Minimizing global catastrophic 430 450 According to the CIA World Factbook. See Butler, Declan.: Work resumes on lethal flu Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 191

194 Endnotes 451 473 avoiding this. “Friendly AI” projects focus on the AI’s See Roco, Mihail C., and William S. Bainbridge.: See Drexler, K. Eric.: Radical Abundance: How a goals directly, and attempt to make these compatible Revolution in Nanotechnology Will Change Civilization. Societal implications of nanoscience and with human survival. The Oracle AI approach attempts PublicAffairs (2013). nanotechnology: Maximizing human benefit. Journal of 474 to “box” the AI (constrain its abilities to influence the Nanoparticle Research 7.1 (2005): 1-13. See Drexler, K. Eric.: Engines of Creation: the Coming 452 world) in order to prevent it from acquiring power. The Era of Nanotechnology. Fourth Estate (1990). See the report of the Center for Responsible reduced impact AI is a novel approach which attempts 475 Nanotechnology,: Dangers of Molecular Manufacturing. See Nanostart’s presentation at the to construct AIs goals that are not friendly per se, but 453 EuroNanoForum 2013, Dublin: Venture Capital See Drexler, K. Eric.: Radical Abundance: How a that motivate the AI to have a very limited impact on the Investments in Nanotech (2013). Revolution in Nanotechnology Will Change Civilization. world, thus constraining the potential damage it could 476 PublicAffairs (2013). See Nanostart’s newsletter, A space elevator would do. 454 be a tremendous benefit for humankind (2012). See Barber, Kevin, et al.: Nanotechnology for Uranium 494 There are many different designs for AIs, and 477 Separations and Immobilization. Advances in Materials See Levine, Michael E., and Jennifer L. Forrence.: disagreements about whether they could be considered Science for Environmental and Nuclear Technology: Regulatory capture, public interest, and the public moral agents or capable of suffering. At one end of Ceramic Transactions 107 (2010): 177. agenda: Toward a synthesis. Journal of Law, Economics, the scale is the theoretical AIXI(tl), which is essentially 455 & Organization 6 (1990): 167-198. See Gubrud, Mark Avrum.: Nanotechnology and nothing more than an equation with vast computing 478 international security. Fifth Foresight Conference on See the Congressional Research Service report, power; at the other lie whole brain emulations, copies of Molecular Nanotechnology (1997). Nanotechnology: a Policy Primer (2013). human minds implemented inside a computer. Suffering 456 479 See Altmann, Jürgen.: Military uses of Nano-replicator: A system able to build copies of itself for the first design seems unlikely; suffering for the nanotechnology: perspectives and concerns. Security when provided with raw materials and energy second seems very possible. Determining whether a Dialogue 35.1 (2004): 61-79. 480 http://en.wikipedia.org/wiki/Artificial_intelligence specific AI design can suffer will be an important part of 457 See the Oxford Martin School publication by Drexler, 481 figuring out what to do with it; it seems unlikely that we’ll See Armstrong, Stuart, and Kaj Sotala. 2012.: How K. Eric and Pamlin, Dennis: Nano-solutions for the 21st have hard and fast rules in advance. We’re Predicting AI—or Failing To. In Beyond AI: Artificial Century, (2013). 495 Dreams, edited by Jan Romportl, Pavel Ircing, Eva The idea behind uploads, also called whole brain 458 See Wilson, Grant.: Minimizing global catastrophic Zackova, Michal Polak, and Radek Schuster, 52–75 emulations, is to take a human brain, copy the position and existential risks from emerging technologies through 482 of its neurones and connections to sufficient precision, See Chalmers, David, The singularity: A philosophical international law. Available at SSRN 2179094 (2012). and then run the emulation forwards according to the analysis. Journal of Consciousness Studies 17.9-10 459 See Drexler, K. Eric.: Engines of Creation: the Coming laws of physics and chemistry. It should then behave (2010): 9-10. Era of Nanotechnology. Fourth Estate (1990). 483 exactly as a human brain in the real world. See http:// See Muehlhauser, Luke, and Anna Salamon.: 460 www.fhi.ox.ac.uk/brain-emulation-roadmap-report.pdf See Phoenix, Chris, and Eric Drexler.: Safe Intelligence explosion: Evidence and import. Singularity for more details, which also goes into the requirements exponential manufacturing. Nanotechnology 15.8 (2004): Hypotheses. Springer Berlin Heidelberg (2012) 15-42. and tries to estimate the likelihood off success of such 869. 484 See Yudkowsky, Eliezer.: Intelligence Explosion 461 an approach (it depends on certain likely but uncertain See Freitas Jr, Robert A.: Some limits to global Microeconomics. Technical report 2013-1 Berkeley, CA: assumptions about how the brain works at the small ecophagy by biovorous nanoreplicators, with public Machine Intelligence Research Institute (2013). scale; ie that most things below a certain scale can policy recommendations. Foresight Institute (2000). 485 See Wilson, Grant.: Minimizing global catastrophic be handled statistically rather than through detailed 462 Such as computer viruses today, which endure and existential risks from emerging technologies through modelling). despite efforts at eradication, but don’t threaten the international law. Available at SSRN 2179094 (2012). 496 There’s two ways there could be diminishing returns system as a whole. 486 See Omohundro, Stephen M.: The basic AI drives. to intelligence: first, it might not be possible for an AI 463 See Leary, Scott P., Charles Y. Liu, and Michael LJ Frontiers in Artificial Intelligence and applications 171 to improve its own intelligence strongly. Maybe it could Apuzzo.: Toward the emergence of nanoneurosurgery: (2008): 483. find some ways of better executing its algorithms, and Part III-Nanomedicine: Targeted nanotherapy, 487 See Muehlhauser, Luke, and Louie Helm.: a few other low-hanging fruits, but it’s very possible that nanosurgery, and progress toward the realization of Intelligence Explosion and Machine Ethics. In Singularity after doing this, it will find that the remaining intelligence- nanoneurosurgery. Neurosurgery 58.6 (2006): 1009- Hypotheses: A Scientific and Philosophical Assessment, increase problems are hard, and that it hasn’t improved 1026.. edited by Amnon Eden, Johnny Søraker, James H. Moor, itself sufficiently to make them easy (you could analogise 464 See the later part of the scenario in Tonn, Bruce, and and Eric Steinhart. Berlin: Springer (2012) this to dampened resonance - initially the AI can improve Donald MacGregor.: A singular chain of events. Futures 488 Dealing with most risks comes under the category its intelligence quite easily, and this gives it more 41.10 (2009): 706-714. of decision theory: finding the right approaches to intelligence with which to work, allowing it to improve its 465 See Tetlock, Philip E.: Expert political judgment: How maximise the probability of the most preferred options. intelligence still further; but each time, the effect is less, good is it? How can we know? Princeton University But an intelligent agent can react to decisions in a way and it soon plateaus). Press (2005). the environment cannot, meaning that interactions The other way there could be diminishing returns to 466 with AIs are better modelled by the more complicated See Lewandowski, Bartosz, et al.: Sequence- intelligence is if intelligence doesn’t translate into power discipline of game theory. specific peptide synthesis by an artificial small-molecule effectively. This is more likely with things like social 489 machine. Science 339.6116 (2013): 189-193. See Yudkowsky, Eliezer.: Artificial intelligence as intelligence: it seems plausible that no entity could 467 a positive and negative factor in global risk. Global See Freitas, Robert A., and Ralph Charles Merkle.: convince, say, 99% of Britain to vote for their party, catastrophic risks 1 (2008): 303. Kinematic self-replicating machines. Georgetown, TX: no matter how socially intelligent it was. Scientific and 490 Landes Bioscience/Eurekah. com (2004). See Muehlhauser, Luke, and Nick Bostrom.: Why we technological intelligence could conceivably be limited 468 need friendly AI. Think: Philosophy for Everyone (2014). Defined by Eric Drexler as a “device able to guide by creativity (unlikely), constraints on the speed of 491 chemical reactions by positioning reactive molecules making experiments, or just the fact that humans have The balance is currently very uneven, with only three with atomic precision.” solved the easy problems already. small organisations – the Future of Humanity Institute, 469 497 the Machine Intelligence Research Institute and the See Vugt, Gert Van.: The Killer Idea: How Some See DARPA’s briefing, DARPA-SN-13-30: Cambridge Centre for Existential Risk – and focused Gunslinging Anarchists Held Freedom of Speech at Probabilistic Programming for Advancing Machine mainly on the risks of AI, along with some private Gunpoint. Proceedings of the eLaw Conference 3D Learning (PPAML) (2013). 498 individuals. Printing: Destiny, Doom or Dream (2013). See for instance Curtis, Jon, Gavin Matthews, and 492 470 See the report of the Center for Responsible See Sandberg, Anders, and Nick Bostrom.: Whole David Baxter.: On the effective use of Cyc in a question Nanotechnology, Dangers of Molecular Manufacturing. brain emulation: A roadmap. Future of Humanity Institute answering system. Proc Workshop on Knowledge and 471 Technical Report #2008-3, Oxford University (2008). Reasoning for Answering Questions. (2005). See Excell, John, and Stuart Nathan.: The rise of 493 499 additive manufacturing. The Engineer (2010). A dangerous AI would be a very intelligent that one See The Rise of Industrial Big Data, GE Intelligent 472 could acquire great power in the world, and would Platforms (2013). The gun designer, Cody Wilson, who describes himself 500 have goals that were not compatible with full human as a crypto-anarchist, said his plans to make the design Such as Watson’s triumph on “Jeopardy!” (source: survival and flourishing. There are a variety of ways of available were “about liberty” (source: BBC News). Ferrucci, David, et al.: Building Watson: An overview of Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 192

195 Endnotes life and sharpening the Fermi paradox. Acta Astronautica Steve.: Rational artificial intelligence for the greater good. the DeepQA project. AI magazine 31.3 (2010): 59-79). 501 (2013). Singularity Hypotheses. Springer Berlin Heidelberg. See the Wired report by Steven Levy, How Ray 540 (2012) 161-179. Kurzweil Will Help Google Make the Ultimate AI Brain See Bostrom, Nick.: Anthropic bias: Observation 519 (2013). selection effects in science and philosophy. Psychology See James Barrat.: Our Final Invention: Artificial 502 Press (2002). Intelligence and the End of the Human Era. (2013). “Soon after [1988] came a period called the ‘AI 541 520 Winter,’ in which many companies fell by the wayside as See Omohundro, Stephen M.: The basic AI drives. Maher, Timothy M., and Seth D. Baum.: Adaptation to they failed to deliver on extravagant promises.” (Source: Frontiers in Artificial Intelligence and applications 171 and recovery from global catastrophe. Sustainability 5.4 Stuart, Russell, and Norvig Peter. “Artificial intelligence: (2013): 1461-1479. (2008): 483. 542 521 a modern approach.” Upper Saddle River, New Jersey: See Wilson, Grant.: Minimizing global catastrophic Directly coding safe behaviour is seen to be an Prentice Hall (2003) pp24). and existential risks from emerging technologies through extremely difficult challenge. 503 522 international law. Available at SSRN 2179094 (2012). “At its low point, some computer scientists and See Armstrong, Stuart, Bostrom, Nick, and Shulman, 543 software engineers avoided the term artificial intelligence Carl.: Racing to the precipice: a model of artificial See Maher, Timothy M., and Seth D. Baum: for fear of being viewed as wild-eyed dreamers.” intelligence development. Technical Report #2013-1, Adaptation to and recovery from global catastrophe. (Source: New York Times report by John Markov: Behind Future of Humanity Institute, Oxford University (2013).: Sustainability 5.4 (2013): 1461-1479. Artificial Intelligence, a Squadron of Bright Real People 544 http://www.fhi.ox.ac.uk/wp-content/uploads/Racing- One possible exception could be James Lovelock: (2005)). to-the-precipice-a-model-of-artificial-intelligence- The Revenge of Gaia. Why the Earth is fighting back— 504 development.pdf See Ferrucci, David, et al.: Building Watson: An and how we can still save humanity. (2006). 523 overview of the DeepQA project. AI magazine 31.3 545 See Smith, Theresa Clair.: Arms race instability and See Tonn, Bruce, and Dorian Stiefel.: Evaluating (2010): 59-79. war. Journal of Conflict Resolution 24.2 (1980): 253-284. methods for estimating existential risks. Risk Analysis 505 http://jcr.sagepub.com/content/24/2/253.full.pdf+html See Guizzo, Erico.: How Google’s self-driving car (2013). 524 works. IEEE Spectrum Online (2011). 546 See Rapoport, Anatol.: Prisoner’s dilemma: A study For more information about the different methods: 506 in conflict and cooperation. University of Michigan Press See Kurzweil, Ray.: The Age of Spiritual Machines: Tonn, Bruce, and Dorian Stiefel. “Evaluating methods (1965). How we will live, work and think in the new age of for estimating existential risks.” Risk Analysis 33.10 525 intelligent machines. Orion (1999). (2013): 1772-1787. http://www.ncbi.nlm.nih.gov/ See the special events on the program of the 23rd 507 pubmed/23551083 International Joint Conference on Artificial Intelligence. See Sotala, Kaj and Roman V. Yampolskiy.: 547 http://ijcai13.org/program/special_events_the_impact_ Responses to Catastrophic AGI Risk: A Survey. See Tonn, Bruce E.: A methodology for organizing of_ai_on_society and quantifying the results of environmental scanning Technical report 2013-2. Berkeley, CA: Machine 526 exercises. Technological Forecasting and Social Change See Bostrom, Nick.: Existential Risk Prevention as Intelligence Research Institute (2013) 508 75.5 (2008): 595-609. Global Priority. Global Policy 4.1 (2013): 15-31. The term AI is now ambiguous, with various “narrow 548 527 AIs” being produced (such as self-driving cars or See Armstrong, Stuart, and Anders Sandberg.: See Sherrill, Michelle.: Self-Serving Bias. American stock-trading programmes). An “AGI” returns to the Eternity in six hours: Intergalactic spreading of intelligent Psychologist 41: 954-969 (2008). https://facultystaff. original meaning of AI, that of an artificial being with life and sharpening the Fermi paradox. Acta Astronautica richmond.edu/~dforsyth/pubs/forsyth2008selfserving. general intelligence skills. This report will use the term AI (2013). pdf 549 528 interchangeably with AGI, however. See Yudkowsky, Eliezer.: Artificial intelligence as See Webb, Stephen.: If the Universe is teeming with 509 a positive and negative factor in global risk. Global aliens... Where is everybody?: Fifty solutions to the Fermi See Armstrong, Stuart, and Kaj Sotala. 2012.: How paradox and the problem of extraterrestrial life. Springer (2002). catastrophic risks 1 (2008): 303. http://intelligence.org/ We’re Predicting AI—or Failing To. In Beyond AI: Artificial 550 files/AIPosNegFactor.pdf Dreams, edited by Jan Romportl, Pavel Ircing, Eva See Hanson, Robin.: The great filter - are we almost 529 Zackova, Michal Polak, Radek Schuster, 52–75 past it? (1998). One of the authors of Artificial Intelligence: a 510 551 modern approach. http://www.amazon.co.uk/ See Yudkowsky, Eliezer.: Levels of Organization in See Lineweaver, Charles H., Yeshe Fenner, and Brad Artificial-Intelligence-Approach-International-Edition/ General Intelligence. In Artificial General Intelligence, K. Gibson.: The galactic habitable zone and the age dp/0130803022 edited by Ben Goertzel and Cassio Pennachin 389–501 distribution of complex life in the Milky Way. Science 530 (2007). 303.5654 (2004): 59-62. See CSER’s list of external advisors on their website. 511 552 http://cser.org/ See Bostrom, Nick.: The superintelligent will: http://www.fhi.ox.ac.uk/intergalactic-spreading.pdf 531 Motivation and instrumental rationality in advanced 553 See for instance the “X factors” section of Global See The Philippine Star news report Top artificial agents. Minds and Machines 22.2 (2012): 71-85, Risks 2013 - Eighth Edition. World Economic Forum conglomerates spearhead creation of private sector and Armstrong, Stuart.:General purpose intelligence: (2013). disaster response body (2013). arguing the Orthogonality thesis.” Analysis and 532 554 See Ellis, John, et al.: Review of the safety of LHC See the UNISDR report Private Sector Strengths Metaphysics (2013). collisions. Journal of Physics G: Nuclear and Particle Applied (2013). http://www.philstar.com:8080/ 512 See the Less Wrong summary post Complexity of Physics 35.11 (2008): 115004. business/2013/12/10/1266103/top-conglomerates- value for a discussion on this issue. 533 spearhead-creation-private-sector-disaster-response See S Korean dies after games session, BBC News. 513 See Yampolskiy, Roman V.: Leakproofing the 534 http://www.unisdr.org/files/33594_ See Alex Knapp, Is It Ethical to Make Animals As Singularity. Journal of Consciousness Studies 19.1-2 privatesectorstrengthsapplied2013di.pdf Smart As People? Forbes. (2012): 194-214. 555 535 See George Dvorsky, New Project to Message Aliens http://www.who.int/trade/glossary/story038/en/ 514 See Armstrong, Stuart, Anders Sandberg, and Nick 556 is Both Useless and Potentially Reckless, io9. There are known effective strategies for poverty Bostrom.: Thinking inside the box: Using and controlling 536 reduction. See for instance Ravallion, Martin.: A See Shanteau, James.: Competence in experts: The an Oracle AI. Minds and Machines (2012). comparative perspective on poverty reduction in Brazil, role of task characteristics. Organisational behavior and 515 See Armstrong, Stuart, Anders Sandberg, and Nick China and India. World Bank Policy Research Working human decision processes 53.2 (1992): 252-266. Bostrom.: Thinking inside the box: Using and controlling 537 Paper Series (2009). See Ord, Toby, Rafaela Hillerbrand, and Anders an Oracle AI. Minds and Machines (2012). 557 Sandberg.: Probing the improbable: Methodological See Bostrom, Nick.: What is a singleton? Linguistic 516 See Dewey, Daniel. “Learning what to value.” Artificial challenges for risks with low probabilities and high and Philosophical Investigations 5.2 (2006): 48-54, and General Intelligence. Springer Berlin Heidelberg, (2011). stakes. Journal of Risk Research 13.2 (2010): 191-205. Caplan, Bryan.: The totalitarian threat. (2008). 309-314. 558 538 See Tetlock, Philip E.: Expert political judgment: How See Ćirković, Milan M.: Small Theories and Large 517 See Thomas, Michael SC, and James L. McClelland. good is it? How can we know? Princeton University Risks—Is Risk Analysis Relevant for Epistemology?. Risk “Connectionist models of cognition.” The Cambridge Analysis 32.11 (2012): 1994-2004. Press (2005). handbook of computational psychology (2008): 23-58. 559 539 See Tetlock, Philip E.: Thinking the unthinkable: This paper demonstrates the relative ease of moving 518 See Yudkowsky, Eliezer.: Artificial intelligence as between galaxies, making the Fermi paradox more Sacred values and taboo cognitions. Trends in cognitive a positive and negative factor in global risk. Global problematic: Armstrong, Stuart, and Anders Sandberg.: sciences 7.7 (2003): 320-324. catastrophic risks 1 (2008): 303, and Omohundro, 560 Eternity in six hours: Intergalactic spreading of intelligent See for instance the multiplicity of values in the World Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 193

196 Endnotes Cameron Heath (Chair), John Campbell, Daniel Clarke, 11-17. Values Survey. Those countries with stronger survival, 579 Darren Farr, Gladys Hosken, Gillian James, Andrew and weaker self expression, values would be presumably See Zielinski, G. A., et al.: Potential atmospheric Newman, David Simmons, and Hannes Van Rensburg more likely to tolerate dictatorships in exchange for impact of the Toba Mega‐Eruption 71,000 years ago. www.actuaries.org.uk/system/files/documents/pdf/1- reduced casualties. Geophysical Research Letters 23.8 (1996): 837-840. 561 200-final-paper.pdf 580 These papers provide an overview of the potential See Jablonski, David, and W. G. Chaloner.: 600 resources for future human generations, hinting at their http://www.itm.su.se/page.php?pid=1081 Extinctions in the Fossil Record [and Discussion]. 601 potential size: Bostrom, Nick.: Astronomical waste: The Philosophical Transactions of the Royal Society of http://www.ipcc.ch opportunity cost of delayed technological development. 602 London. Series B: Biological Sciences 344.1307 (1994): Schaeffer, Michiel; Hare, William; Rahmstorf, Stefan; Utilitas 15.03 (2003): 308-314, and Armstrong, Stuart, 11-17. Vermeer, Martin (2009): Long-term sea-level rise implied and Anders Sandberg.: Eternity in six hours: Intergalactic 581 by 1.5 C and 2 C warming levels. Nature http://www. See Hall, Christopher D., and I. Michael Ross.: spreading of intelligent life and sharpening the Fermi nature.com/nclimate/journal/v2/n12/full/nclimate1584. Dynamics and control problems in the deflection of paradox. Acta Astronautica (2013). html Near-Earth Objects. Adv. Astronaut. Sci 67.640 (1997): 562 See Bostrom, Nick.: Existential Risk Prevention as 603 1-18. Sources for the estimates include: Global Priority. Global Policy 4.1 (2013): 15-31. http:// 582 See Parmesan, Camille.: Ecological and evolutionary Fekete, Hanna; Vieweg, Marion; Rocha, Marcia; www.existential-risk.org/concept.pdf responses to recent climate change. Annu. Rev. Ecol. Braun, Nadine; Lindberg, Marie; Gütschow, Johannes; 563 See Oxford Poverty and Human Development Evol. Syst. 37 (2006): 637-669. Jefferey, Louise; Höhne, Niklas ; Hare, Bill; Schaeffer, Initiative (2013), Multidimensional Poverty Index (MPI) 583 Michiel; Macey, Kirsten and Larkin, Julia (2013): See Epstein, Paul R.: Climate and health. Science Data Bank. OPHI, University of Oxford. Analysis of current greenhouse gas emission trends 285.5426 (1999): 347-348. 564 See Ravallion, Martin.: A comparative perspective on 584 http://climateactiontracker.org/assets/publications/ See the World Health Organization Global Influenza poverty reduction in Brazil, China and India. World Bank publications/CAT_Trend_Report.pdf Programme.: Pandemic influenza preparedness and Policy Research Working Paper Series (2009). response: a WHO guidance document. World Health New, Mark G.; Liverman, Diana M.; Betts, Richard 565 See for instance Jeanneney, Sylviane Guillaumont, Organization (2009). A.; Anderson, Kevin L. and West, Chris C. (2011): and Kangni Kpodar.: Financial development and poverty 585 Four degrees and beyond: the potential for a global See Yudkowsky, Eliezer.:Artificial intelligence as reduction: Can there be a benefit without a cost? The temperature increase of four degrees and its implications a positive and negative factor in global risk. Global Journal of Development Studies 47.1 (2011): 143-163. catastrophic risks 1 (2008): 303. http://rsta.royalsocietypublishing.org/content/369/1934. 566 See the Guardian’s report, The NSA Files. 586 toc See Freitas Jr, Robert A.: Some limits to global 567 The reaction was both within the USA and without; ecophagy by biovorous nanoreplicators, with public Rogelj, Joeri (2013): Risk shifts under changing climate see for instance Der Spiegel’s report, Out of Hand: policy recommendations. Foresight Institute (2000). sensitivity estimates, http://global-risk-indicator.net/data/ Europe Furious Over US Spying Allegations. 587 pdf_01.pdf See the report of the Center for Responsible 568 Many news stories raised the issue; see for instance Nanotechnology, Dangers of Molecular Manufacturing. Schneider, Stephen H. (2005): What is the Probability of the PBS report NSA repeatedly failed to obey court 588 ‘Dangerous’ Climate Change? http://stephenschneider. See for instance Imai, Masaki, et al.: Experimental orders on how to conduct surveillance on Americans. stanford.edu/Climate/Climate_Impacts/ adaptation of an influenza H5 HA confers respiratory 569 WhatIsTheProbability.html See the UNEP press release UNDP-UNEP Poverty- droplet transmission to a reassortant H5 HA/H1N1 virus Environment Initiative Launches New Five-Year Phase to in ferrets. Nature 486.7403 (2012): 420-428. Sokolov, A. P., and Co-authors, 2009: Probabilistic Meet Growing Demand from Member States (2013). 589 Forecast for Twenty-First-Century Climate Based on See Drexler, K. Eric.: Radical Abundance: How a 570 Uncertainties in Emissions (Without Policy) and Climate See the UNEP news report Sharing the waste, Revolution in Nanotechnology Will Change Civilization. Parameters. J. Climate, 22, 5175–5204. sharing the wealth: Uruguay uses the law to catalyse the Public Affairs (2013). transition to an inclusive green economy. 590 http://journals.ametsoc.org/doi/ See Drexler, K. Eric.: Radical Abundance: How a 571 abs/10.1175/2009JCLI2863.1 See the UNEP news report Malawi changes course Revolution in Nanotechnology Will Change Civilization. after analysing the real costs and benefits of policy Public Affairs (2013). World Bank (2013): Turn Down the Heat: Climate choices. 591 Extremes, Regional Impacts, and the Case for Resilience See Sailor, William C., et al.: A nuclear solution to 572 See Ravallion, Martin.: A comparative perspective on http://www.worldbank.org/content/dam/Worldbank/ climate change? Science (Washington) 288.5469 (2000): poverty reduction in Brazil, China and India. World Bank document/Full_Report_Vol_2_Turn_Down_The_ 1177-1178. http://faculty.publicpolicy.umd.edu/sites/ Policy Research Working Paper Series (2009). Heat_%20Climate_Extremes_Regional_Impacts_Case_ default/files/fetter/files/2000-Science-NE.pdf 573 for_Resilience_Print%20version_FINAL.pdf 592 See Dollar, David, and Aart Kraay.: Trade, growth, See Harding, Jim.: Economics of nuclear power 604 and poverty World Bank, Development Research Group, http://news.stanford.edu/news/2003/september24/ and proliferation risks in a carbon-constrained Macroeconomics and Growth (2001). tellerobit-924.html world. The Electricity Journal 20.10 (2007): 65-76. 574 605 http://www.sciencedirect.com/science/article/pii/ World Bank address to a ministerial conference on Robock, A., L. Oman, and G. L. Stenchikov S1040619007001285 (2007), Nuclear winter revisited with a modern climate responses to pandemic threat, 2008, in Background 593 Paper, Pandemic risk, by Olga B. Jonas, The World model and current nuclear arsenals: Still catastrophic Morse, Stephen S., et al.: Prediction and prevention consequences, J. Geophys. Res., 112, D13107, Bank: http://un-influenza.org/sites/default/files/WDR14_ of the next pandemic zoonosis. The Lancet 380.9857 doi:10.1029/2006JD008235. http://www.envsci.rutgers. bp_Pandemic_Risk_Jonas.pdf (2012): 1956-1965. http://www.thelancet.com/journals/ 575 edu/~gera/nwinter/nw6accepted.pdf lancet/article/PIIS0140-6736(12)61684-5/abstract See Esser, James K.: Alive and well after 25 years: A 606 594 review of groupthink research. Organizational behavior Sources for the estimates include: See this text by Nick Bostrom discussing difficulties and human decision processes 73.2 (1998): 116-141 estimating existential risks.: http://www.nickbostrom. Brams, Steven J.;Kilgour, D. Marc; Fichtner, John; & and Kerr, Norbert L., and R. Scott Tindale.: Group com/existential/risks.html Avenhaus, Rudolf (1989): The Probability of Nuclear War, performance and decision making. Annu. Rev. Psychol. 595 Journal of Peace Research 26(1): 91–99. http://econ. For an explanation of how to calculate probabilities 55 (2004): 623-655. as.nyu.edu/docs/IO/9397/RR86-24.pdf see: http://en.wikipedia.org/wiki/100-year_flood 576 See Homer-Dixon, Thomas.: The upside of down: 596 Helfand, Ira. “NUCLEAR FAMINE: TWO BILLION http://www.lloyds.com/the-market/business- catastrophe, creativity, and the renewal of civilization. PEOPLE AT RISK?” International Physicians for the timetable/solvency/ica Island Press (2008). Prevention of Nuclear War (2013). http://www.psr.org/ 597 http://en.wikipedia.org/wiki/Solvency_II 577 assets/pdfs/two-billion-at-risk.pdf See Turco, Richard P., et al.: Nuclear winter: global 598 What is a 1-in-200? Presented at GIRO 2009 by consequences of multiple nuclear explosions. Science Hellman, Martin E. (2008); ”Risk Analysis of Nuclear Cameron Heath (Chair), John Campbell, Daniel Clarke, 222.4630 (1983): 1283-1292. Deterrence” in THE BENT OF TAU BETA PI http:// Darren Farr, Gladys Hosken, Gillian James, Andrew 578 nuclearrisk.org/paper.pdf See Jablonski, David, and W. G. Chaloner.: Newman, David Simmons, and Hannes Van Rensburg Extinctions in the Fossil Record [and Discussion]. Lugar, Richard G (2005): ”The Lugar Survey On www.actuaries.org.uk/system/files/documents/pdf/1- Philosophical Transactions of the Royal Society of Proliferation Threats and Responses” https://www.fas. 200-final-paper.pdf London. Series B: Biological Sciences 344.1307 (1994): org/irp/threat/lugar_survey.pdf 599 What is a 1-in-200? Presented at GIRO 2009 by Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 194

197 Endnotes 612 http://en.wikipedia.org/wiki/Bullwhip_effect Robock, A., L. Oman, and G. L. Stenchikov and the future of biosecurity.” Politics and the Life 613 (2007), Nuclear winter revisited with a modern Sciences 28.2 (2009): 2-26. See for example Gao, Qin and Ma, Junhai (2009): climate model and current nuclear arsenals: Still Chaos and Hopf bifurcation of a finance system and Russ, Zachary N. “Synthetic biology: enormous catastrophic consequences, J. Geophys. Res., 112, Hommes, Cars, and Florian Wagener (2009): Complex possibility, exaggerated perils.” Journal of biological D13107, doi:10.1029/2006JD008235. http://web. evolutionary systems in behavioral finance. Handbook of engineering 2.7 (2008). kaust.edu.sa/faculty/GeorgiyStenchikov/nwinter/ financial markets: Dynamics and evolution Radosavljevic, Vladan, and Goran Belojevic. “A new RobockJgr2006JD008235.pdf 614 Sleep, Norman H., et al. (1989): Annihilation of model of bioterrorism risk assessment.” Biosecurity and Shulman, Carl (2012): Nuclear winter and human ecosystems by large asteroid impacts on the early Earth. bioterrorism: biodefense strategy, practice, and science extinction: Q&A with Luke Oman http://www. Nature 342.6246 7.4 (2009): 443-451. overcomingbias.com/2012/11/nuclear-winter-and- 615 http://neo.jpl.nasa.gov/risks/doc/sentry.html Tucker, Jonathan B., and Raymond A. Zilinskas. “The human-extinction-qa-with-luke-oman.html 616 promise and perils of synthetic biology.” New Atlantis https://b612foundation.org/ 607 See for example: http://metabiota.com/ and http:// 12.1 (2006): 25-45. 617 http://neo.jpl.nasa.gov/risks/ instedd.org/ 623 Altmann, Jürgen, and Mark A. Gubrud.: Military, 618 Sources for the estimates include: 608 Sources for the estimates include: arms control, and security aspects of nanotechnology. Jablonski, David, and W. G. Chaloner.: Extinctions Discovering the Nanoscale (2004): 269 http:// Bagus, Ghalid (2008): Pandemic Risk Modeling in the Fossil Record [and Discussion]. Philosophical nanoinvesting.my-place.us/altmann-gubrud.pdf http://www.chicagoactuarialassociation.org/CAA_ Transactions of the Royal Society of London. Series B: 624 Quoted in Kaku, Michio: Visions: how science will PandemicRiskModelingBagus_Jun08.pdf Biological Sciences 344.1307 (1994): 11-17. revolutionize the 21st century. Oxford University Press, Broekhoven, Henk van, Hellman, Anni (2006): Actuarial Collins, Gareth S., H. Jay Melosh, and Robert A. 1999 reflections on pandemic risk and its consequences Marcus.: Earth Impact Effects Program: A Web‐ 625 Altmann, Jürgen, and Mark A. Gubrud.: Military, http://actuary.eu/documents/pandemics_web.pdf based computer program for calculating the regional arms control, and security aspects of nanotechnology. Brockmann, Dirk and Helbing, Dirk (2013): The Hidden environmental consequences of a meteoroid impact Discovering the Nanoscale (2004): 269 http:// Geometry of Complex, Network-Driven Contagion on Earth. Meteoritics & planetary science 40.6 (2005): nanoinvesting.my-place.us/altmann-gubrud.pdf Phenomena SCIENCE VOL 342 http://rocs.hu-berlin.de/ 817-840. Berube, David M., et al. “Communicating Risk in the resources/HiddenGeometryPaper.pdf Monthly Notices of the Royal Astronomical Society 327.1 21st Century: The case of nanotechnology.” National W. Bruine de Bruin, B. Fischhoff; L. Brilliant and D. (2001): 126-132, and Board, Space Studies.: Defending Nanotechnology Coordination Office, Arlington (2010). Caruso (2006): Planet Earth: Near-Earth Object Surveys and Hazard Bostrom, Nick and Sandberg, Anders (2008): Global Mitigation Strategies. National Academies Press (2010). Expert judgments of pandemic influenza risks, Global catastrophic risks survey civil wars 98.3 http://www.fhi. Public Health, June 2006; 1(2): 178193 http://www.cmu. http://neo.jpl.nasa.gov/risks/ ox.ac.uk/gcr-report.pdf edu/dietrich/sds/docs/fischhoff/AF-GPH.pdf Neukum, G., and B. A. Ivanov. (1994): Crater size Turchin, Alexey.: Structure of the global catastrophe. Khan K, Sears J, Hu VW, Brownstein JS, Hay S, distributions and impact probabilities on Earth from lunar, Risks of human extinction in the XXI century. Lulu. com, Kossowsky D, Eckhardt R, Chim T, Berry I, Bogoch terrestrial-planet, and asteroid cratering data. Hazards 2008. I, Cetron M.: Potential for the International Spread due to Comets and Asteroids 359 Williams, Richard A., et al. “Risk characterization for of Middle East Respiratory Syndrome in Association Chodas, P., and D. Yeomans. (1999): Orbit determination nanotechnology.” Risk Analysis 30.11 (2010): 1671-1679. with Mass Gatherings in Saudi Arabia. PLOS Currents and estimation of impact probability for near-Earth 626 Outbreaks. 2013 Jul 17. Edition 1. doi: 10.1371/currents. http://www.fhi.ox.ac.uk/wp-content/uploads/Racing- objects. outbreaks.a7b70897ac2fa4f79b59f90d24c860b8. to-the-precipice-a-model-of-artificial-intelligence- Chapman, Clark R., and David Morrison. (1994): Impacts development.pdf http://currents.plos.org/outbreaks/article/assessing-risk- on the Earth by asteroids and comets: assessing the 627 for-the-international-spread-of-middle-east-respiratory- Bostrom, Nick and Sandberg, Anders (2008): Global hazard. Nature 367.6458 syndrome-in-association-with-mass-gatherings-in- catastrophic risks survey civil wars 98.3 http://www.fhi. Chapman, Clark R., Daniel D. Durda, and Robert E. saudi-arabia/ ox.ac.uk/gcr-report.pdf Gold. (2001): The comet/asteroid impact hazard: a Murray, Christopher JL, et al.: Estimation of potential Kruel Alexander (2013): Probability of unfriendly and systems approach. Office of Space Studies, Southwest global pandemic influenza mortality on the basis of vital friendly AI, including comments to the blog post Research Institute, Boulder CO 80302 registry data from the 1918–20 pandemic: a quantitative 619 http://kruel.co/2013/09/23/probability-of-unfriendly-and- Sources for the estimations include: analysis. The Lancet 368.9554 (2007): 2211-2218. http:// friendly-ai/#sthash.xRhFOGHW.dpbs 620 E.g. US Government laboratories had 395 incidents www.thelancet.com/journals/lancet/article/PIIS0140- 628 Bostrom, Nick and Sandberg, Anders (2008): Global that involved the potential release of selected agents 6736(06)69895-4/fulltext catastrophic risks survey civil wars 98.3 http://www.fhi. between 2003 and 2009, though only seven related Sandman , Peter M. (2007): Talking about a flu pandemic ox.ac.uk/gcr-report.pdf infections were reported. The accidents, including worst-case scenario, http://www.cidrap.umn.edu/news- 629 animal bites, needle sticks, and other mishaps, are http://www.who.int/trade/glossary/story038/en/ perspective/2007/03/talking-about-flu-pandemic-worst- 630 mentioned briefly in an NRC report on the plans for a One of those who led the development of how case-scenario risk assessment for an Army biodefense lab to be built at organisations can become destructive is Philip 609 Sources for the estimates include: Peterson, G. D., S. Ft. Detrick in Frederick, Md. National Research Council. Zimbardo, Stanford University http://www.lucifereffect. R. Carpenter, and William A. Brock.: Uncertainty and the Review of Risk Assessment Work Plan for the Medical com/ management of multistate ecosystems: an apparently Countermeasures Test and Evaluation Facility at Fort 631 http://www.economist.com/news/briefing/21578643- rational route to collapse. Ecology 84.6 (2003): 1403- Detrick: A Letter Report. Washington, DC: The National world-has-astonishing-chance-take-billion-people- 1411. Academies Press, 2011: http://www.nap.edu/catalog. out-extreme-poverty-2030-not?fsrc=scn/tw/te/pe/im/ Butler, Colin D., Carlos F. Corvalan, and Hillel S. Koren.: php?record_id=13265 notalwayswithus Human health, well-being, and global ecological 621 Clauset, Aaron, and Ryan Woodard. (2013): 632 http://www.worldbank.org/en/news/press- scenarios. Ecosystems 8.2 (2005): 153-162.Brook, Barry Estimating the historical and future probabilities of large release/2013/10/09/world-bank-sets-interim-poverty- W., Navjot S. Sodhi, and Corey J A Bradshaw.: Synergies terrorist events. The Annals of Applied Statistics 7.4: target-at-9-percent-in-2020 among extinction drivers under global change. Trends in 1838-1865. http://arxiv.org/pdf/1209.0089v3.pdf 633 http://www.worldbank.org/en/news/press- ecology & evolution 23.8 (2008): 453-460. 622 Bennett, Gaymon, et al. “From synthetic biology to release/2013/10/09/world-bank-sets-interim-poverty- Conversations with Johan Rockstrom, Executive director biohacking: are we prepared?.” Nature Biotechnology target-at-9-percent-in-2020 of the Stockholm Resilience Centre. 27.12 (2009): 1109-1111. 634 http://econ.worldbank.org/external/default/ 610 This is true for all complex systems. E.g. Sage, A.P., Bostrom, Nick and Sandberg, Anders (2008): Global main?pagePK=64165259&theSitePK=469372&piPK=64 Systems engineering: Fundamental limits and future catastrophic risks survey: civil wars 98.3 http://www.fhi. 165421&menuPK=64166093&entityID=000094946_030 prospects, Proceedings of the IEEE , vol.69, no.2, ox.ac.uk/gcr-report.pdf 11004010512 pp.158,166, Feb. 1981 Mukunda, Gautam, Kenneth A. Oye, and Scott C. Mohr. 635 611 http://en.wikipedia.org/wiki/History_of_Russia_ Sources for the estimates include: “What rough beast? Synthetic biology, uncertainty, Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 195

198 Endnotes 656 Nagy, Béla; Doyne Farmer, J.; M. Bui, Quan; E. (1992%E2%80%93present) 636 Trancik, Jessika (2013): A projection of future PV http://www.brookings.edu/research/interactives/2013/ electricity costs from the Photovoltaics2 historical data ending-extreme-poverty set (1977–2009) using Moore’s exponential functional 637 http://www.brookings.edu/research/interactives/2013/ form. Figure_6.tif. PLOS ONE. 10.1371/journal. ending-extreme-poverty pone.0052669.g006. 638 http://www.economist.com/news/briefing/21578643- 657 http://www.itu.int/en/ITU-D/Statistics/Pages/intlcoop/ world-has-astonishing-chance-take-billion-people- mdg/default.aspx out-extreme-poverty-2030-not?fsrc=scn/tw/te/pe/im/ 658 http://www.un.org/esa/population/publications/ notalwayswithus worldageing19502050/pdf/90chapteriv.pdf 639 http://en.wikipedia.org/wiki/Malthusian_catastrophe 659 http://www.un.org/esa/population/publications/ 640 http://www.worldbank.org/depweb/english/beyond/ worldageing19502050/pdf/90chapteriv.pdf global/chapter3.html 660 http://www.un.org/esa/population/publications/ 641 http://www.worldbank.org/depweb/english/beyond/ worldageing19502050/pdf/90chapteriv.pdf global/chapter3.html 661 http://www.un.org/esa/population/publications/ 642 http://www.worldbank.org/depweb/english/beyond/ worldageing19502050/pdf/90chapteriv.pdf global/chapter3.html 662 http://en.wikipedia.org/wiki/Big_data 643 http://esa.un.org/wpp/Documentation/pdf/WPP2012_ 663 For some examples of visualisation tools: http://www. HIGHLIGHTS.pdf creativebloq.com/design-tools/data-visualization-712402 644 http://esa.un.org/wpp/Documentation/pdf/WPP2012_ 664 For a discussion about risk and the importance HIGHLIGHTS.pdf of probability curves of risk, see Kaplan, Stanley and 645 http://esa.un.org/wpp/Documentation/pdf/WPP2012_ Garrick, John B., On the Quantitative Definition of HIGHLIGHTS.pdf Risk, Risk Analysis, Vol. I, No. 1, 1981, in which the 646 http://www.worldometers.info/world-population/ authors state (pp. 11–27): “We prefer to say that ‘risk is 647 http://esa.un.org/unpd/wpp/Documentation/pdf/ probability and consequence.’ In the case of a single WPP2010_Wallchart_Text.pdf scenario the probability times consequence viewpoint 648 http://wwf.panda.org/about_our_earth/all_ would equate a low-probability high-damage scenario publications/living_planet_report/ with a high-probability low-damage scenario – clearly not 649 http://www.worldbank.org/depweb/english/beyond/ the same thing at all. In the case of multiple scenarios the global/chapter8.html probability times consequence view would correspond 650 to saying the risk is the expected value of damage, i.e., http://en.wikipedia.org/wiki/List_of_wars_and_ the mean of the risk curve. We say it is not the mean of anthropogenic_disasters_by_death_toll 651 the curve, but the curve itself which is the risk. A single Van Evera, Stephen (2013): Causes of war: Power number is not a big enough concept to communicate and the roots of conflict. Cornell University Press, 2013. the idea of risk. It takes a whole curve. Now the truth is Xu Jin (2006): The Strategic Implications of Changes that a curve is not a big enough concept either. It takes a in Military Technology, Chinese Journal of International whole family of curves to fully communicate the idea of Politics 2006 1: 163-193. http://cjip.oxfordjournals.org/ risk.” content/1/2/163.full 665 http://www.ipcc.ch/pdf/supporting-material/ 652 http://www.plosone.org/article/ uncertainty-guidance-note.pdf info%3Adoi%2F10.1371%2Fjournal.pone.0052669 666 SNRA covers risks such as school shootings, 653 http://www.kurzweilai.net/the-law-of-accelerating- heatwaves and collapses of hydroelectric dams returns 667 https://www.msb.se/RibData/Filer/pdf/26561.pdf 654 Lloyd, Seth (2000): Ultimate physical limits to 668 http://www.ipcc.ch/pdf/supporting-material/ computation, http://arxiv.org/pdf/quant-ph/9908043. uncertainty-guidance-note.pdf pdf?origin=publication_detail 669 Richard Possner has described key elements of such http://www.nature.com/nature/journal/v406/n6799/ a centre in the book Catastrophe: risk and response. full/4061047a0.html Posner, Richard A. Catastrophe: risk and response. 655 E.g. Claw E. (1998) The natural limits of technological Oxford University Press, 2004. innovation: http://www.sciencedirect.com/science/ article/pii/S0160791X98000050 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 196

199 Endnotes Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 197

200 Appendix 1 – Global Challenges Bibliography Appendix 1 – Global Challenges Bibliography The Global Catastrophic Risk Insti- Further, the term is used by re- More publications were identified by searching scholarly databases searchers from a variety of different tute (GCRI) published a GCR bibliog- - (mainly Web of Science and Google raphy compiled in July 2011 by Seth backgrounds. This makes it a par Baum, available at http://gcrinstitute. Scholar) and databases of popular ticularly fruitful term for discovering new global challenges research. org/bibliography. This contains 115 literature (mainly Amazon and the New York Public Library) for relevant entries, emphasising publications keywords and for citations of the One hallmark of the global chal- surveying the breadth of the risks or discussing other topics of general lenges topic is that it is studied by publications already identified. distinct research communities that interest to the study of GCR, with have limited interaction with each less emphasis on analysis of spe- Several keywords and phrases cific global challenges. It has been other. As research communities often were searched for in the databases: “existential catastrophe”; “exis- develop their own terminology, it can updated for this Global Challenges - Foundation report and now contains be difficult to discover one commu tential risk”; “global catastrophe”; nity by searching for another’s terms. 178 entries. “global catastrophic risk”; “greatest global challenges”; “human extinc- For example, “existential risk” is tion”; “xrisk”; and “infinite risk”. The used heavily by researchers studying The reason for focusing on general risk from artificial intelligence and interest publications is because the results of these searches were then literature on specific global challeng screened for relevant publications. other emerging technologies, but it is - rarely used by researchers studying es is far too voluminous to cata- Many of the results were not rele- environmental risks. Discovering and vant, because these terms are used logue. It would include, for example, connecting the disparate corners in other ways. For example, “existen- a significant portion of the literatures tial risk” is sometimes used to refer on climate change, energy, nuclear of the GCR research is an ongoing to risks to the existence of business- challenge for the GCR community. weapons, infectious diseases and es, countries or other entities; “hu- biodiversity, all topics that receive Bibliography searches such as these man extinction” is used in the study extensive research attention. Thus are an important way to meet this of memory. The publications that challenge. the full bibliography compiled by use these terms in the same sense GCRI is only a small portion of the as the bibliography were then further total global challenges literature. screened for publications of general Publications for the full bibliography global challenges interest, not for specific global challenges. were identified in several ways. The bibliography began with publications The most productive search term for already known to fit the selection the database searches turned out to criteria. Additional publications were identified by examining the reference be “global catastrophe”. This term lists of the initial publications. produced a relatively large number of hits and relatively few publications on unrelated topics. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 198

201 Appendix 1 – Global Challenges Bibliography – Andersen, Ross, 2012. We’re under – Bloom, Barry, 2003. Bioterrorism and - – Baum, Seth D., 2009. Global Cat- astrophic Risks (book review). Risk estimating the risk of human extinc- the university: The threats to security – tion. The Atlantic, March 6. http:// Analysis, vol. 29, no. 1 (January), and to openness. Harvard Magazine, pages 155-156. http://sethbaum.com/ www.theatlantic.com/technology/ November-December, pages 48-52. archive/2012/03/were-underesti- ac/2009_Rev-GCR.pdf mating-the-risk-of-human-extinc- – Bommier, Antoine and Stéphane Zuber, 2008. Can preferences for – Baum, Seth D., 2009. Cost-ben- tion/253821 efit analysis of space exploration: catastrophe avoidance reconcile social – Andersen, Ross, 2013. Omens. discounting with intergenerational eq- Some ethical considerations. Space Aeon, February 25. http://aeon.co/ uity?. Social Choice and Welfare, vol. Policy, vol. 25, no. 2 (May), pages 31, no. 3 (October), pages 415-434. magazine/world-views/ross-anders- 75-80. http://sethbaum.com/ac/2009_ CBA-SpaceExploration.html en-human-extinction – Bostrom, Nick 1999. The doomsday – Baum, Seth D., 2010. Is humanity argument is alive and kicking. Mind, – Asimov, Isaac, 1981. A Choice of Ca- tastrophes: The Disasters That Threat- vol. 108, no. 431, pages 539-550. doomed? Insights from astrobiology. Sustainability, vol. 2, no. 2 (February), en Our World. New York: Ballantine http://www.anthropic-principle.com/ Books. preprints/ali/alive.html pages 591-603. – Bostrom, Nick, 2001. The doomsday – Barrett, Scott, 2006. The problem of – Baum, Seth D., 2013. Teaching argument, Adam & Eve, UN++, and averting global catastrophe. Chicago astrobiology in a sustainability course. Quantum Joe. Synthese, vol. 127, Journal of International Law, vol. 6, no. Journal of Sustainability Education, February (online): http://www.jsed- 2, pages 527-552. no. 3, pages 359-387. http://www. anthropic-principle.com/preprints/cau/ imensions.org/wordpress/content/ - paradoxes.html teaching-astrobiology-in-a-sustainabil- – Barrett, Scott, 2007. Why Cooper ate? The Incentive to Supply Global ity-course_2013_02. - Public Goods. Oxford: Oxford Univer – Bostrom, Nick, 2002. Existential – Baum, Seth D., Timothy M. Maher sity Press. risks: Analyzing human extinction scenarios and related hazards. Journal Jr., and Jacob Haqq-Misra, 2013. Double catastrophe: Intermittent – Barrett, Anthony M., Seth D. Baum, of Evolution and Technology, 9. http:// www.jetpress.org/volume9/risks.pdf and Kelly R. Hostetler, 2013. Analyzing stratospheric geoengineering induced by societal collapse. Environment, and reducing the risks of inadvertent Systems, and Decisions, vol. 33, no. 1 – Bostrom, Nick, 2003. Astronomical nuclear war between the United States waste: The opportunity cost of delayed (March), pages 168-180. and Russia. Science and Global Secu- technological development. Utilitas, rity, vol. 21, no. 2, pages 106-133. Vol. 15, No. 3, pp. 308-314. http:// – Baum, Seth D. and Grant S. Wilson, 2013. The ethics of global catastrophic www.nickbostrom.com/astronomical/ – Baudiment, Fabienne Goux, 2009. waste.pdf Tomorrow will die. Futures, vol. 41, no. risk from dual-use bioengineering. Eth- 10 (December), pages 746-753. ics in Biology, Engineering and Medi- – Bostrom, Nick, 2006. Dinosaurs, cine, vol. 4, no. 1, pages 59-72. dodos, humans?. Global Agenda, – Baum, Seth, 2008. Reducing cat- January, pages 230-231. http://www. astrophic risk through integrative – Beckstead, Nicholas, 2013. On The assessment. Hawaii Reporter, August Overwhelming Importance Of Shaping nickbostrom.com/papers/globalagen- - The Far Future. Doctoral Dissertation, 25; Daily Camera (Boulder, CO), Sep da.pdf Department of Philosophy, Rutgers tember 2 http://www.amergeog.org/ – Bostrom, Nick, 2011. Existential risk University. newsrelease/baum-hawaii08.pdf FAQ. http://www.existentialrisk.com/ – Benatar, David, 2006. Better Never to – Baum, Seth D., 2008. Better to exist: faq.html Have Been: The Harm of Coming Into A reply to Benatar. Journal of Medical Ethics, vol. 34, no. 12 (December), Existence. Oxford: Clarendon Press. – Bostrom, Nick (2013) Existential risk pages 875-876. http://sethbaum.com/ prevention as a global priority. Global Policy 4(1) 15-31. ac/2008_BetterToExist.pdf Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 199

202 Appendix 1 – Global Challenges Bibliography – Bostrom, Nick and Milan M. Ćirković, – Ćirković, Milan, Anders Sandberg, – Charles, Daniel, 2006. A ‘forever’ and Nick Bostrom, 2010. Anthropic 2003. The doomsday argument and seed bank takes root in the Arctic. the self-indication assumption: Reply shadow: Observation selection effects Science, vol. 312, no. 5781 (23 June), pages 1730-1731. to Olum. Philosophical Quarterly, vol. and human extinction risks. Risk Anal- 53, no. 210 (January), pages 83-91. ysis, vol. 30, no. 10 (October), pages – Chichilnisky, Graciela, 2009. Avoid- http://www.anthropic-principle.com/ 1495-1506. http://www.nickbostrom. com/papers/anthropicshadow.pdf preprints/olum/sia.pdf ing extinction: Equal treatment of the present and the future. Economics, – Cloninger, C. Robert, 2013. What – Bostrom, Nick and Milan Ćirković, vol. 3, article 2009-32. http://www. economics-ejournal.org/economics/ Makes People Healthy, Happy, and 2008. Global Catastrophic Risks. journalarticles/2009-32 Oxford: Oxford University Press. http:// Fulfilled In The Face Of Current World www.global-catastrophic-risks.com/ Challenges? Mens Sana Monographs book.html 11(1): 16–24. http://www.ncbi.nlm.nih. – Chichilnisky, Graciela, 2009. Avoiding gov/pmc/articles/PMC3653221 extinction: The future of economics. International Journal of Green Eco- – Bostrom, Nick and Milan Ćirković, nomics, vol. 3, no. 1, pages 1-18. – Coates, Joseph F. et al., 1979. Risks 2008. Introduction. In Nick Bostrom and Milan Ćirković (eds), Global Cata - to Humankind. Washington, DC: U.S. strophic Risks. Oxford: Oxford Uni- - Congress, Office of Technology As – Chichilnisky, Graciela, 2009. Cata- versity Press http://www.global-cata- sessment, March. strophic risks. International Journal of strophic-risks.com/docs/Chap01.pdf Green Economics, vol. 3, no. 2, pages – Coates, Joseph F., 2009. Risks and 130-141. – Buchholz, Wolfgang, Michael Schy- threats to civilization, humankind, and the earth. Futures, vol. 41, no. 10 (De- mura, 2012. Expected utility theory – Chichilnisky, Graciela and Peter Ei- cember), pages 694-705. senberger, 2010. Asteroids: Assessing and the tyranny of catastrophic risks. catastrophic risks. Journal of Proba- Ecological Economics, vol. 77, pages bility and Statistics, vol. 2010, article 234–239. – Colebrook, Claire, 2013. Framing the 954750. http://www.hindawi.com/jour - End of the Species: Images without Bodies. symploke, Volume 21, Num- – Butzer, Karl W., Georgina H. Endfield, nals/jps/2010/954750 editors, 2012. Critical Perspectives bers 1-2, pages 51-63. - on Historical Collapse. Proceedings – Ćirković, Milan M., 2002. Cosmo of the National Academy of Scienc- – Cooke, Roger M. and Carolyn logical Forecast and its Practical Kousky, 2009. Climate change and risk Significance. Journal of Evolution and es (Special Feature), vol. 109, pages - 3628-3681. management: challenges for insur Technology, xii. http://www.jetpress. ance, adaptation, and loss estimation. org/volume12/CosmologicalForecast. Resources for the Future Discussion – Cairns, John Jr., 2010. Global Crisis pdf Paper 09-03. Collaboration: The Key to the Surviv- - al of Civilization in the 21st Century. – Ćirković, Milan M., 2007. Evolu tionary catastrophes and the Goldi- – Coughlan, Sean, 2013. How are hu- Asian Journal of Experimental Scienc- es, Vol. 24, No. 1, 33-38. locks problem. International Journal mans going to become extinct? BBC News, April 24. http://www.bbc.co.uk/ of Astrobiology, vol. 6, no. 4, pages - news/business-22002530 – Carpenter, P. A. and P. C. Bishop, 325-329. http://arxiv.org/ftp/arxiv/pa 2009. A review of previous mass pers/0709/0709.2309.pdf – Curry, Nathan, 2013. Some Cred- extinctions and historic catastrophic events. Futures, vol. 41, no. 10 (De- ible Scientists Believe Humanity is - –Ćirković, Milan M., 2012. Small Theo ries and Large Risks—Is Risk Analysis Irreparably Close to Destruction. Vice cember), pages 676-682. Canada. http://www.vice.com/en_ca/ Relevant for Epistemology? Risk Anal- ysis, Vol. 32, No. 11, p. 1994-2004. read/near-term-extinctionists-believe- – Carpenter, P. A. and P. C. Bishop, 2009. The seventh mass extinction: the-world-is-going-to-end-very-soon Human-caused events contribute to a fatal consequence. Futures, vol. 41, no. 10 (December), pages 715-722. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 200

203 Appendix 1 – Global Challenges Bibliography – de Neufville, Robert, 2013. Can the – Haller, Stephen F., 2002. Apoca- – Eckhardt, William, 1993. Probability lypse Soon? Wagering on Warnings of human race survive? Anthropocene, theory and the doomsday argument. January 31. http://anthropoceneblog. Global Catastrophe. Montreal: McGill- Mind, vol. 102, no. 407 (July), pages Queens University Press. 483-488. http://williameckhardt.com. wordpress.com/2013/01/31/can-the- human-race-surviv p2.hostingprod.com/yahoo_site_ad- – Hanson, Robin, 1998. Critiquing the min/assets/docs/William_Eckhardt_-_ Probability_Theory_and_the_Dooms- doomsday argument. Unpublished – Dieks, Dennis, 1992. Doomsday - Or: day_Argument.131122037.pdf manuscript, 27 August. http://hanson. The dangers of statistics. Philosophical gmu.edu/nodoom.html Quarterly, vol. 42, no. 166 (January), pages 78-84. – Epstein, Richard J. and Y. Zhao, 2009. The threat that dare not speak – Hayne, David M. (ed), 1999. Human its name: Human extinction. Perspec- Survivability in the 21st Century. Toron- – Draper, John William, 2013. Ex- tinction and Risk. U of Penn Law tives in Biology and Medicine, vol. 52, to: University of Toronto Press. Hempsell, C. M., 2004. The investi- School, Public Law Research Paper no. 1 (Winter), pages 116-125. No. 12-24; U of Penn, Inst for Law gation of natural global catastrophes. & Econ Research Paper No. 12-14. Journal of the British Interplanetary – Faber, M.H., 2010. Critical Issues in Available at SSRN: http://ssrn.com/ Society, vol. 57, pages 2-13. the Management of Catastrophic and abstract=2030241 Global Risks, ISRERM 2010. http:// – Hempsell, C. M., 2004. The poten- www.adaptationlearning.net/system/ – Dvorsky, George, 2012. Does a tial for space intervention in global files/GlobalCatastrophicRisks.pdf catastrophes. Journal of the British galaxy filled with habitable planets Interplanetary Society, vol. 57, pages – Faber, M.H., 2011. On the govern- mean humanity is doomed? io9, June 18. http://io9.com/5919110/does-a- 14-21. ance of global and catastrophic risks. galaxy-filled-with-habitable-planets- International Journal of Risk Assess- – Herz, John H., 2003. On human sur mean-humanity-is-doomed ment and Management, vol. 15, no. - vival: Reflections on survival research 5-6, pages 400-416. and survival policies. World Futures, – Dvorsky, George, 2012. 9 ways vol. 59, no. 3-4 (April-June), pages – Franck, Thomas M., 2006. Collective humanity could bring about its own destruction. io9, December 12. http:// security and UN reform: Between the 135-143. io9.com/5967660/9-ways-humanity- necessary and the possible. Chicago could-bring-about-our-own-destruc- – Horton, Richard, 2005. Threats to Journal of International Law, vol. 6, no. human survival: A WIRE to warn the 2, pages 597-612. tion world. Lancet, vol. 365 (15 January), 191-193. – Dvorsky, George, 2013. Can one – German Advisory Council on Global Change, 1998. World in Transition: single person destroy the entire – Iklé, Fred Charles, 2006. Annihilation world? io9, http://io9.com/could-a-sin- Strategies for Managing Global Envi- gle-individual-really-destroy-the-wor from Within: The Ultimate Threat to ronmental Risks. Berlin: Springer Nations. New York: Columbia Univer - ld-1471212186/1471327744 sity Press. – Gott, J. Richard III, 1993. Implica- – Economist, The, 1998. Human ex- tions of the Copernican Principle for our future prospects. Nature, vol. 363, tinction: Sui genocide. The Economist, – Jablonowski, Mark, 2012. Surviving progress: Managing the collective risks pages 315-319. http://www-psych. vol. 349, no. 8099 (December 19), pages 130-131. http://www.econo- stanford.edu/~jbt/224/Gott_93.pdf of civilization. real-world economics review, 62, p. 121-131. http://www. – Grausam, Daniel, 2011. On Endings: mist.com/node/179963 paecon.net/PAEReview/issue62/Jablo- American Postmodern Fiction and the nowski62.pdf Cold War. Charlottesville: University of Virginia Press. – Jha, Alok, 2011. The Doomsday – Guterl, Fred, 2012. The Fate of the Handbook - 50 Ways to the End of the Species: Why the Human Race May World. London: Quercus. Cause Its Own Extinction and How We Can Stop It. New York: Bloomsbury USA. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 201

204 Appendix 1 – Global Challenges Bibliography – Jones, Andrew R., 2009. The next – Leslie, John, 1997. Observer-relative – Koopmans, Tjalling C., 1974. Proof mass extinction: Human evolution or for a case where discounting advanc- chances and the doomsday argument. Inquiry, vol. 40, no. 4, pages 427-436. es the doomsday. Review of Economic human eradication. Journal of Cos- Studies, vol. 41, pages 117-120. mology, vol. 2, pages 316-333. http:// – Leslie, John, 2010. The Risk that journalofcosmology.com/Extinc- tion108.html – Kuznick, Peter J., 2007. Prophets of Humans Will Soon Be Extinct. Philoso- doom or voices of sanity? The evolv- phy, Volume 85, Issue 04, p. 447-463. ing discourse of annihilation in the first – Jones, Charles I., 2011. Life and growth. Working paper, Stanford decade and a half of the nuclear age. – Levin, Josh, 2009. How is America going to end? Slate’s ‘Choose your Journal of Genocide Research, vol. 9, University Graduate School of Busi- own apocalypse’ lets you map out ness. http://www.stanford.edu/~chadj/ no. 3 (September), pages 411–441. the death of the United States. Slate, life200.pdf 7 August. http://www.slate.com/ – Leggett, Mark, 2006. An indicative id/2223285 – Jones, Christopher B., 2009. Gaia costed plan for the mitigation of global bites back: Accelerated warming. risks. Futures, 38, 778-809. Futures, vol. 41, no. 10 (December), Lenman, James 2002. On becoming – Levin, Josh, 2009. How is America going to end? The world’s leading extinct. Pacific Philosophical Quarter - pages 723-730. futurologists have four theories. Slate, ly, vol. 83, no. 3 (September), pages – Joy, Bill, 2000. Why the future 7 August. http://www.slate.com/ 253-269. doesn’t need us. Wired, April, pages id/2223962 – Leslie, John, 1989. Risking the 238-262. http://www.wired.com/wired/ archive/8.04/joy.html world’s end. Canadian Nuclear Society - Levin, Josh, 2009. How is America Bulletin, May, pages 10-15. going to end? The apocalypse you chose. Plus, the end-of-America social - – Kaplan, Ralph and Harvey Silver glate, 2006. An urgent cause for network. Slate, 7 August. http://www. – Leslie, John, 1990. Is the end of the world nigh?. Philosophical Quarterly, philanthropy. Boston Globe, July slate.com/id/2224425 vol. 40, no. 158 (January), pages 65- 23. http://www.boston.com/news/ - Lisboa, Maria Manuel, 2011. The End 72. globe/editorial_opinion/oped/arti- - of the World: Apocalypse and Its After cles/2006/07/23/an_urgent_cause_for_ – Leslie, John, 1990. Risking the math in Western Culture. Cambridge, philanthropy world’s end. Interchange, vol. 21, no. 1 UK: Open Book Publishers (Spring), pages 49-58. – Kaplan, Ralph and Harvey Silver - glate, 2007. The next generation of - Lopes, Tobin, Thomas J. Chermack, – Leslie, John, 1992. Time and the Deb Demers, Madhavi Kari, Berna- threats. The Boston Globe, February 24. http://www.boston.com/news/ anthropic principle. Mind, vol. 101, no. dette Kasshanna, Tiffani Payne, 2009. 403 (July), pages 521-540. globe/editorial_opinion/oped/arti- Human Extinction Scenario Frame- cles/2007/02/24/the_next_genera- works. Futures, vol. 41, no. 10 (De- – Leslie, John, 1992. Doomsday revis- tion_of_threats cember), pages 731-737. ited. Philosophical Quarterly, vol. 42, – Maher, Timothy M., Jr. and Seth no. 166 (January), pages 85-89. – Kent, Adrian, 2003. A critical look at D. Baum, 2013. Adaptation to and risk assessments for global catastro- recovery from global catastrophe. Sus- phes. Risk Analysis, vol. 24, no. 1 – Leslie, John, 1993. Doom and prob- tainability, vol. 5, no. 4 (April), pages abilities. Mind, vol. 102, no. 407 (July) (February), pages 157-168. Mind, vol. 102, no. 407 (July), pages 1461-1479. – Kock, R. A., 2013. Will the damage 489-491. be done before we feel the heat? – Mann, Charles C., 2012. State of the species. Orion Magazine, November/ – Leslie, John, 1996. The End of the Infectious disease emergence and World: The Science and Ethics of Hu- human response. Animal Health Re- December, pages 16-27. http://www. search Reviews, Volume 14, Issue 2, man Extinction. London: Routledge. orionmagazine.org/index.php/articles/ article/7146 pp 127-132 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 202

205 Appendix 1 – Global Challenges Bibliography – Martin, James, 2007. The Meaning of – Posner, Richard A., 2006. Efficient – Nordhaus, William D., 2009. An anal- ysis of the dismal theorem. Cowles the 21st Century. New York: Riverhead responses to catastrophic risk. Chica- Foundation Discussion Paper, No. Penguin. go Journal of International Law, vol. 6, no. 2, pages 511-526. 1686. http://cowles.econ.yale.edu/P/ – Mason, Colin, 2003. The 2030 Spike: cd/d16b/d1686.pdf – Powell, Corey S., 2000. 20 ways the Countdown to Global Catastrophe. London: Earthscan. – Norgaard, Richard B. and Paul Baer, world could end swept away. Discover, 2003. Seeing the whole picture. World vol. 21, no. 10 (October). Futures, vol. 59, no. 3-4 (April-June), – Matheny, Jason G., 2007. Reducing – Randers, Jorgen, 2008. Global pages 225-239. the risk of human extinction. Risk Anal- collapse — Fact or fiction?. Futures, ysis vol. 27, no. 5, pages 1335-1344. – Ord, Toby, Rafaela Hillerbrand, and vol. 40, no. 10 (December), pages http://jgmatheny.org/matheny_extinc- Anders Sandberg, 2010. Probing the 853-864. tion_risk.htm improbable: Methodological challeng- – Matheny, Jason G., undated. Ought es for risks with low probabilities and – Rees, Martin, 2003. Our Final Cen- tury: Will the Human Race Survive the we worry about human extinction?. high stakes. Journal of Risk Research, vol. 13, no. 2 (March), pages 191-205. Unpublished working paper, http://jg- Twenty-first Century? Oxford: William http://www.fhi.ox.ac.uk/__data/assets/ matheny.org/extinctionethics.htm Heinemann. - pdf_file/0006/4020/probing-the-im – Rees, Martin, 2013. Denial of Cat- – Matson, John and John Pavlus, probable.pdf 2010. Laying odds on the apocalypse. astrophic Risks. Science, vol. 339 (8 – Parson, Edward A, 2007. The big March), page 1123 Scientific American, vol. 303, no. 3 one: A review of Richard Posner’s Ca- (September), pages 82-83. – Rees, Martin, 2013. We are in denial tastrophe: Risk and Response. Journal of Economic Literature, XLV (March), - – McGuire, Bill, 2005. Global Ca about catastrophic risks. Edge. http:// tastrophes: A Very Short Introduction. edge.org/response-detail/23864 pp. 147-164 Oxford: Oxford University Press. – Pindyck, Robert S., 2011. Fat Tails, – Rifkin, Lawrence, 2013. The survival Thin Tails, and Climate Change Policy. of humanity. Scientific American, Sep – Morgan, Dennis Ray, 2009. World on - - Review of Environmental Economics fire: two scenarios of the destruction tember 13. http://blogs.scientificamer and Policy, 5 (2) 258-274. ican.com/guest-blog/2013/09/13/ of human civilization and possible the-survival-of-humanity extinction of the human race. Futures, – Pindyck, Robert S. and Neng Wang, vol. 41, no. 10 (December), pages 2011. The economic and policy con- – Roberts, Patrick S., 2008. Catastro- 683-693. sequences of catastrophes. NBER phe: Risk and Response, by Richard A. Posner. Homeland Security Affairs, Working Papers, no. 15373. http:// – Ng, Yew-Kwang, 1991. Should we be very cautious or extremely cautious vol. 4, no. 1 (January). www.nber.org/papers/w15373. http:// on measures that may involve our de- web.mit.edu/rpindyck/www/Papers/ struction?. Social Choice and Welfare, EconDisastersJanuary2011.pdf vol. 8, pages 79-88. – Posner, Richard, 2004. Catastrophe: – Ng, Yew-Kwang, 2011. Consumption Risk and Response. Oxford: Oxford tradeoff vs. catastrophes avoidance: University Press. Implications of some recent results in – Posner, Richard A., 2005. Cata happiness studies on the economics - strophic risks, resource allocation, and of climate change. Climatic change, homeland security. Journal of Home- Volume 105, Issue 1-2, pp 109-127. land Security, October. http://www. homelandsecurity.org/journal/Default. aspx?oid=133&ocat=1 Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 203

206 Appendix 1 – Global Challenges Bibliography – Rockström, Johan, Will Steffen, – Shapovalov, Viktor I., Nickolay V. – Sandberg, Anders and Nick Bo- Kazakov, 2013. The fundamental rea- Kevin Noone, Åsa Persson, F. Stuart III strom, 2008. Global catastrophic Chapin, Eric Lambin, Timothy M. Len- sons for global catastrophes. Natural risks survey. Technical Report 2008/1, Future of Humanity Institute, Ox- ton, Marten Scheffer, Carl Folke, Hans Science Vol.5, No.6, 673-677. ford University. http://www.fhi.ox.ac. Joachim Schellnhuber, Björn Nykvist, Cynthia A. de Wit, Terry Hughes, – Shapiro, Robert, 2009. A new uk/Reports/2008-1.pdf rationale for returning to the Moon? Sander van der Leeuw, Henning Rod- – Sandberg, Anders, Jason G. Mathe- Protecting civilization with a sanctuary. he, Sverker Sörlin, Peter K. Snyder, Robert Costanza, Uno Svedin, Malin ny, and Milan M. Ćirković, 2008. How Space Policy, vol. 25, no. 1 (February), Falkenmark, Louise Karlberg, Robert pages 1-5. can we reduce the risk of human extinction?. Bulletin of the Atomic W. Corell, Victoria J. Fabry, James - – Shuckburgh, Emily, 2008, editor. Sur Scientists, September. http://www. Hansen, Brian Walker, Diana Liverman, vival: The Survival of the Human Race. thebulletin.org/web-edition/features/ Katherine Richardson, Paul Crutzen, Cambridge, UK: Cambridge University how-can-we-reduce-the-risk-of-hu- and Jonathan Foley, 2009. Planetary - man-extinction boundaries: Exploring the safe oper Press. ating space for humanity. Ecology and Society, vol. 14, no. 2, article 32, http:// – Singer, Peter, 2005. ‘Catastrophe’: – Scheffler, Samuel, 2013. The im - www.ecologyandsociety.org/vol14/ portance of the afterlife. Seriously. Apocalypse when? (book review The New York Times, September iss2/art32 of R. Posner). New York Times, 2 January 2005 http://www.nytimes. 21. http://opinionator.blogs.ny- - com/2005/01/02/books/review/02SIN times.com/2013/09/21/the-impor - – Rockström, Johan, Will Steffen, GERL.html tance-of-the-afterlife-seriously Kevin Noone, Åsa Persson, F. Stuart III Chapin, Eric Lambin, Timothy M. – Schild, Rudy, 2009. Asteroids, Lenton, Marten Scheffer, Carl Folke, – Smil, Vaclav, 2005. The next 50 years: Fatal discontinuities. Population Meteors, Comets, Climate and Mass Hans Joachim Schellnhuber, Björn & Development Review, 31:201-236 Nykvist, Cynthia A. de Wit, Terry Extinctions. Journal of Cosmology Hughes, Sander van der Leeuw, Hen- http://home.cc.umanitoba.ca/~vsmil/ - (special issue), November. http://jour ning Rodhe, Sverker Sörlin, Peter K. publications_pdf.html nalofcosmology.com/Contents2.html Snyder, Robert Costanza, Uno Svedin, – Smil, Vaclav, 2008. Global Catastro- – Seidel, Peter, 2003. Introduction. Malin Falkenmark, Louise Karlberg, World Futures, vol. 59, no. 3-4 (April- Robert W. Corell, Victoria J. Fabry, phes and Trends: The Next Fifty Years. James Hansen, Brian Walker, Diana Cambridge, MA: MIT Press. June), pages 127-128. Liverman, Katherine Richardson, Paul – Seidel, Peter, 2003. ‘Survival re- – Sunstein, Cass R., 2005-2006. Crutzen, and Jonathan Foley, 2009. A safe operating space for humanity. Na- Irreversible and catastrophic: Glob- search’: A new discipline needed now. al warming, terrorism, and other World Futures, vol. 59, no. 3-4 (April- ture, vol. 461 (24 September), pages 472-475. June), pages 129-133. problems. Pace Environmental Law Review, vol. 23, no. 1 (Winter), pages – Russell, Bertrand, Albert Einstein, – Seidel, Peter, 2009. Is it inevitable 3-19. Max Born, Percy W. Bridgman, that evolution self-destruct?. Futures, vol. 41, no. 10 (December), pages Leopold Infeld, Frederic Joliot Cu- – Sunstein, Cass R., 2006. Irreversible 754-759. rie, Herman J. Muller, Linus Pauling, and catastrophic. Cornell Law Review, Cecil F. Powell, Joseph Rotblat, Hideki vol. 91, no. 4, pages 841-897. – Sevilla, J. P., 2012. On the Underlying Yukawa, 1955. The Russell-Einstein – Sunstein, Cass R, 2007. Worst-Case Ontology of the View that Extinction manifesto. July 9. http://www.pug- is Bad. Available at SSRN: http://ssrn. Scenarios. Cambridge, MA: Harvard wash.org/about/manifesto.htm University Press. com/abstract=1755298 or http://dx. doi.org/10.2139/ssrn.1755298 – Tännsjö, Torbjörn, 1997. Doom soon?. Inquiry, vol. 40, no. 2, pages 243-252. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 204

207 Appendix 1 – Global Challenges Bibliography – Taylor, Kenneth B., 2013. Demise – Weitzman, Martin L., 2009. Structural – Tonn, Bruce and Donald MacGregor, 2009. Are we doomed?. Futures, vol. of the Modern Human. World Future uncertainty and the value of statistical 41, no. 10 (December), pages 673- Review, vol. 5, no. 2, p. 85-93. life in the economics of catastrophic 675. climate change. Review of Economics and Statistics, vol. 91, no. 1, pages – Taylor, Matthew A., 2012. Whither – Tonn, Bruce and Donald MacGre- life? Subjectivity (2012) 5, 276–289. 1-19. gor, 2009. A singular chain of events. – Tegmark, Max and Nick Bostrom, – Wiener, Jonathan B., 2005. Review Futures, vol. 41, no. 10 (December), 2005. Is a doomsday catastrophe pages 706-714. of: Catastrophe: Risk and Response and Collapse: How Societies Choose likely?. Nature, vol. 438 (8 December), – Tonn, Bruce and Jenna Tonn, 2009. to Fail or Succeed. Journal of Policy page 754. Analysis and Management, vol. 24, no. A literary human extinction scenario. 4, pages 885-890. – Thacker, Eugene, 2012. Notes on Ex- Futures, vol. 41, no. 10 (December), pages 760-765. tinction and Existence. Configurations, – Wilson, E.O., 1993. Is humanity sui- Volume 20, Number 1-2, 137-148 – Tonn, Bruce, Dorian Stiefel, 2013. cidal? The New York Times Magazine, Evaluating Methods for Estimating – Tonn, Bruce E., 1999. Transcending May 30, pages 24-28. Existential Risks. Risk Analysis, Vol. oblivion. Futures, vol. 31, pages 351- 359. – Wilson, Grant S., 2013. Minimizing 33, No. 10, pages 1772-1787. global catastrophic and existential risks from emerging technologies – Trisel, Brooke Allen, 2004. Human – Tonn, Bruce E., 2000. Ensuring the extinction and the value of our efforts. future. Futures, vol. 32, pages 17-26. through international law. Virginia Envi- The Philosophical Forum, vol. 35, no. 3 ronmental Law Journal, vol. 31, no. 2. – Tonn, Bruce E., 2002. Distant futures (September), pages 371-391. and the environment. Futures, vol. 34, – Wuthnow, Robert, 2010. Be Very pages 117-132. – Tudge, C., 1989. The rise and fall of Afraid: The Cultural Response to Homo sapiens sapiens. Philosophical Terror, Pandemics, Environmental – Tonn, Bruce E., 2004. Research so- Transactions of the Royal Society of Devastation, Nuclear Annihilation, and ciety: science and technology for the - London, Series B, vol. 325, no. 1228 (6 Other Threats. Oxford: Oxford Univer ages. Futures, 36, 335–346 November), pages 479-488. sity Press. – Tonn, Bruce E., 2007. Futures sus- – Union of Concerned Scientists, tainability. Futures 39, pages 1097- 1993. World Scientists’ Warning To 1116. Humanity. Cambridge, MA: Union of Concerned Scientists. – Tonn, Bruce E., 2009. Obligations to future generations and acceptable risks of human extinction. Futures, vol. – Vikulin, A. V., N. V. Semenets, M. A. Vikulina, 2013. Global disasters: 41, no. 7, pages 427-435. Geodynamics and society. Izvestiya, – Tonn, Bruce E., 2009. Beliefs about Atmospheric and Oceanic Physics, human extinction. Futures, vol. 41, no. Volume 49, Issue 7, pp 691-714 10 (December), pages 766-773. – Tonn, Bruce, 2009. Preventing the – Waldman, Ronald, 2006. Respond- next mass extinction: Ethical obliga- ing to catastrophes: A public health perspective. Chicago Journal of tions. Journal of Cosmology, vol. 2, International Law, vol. 6, no. 2, pages pages 334-343. 553-572. – Tonn, Bruce, Don MacGregor, edi- tors, 2009. Human Extinction. Futures - Wells, Willard, 2009. Apocalypse (special issue), Volume 41, Issue 10, When? Calculating How Long the Pages 673-774. Human Race Will Survive. Chichester, UK: Springer-Praxis. Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 205

208 Appendix 2 – Workshops Appendix 2 – Workshops Workshop 1 14 January 2014, at the Future of Humanity Institute (FHI), University of Oxford: Participants Jules Peck, Founding Partner, Jericho Stuart Armstrong, James Martin Re- Patrick McSharry, head of Smith Chambers; Trustee, New Economics School’s Catastrophe Risk Financing search Fellow, FHI, Oxford Foundation research area. Seth Baum, Executive Director of the Vincent Müller, James Martin Research Anders Sandberg, James Martin Re- Global Catastrophic Risk Institute Fellow, FHI, Oxford Nick Beckstead, Research Fellow, search Fellow, FHI, Oxford Robert de Neufville, Professional As- Andrew Simms, Author, Fellow at the FHI, Oxford New Economics Foundation and Chief Eric Drexler, Academic Visitor, James sociate, Global Catastrophic Analyst at Global Witness Risk Institute Martin Research Fellow, FHI, Oxford Toby Ord, James Martin Research Andrew Snyder-Beattie, Academic Madeleine Enarsson, Transformative Fellow, FHI, Oxford Catalyst, 21st Century Frontiers Project Manager, FHI, Oxford James Taplan, Principal Sustainability Seán Ó hÉigeartaigh, James Martin Aca- Dennis Pamlin, Executive Project Manager, Global Challenges demic Project Manager, FHI, Oxford and Advisor, Forum for the Future project manager, Cambridge’s Centre for Foundation Raj Thamotheram, CEO, Preventable Surprises Study of Existential Risk, Cambridge 14.30-15.30 Agenda 10.40-11.40 Emerging challenges Major events during 2013 and links to 2014 Probabilities and impacts, discussions Discussing the draft list and structure. 09.30 Welcome regarding the challenges. Who can What is missing, especially with regards to emerging countries? What could happen 09.35-09.50: Round of introductions provide an estimation of the risk and who can continue to work on these? Are there during 2014 that we think should be synergies between them and between reflected in the report? 09.50-10.10 them and today’s major challenges? Introducing the Global Challenges Foun- dation and the work with the “Global 11.40-12.30 Challenges Report” 15.30-16.00 Infinite impacts How can infinite impacts be defined and What is the background for this work, Natural challenges and policy challenges presented in ways that make policy mak- How to deal with these challenges that are draft structure of the report and agenda for the day? ers take them seriously? What models different, especially policy challenges? and narratives exist, what coalitions can work with such questions? 16.00-16.30 10.10-10.40 Global Challenges in society today and Next steps and need for funding 12.30-13.30 Lunch Based on the discussion during the day, global risks what are possible next steps? Are there Who cares about and works with the major global challenges that pose an 13.30-14.30 initiatives/work in need of funding? What role could a quarterly Global Challenges/ Major challenges existential threat and who wants to cancel the apocalypse? In short: In what context Probabilities and impacts, discussions risk report have? will the report be received? regarding the challenges. Who can provide an estimation of the risk and who can continue to work on these? Are there synergies between them? Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 206

209 Appendix 2 – Workshops Workshop 2 15 January 2014 at the Munich RE office in London: Participants Nick Silver, director of Callund Con- Oliver Bettis, pricing actuary Mu- sulting and founder and director of the nich RE and fellow of the Chartered Insurance Institute and the Institute & Climate Bonds Initiative (CBI) Faculty of Actuaries. Liang Yin, Investment Consultant at Towers Watson Madeleine Enarsson, Transformative Catalyst, 21st Century Frontiers Jennifer Morgan, Founder & Co-Con- vener, the finance lab Dennis Pamlin, Executive Project Man- ager, Global Challenges Foundation 12.45-13.00 Agenda How can infinite impacts be included in mainstream actuary work? How can 11.00-11.05 thought leaders in the actuarial profes- Welcome sion begin to include infinite impacts? 11.05-11.15 13.00-13.30 Round of introductions What should the report include to be relevant for actuaries/financial sector? 11.15-11.30 The Global Challenges Foundation and 13.30-14.00 a risk report Possible ways forward 11.30-12.00 Infinite impacts and global challenges. 12.00-12.15 What risks with a possible infinite impact are on the radar screen of actuaries today (comparing the list from Oxford with current actuary work) and how can “Infinite impacts” be addressed? 12.15-12.45 Relations and overview Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 207

210 Notes Notes Global Challenges – Twelve risks that threaten human civilisation – The case for a new category of risks 208

211 design by wearebwa.co.uk

212 Published February 2015 by Global Challenges Foundation globalchallenges.org Global Challenges Foundation Stureplan 4C 114 35 Stockholm For comments and/or queries on this report, please contact the co-authors: Dennis Pamlin Dr Stuart Armstrong Executive Project Manager James Martin Research Fellow Global Risks Future of Humanity Institute Global Challenges Foundation Oxford Martin School & Faculty of Philosophy [email protected] University of Oxford globalchallenges.org [email protected] fhi.ox.ac.uk

Related documents

RIE Tenant List By Docket Number

RIE Tenant List By Docket Number

SCRIE TENANTS LIST ~ By Docket Number ~ Borough of Bronx SCRIE in the last year; it includes tenants that have a lease expiration date equal or who have received • This report displays information on ...

More info »
CityNT2019TentRoll 1

CityNT2019TentRoll 1

STATE OF NEW YORK 2 0 1 9 T E N T A T I V E A S S E S S M E N T R O L L PAGE 1 VALUATION DATE-JUL 01, 2018 COUNTY - Niagara T A X A B L E SECTION OF THE ROLL - 1 CITY - North Tonawanda TAX MAP NUMBER ...

More info »
CRPT 116hrpt9 u2

CRPT 116hrpt9 u2

U:\2019CONF\HJRes31Front.xml APPRO. SEN. [COMMITTEE PRINT] REPORT { } CONGRESS 116TH 1st HOUSE OF REPRESENTATIVES Session 116- FURTHER APPROPRIATIONS FOR MAKING CONTINUING OF HOMELAND SECURITY FOR THE...

More info »
Fourth National Report on Human Exposure to Environmental Chemicals Update

Fourth National Report on Human Exposure to Environmental Chemicals Update

201 8 Fourth National Report on Human Exposure to Environmental Chemicals U pdated Tables, March 2018 , Volume One

More info »
CalCOFI Atlas 33

CalCOFI Atlas 33

THE EARLY STAGES IN OF THE FISHES CALIFORNIA CURRENT REGION CALIFORNIA FISHERIES COOPERATIVE OCEANIC INVESTIGATIONS ATLAS NO. 33 BY THE SPONSORED STATES OF COMMERCE DEPARTMENT UNITED OCEANIC AND ATMOS...

More info »
MPI: A Message Passing Interface Standard

MPI: A Message Passing Interface Standard

MPI : A Message-Passing Interface Standard Version 3.0 Message Passing Interface Forum September 21, 2012

More info »
JO 7400.11C   Airspace Designations and Reporting Points

JO 7400.11C Airspace Designations and Reporting Points

U.S. DEPARTMENT OF TRANSPORTATION ORDER FEDERAL AVIATION ADMINISTRATION 7400.11C JO Air Traffic Organization Policy August 13, 2018 SUBJ: Airspace Designations and Reporting Points . This O rder, publ...

More info »
tsa4

tsa4

i i “tsa4_trimmed” — 2017/12/8 — 15:01 — page 1 — #1 i i Springer Texts in Statistics Robert H. Shumway David S. Sto er Time Series Analysis and Its Applications With R Examples Fourth Edition i i i ...

More info »
ayout 1

ayout 1

0465039146-FM:FM 12/5/06 12:25 AM Page i C O D E

More info »
NB18

NB18

Table of Contents National Board Pressure Relief Device Certificati ons NB-18 FOREWARD... 1 NATIONAL BOARD PRESSURE RELIEF DEVICE CERTIFICATION... 2 DETERMINATION OF CERTIFIED RELIEVING CAPACITIES... ...

More info »
OperatorHoursReport

OperatorHoursReport

John Bel Edwards Rebekah E. Gee MD, MPH SECRETARY GOVERNOR State of Louisiana Louisiana Department of Health Office of Public Health Certified Water and Wastewater Operators 2018 - 2019 Hours Hours li...

More info »
The Health Consequences of Smoking   50 Years of Progress: A Report of the Surgeon General

The Health Consequences of Smoking 50 Years of Progress: A Report of the Surgeon General

The Health Consequences of Smoking—50 Years of Progress A Report of the Surgeon General U.S. Department of Health and Human Services

More info »
pisa 2012 results volume I

pisa 2012 results volume I

PISA 2012 Results: What Students Know and Can Do tICS, themA StuDent PeRfoRmAnCe In mA ReADIng AnD SCIenCe Volume I rogramme for ssessment A tudent S nternational I P

More info »
untitled

untitled

G:\P\16\HR1\INTRO.XML ... (Original Signature of Member) TH 116 CONGRESS 1 ST S ESSION H. R. 1 To expand Americans’ access to the ballot box, reduce the influence of big money in politics, and strengt...

More info »
June2018CUR

June2018CUR

CHANCELLOR'S UNIVERSITY REPORT JUNE 25 2018

More info »
Second National Report on Biochemical Indicators of Diet and Nutrition in the U.S. Population

Second National Report on Biochemical Indicators of Diet and Nutrition in the U.S. Population

Second National Report on Biochemical Indicators of Diet and Nutrition in the U.S. Population Second National Report on Biochemical Indicators of Diet and Nutrition in the U.S. Population 2012 Nationa...

More info »
Department of Defense   Law of War Manual (June 2015)

Department of Defense Law of War Manual (June 2015)

D E A R T M E N T O F D E F E N S E P N A L O F W A R M A W U A L J U N E 2 0 1 5 O F F I C E O F G E N ER A L C O U N S E L D P A R T M E N T E O F D E F E N S E

More info »
Why Forests? Why Now? The Science, Economics and Politics of Tropical Forests and Climate Change

Why Forests? Why Now? The Science, Economics and Politics of Tropical Forests and Climate Change

WHY FORESTS? WHY NOW? The Science, Economics and Politics of Tropical Forests and Climate Change Frances Seymour Jonah Busch

More info »
me bpd eng

me bpd eng

2013–14 Estimates Amendment Atlantic Canada Opportunity Agency: On February 22, 2013, the Prime-Minister announ ced changes to the Ministry. Effective immediately, the Minister of National Revenue is ...

More info »