1 Networks, Crowds, and Markets: Reasoning about a Highly Connected World David Easley Jon Kleinberg Dept. of Economics Dept. of Computer Science Cornell University Cornell University Cambridge University Press, 2010 Draft version: June 10, 2010.

2 2

3 Contents Preface i 1 1 Overview 2 1.1 Aspects of Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Central Themes and Topics . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 I Graph Theory and Social Networks 21 2 Graphs 23 23 2.1 Basic Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 2.2 Paths and Connectivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3 Distance and Breadth-First Search . . . . . . . . . . . . . . . . . . . . . . . 32 2.4 Network Datasets: An Overview . . . . . . . . . . . . . . . . . . . . . . . . . 40 2.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 3 Strong and Weak Ties 47 3.1 Triadic Closure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 3.2 The Strength of Weak Ties . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 56 3.3 Tie Strength and Network Structure in Large-Scale Data . . . . . . . . . . . 3.4 Tie Strength, Social Media, and Passive Engagement . . . . . . . . . . . . . 60 3.5 Closure, Structural Holes, and Social Capital . . . . . . . . . . . . . . . . . . 64 3.6 Advanced Material: Betweenness Measures and Graph Partitioning . . . . . 69 3.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 4 Networks in Their Surrounding Contexts 85 4.1 Homophily . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86 4.2 Mechanisms Underlying Homophily: Selection and Social Influence . . . . . . 90 4.3 Affiliation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93 4.4 Tracking Link Formation in On-Line Data . . . . . . . . . . . . . . . . . . . 97 4.5 A Spatial Model of Segregation . . . . . . . . . . . . . . . . . . . . . . . . . 107 4.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116 3

4 4 CONTENTS 119 5 Positive and Negative Relationships 5.1 Structural Balance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120 5.2 Characterizing the Structure of Balanced Networks . . . . . . . . . . . . . . 123 5.3 Applications of Structural Balance . . . . . . . . . . . . . . . . . . . . . . . 126 5.4 A Weaker Form of Structural Balance . . . . . . . . . . . . . . . . . . . . . . 129 5.5 Advanced Material: Generalizing the Definition of Structural Balance . . . . 132 5.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148 II Game Theory 153 6 Games 155 6.1 What is a Game? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 156 6.2 Reasoning about Behavior in a Game . . . . . . . . . . . . . . . . . . . . . . 158 6.3 Best Responses and Dominant Strategies . . . . . . . . . . . . . . . . . . . . 163 6.4 Nash Equilibrium . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166 6.5 Multiple Equilibria: Coordination Games . . . . . . . . . . . . . . . . . . . . 168 6.6 Multiple Equilibria: The Hawk-Dove Game . . . . . . . . . . . . . . . . . . . 172 6.7 Mixed Strategies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173 6.8 Mixed Strategies: Examples and Empirical Analysis . . . . . . . . . . . . . . 179 6.9 Pareto-Optimality and Social Optimality . . . . . . . . . . . . . . . . . . . . 184 6.10 Advanced Material: Dominated Strategies and Dynamic Games . . . . . . . 186 6.11 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200 7 Evolutionary Game Theory 209 7.1 Fitness as a Result of Interaction . . . . . . . . . . . . . . . . . . . . . . . . 210 7.2 Evolutionarily Stable Strategies . . . . . . . . . . . . . . . . . . . . . . . . . 211 7.3 A General Description of Evolutionarily Stable Strategies . . . . . . . . . . . 216 7.4 Relationship Between Evolutionary and Nash Equilibria . . . . . . . . . . . . 218 7.5 Evolutionarily Stable Mixed Strategies . . . . . . . . . . . . . . . . . . . . . 220 7.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225 8 Modeling Network Traffic using Game Theory 229 8.1 Traffic at Equilibrium . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229 8.2 Braess’s Paradox . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231 8.3 Advanced Material: The Social Cost of Traffic at Equilibrium . . . . . . . . 233 8.4 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243 9 Auctions 249 9.1 Types of Auctions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249 9.2 When are Auctions Appropriate? . . . . . . . . . . . . . . . . . . . . . . . . 251 9.3 Relationships between Different Auction Formats . . . . . . . . . . . . . . . 252 9.4 Second-Price Auctions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 254 9.5 First-Price Auctions and Other Formats . . . . . . . . . . . . . . . . . . . . 257 9.6 Common Values and The Winner’s Curse . . . . . . . . . . . . . . . . . . . . 258

5 CONTENTS 5 9.7 Advanced Material: Bidding Strategies in First-Price and All-Pay Auctions . 260 9.8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 268 275 III Markets and Strategic Interaction in Networks 10 Matching Markets 277 10.1 Bipartite Graphs and Perfect Matchings . . . . . . . . . . . . . . . . . . . . 277 10.2 Valuations and Optimal Assignments . . . . . . . . . . . . . . . . . . . . . . 282 10.3 Prices and the Market-Clearing Property . . . . . . . . . . . . . . . . . . . . 284 10.4 Constructing a Set of Market-Clearing Prices . . . . . . . . . . . . . . . . . . 288 10.5 How Does this Relate to Single-Item Auctions? . . . . . . . . . . . . . . . . 291 10.6 Advanced Material: A Proof of the Matching Theorem . . . . . . . . . . . . 293 10.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 302 11 Network Models of Markets with Intermediaries 311 11.1 Price-Setting in Markets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 311 11.2 A Model of Trade on Networks . . . . . . . . . . . . . . . . . . . . . . . . . 315 11.3 Equilibria in Trading Networks . . . . . . . . . . . . . . . . . . . . . . . . . 322 11.4 Further Equilibrium Phenomena: Auctions and Ripple Effects . . . . . . . . 326 11.5 Social Welfare in Trading Networks . . . . . . . . . . . . . . . . . . . . . . . 330 11.6 Trader Profits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 332 11.7 Reflections on Trade with Intermediaries . . . . . . . . . . . . . . . . . . . . 334 11.8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 334 339 12 Bargaining and Power in Networks 12.1 Power in Social Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 339 12.2 Experimental Studies of Power and Exchange . . . . . . . . . . . . . . . . . 342 12.3 Results of Network Exchange Experiments . . . . . . . . . . . . . . . . . . . 344 12.4 A Connection to Buyer-Seller Networks . . . . . . . . . . . . . . . . . . . . . 348 12.5 Modeling Two-Person Interaction: The Nash Bargaining Solution . . . . . . 349 12.6 Modeling Two-Person Interaction: The Ultimatum Game . . . . . . . . . . . 352 12.7 Modeling Network Exchange: Stable Outcomes . . . . . . . . . . . . . . . . 355 12.8 Modeling Network Exchange: Balanced Outcomes . . . . . . . . . . . . . . . 359 12.9 Advanced Material: A Game-Theoretic Approach to Bargaining . . . . . . . 361 12.10Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 369 IV Information Networks and the World Wide Web 373 13 The Structure of the Web 375 13.1 The World Wide Web . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 376 13.2 Information Networks, Hypertext, and Associative Memory . . . . . . . . . . 378 13.3 The Web as a Directed Graph . . . . . . . . . . . . . . . . . . . . . . . . . . 384 13.4 The Bow-Tie Structure of the Web . . . . . . . . . . . . . . . . . . . . . . . 388

6 6 CONTENTS 13.5 The Emergence of Web 2.0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 392 13.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 394 397 14 Link Analysis and Web Search 14.1 Searching the Web: The Problem of Ranking . . . . . . . . . . . . . . . . . . 397 14.2 Link Analysis using Hubs and Authorities . . . . . . . . . . . . . . . . . . . 399 14.3 PageRank . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 406 14.4 Applying Link Analysis in Modern Web Search . . . . . . . . . . . . . . . . 412 14.5 Applications beyond the Web . . . . . . . . . . . . . . . . . . . . . . . . . . 415 14.6 Advanced Material: Spectral Analysis, Random Walks, and Web Search . . . 417 14.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 429 437 15 Sponsored Search Markets 15.1 Advertising Tied to Search Behavior . . . . . . . . . . . . . . . . . . . . . . 437 15.2 Advertising as a Matching Market . . . . . . . . . . . . . . . . . . . . . . . . 440 15.3 Encouraging Truthful Bidding in Matching Markets: The VCG Principle . . 444 15.4 Analyzing the VCG Procedure: Truth-Telling as a Dominant Strategy . . . . 449 15.5 The Generalized Second Price Auction . . . . . . . . . . . . . . . . . . . . . 452 15.6 Equilibria of the Generalized Second Price Auction . . . . . . . . . . . . . . 456 15.7 Ad Quality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 459 15.8 Complex Queries and Interactions Among Keywords . . . . . . . . . . . . . 461 15.9 Advanced Material: VCG Prices and the Market-Clearing Property . . . . . 462 15.10Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 478 V Network Dynamics: Population Models 481 483 16 Information Cascades 16.1 Following the Crowd . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 483 16.2 A Simple Herding Experiment . . . . . . . . . . . . . . . . . . . . . . . . . . 485 16.3 Bayes’ Rule: A Model of Decision-Making Under Uncertainty . . . . . . . . . 489 16.4 Bayes’ Rule in the Herding Experiment . . . . . . . . . . . . . . . . . . . . . 494 16.5 A Simple, General Cascade Model . . . . . . . . . . . . . . . . . . . . . . . . 496 16.6 Sequential Decision-Making and Cascades . . . . . . . . . . . . . . . . . . . 500 16.7 Lessons from Cascades . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 503 16.8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 505 17 Network Effects 509 17.1 The Economy Without Network Effects . . . . . . . . . . . . . . . . . . . . . 510 17.2 The Economy with Network Effects . . . . . . . . . . . . . . . . . . . . . . . 514 17.3 Stability, Instability, and Tipping Points . . . . . . . . . . . . . . . . . . . . 517 17.4 A Dynamic View of the Market . . . . . . . . . . . . . . . . . . . . . . . . . 519 17.5 Industries with Network Goods . . . . . . . . . . . . . . . . . . . . . . . . . 526 17.6 Mixing Individual Effects with Population-Level Effects . . . . . . . . . . . . 528 17.7 Advanced Material: Negative Externalities and The El Farol Bar Problem . 533

7 CONTENTS 7 17.8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 541 18 Power Laws and Rich-Get-Richer Phenomena 543 18.1 Popularity as a Network Phenomenon . . . . . . . . . . . . . . . . . . . . . . 543 18.2 Power Laws . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 545 18.3 Rich-Get-Richer Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 547 18.4 The Unpredictability of Rich-Get-Richer Effects . . . . . . . . . . . . . . . . 549 18.5 The Long Tail . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 551 18.6 The Effect of Search Tools and Recommendation Systems . . . . . . . . . . . 554 18.7 Advanced Material: Analysis of Rich-Get-Richer Processes . . . . . . . . . . 555 18.8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 559 561 VI Network Dynamics: Structural Models 19 Cascading Behavior in Networks 563 19.1 Diffusion in Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 563 19.2 Modeling Diffusion through a Network . . . . . . . . . . . . . . . . . . . . . 565 19.3 Cascades and Clusters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 573 19.4 Diffusion, Thresholds, and the Role of Weak Ties . . . . . . . . . . . . . . . 578 19.5 Extensions of the Basic Cascade Model . . . . . . . . . . . . . . . . . . . . . 580 19.6 Knowledge, Thresholds, and Collective Action . . . . . . . . . . . . . . . . . 583 19.7 Advanced Material: The Cascade Capacity . . . . . . . . . . . . . . . . . . . 587 19.8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 603 20 The Small-World Phenomenon 611 20.1 Six Degrees of Separation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 611 20.2 Structure and Randomness . . . . . . . . . . . . . . . . . . . . . . . . . . . . 612 20.3 Decentralized Search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 616 20.4 Modeling the Process of Decentralized Search . . . . . . . . . . . . . . . . . 619 20.5 Empirical Analysis and Generalized Models . . . . . . . . . . . . . . . . . . 622 20.6 Core-Periphery Structures and Difficulties in Decentralized Search . . . . . . 629 20.7 Advanced Material: Analysis of Decentralized Search . . . . . . . . . . . . . 631 20.8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 642 21 Epidemics 645 21.1 Diseases and the Networks that Transmit Them . . . . . . . . . . . . . . . . 645 21.2 Branching Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 647 21.3 The SIR Epidemic Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 650 21.4 The SIS Epidemic Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 656 21.5 Synchronization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 659 21.6 Transient Contacts and the Dangers of Concurrency . . . . . . . . . . . . . . 662 21.7 Genealogy, Genetic Inheritance, and Mitochondrial Eve . . . . . . . . . . . . 666 21.8 Advanced Material: Analysis of Branching and Coalescent Processes . . . . . 672 21.9 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 685

8 8 CONTENTS 689 VII Institutions and Aggregate Behavior 691 22 Markets and Information 22.1 Markets with Exogenous Events . . . . . . . . . . . . . . . . . . . . . . . . . 692 22.2 Horse Races, Betting, and Beliefs . . . . . . . . . . . . . . . . . . . . . . . . 694 22.3 Aggregate Beliefs and the “Wisdom of Crowds” . . . . . . . . . . . . . . . . 700 22.4 Prediction Markets and Stock Markets . . . . . . . . . . . . . . . . . . . . . 704 22.5 Markets with Endogenous Events . . . . . . . . . . . . . . . . . . . . . . . . 708 22.6 The Market for Lemons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 709 22.7 Asymmetric Information in Other Markets . . . . . . . . . . . . . . . . . . . 714 22.8 Signaling Quality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 718 22.9 Quality Uncertainty On-Line: Reputation Systems and Other Mechanisms . 720 22.10Advanced Material: Wealth Dynamics in Markets . . . . . . . . . . . . . . . 723 22.11Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 730 23 Voting 735 23.1 Voting for Group Decision-Making . . . . . . . . . . . . . . . . . . . . . . . 735 23.2 Individual Preferences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 737 23.3 Voting Systems: Majority Rule . . . . . . . . . . . . . . . . . . . . . . . . . 740 23.4 Voting Systems: Positional Voting . . . . . . . . . . . . . . . . . . . . . . . . 745 23.5 Arrow’s Impossibility Theorem . . . . . . . . . . . . . . . . . . . . . . . . . 748 23.6 Single-Peaked Preferences and the Median Voter Theorem . . . . . . . . . . 750 23.7 Voting as a Form of Information Aggregation . . . . . . . . . . . . . . . . . . 756 23.8 Insincere Voting for Information Aggregation . . . . . . . . . . . . . . . . . . 758 23.9 Jury Decisions and the Unanimity Rule . . . . . . . . . . . . . . . . . . . . . 761 23.10Sequential Voting and the Relation to Information Cascades . . . . . . . . . 766 23.11Advanced Material: A Proof of Arrow’s Impossibility Theorem . . . . . . . . 767 23.12Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 772 24 Property Rights 775 24.1 Externalities and the Coase Theorem . . . . . . . . . . . . . . . . . . . . . . 775 24.2 The Tragedy of the Commons . . . . . . . . . . . . . . . . . . . . . . . . . . 780 24.3 Intellectual Property . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 783 24.4 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 786

9 Preface Over the past decade there has been a growing public fascination with the complex “connectedness” of modern society. This connectedness is found in many incarnations: in the rapid growth of the Internet and the Web, in the ease with which global communication now takes place, and in the ability of news and information as well as epidemics and financial crises to spread around the world with surprising speed and intensity. These are phenomena that involve networks, incentives, and the aggregate behavior of groups of people; they are based on the links that connect us and the ways in which each of our decisions can have subtle consequences for the outcomes of everyone else. Motivated by these developments in the world, there has been a coming-together of mul- tiple scientific disciplines in an effort to understand how highly connected systems operate. Each discipline has contributed techniques and perspectives that are characteristically its own, and the resulting research effort exhibits an intriguing blend of these different flavors. From computer science and applied mathematics has come a framework for reasoning about how complexity arises, often unexpectedly, in systems that we design; from economics has come a perspective on how people’s behavior is affected by incentives and by their expec- tations about the behavior of others; and from sociology and the social sciences have come insights into the characteristic structures and interactions that arise within groups and pop- ulations. The resulting synthesis of ideas suggests the beginnings of a new area of study, focusing on the phenomena that take place within complex social, economic, and technolog- ical systems. This book grew out of a course that we developed at Cornell, designed to introduce this topic and its underlying ideas to a broad student audience at an introductory level. The central concepts are fundamental and accessible ones, but they are dispersed across the research literatures of the many different fields contributing to the topic. The principal goal of this book is therefore to bring the essential ideas together in a single unified treatment, and to present them in a way that requires as little background knowledge as possible. D. Easley and J. Kleinberg. Networks, Crowds, and Markets: Reasoning about a Highly Connected World . Cambridge University Press, 2010. Draft version: June 10, 2010. i

10 ii PREFACE The book is intended to be used at the introductory undergraduate level, and Overview. as such it has no formal prerequisites beyond a level of comfort with basic mathematical definitions at a pre-calculus level. In keeping with the introductory style, many of the ideas are developed in special cases and through illustrative examples; our goal is to take concepts and theories that are complex in their full generality, and to provide simpler formulations where the essential ideas still come through. In our use of the book, we find that many students are also interested in pursuing some of these topics more deeply, and so it is useful to provide pathways that lead from the introductory formulations into the more advanced literature on these topics. With this in mind, we provide optional sections labeled at the ends of most chapters. Advanced Material These advanced sections are qualitatively different from the other sections in the book; some draw on more advanced mathematics, and their presentation is at a more challenging level of conceptual complexity. Aside from the additional mathematical background required, however, even these advanced sections are self-contained; they are also strictly optional, in the sense that nothing elsewhere in the book depends on them. Synopsis. The first chapter of the book provides a detailed description of the topics and issues that we cover. Here we give a briefer summary of the main focus areas. The book is organized into seven parts of three to four chapters each. Parts I and II discuss the two main theories that underpin our investigations of networks and behavior: graph theory game theory , which formulates models , which studies network structure, and of behavior in environments where people’s decisions affect each other’s outcomes. Part III integrates these lines of thought into an analysis of the network structure of markets, and the notion of power in such networks. Part IV pursues a different integration, discussing the World Wide Web as an information network, the problem of Web search, and the de- velopment of the markets that currently lie at the heart of the search industry. Parts V and VI study the dynamics of some of the fundamental processes that take place within networks and groups, including the ways in which people are influenced by the decisions of others. Part V pursues this topic at an aggregate scale, where we model interactions between an individual and the population as a whole. Part VI continues the analysis at the more fine-grained level of network structure, beginning with the question of influence and moving on to the dynamics of search processes and epidemics. Finally, Part VII considers how we can interpret fundamental social institutions — including markets, voting systems, and property rights — as mechanisms for productively shaping some of the phenomena we’ve been studying. Use of the Book. The book is designed for teaching, as well as for any reader who finds these topics interesting and wants to pursue them independently at a deeper level.

11 iii Several different types of courses can be taught from this book. When we teach from it at Cornell, the students in our class come from many different majors and have a wide variety of technical backgrounds; this diversity in the audience has served as our primary calibration in setting the introductory level of the book. Our course includes a portion of the material from each chapter; for the sake of concreteness, we provide the approximate weekly schedule we follow below. (There are three 50-minute lectures each week, except that weeks 6 and 7 of our course contain only two lectures each. In each lecture, we don’t necessarily include all the details from each indicated section.) Week 1: Ch. 1; Ch. 2.1-2.3; Ch. 3.1-3.3,3.5,4.1 Week 2: Ch. 5.1-5.3; Ch. 6.1-6.4; Ch. 6.5-6.9 Week 3: Ch. 8.1-8.2; Ch. 9.1-9.6; Ch. 10.1-10.2 Week 4: Ch. 10.3; Ch. 10.4-10.5; Ch. 11.1-11.2 Week 5: Ch. 11.3-11.4; Ch. 12.1-12.3; Ch. 12.5-12.6 Week 6: Ch. 12.7-12.8; Ch. 13 Week 7: Ch. 14.1-14.2; Ch. 14.3-14.4 Week 8: Ch. 15.1-15.2; Ch. 15.3-15.4; Ch. 15.5-15.6,15.8 Week 9: Ch. 16.1-16.2; Ch. 16.3-16.4; Ch. 16.5-16.7 Week 10: Ch. 17.1-17.2; Ch. 17.3-17.5; Ch. 18 Week 11: Ch. 19.1-19.2; Ch. 19.3; Ch. 19.4,19.6 Week 12: Ch. 22.1-22.4; Ch. 22.5-22.9; Ch. 7.1-7.4 Week 13: Ch. 20.1-20.2; Ch. 20.3-20.6; Ch. 21.1-21.3 Week 14: Ch. 23.1-23.5; Ch. 23.6-23.9; Ch. 24 There are many other paths that a course could follow through the book. First, a number of new courses are being developed at the interface of computer science and economics, focusing particularly on the role of economic reasoning in the design and behavior of modern computing systems. The book can be used for such courses in several ways, building on four chapters as a foundation: Chapter 2 on graphs, Chapter 6 on games, Chapter 9 on auctions, and Chapter 10 on matching markets. From here, a more expansive version of such a course could cover the remainder of Parts II and III, all of Parts IV and V, Chapter 19, and portions of Part VII. A more focused and potentially shorter version of such a course concerned principally with auctions, markets, and the on-line applications of these ideas could be constructed from Chapters 2, 6, 9, 10, 13, 15, 17, 18, and 22, and drawing on parts of Chapters 11, 12, 14, 16, and 19. When these courses are taught at a more advanced level, the advanced sections at the ends of most of these chapters would be appropriate material; depending on the exact level of the course, the text of many of these chapters could be used to lead into the more advanced analysis in their respective final sections. In a different but related direction, new courses are also being developed on the topic of social computing and information networks. The book can be used for courses of this type by

12 iv PREFACE emphasizing Chapters 2-6, 13-14, 17-20, and 22; many such courses also include sponsored search markets as part of their coverage of the Web, which can be done by including Chapters 9, 10, and 15 as well. The advanced sections in the book can play a role here too, depending on the level of the course. Finally, portions of the book can serve as self-contained “modules” in courses on broader topics. To pick just a few examples, one can assemble such modules on network algorithms (Sections 2.3, 3.6, 5.5, 8.3, 10.6, 14.2-3, 14.6, 15.9, 20.3-4, and 20.7); applications of game theory (Chapters 6-9, 11, Sections 12.9, 15.3-15.6, 19.2-19.3, 19.5-19.7, 23.7-23.9); social net- work analysis (Chapters 2-5, 12.1-12.3, 12.5-12.8, 18-20); the role of information in economic settings (Chapters 16, 22, Sections 23.6-23.10); and the analysis of large-scale network data sets (Sections 2.3, 3.2-3.3, 3.6, 4.4, 5.3, 13.3-13.4, 14.2-14.5, 18.2, 18.5, 20.5). Most of these modules use graphs and/or games as fundamental building blocks; for students not already familiar with these topics, Chapters 2 and 6 respectively provide self-contained introductions. Acknowledgements. Our work on this book took place in an environment at Cornell that was particularly conducive to interaction between computing and the social sciences. Our collaboration began as part of a project with Larry Blume, Eric Friedman, Joe Halpern, ́ Dan Huttenlocher, and Eva Tardos funded by the National Science Foundation, followed by a campus-wide “theme project” on networks sponsored by Cornell’s Institute for the Social Sciences, with a group that included Larry and Dan together with John Abowd, Geri Gay, Michael Macy, Kathleen O’Connor, Jeff Prince, and David Strang. Our approach to the material in the book draws on perspectives — ways of thinking about these topics and ways of talking about them — that we’ve learned and acquired from this interdisciplinary set of colleagues, a group that includes some of our closest professional collaborators. The course on which the book is based grew out of discussions that were part of the Cornell theme project; the two of us had taught distinct portions of this material separately in graduate courses that we had developed, and Michael Kearns’s Networked Life course at Penn demonstrated the vibrancy and relevance this material could have for an introductory undergraduate audience as well. We were intrigued by the prospect of combining different perspectives that hadn’t previously appeared together — a process that would be educational not only to the students in the course but to us as well. Creating and teaching this new interdisciplinary course was made possible by the support of our departments, Computer Science and Economics, and by support from the Solomon Fund at Cornell University. Once the book had begun to take shape, we benefitted enormously from the feedback, suggestions, and experiences of colleagues who taught from early drafts of it. In particu- lar, we thank Daron Acemoglu (MIT), Lada Adamic (Michigan), Allan Borodin (Toronto), Noshir Contractor (Northwestern), Jason Hartline (Northwestern), Nicole Immorlica (North- western), Ramesh Johari (Stanford), Samir Khuller (Maryland), Jure Leskovec (Stanford),

13 v David Liben-Nowell (Carleton), Peter Monge (USC), Asu Ozdaglar (MIT), Vijay Ramachan- dran (Colgate), R. Ravi (CMU), Chuck Severance (Michigan), Aravind Srinivasan (Mary- land), and Luis von Ahn (CMU). The graduate and undergraduate teaching assistants in our own teaching of this subject have been very helpful as well; we thank Alex Ainslie, Lars Backstrom, Jacob Bank, Vlad Barash, Burak Bekdemir, Anand Bhaskar, Ben Cole, Bistra Dilkina, Eduard Dogaru, Ram Dubey, Ethan Feldman, Ken Ferguson, Narie Foster, Eric Frackleton, Christie Gibson, Vaibhav Goel, Scott Grabnic, Jon Guarino, Fahad Karim, Ko- ralai Kirabaeva, Tian Liang, Austin Lin, Fang Liu, Max Mihm, Sameer Nurmohamed, Ben Pu, Tal Rusak, Mark Sandler, Stuart Tettemer, Ozgur Yonter, Chong-Suk Yoon, and Yisong Yue. In addition to the instructors who used early drafts, a number of other people provided extensive comments on portions of the book, leading to many improvements in the text: Lada Adamic, Robert Kerr, Evie Kleinberg, Gueorgi Kossinets, Stephen Morris, David Parkes, Rahul Sami, Andrew Tomkins, and Johan Ugander. We also thank a further set of colleagues, in addition to those already listed, who have provided very useful advice and suggestions on this project as it has proceeded: Bobby Kleinberg, Gene Kleinberg, Lillian Lee, Maureen O’Hara, Prabhakar Raghavan, and Steve Strogatz. It has been a pleasure to be able to work with the editorial team at Cambridge University Press. Lauren Cowles, our main point of contact at Cambridge, has been an amazing source of advice and help, and we likewise very much appreciate the contributions of Scott Parris and David Tranah to this project, and Peggy Rote and her colleagues at Aptara for their work on the production of the book. Finally, a profound thanks to our families, in continuing appreciation of their support and many other contributions. David Easley Jon Kleinberg Ithaca, 2010

14 vi PREFACE

15 Chapter 1 Overview Over the past decade there has been a growing public fascination with the complex “connectedness” of modern society. At the heart of this fascination is the idea of a network — a pattern of interconnections among a set of things — and one finds networks appearing in discussion and commentary on an enormous range of topics. The diversity of contexts in which networks are invoked is in fact so vast that it’s worth deferring precise definitions for a moment while we first recount a few of the more salient examples. To begin with, the social networks we inhabit — the collections of social ties among friends — have grown steadily in complexity over the course of human history, due to technological advances facilitating distant travel, global communication, and digital interaction. The past half-century has seen these social networks depart even more radically from their geographic underpinnings, an effect that has weakened the traditionally local nature of such structures but enriched them in other dimensions. The information we consume has a similarly networked structure: these structures too have grown in complexity, as a landscape with a few purveyors of high-quality informa- tion (publishers, news organizations, the academy) has become crowded with an array of information sources of wildly varying perspectives, reliabilities, and motivating intentions. Understanding any one piece of information in this environment depends on understanding the way it is endorsed by and refers to other pieces of information within a large network of links. Our technological and economic systems have also become dependent on networks of enormous complexity. This has made their behavior increasingly difficult to reason about, and increasingly risky to tinker with. It has made them susceptible to disruptions that spread through the underlying network structures, sometimes turning localized breakdowns into cascading failures or financial crises. D. Easley and J. Kleinberg. Networks, Crowds, and Markets: Reasoning about a Highly Connected World . Cambridge University Press, 2010. Draft version: June 10, 2010. 1

16 2 CHAPTER 1. OVERVIEW 23 27 15 10 20 4 16 13 31 1 1 14 34 30 6 12 17 1 9 33 21 7 29 3 18 5 22 19 2 28 25 8 24 32 26 Figure 1.1: The social network of friendships within a 34-person karate club [421]. The imagery of networks has made its way into many other lines of discussion as well: Global manufacturing operations now have networks of suppliers, Web sites have networks of users, and media companies have networks of advertisers. In such formulations, the emphasis is often less on the structure of the network itself than on its complexity as a large, diffuse population that reacts in unexpected ways to the actions of central authorities. The terminology of international conflict has come to reflect this as well: for example, the picture of two opposing, state-supported armies gradually morphs, in U.S. Presidential speeches, into images of a nation facing “a broad and adaptive terrorist network” [296], or “at war against a far-reaching network of violence and hatred” [328]. 1.1 Aspects of Networks How should we think about networks, at a more precise level, so as to bring all these issues together? In the most basic sense, a network is any collection of objects in which some pairs links . This definition is very flexible: depending on the of these objects are connected by setting, many different forms of relationships or connections can be used to define links. Because of this flexibility, it is easy to find networks in many domains, including the ones we’ve just been discussing. As a first example of what a network looks like, Figure 1.1 depicts the social network among 34 people in a university karate club studied by the anthropologist Wayne Zachary in the 1970s. The people are represented by small circles, with lines joining the pairs of people who are friends outside the context of the club. This is the typical way in which networks will be drawn, with lines joining the pairs of objects that are connected

17 1.1. ASPECTS OF NETWORKS 3 Social networks based on communication and interaction can also be Figure 1.2: In this case, the pattern of e- constructed from the traces left by on-line data. mail communication among 436 employees of Hewlett Packard Research Lab is su- perimposed on the official organizational hierarchy [6]. (Image from http://www- personal.umich.edu/ ladamic/img/hplabsemailhierarchy.jpg) by links. Later in this chapter we’ll discuss some of the things one can learn from a network such as the one in Figure 1.1, as well as from larger examples such as the ones shown in Figures 1.2– 1.4. These larger examples depict, respectively, e-mail exchanges among employees of a company; loans among financial institutions; and links among blogs on the Web. In each case, links indicate the pairs who are connected (specifically, people connected by e-mail exchange, financial institutions by a borrower-lender relationship, and blogs through a link on the Web from one to the other). Simply from their visual appearance, we can already see some of the complexity that net- work structures contain. It is generally difficult to summarize the whole network succinctly; there are parts that are more or less densely interconnected, sometimes with central “cores” containing most of the links, and sometimes with natural splits into multiple tightly-linked regions. Participants in the network can be more central or more peripheral; they can strad- dle the boundaries of different tightly-linked regions or sit squarely in the middle of one. Developing a language for talking about the typical structural features of networks will be an important first step in understanding them.

18 4 CHAPTER 1. OVERVIEW Figure 1.3: The network of loans among financial institutions can be used to analyze the roles that different participants play in the financial system, and how the interactions among these roles affect the health of individual participants and the system as a whole The network here is annotated in a way that reveals its dense core, according to a scheme we will encounter in Chapter 13. (Image from Bech and Atalay [50].) Behavior and Dynamics. But the structure of the network is only a starting point. When people talk about the “connectedness” of a complex system, in general they are really talking about two related issues. One is connectedness at the level of structure — who is linked to whom — and the other is connectedness at the level of behavior — the fact that each individual’s actions have implicit consequences for the outcomes of everyone in the system. This means that in addition to a language for discussing the structure of networks, we also need a framework for reasoning about behavior and interaction in network contexts. And just as the underlying structure of a network can be complex, so too can the coupled behavior of its inhabitants. If individuals have strong incentives to achieve good outcomes,

19 1.1. ASPECTS OF NETWORKS 5 Figure 1.4: The links among Web pages can reveal densely-knit communities and prominent sites. In this case, the network structure of political blogs prior to the 2004 U.S. Presiden- tial election reveals two natural and well-separated clusters [5]. (Image from http://www- personal.umich.edu/ ladamic/img/politicalblogs.jpg) then not only will they appreciate that their outcomes depend on how others behave, but they will take this into account in planning their own actions. As a result, models of networked behavior must take strategic behavior and strategic reasoning into account. A fundamental point here is that in a network setting, you should evaluate your actions not in isolation, but with the expectation that the world will react to what you do. This means that cause-effect relationships can become quite subtle. Changes in a product, a Web site, or a government program can seem like good ideas when evaluated on the assumption that everything else will remain static, but in reality such changes can easily create incentives that shift behavior across the network in ways that were initially unintended. Moreover, such effects are at work whether we are able to see the network or not. When a large group of people is tightly interconnected, they will often respond in complex ways that are only apparent at the population level, even though these effects may come from implicit networks that we do not directly observe. Consider, for example, the way in which new products, Web sites, or celebrities rise to prominence — as illustrated, for example, by Figures 1.5 and 1.6, which show the growth in popularity of the social media sites YouTube

20 6 CHAPTER 1. OVERVIEW Y ube Search volume for ouT 2.0 1.0 Figure 1.5: The rapidly growing popularity of YouTube is characteristic of the way in which new products, technologies, or innovations rise to prominence, through feedback effects in the behavior of many individuals across a population. The plot depicts the number of Google queries for YouTube over time. The image comes from the site Google Trends (http://www.google.com/trends?q=youtube); by design, the units on the y-axis are sup- pressed in the output from this site. and Flickr over the past several years. What we see in these figures is a growing awareness and adoption of a new innovation that is visible in aggregate, across a whole population. What are the underlying mechanisms that lead to such success? Standard refrains are often invoked in these situations: the rich get richer; winners take all; small advantages are magnified to a critical mass; new ideas get attention that becomes “viral.” But the rich don’t always get richer and small advantages don’t always lead to success. Some social networking sites flourish, like Facebook, while others, like SixDegrees.com, vanish. To understand how these processes work, and how they are realized through the interconnected actions of many people, we need to study the dynamics of aggregate behavior. A Confluence of Ideas. Understanding highly connected systems, then, requires a set of ideas for reasoning about network structure, strategic behavior, and the feedback effects they produce across large populations. These are ideas that have traditionally been dis- persed across many different disciplines. However, in parallel with the increasing public interest in networks, there has been a coming-together of scientific fields around the topic of network research. Each of these fields brings important ideas to the discussion, and a full understanding seems to require a synthesis of perspectives from all of them. One of our central goals in this book is to help bring about such a synthesis, combining approaches that have traditionally been pursued separately. From computer science, ap- plied mathematics, and operations research we draw on a language for talking about the complexity of network structure, information, and systems with interacting agents. From

21 1.1. ASPECTS OF NETWORKS 7 Search volume for Flickr 2.0 1.0 Figure 1.6: This companion to Figure 1.5 shows the rise of the social media site Flickr; the growth in popularity has a very similar pattern to that of other sites including YouTube. (Image from Google Trends, http://www.google.com/trends?q=flickr) economics we draw on models for the strategic behavior of individuals who interact with each other and operate as members of larger aggregates. From sociology — particularly the more mathematical aspects concerned with social networks — we draw on a broad set of theoretical frameworks for talking about the structure and dynamics of social groups. And the overall picture can help fill in pieces that are arguably missing from the intel- lectual landscape of each of these disciplines. Economics has developed rich theories for the strategic interaction among small numbers of parties, as well as for the cumulative behavior of large, homogeneous populations. The challenge it faces is that much of economic life takes place in the complex spectrum between these extremes, with macroscopic effects that arise from an intricate pattern of localized interactions. Sociology has developed some of the fundamental insights into the structure of social networks, but its network methodology has been refined in the domains and scales where data-collection has traditionally been possible — primarily, well-defined groups with tens to hundreds of people. The explosion of new con- texts where we find network data and network applications — including enormous, digitally mediated ones — leads to new opportunities for how we can pose questions, formulate theo- ries, and evaluate predictions about social networks. Computer science, with the rise of the Web and social media, has had to deal with a world in which the design constraints on large computing systems are not just technological ones but also human ones — imposed by the complex feedback effects that human audiences create when they collectively use the Web for communication, self-expression, and the creation of knowledge. A fully satisfactory theory of network structure and behavior has the potential to address the simultaneous challenges that all these fields are encountering. A recurring theme underlying these challenges is the way in which networks span many different levels of scale and resolution. There are interesting questions that reach from the

22 8 CHAPTER 1. OVERVIEW 23 27 15 10 20 4 16 13 31 1 1 30 14 34 6 17 12 1 9 33 21 7 29 3 5 18 22 19 2 28 25 8 24 32 26 Figure 1.7: From the social network of friendships in the karate club from Figure 1.1, we can find clues to the latent schism that eventually split the group into two separate clubs (indicated by the two different shadings of individuals in the picture). scale of small groups, such as the 34-person social network in Figure 1.1, all the way up to the level of whole societies or economies, or to the body of global knowledge represented by the Web. We will think of networks both at the level of explicit structures, like those in Figures 1.1–1.4, and at the level of aggregate effects, like the popularity curves in Figures 1.5 and 1.6. As we look at networks of increasing scales, it becomes correspondingly more appropriate to take aggregate models into account. But the ability to work with massive network datasets has also enriched the picture, making it possible to study networks with billions of interacting items at a level of resolution where each connection is recorded. When an Internet search engine identifies the most useful pages from an index of the entire Web, for example, it is doing precisely this in the context of a specific task. Ultimately, it is an ongoing and challenging scientific problem to bridge these vastly different levels of scale, so that predictions and principles from one level can be reconciled with those of others. 1.2 Central Themes and Topics With this set of ideas in mind, we now introduce some of the main topics the book will consider, and the ways in which these topics reinforce the underlying principles of networks. We begin with the two main bodies of theory that we will be building on — graph theory and game theory. These are theories of structure and behavior respectively: Graph theory is the study of network structure, while game theory provides models of individual behavior

23 1.2. CENTRAL THEMES AND TOPICS 9 in settings where outcomes depend on the behavior of others. Graph Theory. In our discussion of graph theory, we will focus particularly on some of the fundamental ideas from social network analysis, framing a number of graph-theoretic concepts in these terms. The networks in Figures 1.1 and 1.2 hint at some of these ideas. In the corporate e-mail communication network from Figure 1.2, for example, we can see how the communication is balanced between staying within small organizational units and cutting across organizational boundaries. This is an example of a much more general principle in social networks — that strong ties , representing close and frequent social contacts, tend to be embedded in tightly-linked regions of the network, while , representing more weak ties casual and distinct social contacts, tend to cross between these regions. Such a dichotomy suggests a way of thinking about social networks in terms of their dense pockets of strong ties, and the ways in which they interact with each other through weaker ties. In a professional setting, it suggests a strategy for navigating one’s way through the social landscape of a large organization, by finding the structural holes between parts of the network that interact very little with each other. At a global scale, it suggests some of the ways in which weak ties can act as “short-cuts” that link together distant parts of the world, resulting in the phenomenon colloquially known as the six degrees of separation . Social networks can also capture the sources of conflict within a group. For example, latent conflicts are at work in the karate-club social network from Figure 1.1. The people labeled 1 and 34 (the darker circles) are particularly central in the network of friendships, with many connections to other people. On the other hand, they are not friends with each other, and in fact most people are only friends with one or the other of them. These two central people were, respectively, the instructor and the student founder of the club, and this pattern of non-interacting clusters was the most visible symptom of a conflict between them and their factions that ultimately splintered the group into two rival karate clubs, as shown in Figure 1.7. Later, we will see how the theory of structural balance can be used to reason about how fissures in a network may arise from the dynamics of conflict and antagonism at a purely local level. Game Theory. Our discussion of game theory starts from the observation that there are numerous settings in which a group of people must simultaneously choose how to act, knowing that the outcome will depend on the joint decisions made by all of them. One natural example is the problem of choosing a driving route through a network of highways at a time when traffic is heavy. If you’re a driver in such a situation, the delays you experience depend on the pattern of traffic congestion arising not just from your choice of route, but from the choices made by all other drivers as well. In this example, the network plays the role of a shared resource, and the combined actions of its users can either congest this

24 10 CHAPTER 1. OVERVIEW Figure 1.8: In a network representing international trade, one can look for countries that occupy powerful positions and derive economic benefits from these positions [262]. (Image from http://www.cmu.edu/joss/content/articles/volume4/KrempelPlumper.html) resource or use it more efficiently. In fact, the interactions among people’s behavior can lead to counter-intuitive effects here: for instance, adding resources to a transportation network can in fact create incentives that seriously undermine its efficiency, in a phenomenon known as Braess’s Paradox [76]. Another example that will recur in several settings throughout the book is the problem of bidding in an auction. If a seller is trying to sell a single item using an auction, then the success of any one bidder in the auction (whether she gets the item, and how much she pays) depends not just on how she bids but on how everyone else bids as well — and so an optimal bidding strategy should take this into account. Here too there are counter-intuitive effects at work: for example, if the seller introduces more aggressive pricing rules into the auction, he can make the strategic behavior of the bidders much more complex, and in particular induce optimal bidding that offsets whatever gains he might have expected to make from the new rules. We will find that auctions represent a basic kind of economic interaction that can be

25 1.2. CENTRAL THEMES AND TOPICS 11 phys- 1.9: some settings, such as this map of Medieval trade Figure In routes, ical constrain the patterns networks interaction, giving certain participants of an intrinsic economic advantage based on their network position. (Image from Trade Routes.jpg.) Medieval http://upload.wikimedia.org/wikipedia/commons/e/e1/Late directly generalized to more complex patterns of interactions on networks. As a general part of our investigation of game theory, we will abstract such situations with inter-dependent behavior into a common framework, where a collection of individuals must each commit to a strategy , thereby receiving a payoff that depends on the strategies chosen by everyone. Interpreting our preceding examples in this light, the strategies available to a driver on a set of highways consist of the different options for routes he can take, and the payoff to this driver is based on his resulting travel time. In an auction, the strategies are the different choices for how to bid, and the payoff to a bidder is the difference between the value of the goods she receives and the price she pays. This general framework allows us to make predictions about how people will behave in a range of such situations. A fundamental part of this framework will be the notion of equilibrium — a state that is “self-reinforcing,”

26 12 CHAPTER 1. OVERVIEW in that it provides no individual with an incentive to unilaterally change his or her strategy, even knowing how others will behave. Once we’ve developed graph theory Markets and Strategic Interaction on Networks. and game theory, we can combine them to produce richer models of behavior on networks. One natural setting where we can explore this is in models of trade and other forms of economic activity. The interactions among buyers and sellers, or pairs of counterparties to a trade or loan, naturally forms a network. In Figure 1.3 we saw an example of such a network, with links between banks engaging in a loan. Figure 1.8 shows another example: a network representation of international trade among 28 countries [262], with the size of each country depicting its total amount of trade, and the thickness of each link connecting two countries indicating the amount of trade between them. Where do these networks come from? In some cases, they are the traces of what happens when each participant seeks out the best trading partner they can, guided by how highly they value different trading opportunities. In other cases, they also reflect fundamental underlying constraints in the market that limit the access of certain participants to each other. In modern markets, these constraints could be institutional restrictions based on regulations; in other settings, they could be based on physical constraints like geography. For example, Figure 1.9 shows a map of trade routes in medieval Europe: when the physical movement of goods is costly and difficult, the economic outcome for different cities can depend significantly on where they are located in the underlying transportation network. In all these settings, then, the network structure encodes a lot about the pattern of trade, with the success levels of different participants affected by their positions in the network. Having a powerful position, however, depends not just on having many connections providing different options, but also on more subtle features — such as the power of the other individuals to which one is connected. We will see that this idea of network positions conferring power has been extended much more broadly, reaching beyond just economic exchange to suggest how power imbalances in many forms of social relationships may have their roots in the network patterns that the relationships form. Information networks. The information we deal with on-line has a fundamental network structure. Links among Web pages, for example, can help us to understand how these pages are related, how they are grouped into different communities, and which pages are the most prominent or important. Figure 1.4 illustrates some of these issues: it shows a network of links among political blogs constructed by Lada Adamic and Natalie Glance in the period leading up to the 2004 U.S. Presidential election [5]. Although the network is too large here to be able to really see the detailed structure around individual blogs, the image and its layout does convey the clear separation of the blogging network into two large clusters,

27 1.2. CENTRAL THEMES AND TOPICS 13 which turn out to closely correspond to the sets of liberal and conservative blogs respectively. From more detailed analysis of the raw linkage data underlying the image, it is possible to pick out the prominent blogs within each of these clusters. Current Web search engines such as Google make extensive use of network structure in evaluating the quality and relevance of Web pages. For producing search results, these sites evaluate the prominence of a Web page not simply based on the number of links it receives, but based on more subtle aspects of its position in the network. For example, a page can be viewed as more prominent if it receives links from pages that are themselves prominent; this is a circular kind of notion in which prominence is defined in terms of itself, but we will see that this circularity can be resolved through careful definitions that are based on a kind of equilibrium in the link structure. The interaction between search engines and the authors of Web pages is also a compelling example of a system where “connectedness” at the level of behavior produces interesting effects. Whenever a search engine introduces a new method for evaluating Web pages, deciding which pages to rank highly in its results, the creators of Web content react to this: they optimize what they put on the Web so as to try achieving a high rank under the new method. As a result, changes to a search engine can never be designed under the assumption that the Web will remain static; rather, the Web inevitably adapts to the ways in which search engines evaluate content, and search methods must be developed with these feedback effects in mind. This inherently game-theoretic interaction existed in latent form even in the early days of the Web. Over time it became more explicit and formalized, through the design of markets for advertising based on search, with advertising space allocated by auction mechanisms. Today, such markets are a principal source of revenue for the main search engines. Network Dynamics: Population Effects. If we observe a large population over time, we’ll see a recurring pattern by which new ideas, beliefs, opinions, innovations, technologies, products, and social conventions are constantly emerging and evolving. Collectively, we can refer to these as social practices [382] (holding opinions, adopting products, behaving according to certain principles) that people can choose to adopt or not. As we watch a group or society over time, we’ll see that new practices can be introduced and either become popular or remain obscure; meanwhile, established practices can persist or potentially fade over time. If we think back to Figures 1.5 and 1.6, they show the adoption of particular practices over time — the use of two very popular social media sites (taking the total number of Google queries for these sites over time as proxies for their popularity). Figure 1.10 depicts an analogous curve for the social-networking site MySpace, where we see a life cycle of rapid adoption followed by a slower period of decline, as MySpace’s dominance was challenged by newer competitors including Facebook.

28 14 CHAPTER 1. OVERVIEW Figure 1.10: Cascading adoption of a new technology or service (in this case, the social- networking site MySpace in 2005-2006) can be the result of individual incentives to use the most widespread technology — either based on the informational effects of seeing many other people adopt the technology, or the direct benefits of adopting what many others are already using. (Image from Google Trends, http://www.google.com/trends?q=myspace) The way in which new practices spread through a population depends in large part on influence each other’s behavior. In short, as you see more and more the fact that people people doing something, you generally become more likely to do it as well. Understanding why this happens, and what its consequences are, is a central issue for our understanding of networks and aggregate behavior. At a surface level, one could hypothesize that people imitate the decisions of others simply because of an underlying human tendency to conform : we have a fundamental inclination to behave as we see others behaving. This is clearly an important observation, but as an explanation it leaves some crucial questions unresolved. In particular, by taking imitation as a given, we miss the opportunity to ask people are influenced by the behavior of others. why This is a broad and difficult question, but in fact it is possible to identify multiple reasons why even purely rational agents — individuals with no a priori desire to conform to what others are doing — will nonetheless copy the behavior of others. One class of reasons is based on the fact that the behavior of others conveys information . You may have some private information on which to base a decision between alternatives, but if you see many people making a particular choice, it is natural to assume that they too have their own information, and to try inferring how people are evaluating different choices from how they are behaving. In the case of a Web site like YouTube or Flickr, seeing a lot of people using it can suggest that these people know something about its quality. Similarly, seeing that a certain restaurant is extremely crowded every weekend can suggest that many people think highly of it. But this sort of reasoning raises surprisingly subtle issues: as many people make decisions sequentially over time, the later decisions can be based in complex

29 1.2. CENTRAL THEMES AND TOPICS 15 ways on a mixture of private information and inferences from what has already happened, so that the actions of a large set of people can in fact be based on surprisingly little genuine , information. In an extreme form of this phenomenon we may get information cascades where even rational individuals can choose to abandon their private information and follow a crowd. There is a completely different but equally important class of reasons why people might imitate the behavior of others — when there is a direct benefit from aligning your behavior with that of others, regardless of whether they are making the best decision. Let’s go back to our examples of social-networking and media-sharing sites. If the value of such sites is in the potential to interact with others, to have access to a wide range of content, and to have a large audience for the content you post, then these types of sites become more and more valuable as people join them. In other words, regardless of whether YouTube had better features than its competitors, once it became the most popular video-sharing site, there was almost by definition an added value in using it. Such network effects amplify the success of products and technologies that are already doing well; in a market where network effects are at work, the leader can be hard to displace. Still, this type of dominance is not necessarily permanent; as we will see, it is possible for a new technology to displace an old one if it offers something markedly different — and often when it starts in a part of the network where there is room for it to take hold. These considerations show how popularity — as a general phenomenon — is governed by a “rich-get-richer” feedback process in which popularity tends to build on itself. It is possible to build mathematical models for this process, with predictions for the distribution of popularity that are borne out by empirical data — a picture in which society’s attention is divided between a small number of prominent items and a “long tail” of more obscure ones. Network Dynamics: Structural Effects. As we’ve just seen, the question of how people influence each other’s behavior is already quite subtle even when the actual structure of the underlying network is left implicit. But taking network structure into account provides important further insights into how such kinds of influence take place. The underlying mechanisms — based on information and direct benefits — are present both at the level of whole populations, and also at a local level in the network, between an individual and his or her set of friends or colleagues. In many cases you care more about aligning your own behavior with the behavior of your immediate neighbors in the social network, rather than with the population as a whole. When individuals have incentives to adopt the behavior of their neighbors in the network, we can get cascading effects, where a new behavior starts with a small set of initial adopters, and then spreads radially outward through the network. Figure 1.11 shows a small example,

30 16 CHAPTER 1. OVERVIEW Figure 1.11: When people are influenced by the behaviors their neighbors in the network, the adoption of a new product or innovation can cascade through the network structure. Here, e-mail recommendations for a Japanese graphic novel spread in a kind of informational or social contagion. (Image from Leskovec et al. [271].) in which e-mail recommendations for a particular Japanese graphic novel spread outward from four initial purchasers. By reasoning about the underlying network structure, we will see how it becomes possible for a superior technology to displace a universally-used but inferior one, if the superior technology starts in a portion of the network where it can make progress incrementally, a few people at a time. We will also find that the diffusion of technologies can be blocked by the boundary of a densely-connected cluster in the network — a “closed community” of individuals who have a high amount of linkage among themselves, and hence are resistant to outside influences. Cascading behavior in a network is sometimes referred to as “social contagion,” because it spreads from one person to another in the style of a biological epidemic. Figure 1.12 reinforces this analogy; it shows the beginning of a tuberculosis outbreak [16] and forms a visual counterpart to the social cascade in Figure 1.11. There are fundamental differences in the underlying mechanisms between social and biological contagion — social contagion tends to involve decision-making on the part of the affected individuals, whereas biological contagion is based on the chance of catching a disease-causing pathogen through contact with another individual. But the network-level dynamics are similar, and insights from the study of biological epidemics are also useful in thinking about the processes by which things spread on networks. The act of spreading, which transmits both ideas and diseases, is just one kind of dynamic

31 1.2. CENTRAL THEMES AND TOPICS 17 Figure 1.12: The spread of an epidemic disease (such as the tuberculosis outbreak shown here) is another form of cascading behavior in a network. The similarities and contrasts between biological and social contagion lead to interesting research questions. (Image from Andre et al. [16].) process that takes place on networks. A different process that we also consider is — search the way people can explore chains of social contacts for information or referrals to others. The surprising effectiveness with which people are able to accomplish such tasks, confirmed both by experiments and everyday experience, suggests characteristic patterns of structure at the network level that help facilitate these types of activities. Institutions and Aggregate Behavior. Once we have developed some of the basic forces underlying networks and strategic behavior, we can ask how the institutions a society designs can, in effect, channel these forces to produce certain kinds of overall outcomes. Our notion of an institution here is very broad — it can be any set of rules, conventions, or mechanisms that serve to synthesize individual actions into a pattern of aggregate behavior. We’ve already discussed particular examples of this process: for example, in the way in which a particular auction mechanism leads to bidding behavior and hence prices; or the way in which the Internet search industry has become a significant influence on how Web content is created.

32 18 CHAPTER 1. OVERVIEW 100 90 80 70 60 50 40 30 20 10 0 Figure 1.13: Prediction markets, as well as markets for financial assets such as stocks, can synthesize individual beliefs about future events into a price that captures the aggregate of these beliefs. The plot here depicts the varying price over time for two assets that paid $1 in the respective events that the Democratic or Republican nominee won the 2008 U.S. Presidential election. (Image from Iowa Electronic Markets, PRES08 WTA.cfm.) http://iemweb.biz.uiowa.edu/graphs/graph There are a number of settings in which this kind of analysis, applied to fundamental social institutions, can be very informative. One such setting is to think about markets and their role in aggregating and conveying information. In a financial market, for example, the market price serves as an aggregator of individuals’ beliefs about the value of the assets being traded. In this sense, the overall behavior of the market serves to synthesize the information that is held by many participants; consequently, when people speak of what the market “expects,” they are really referring to the expectations that can be read out of this composite of information. How this synthesis works depends on how the market is designed, and on the kind of individual and aggregate behavior that results. Nor are such issues restricted to markets for financial assets such as stocks. Recent work, for example, has explored the design of

33 1.2. CENTRAL THEMES AND TOPICS 19 that use a market mechanism to provide predictions of future events such prediction markets as the outcomes of elections. Here, participants in the market purchase assets that pay a fixed amount if a certain event takes place. In this way, the price of the asset reflects an aggregate estimate for the probability of the event, and such estimates have been found to be highly accurate in a number of cases — with the market’s aggregate predictions often outperforming the opinions of expert analysts. Figure 1.13 shows an example from the 2008 U.S. Presidential Election: the upper curve depicts the price over time for an asset that paid $1 in the event that the Democratic Party’s nominee won the election, and the lower curve depicts the corresponding price for the Republican Party’s nominee. Note that the market was already functioning before the identities of these nominees were known, and it shows a clear aggregate reaction to certain events such as the contentious end of the Democratic primary process between Obama and Clinton (in early May) and the Republican National Convention (in early September), both of which brought the prices for the opposing predictions close to equality, before they diverged once and for all as the actual election neared. Voting is another social institution that aggregates behavior across a population. While markets and voting systems both seek a synthesis of individual beliefs or preferences, there are some fundamental contrasts in the settings where they are generally applied. We have just outlined a view of markets as aggregators of beliefs about the probabilities of future events. In this view, each individual belief that forms an ingredient of the market’s consensus will ultimately be confirmed as correct or incorrect, based on whether certain relevant future events actually happen or not. Voting systems, on the other hand, are typically applied to cases where each individual has a preference or prioritization over a set of arbitrary and subjective choices for which there may be no eventual way to say that any one is “right” or “wrong.” The question is then to synthesize a cumulative social preference that reconciles, as well as possible, the conflicting priorities of the individuals in the population. In our analysis of voting, we will explore a long history of work showing that the task of producing such a social preference is fraught with unavoidable difficulties — results that formalize such difficulties began with work of 18th-century French philosophers, and came fully into focus with Arrow’s Impossibility Theorem in the 1950s. This perspective on institutions is a natural one for social systems that are highly inter- connected. Whenever the outcomes across a population depend on an aggregate of everyone’s behavior, the design of the underlying institutions can have a significant effect on how this behavior is shaped, and on the resulting consequences for society. Looking ahead. Examples, phenomena, and principles such as these will motivate the ways in which we analyze networks, behavior, and population-level dynamics throughout the book. Understanding whether a principle holds across many settings will involve formulating

34 20 CHAPTER 1. OVERVIEW and reasoning about mathematical models, and also reasoning qualitatively about these models and searching for their broader implications. In this way, we can hope to develop a network perspective as a powerful way of looking at complex systems in general — a way of thinking about social dynamics, economic interaction, on-line information, designed technology, and natural processes, and approaching such systems with an eye toward their patterns of internal structure and the rich feedback effects that result.

35 Part I Graph Theory and Social Networks 21

36

37 Chapter 2 Graphs In this first part of the book we develop some of the basic ideas behind graph theory, the study of network structure. This will allow us to formulate basic network properties in a unifying language. The central definitions here are simple enough that we can describe them relatively quickly at the outset; following this, we consider some fundamental applications of the definitions. 2.1 Basic Definitions Graphs: Nodes and Edges. A is a way of specifying relationships among a collec- graph tion of items. A graph consists of a set of objects, called nodes , with certain pairs of these objects connected by links called . For example, the graph in Figure 2.1(a) consists edges of 4 nodes labeled , B , C A D , with B connected to each of the other three nodes by , and edges, and C and D connected by an edge as well. We say that two nodes are neighbors if they are connected by an edge. Figure 2.1 shows the typical way one draws a graph — with little circles representing the nodes, and a line connecting each pair of nodes that are linked by an edge. In Figure 2.1(a), you should think of the relationship between the two ends of an edge as being symmetric; the edge simply connects them to each other. In many settings, however, A points to we want to express asymmetric relationships — for example, that but not B vice versa. For this purpose, we define a directed graph to consist of a set of nodes, as before, together with a set of directed edges ; each directed edge is a link from one node to another, with the direction being important. Directed graphs are generally drawn as in Figure 2.1(b), with edges represented by arrows. When we want to emphasize that a graph is not directed, we can refer to it as an undirected graph ; but in general the graphs we discuss D. Easley and J. Kleinberg. Networks, Crowds, and Markets: Reasoning about a Highly Connected World . Cambridge University Press, 2010. Draft version: June 10, 2010. 23

38 24 CHAPTER 2. GRAPHS A A B B D D C C A graph on 4 nodes. (b) A directed graph on 4 nodes. (a) Figure 2.1: Two graphs: (a) an undirected graph, and (b) a directed graph. will be undirected unless noted otherwise. Graphs are useful because they serve as mathematical Graphs as Models of Networks. models of network structures. With this in mind, it is useful before going further to replace the toy examples in Figure 2.1 with a real example. Figure 2.2 depicts the network structure of the Internet — then called the Arpanet — in December 1970 [214], when it had only 13 sites. Nodes represent computing hosts, and there is an edge joining two nodes in this picture if there is a direct communication link between them. Ignoring the superimposed map of the U.S. (and the circles indicating blown-up regions in Massachusetts and Southern California), the rest of the image is simply a depiction of this 13-node graph using the same dots-and-lines style that we saw in Figure 2.1. Note that for showing the pattern of connections, the actual placement or layout of the nodes is immaterial; all that matters is which nodes are linked to which others. Thus, Figure 2.3 shows a different drawing of the same 13-node Arpanet graph. Graphs appear in many domains, whenever it is useful to represent how things are either physically or logically linked to one another in a network structure. The 13-node Arpanet in Figures 2.2 and 2.3 is an example of a communication network , in which nodes are computers or other devices that can relay messages, and the edges represent direct links along which messages can be transmitted. In Chapter 1, we saw examples from two other broad classes of graph structures: social networks , in which nodes are people or groups of people, and edges represent some kind of social interaction; and information networks , in which the nodes are information resources such as Web pages or documents, and edges represent logical

39 2.2. PATHS AND CONNECTIVITY 25 Figure 2.2: A network depicting the sites on the Internet, then known as the Arpanet, in December 1970. (Image from F. Heart, A. McKenzie, J. McQuillian, and D. Walden [214]; on-line at http://som.csudh.edu/cis/lpress/history/arpamaps/.) connections such as hyperlinks, citations, or cross-references. The list of areas in which graphs play a role is of course much broader than what we can enumerate here; Figure 2.4 gives a few further examples, and also shows that many images we encounter on a regular basis have graphs embedded in them. 2.2 Paths and Connectivity We now turn to some of the fundamental concepts and definitions surrounding graphs. Per- haps because graphs are so simple to define and work with, an enormous range of graph- theoretic notions have been studied; the social scientist John Barnes once described graph theory as a “terminological jungle, in which any newcomer may plant a tree” [45]. Fortu- nately, for our purposes, we will be able to get underway with just a brief discussion of some of the most central concepts.

40 26 CHAPTER 2. GRAPHS LINC SRI AH UT CASE MIT SDC AN ST UCSB CARN BBN V HAR RAND UCLA Figure 2.3: An alternate drawing of the 13-node Internet graph from December 1970. Paths. Although we’ve been discussing examples of graphs in many different areas, there are clearly some common themes in the use of graphs across these areas. Perhaps foremost among these is the idea that things often travel across the edges of a graph, moving from node to node in sequence — this could be a passenger taking a sequence of airline flights, a piece of information being passed from person to person in a social network, or a computer user or piece of software visiting a sequence of Web pages by following links. This idea motivates the definition of a path in a graph: a path is simply a sequence of nodes with the property that each consecutive pair in the sequence is connected by an edge. Sometimes it is also useful to think of the path as containing not just the nodes but also the sequence of edges linking these nodes. For example, the sequence of nodes mit, bbn, rand, ucla is a path in the Internet graph from Figures 2.2 and 2.3, as is the sequence case, . As we have defined it here, a path can repeat nodes: for lincoln, mit, utah, sri, ucsb example, sri, stan, ucla, sri, utah, mit is a path. But most paths we consider will not do this; if we want to emphasize that the path we are discussing does not repeat nodes, we can refer to it as a simple path . Cycles. A particularly important kind of non-simple path is a cycle , which informally is a “ring” structure such as the sequence of nodes linc, case, carn, harv, bbn, mit, linc on the right-hand-side of Figure 2.3. More precisely, a cycle is a path with at least three edges, in which the first and last nodes are the same, but otherwise all nodes are distinct. There are many cycles in Figure 2.3: sri, stan, ucla, sri is as short an example as possible according to our definition (since it has exactly three edges), while sri, stan, ucla, rand, bbn, mit, utah, sri is a significantly longer example. In fact, every edge in the 1970 Arpanet belongs to a cycle, and this was by design: it means that if any edge were to fail (e.g. a construction crew accidentally cut through the cable), there would still be a way to get from any node to any other node. More generally, cycles

41 2.2. PATHS AND CONNECTIVITY 27 Airline routes Subway map (a) (b) Flowchart of college courses (d) Tank Street Bridge in Brisbane (c) Images of graphs arising in different domains. The depictions of airline and subway systems Figure 2.4: in (a) and (b) are examples of transportation networks , in which nodes are destinations and edges represent direct connections. Much of the terminology surrounding graphs derives from metaphors based on transporta- tion through a network of roads, rail lines, or airline flights. The prerequisites among college courses in (c) is an example of a dependency network , in which nodes are tasks and directed edges indicate that one task must be performed before another. The design of complex software systems and industrial processes often requires the analysis of enormous dependency networks, with important consequences for efficient scheduling in these settings. The Tank Street Bridge from Brisbane, Australia shown in (d) is an example of a structural network , with joints as nodes and physical linkages as edges. The internal frameworks of mechanical structures such as buildings, vehicles, or human bodies are based on such networks, and the area of rigidity theory , at the inter- section of geometry and mechanical engineering, studies the stability of such structures from a graph-based perspective [388]. (Images: (a) www.airlineroutemaps.com/USA/Northwest Airlines asia pacific.shtml, (b) www.wmata.com/metrorail/systemmap.cfm, (c) www.cs.cornell.edu/ugrad/flowchart.htm.)

42 28 CHAPTER 2. GRAPHS D C A E F B G H M J I L K Figure 2.5: A graph with three connected components. in communication and transportation networks are often present to allow for redundancy — they provide for alternate routings that go the “other way” around the cycle. In the social network of friendships too, we often notice cycles in everyday life, even if we don’t refer to them as such. When you discover, for example, that your wife’s cousin’s close friend from high school is in fact someone who works with your brother, this is a cycle — consisting of you, your wife, her cousin, his high-school-friend, his co-worker (i.e. your brother), and finally back to you. Connectivity. Given a graph, it is natural to ask whether every node can reach every other node by a path. With this in mind, we say that a graph is connected if for every pair of nodes, there is a path between them. For example, the 13-node Arpanet graph is connected; and more generally, one expects most communication and transportation networks to be connected — or at least aspire to be connected — since their goal is to move traffic from one node to another. On the other hand, there is no a priori reason to expect graphs in other settings to be connected — for example, in a social network, you could imagine that there might exist two people for which it’s not possible to construct a path from one to the other. Figures 2.5 and 2.6 give examples of disconnected graphs. The first is a toy example, while the second is built from the collaboration graph at a biological research center [134]: nodes represent

43 2.2. PATHS AND CONNECTIVITY 29 Structural Genomics of Figure 2.6: The collaboration graph of the biological research center [134], which consists of three distinct connected components. Pathogenic Protozoa (SGPP) This graph was part of a comparative study of the collaboration patterns graphs of nine research centers supported by NIH’s Protein Structure Initiative; SGPP was an intermediate case between centers whose collaboration graph was connected and those for which it was fragmented into many small components. researchers, and there is an edge between two nodes if the researchers appear jointly on a co-authored publication. (Thus the edges in this second figure represent a particular formal definition of collaboration — joint authorship of a published paper — and do not attempt to capture the network of more informal interactions that presumably take place at the research center.) Components. Figures 2.5 and 2.6 make visually apparent a basic fact about disconnected graphs: if a graph is not connected, then it breaks apart naturally into a set of connected “pieces,” groups of nodes so that each group is connected when considered as a graph in isolation, and so that no two groups overlap. In Figure 2.5, we see that the graph consists E of three such pieces: one consisting of nodes and B , one consisting of nodes C , D , and A , and one consisting of the rest of the nodes. The network in Figure 2.6 also consists of three pieces: one on three nodes, one on four nodes, and one that is much larger. To make this notion precise, we we say that a connected component of a graph (often shortened just to the term “component”) is a subset of the nodes such that: (i) every node in the subset has a path to every other; and (ii) the subset is not part of some larger set with the property that every node can reach every other. Notice how both (i) and (ii)

44 30 CHAPTER 2. GRAPHS are necessary to formalize the intuitive definition: (i) says that the component is indeed internally connected, and (ii) says that it really is a free-standing “piece” of the graph, not , a connected part of a larger piece. (For example, we would not think of the set of nodes F , , and J in Figure 2.5 as forming a component, because this set violates part (ii) of the G H definition: although there are paths among all pairs of nodes in the set, it belongs to the F larger set consisting of M , in which all pairs are also linked by paths.) - Dividing a graph into its components is of course only a first, global way of describing its structure. Within a given component, there may be richer internal structure that is important to one’s interpretation of the network. For example, thinking about the largest component from Figure 2.6 in light of the collaborations that it represents, one notices certain suggestive features of the structure: a prominent node at the center, and tightly-knit groups linked to this node but not to each other. One way to formalize the role of the prominent central node is to observe that the largest connected component would break apart into three distinct components if this node were removed. Analyzing a graph this way, in terms of its densely-connected regions and the boundaries between them, is a powerful way of thinking about network structure, and it will be a central topic in Chapter 3. Giant Components. There turns out to be a useful qualitative way of thinking about the connected components of typical large networks, and for this it helps to begin with the following thought experiment. Consider the social network of the entire world, with a link between two people if they are friends. Now, of course, this is a graph that we don’t actually have explicitly recorded anywhere, but it is one where we can use our general intuitions to answer some basic questions. First, is this global friendship network connected? Presumably not. After all, connec- tivity is a fairly brittle property, in that the behavior of a single node (or a small set of nodes) can negate it. For example, a single person with no living friends would constitute a one-node component in the global friendship network, and hence the graph would not be connected. Or the canonical “remote tropical island,” consisting of people who have had no contact with the outside world, would also be a small component in the network, again showing that it is not connected. But there is something more going on here. If you’re a typical reader of this book, then you have friends who grew up in other countries. You’re in the same component as all these friends, since you have a path (containing a single edge) to each of them. Now, if you consider, say, the parents of these friends, your friends’ parents’ friends, their friends and descendants, then all of these people are in the same component as well — and by now, we’re talking about people who have never heard of you, may well not share a language with you, may have never traveled anywhere near where you live, and may have had enormously different life experiences. So even though the global friendship network may not be connected, the

45 2.2. PATHS AND CONNECTIVITY 31 component you inhabit seems very large indeed — it reaches into most parts of the world, includes people from many different backgrounds, and seems in fact likely to contain a significant fraction of the world’s population. This is in fact true when one looks across a range of network datasets — large, complex networks often have what is called a giant component , a deliberately informal term for a connected component that contains a significant fraction of all the nodes. Moreover, when a network contains a giant component, it almost always contains only one. To see why, let’s go back to the example of the global friendship network and try imagining that there were two giant components, each with hundreds of millions of people. All it would take is a single edge from someone in the first of these components to someone in the second, and the two giant components would merge into a single component. Just a single edge — in most cases, it’s essentially inconceivable that some such edge wouldn’t form, and hence two co-existing giant components are something one almost never sees in real networks. When there is a giant component, it is thus generally unique, distinguishable as a component that dwarfs all others. In fact, in some of the rare cases when two giant components have co-existed for a long time in a real network, their merging has been sudden, dramatic, and ultimately catastrophic. For example, Jared Diamond’s book Guns, Germs, and Steel [130] devotes much of its attention to the cataclysm that befell the civilizations of the Western hemisphere when European explorers began arriving in it roughly half a millenium ago. One can view this development from a network perspective as follows: five thousand years ago, the global social network likely contained two giant components — one in the Americas, and one in the Europe-Asia land mass. Because of this, technology evolved independently in the two components, and perhaps even worse, human diseases evolved independently; and so when the two components finally came in contact, the technology and diseases of one quickly and disastrously overwhelmed the other. The notion of giant components is useful for reasoning about networks on much smaller scales as well. The collaboration network in Figure 2.6 is one simple example; another interesting example is depicted in Figure 2.7, which shows the romantic relationships in an American high school over an 18-month period [49]. (These edges were not all present at once; rather, there is an edge between two people if they were romantically involved at any point during the time period.) The fact that this graph contains such a large component is significant when one thinks about the spread of sexually transmitted diseases, a focus of the researchers performing the study. A high-school student may have had a single partner over this time period and nevertheless — without realizing it — be part of this large component and hence part of many paths of potential transmission. As Bearman, Moody, and Stovel note in the paper where they analyze this network, “These structures reflect relationships that may be long over, and they link individuals together in chains far too long to be

46 32 CHAPTER 2. GRAPHS Figure 2.7: A network in which the nodes are students in a large American high school, and an edge joins two who had a romantic relationship at some point during the 18-month period in which the study was conducted [49]. the subject of even the most intense gossip and scrutiny. Nevertheless, they are real: like social facts, they are invisible yet consequential macrostructures that arise as the product of individual agency.” 2.3 Distance and Breadth-First Search In addition to simply asking whether two nodes are connected by a path, it is also interesting in most settings to ask how long such a path is — in transportation, Internet communication, or the spread of news and diseases, it is often important whether something flowing through a network has to travel just a few hops or many. To be able to talk about this notion precisely, we define the length of a path to be the number of steps it contains from beginning to end — in other words, the number of edges in the sequence that comprises it. Thus, for example, the path mit, bbn, rand, ucla in Figure 2.3 has length three, while the path mit, utah has length one. Using the notion of

47 2.3. DISTANCE AND BREADTH-FIRST SEARCH 33 you distance 1 your friends distance 2 friends of friends friends of friends distance 3 of friends all nodes, not already discovered, that have an edge to some node in the previous layer Figure 2.8: Breadth-first search discovers distances to nodes one “layer” at a time; each layer is built of nodes that have an edge to at least one node in the previous layer. a path’s length, we can talk about whether two nodes are close together or far apart in a graph: we define the distance between two nodes in a graph to be the length of the shortest path between them. For example, the distance between and sri is three, though to linc believe this you have to first convince yourself that there is no length-1 or length-2 path between them. Breadth-First Search. For a graph like the one in Figure 2.3, we can generally figure out the distance between two nodes by eyeballing the picture; but for graphs that are even a bit more complicated, we need some kind of a systematic method to determine distances. The most natural way to do this — and also the most efficient way to calculate distances for a large network dataset using a computer — is the way you would probably do it if you

48 34 CHAPTER 2. GRAPHS MIT AH distance 1 LINC BBN UT CASE V HAR RAND distance 2 SRI SDC CARN ST UCSB distance 3 AN UCLA Figure 2.9: The layers arising from a breadth-first of the December 1970 Arpanet, starting at the node mit . really needed to trace out distances in the global friendship network (and had the unlimited patience and cooperation of everyone in the world). This is pictured in Figure 2.8: (1) You first declare all of your actual friends to be at distance 1. (2) You then find all of their friends (not counting people who are already friends of yours), and declare these to be at distance 2. (3) Then you find all of their friends (again, not counting people who you’ve already found at distances 1 and 2) and declare these to be at distance 3. (...) Continuing in this way, you search in successive layers, each representing the next distance out. Each new layer is built from all those nodes that (i) have not already been discovered in earlier layers, and that (ii) have an edge to some node in the previous layer. This technique is called breadth-first search , since it searches the graph outward from a start- ing node, reaching the closest nodes first. In addition to providing a method of determining distances, it can also serve as a useful conceptual framework to organize the structure of a graph, arranging the nodes based on their distances from a fixed starting point.

49 2.3. DISTANCE AND BREADTH-FIRST SEARCH 35 Of course, despite the social-network metaphor we used to describe breadth-first search, the process can be applied to any graph: one just keeps discovering nodes layer-by-layer, building each new layer from the nodes that are connected to at least one node in the previous layer. For example, Figure 2.9 shows how to discover all distances from the node mit in the 13-node Arpanet graph from Figure 2.3. The Small-World Phenomenon. As with our discussion of the connected components in a graph, there is something qualitative we can say, beyond the formal definitions, about distances in typical large networks. If we go back to our thought experiments on the global friendship network, we see that the argument explaining why you belong to a giant compo- nent in fact asserts something stronger: not only do you have paths of friends connecting you to a large fraction of the world’s population, but these paths are surprisingly short . Take the example of a friend who grew up in another country: following a path through this friend, to his or her parents, to their friends, you’ve followed only three steps and ended up in a different part of the world, in a different generation, with people who have very little in common with you. This idea has been termed the small-world phenomenon — the idea that the world looks “small” when you think of how short a path of friends it takes to get from you to almost anyone else. It’s also known, perhaps more memorably, as the ; this six degrees of separation phrase comes from the play of this title by John Guare [200], and in particular from the line uttered by one of the play’s characters: “I read somewhere that everybody on this planet is separated by only six other people. Six degrees of separation between us and everyone else on this planet.” The first experimental study of this notion — and the origin of the number “six” in the pop-cultural mantra — was performed by Stanley Milgram and his colleagues in the 1960s [297, 391]. Lacking any of the massive social-network datasets we have today, and with a budget of only $680, he set out to test the speculative idea that people are really connected in the global friendship network by short chains of friends. To this end, he asked a collection of 296 randomly chosen “starters” to try forwarding a letter to a “target” person, a stockbroker who lived in a suburb of Boston. The starters were each given some personal information about the target (including his address and occupation) and were asked to forward the letter to someone they knew on a first-name basis, with the same instructions, in order to eventually reach the target as quickly as possible. Each letter thus passed through the hands of a sequence of friends in succession, and each thereby formed a chain of people that closed in on the stockbroker outside Boston. Figure 2.10 shows the distribution of path lengths, among the 64 chains that succeeded in reaching the target; the median length was six, the number that made its way two decades later into the title of Guare’s play. That so many letters reached their destination, and by

50 36 CHAPTER 2. GRAPHS Figure 2.10: A histogram from Travers and Milgram’s paper on their small-world experiment [391]. For each possible length (labeled “number of intermediaries” on the x -axis), the plot shows the number of successfully completed chains of that length. In total, 64 chains reached the target person, with a median length of six. such short paths, was a striking fact when it was first discovered, and it remains so today. Of course, it is worth noting a few caveats about the experiment. First, it clearly doesn’t establish a statement quite as bold as “six degrees of separation between us and everyone else on this planet” — the paths were just to a single, fairly affluent target; many letters never got there; and attempts to recreate the experiment have been problematic due to lack of participation [255]. Second, one can ask how useful these short paths really are to people in society: even if you can reach someone through a short chain of friends, is this useful to you? Does it mean you’re truly socially “close” to them? Milgram himself mused about this in his original paper [297]; his observation, paraphrased slightly, was that if we think of each person as the center of their own social “world,” then “six short steps” becomes “six worlds apart” — a change in perspective that makes six sound like a much larger number. Despite these caveats, the experiment and the phenomena that it hints at have formed a crucial aspect in our understanding of social networks. In the years since the initial experiment, the overall conclusion has been accepted in a broad sense: social networks tend to have very short paths between essentially arbitrary pairs of people. And even if your six-

51 2.3. DISTANCE AND BREADTH-FIRST SEARCH 37 Figure 2.11: The distribution of distances in the graph of all active Microsoft Instant Mes- senger user accounts, with an edge joining two users if they communicated at least once during a month-long observation period [273]. step connections to CEOs and political leaders don’t yield immediate payoffs on an everyday basis, the existence of all these short paths has substantial consequences for the potential speed with which information, diseases, and other kinds of contagion can spread through society, as well as for the potential access that the social network provides to opportunities and to people with very different characteristics from one’s own. All these issues — and their implications for the processes that take place in social networks — are rich enough that we will devote Chapter 20 to a more detailed study of the small-world phenomenon and its consequences. Instant Messaging, Paul Erd ̈os, and Kevin Bacon. One reason for the current em- pirical consensus that social networks generally are “small worlds” is that this has been increasingly confirmed in settings where we do have full data on the network structure. Mil- gram was forced to resort to an experiment in which letters served as “tracers” through a global friendship network that he had no hope of fully mapping on his own; but for other kinds of social network data where the full graph structure is known, one can just load it into a computer and perform the breadth-first search procedure to determine what typical

52 38 CHAPTER 2. GRAPHS Figure 2.12: Ron Graham’s hand-drawn picture of a part of the mathematics collaboration graph, centered on Paul Erd ̈os [189]. (Image from http://www.oakland.edu/enp/cgraph.jpg) distances look like. One of the largest such computational studies was performed by Jure Leskovec and Eric Horvitz [273]. They analyzed the 240 million active user accounts on Microsoft Instant Messenger, building a graph in which each node corresponds to a user, and there is an edge between two users if they engaged in a two-way conversation at any point during a month-long observation period. As employees of Microsoft at the time, they had access to a complete snapshot of the system for the month under study, so there were no concerns about missing data. This graph turned out to have a giant component containing almost all of the nodes, and the distances within this giant component were very small. Indeed, the distances in the Instant Messenger network closely corresponded to the numbers from Milgram’s experiment, with an estimated average distance of 6.6, and an estimated median

53 2.3. DISTANCE AND BREADTH-FIRST SEARCH 39 of seven. Figure 2.11 shows the distribution of distances averaged over a random sample of 1000 users: breadth-first search was performed separately from each of these 1000 users, and the results from these 1000 nodes were combined to produce the plot in the figure. The reason for this estimation by sampling users is a computational one: the graph was so large that performing breadth-first search from every single node would have taken an astronomical amount of time. Producing plots like this efficiently for massive graphs is an interesting research topic in itself [338]. In a sense, the plot in Figure 2.11 starts to approximate, in a striking way, what Milgram and his colleagues were trying to understand — the distribution of how far apart we all are in the full global friendship network. At the same time, reconciling the structure of such massive datasets with the underlying networks they are trying to measure is an issue that comes up here, as it will many times throughout the book. In this case, enormous as the Microsoft IM study was, it remains some distance away from Milgram’s goal: it only tracks people who are technologically-endowed enough to have access to instant messaging, and rather than basing the graph on who is truly friends with whom, it can only observe who talks to whom during an observation period. Turning to a smaller scale — at the level of hundred of thousands of people rather than hundreds of millions — researchers have also discovered very short paths in the collaboration networks within professional communities. In the domain of mathematics, for example, people often speak of the itinerant mathematician Paul Erd ̈os — who published roughly 1500 papers over his career — as a central figure in the collaborative structure of the field. To make this precise, we can define a collaboration graph as we did for Figure 2.6, in this case with nodes corresponding to mathematicians, and edges connecting pairs who have jointly authored a paper. (While Figure 2.6 concerned a single research lab, we are now talking about collaboration within the entire field of mathematics.) Figure 2.12 shows a small hand-drawn piece of the collaboration graph, with paths leading to Paul Erd ̈os [189]. Now, a mathematician’s Erd ̈os number is the distance from him or her to Erd ̈os in this graph [198]. The point is that most mathematicians have Erd ̈os numbers of at most 4 or 5, and — extending the collaboration graph to include co-authorship across all the sciences — most scientists in other fields have Erd ̈os numbers that are comparable or only slightly larger; Albert Einstein’s is 2, Enrico Fermi’s is 3, Noam Chomsky’s and Linus Pauling’s are each 4, Francis Crick’s and James Watson’s are 5 and 6 respectively. The world of science is truly a small one in this sense. Inspired by some mixture of the Milgram experiment, John Guare’s play, and a compelling belief that Kevin Bacon was the center of the Hollywood universe, three students at Albright College in Pennsylvania sometime around 1994 adapted the idea of Erd ̈os numbers to the collaboration graph of movie actors and actresses: nodes are performers, an edge connects two performers if they’ve appeared together in a movie, and a performer’s Bacon number is

54 40 CHAPTER 2. GRAPHS his or her distance in this graph to Kevin Bacon [372]. Using cast lists from the Internet Movie Database (IMDB), it is possible to compute Bacon numbers for all performers via breadth-first search — and as with mathematics, it’s a small world indeed. The average Bacon number, over all performers in the IMDB, is approximately 2.9, and it’s a challenge to find one that’s larger than 5. Indeed, it’s fitting to conclude with a network-and-movie enthusiast’s description of his late-night attempts to find the largest Bacon number in the IMDB by hand: “With my life-long passion for movies, I couldn’t resist spending many hours probing the dark recesses of film history until, at about 10 AM on Sunday, I found an Plenniki Morya incredibly obscure 1928 Soviet pirate film, , starring P. Savin with a Bacon number of 7, and whose supporting cast of 8 appeared nowhere else” [197]. One is left with the image of a long exploration that arrives finally at the outer edge of the movie world — in the early history of film, in the Soviet Union — and yet in another sense, only 8 steps from where it started. 2.4 Network Datasets: An Overview The explosion of research on large-scale networks in recent years has been fueled to a large extent by the increasing availability of large, detailed network datasets. We’ve seen examples of such datasets throughout these first two chapters, and it’s useful at this point to step back and think more systematically about where people have been getting the data that they employ in large-scale studies of networks. To put this in perspective, we note first of all that there are several distinct reasons why you might study a particular network dataset. One is that you may care about the actual domain it comes from, so that fine-grained details of the data itself are potentially as interesting as the broad picture. Another is that you’re using the dataset as a proxy for a related network that may be impossible to measure — as for example in the way the Microsoft IM graph from Figure 2.11 gave us information about distances in a social network of a scale and character that begins to approximate the global friendship network. A third possibility is that you’re trying to look for network properties that appear to be common across many different domains, and so finding a similar effect in unrelated settings can suggest that it has a certain universal nature, with possible explanations that are not tied to the specifics of any one of the domains. Of course, all three of these motivations are often at work simultaneously, to varying degrees, in the same piece of research. For example, the analysis of the Microsoft IM graph gave us insight into the global friendship network — but at a more specific level, the re- searchers performing the study were also interested in the dynamics of instant messaging in particular; and at a more general level, the result of the IM graph analysis fit into the broader framework of small-world phenomena that span many domains.

55 2.4. NETWORK DATASETS: AN OVERVIEW 41 large As a final point, we’re concerned here with sources of data on networks that are . If one wants to study a social network on 20 people — say, within a small company, or a fraternity or sorority, or a karate club as in Figure 1.1 — then one strategy is to interview all the people involved and ask them who their friends are. But if we want to study the interactions among 20,000 people, or 20,000 individual nodes of some other kind, then we need to be more opportunistic in where we look for data: except in unusual cases, we can’t simply go out and collect everything by hand, and so we need to think about settings in which the data has in some essential way already been measured for us. With this in mind, let’s consider some of the main sources of large-scale network data that people have used for research. The resulting list is far from exhaustive, nor are the categories truly distinct — a single dataset can easily exhibit characteristics from several. • Collaboration Graphs. Collaboration graphs record who works with whom in a specific setting; co-authorships among scientists and co-appearance in movies by actors and actresses are two examples of collaboration graphs that we discussed in Section 2.3. Another example that has been extensively studied by sociologists is the graph on highly-placed people in the corporate world, with an edge joining two if they have served together on the board of directors of the same Fortune 500 company [301]. The on-line world provides new instances: the Wikipedia collaboration graph (connecting two Wikipedia editors if they’ve ever edited the same article) [122, 246] and the World- of-Warcraft collaboration graph (connecting two W-o-W users if they’ve ever taken part together in the same raid or other activity) [419] are just two examples. Sometimes a collaboration graph is studied to learn about the specific domain it comes from; for example, sociologists who study the business world have a substantive in- terest in the relationships among companies at the director level, as expressed via co-membership on boards. On the other hand, while there is a research community that studies the sociological context of scientific research, a broader community of people is interested in scientific co-authorship networks precisely because they form detailed, pre-digested snapshots of a rich form of social interaction that unfolds over a long period of time [318]. By using on-line bibliographic records, one can often track the patterns of collaboration within a field across a century or more, and thereby at- tempt to extrapolate how the social structure of collaboration may work across a range of harder-to-measure settings as well. • Who-talks-to-Whom Graphs. The Microsoft IM graph is a snapshot of a large commu- nity engaged in several billion conversations over the course of a month. In this way, it captures the “who-talks-to-whom” structure of the community. Similar datasets have been constructed from the e-mail logs within a company [6] or a university [259], as well as from records of phone calls: researchers have studied the structure of call

56 42 CHAPTER 2. GRAPHS in which each node is a phone number, and there is an edge between two if they graphs engaged in a phone call over a given observation period [1, 334]. One can also use the fact that mobile phones with short-range wireless technology can detect other similar devices nearby. By equipping a group of experimental subjects with such devices and studying the traces they record, researchers can thereby build “face-to-face” graphs that record physical proximity: a node in such a graph is a person carrying one of the mobile devices, and there is an edge joining two people if they were detected to be in close physical proximity over a given observation period [141, 142]. In almost all of these kinds of datasets, the nodes represent customers, employees, or students of the organization that maintains the data. These individuals will generally have strong expectations of privacy, not necessarily even appreciating how easily one can reconstruct details of their behavior from the digital traces they leave behind when communicating by e-mail, instant messaging, or phone. As a result, the style of research performed on this kind of data is generally restricted in specific ways so as to protect the privacy of the individuals in the data. Such privacy considerations have also become a topic of significant discussion in settings where companies try to use this type of data for marketing, or when governments try to use it for intelligence-gathering purposes [315]. Related to this kind of “who-talks-to-whom” data, economic network measurements recording the “who-transacts-with-whom” structure of a market or financial commu- nity has been used to study the ways in which different levels of access to market participants can lead to different levels of market power and different prices for goods. This empirical work has in turn motivated more mathematical investigations of how a network structure limiting access between buyers and sellers can affect outcomes [63, 176, 232, 261], a focus of discussion in Chapters 10—12. • Information Linkage Graphs. Snapshots of the Web are central examples of network datasets; nodes are Web pages and directed edges represent links from one page to another. Web data stands out both in its scale and in the diversity of what the nodes represent: billions of little pieces of information, with links wiring them together. And clearly it is not just the information that is of interest, but the social and economic structures that stand behind the information: hundreds of millions of personal pages on social-networking and blogging sites, hundreds of millions more representing companies and governmental organizations trying to engineer their external images in a crowded network. A network on the scale of the full Web can be daunting to work with; simply manipu- lating the data effectively can become a research challenge in itself. As a result, much network research has been done on interesting, well-defined subsets of the Web, includ-

57 2.4. NETWORK DATASETS: AN OVERVIEW 43 ing the linkages among bloggers [264], among pages on Wikipedia [404], among pages on social-networking sites such as Facebook or MySpace [185], or among discussions and product reviews on shopping sites [201]. The study of information linkage graphs significantly predates the Web: the field of has, since the early part of the 20th century, studied the network citation analysis structure of citations among scientific papers or patents, as a way of tracking the evolution of science [145]. Citation networks are still popular research datasets today, for the same reason that scientific co-authorship graphs are: even if you don’t have a substantive interest in the social processes by which science gets done, citation networks are very clean datasets that can easily span many decades. Technological Networks. Although the Web is built on a lot of sophisticated technology, • it would be a mistake to think of it primarily as a technological network: it is really a projection onto a technological backdrop of ideas, information, and social and economic structure created by humans. But as we noted in the opening chapter, there has clearly been a convergence of social and technological networks over recent years, and much interesting network data comes from the more overtly technological end of the spectrum — with nodes representing physical devices and edges representing physical connections between them. Examples include the interconnections among computers on the Internet [155] or among generating stations in a power grid [411]. Even physical networks like these are ultimately economic networks as well, represent- ing the interactions among the competing organizations, companies, regulatory bodies, and other economic entities that shape it. On the Internet, this is made particularly explicit by a two-level view of the network. At the lowest level, nodes are individual routers and computers, with an edge meaning that two devices actually have a physical connection to each other. But at a higher level, these nodes are grouped into what are essentially little “nation-states” termed , each one controlled by a autonomous systems different Internet service-providers. There is then a who-transacts-with-whom graph on the autonomous systems, known as the AS graph , that represents the data transfer agreements these Internet service-providers make with each other. • Networks in the Natural World. Graph structures also abound in biology and the other natural sciences, and network research has devoted particular attention to several different types of biological networks. Here are three examples at three different scales, from the population level down to the molecular level. As a first example, represent the who-eats-whom relationships among species food webs in an ecosystem [137]: there is a node for each species, and a directed edge from node A to node B indicates that members of A consume members of B . Understanding the structure of a food web as a graph can help in reasoning about issues such as

58 44 CHAPTER 2. GRAPHS : if certain species become extinct, then species that rely on them cascading extinctions for food risk becoming extinct as well, if they do not have alternative food sources; these extinctions can propagate through the food web as a chain reaction. Another heavily-studied network in biology is the structure of neural connections within an organism’s brain: the nodes are neurons, and an edge represents a connection between two neurons [380]. The global brain architecture for simple organisms like C. Elegans , with 302 nodes and roughly 7000 edges, has essentially been completely mapped [3]; but obtaining a detailed network picture for brains of “higher” organisms is far beyond the current state of the art. However, significant insight has been gained by studying the structure of specific modules within a complex brain, and understanding how they relate to one another. A final example is the set of networks that make up a cell’s metabolism. There are many ways to define these networks, but roughly, the nodes are compounds that play a role in a metabolic process, and the edges represent chemical interactions among them [43]. There is considerable hope that analysis of these networks can shed light on the complex reaction pathways and regulatory feedback loops that take place inside a cell, and perhaps suggest “network-centric” attacks on pathogens that disrupt their metabolism in targeted ways. 2.5 Exercises 1. One reason for graph theory’s power as a modeling tool is the fluidity with which one can formalize properties of large systems using the language of graphs, and then systematically explore their consequences. In this first set of questions, we will work pivotal through an example of this process using the concept of a node. shortest path First, recall from Chapter 2 that a between two nodes is a path of the minimum possible length. We say that a node X is pivotal for a pair of distinct nodes and Z Y X lies on every shortest path between Y and Z (and X is not equal to if either Y or Z ). For example, in the graph in Figure 2.13, node B is pivotal for two pairs: the pair B A C , and the pair consisting of A and consisting of . (Notice that and is not D pivotal for the pair consisting of D and E since there are two different shortest paths connecting D and E , one of which (using C and F ) doesn’t pass through B . So B D is not on shortest path between D and E .) On the other hand, node every is not pivotal for any pairs. (a) Give an example of a graph in which every node is pivotal for at least one pair of nodes. Explain your answer.

59 2.5. EXERCISES 45 A E F B C D is pivotal for two pairs: the pair consisting of and B A Figure 2.13: In this example, node A and D . On the other hand, node C is not pivotal for any , and the pair consisting of D pairs. (b) Give an example of a graph in which every node is pivotal for at least two different pairs of nodes. Explain your answer. (c) Give an example of a graph having at least four nodes in which there is a single X node every pair of nodes (not counting pairs that include that is pivotal for X ). Explain your answer. 2. In the next set of questions, we consider a related cluster of definitions, which seek to formalize the idea that certain nodes can play a “gatekeeping” role in a network. The X is a gatekeeper if for some other first definition is the following: we say that a node Y and Z two nodes Y to Z passes through X . For example, in the , every path from graph in Figure 2.14, node A is a gatekeeper, since it lies for example on every path from B to E . (It also lies on every path between other pairs of nodes — for example, the pair D E , as well as other pairs.) and This definition has a certain “global” flavor, since it requires that we think about paths in the full graph in order to decide whether a particular node is a gatekeeper. A more “local” version of this definition might involve only looking at the neighbors of a node. X is a local gatekeeper if there Here’s a way to make this precise: we say that a node X , say Y and Z , that are not connected by an edge. (That is, are two neighbors of for X to be a local gatekeeper, there should be two nodes Y and Z so that Y and Z each have edges to , but not to each other.) So for example, in Figure 2.14, node X A is a local gatekeeper as well as being a gatekeeper; node D , on the other hand, is a local gatekeeper but not a gatekeeper. (Node D has neighbors B and C that are not connected by an edge; however, every pair of nodes — including and C — can be B connected by a path that does not go through D .) So we have two new definitions: gatekeeper , and local gatekeeper . When faced with

60 46 CHAPTER 2. GRAPHS B E A D C F A is a gatekeeper. Node D Figure 2.14: Node is a local gatekeeper but not a gatekeeper. new mathematical definitions, a strategy that is often useful is to explore them first through examples, and then to assess them at a more general level and try to relate them to other ideas and definitions. Let’s try this in the next few questions. (a) Give an example (together with an explanation) of a graph in which more than half of all nodes are gatekeepers. (b) Give an example (together with an explanation) of a graph in which there are no gatekeepers, but in which every node is a local gatekeeper. 3. When we think about a single aggregate measure to summarize the distances between the nodes in a given graph, there are two natural quantities that come to mind. One is diameter , which we define to be the maximum distance between any pair of nodes the in the graph. Another is the average distance , which — as the term suggests — is the average distance over all pairs of nodes in the graph. In many graphs, these two quantities are close to each other in value. But there are graphs where they can be very different. (a) Describe an example of a graph where the diameter is more than three times as large as the average distance. (b) Describe how you could extend your construction to produce graphs in which the diameter exceeds the average distance by as large a factor as you’d like. (That is, for every number c , can you produce a graph in which the diameter is more than c times as large as the average distance?)

61 Chapter 3 Strong and Weak Ties One of the powerful roles that networks play is to bridge the local and the global — to offer explanations for how simple processes at the level of individual nodes and links can have complex effects that ripple through a population as a whole. In this chapter, we consider some fundamental social network issues that illustrate this theme: how information flows through a social network, how different nodes can play structurally distinct roles in this process, and how these structural considerations shape the evolution of the network itself over time. These themes all play central roles throughout the book, adapting themselves to different contexts as they arise. Our context in this chapter will begin with the famous “strength of weak ties” hypothesis from sociology [190], exploring outward from this point to more general settings as well. Let’s begin with some backgound and a motivating question. As part of his Ph.D. thesis research in the late 1960s, Mark Granovetter interviewed people who had recently changed employers to learn how they discovered their new jobs [191]. In keeping with earlier research, he found that many people learned information leading to their current jobs through personal contacts. But perhaps more strikingly, these personal contacts were often described by interview subjects as acquaintances rather than close friends. This is a bit surprising: your close friends presumably have the most motivation to help you when you’re between jobs, so why is it so often your more distant acquaintances who are actually to thank for crucial information leading to your new job? The answer that Granovetter proposed to this question is striking in the way it links two different perspectives on distant friendships — one structural, focusing on the way these friendships span different portions of the full network; and the other interpersonal, considering the purely local consequences that follow from a friendship between two people being either strong or weak. In this way, the answer transcends the specific setting of job- D. Easley and J. Kleinberg. Networks, Crowds, and Markets: Reasoning about a Highly Connected World . Cambridge University Press, 2010. Draft version: June 10, 2010. 47

62 48 CHAPTER 3. STRONG AND WEAK TIES G G B B C C F F A A E D E D edge forms. - B Before (a) (b) After B - C C edge forms. Figure 3.1: The formation of the edge between and C illustrates the effects of triadic B A . closure, since they have a common neighbor seeking, and offers a way of thinking about the architecture of social networks more generally. To get at this broader view, we first develop some general principles about social networks and their evolution, and then return to Granovetter’s question. 3.1 Triadic Closure In Chapter 2, our discussions of networks treated them largely as static structures — we take a snapshot of the nodes and edges at a particular moment in time, and then ask about paths, components, distances, and so forth. While this style of analysis forms the basic foundation for thinking about networks — and indeed, many datasets are inherently static, offering us only a single snapshot of a network — it is also useful to think about how a network evolves over time. In particular, what are the mechanisms by which nodes arrive and depart, and by which edges form and vanish? The precise answer will of course vary depending on the type of network we’re considering, but one of the most basic principles is the following: If two people in a social network have a friend in common, then there is an increased likelihood that they will become friends themselves at some point in the future [347]. We refer to this principle as triadic closure , and it is illustrated in Figure 3.1: if nodes B and C A in common, then the formation of an edge between B and C produces have a friend a situation in which all three nodes A , B , and C have edges connecting each other — a structure we refer to as a triangle in the network. The term “triadic closure” comes from

63 3.1. TRIADIC CLOSURE 49 G G B B C C F F A A E D D E (b) After new edges form. Before new edges form. (a) Figure 3.2: If we watch a network for a longer span of time, we can see multiple edges forming D - G edge) form even though — some form through triadic closure while others (such as the the two endpoints have no neighbors in common. B the fact that the C edge has the effect of “closing” the third side of this triangle. If - we observe snapshots of a social network at two distinct points in time, then in the later snapshot, we generally find a significant number of new edges that have formed through this triangle-closing operation, between two people who had a common neighbor in the earlier snapshot. Figure 3.2, for example, shows the new edges we might see from watching the network in Figure 3.1 over a longer time span. The Clustering Coefficient. The basic role of triadic closure in social networks has motivated the formulation of simple social network measures to capture its prevalence. One clustering coefficient of these is the A is [320, 411]. The clustering coefficient of a node defined as the probability that two randomly selected friends of A are friends with each other. In other words, it is the fraction of pairs of A ’s friends that are connected to each other by edges. For example, the clustering coefficient of node A in Figure 3.2(a) is 1 / 6 B (because there is only the single D edge among the six pairs of friends B - C , - - D , B - E , C C - D , C - E , and D - E ), and it has increased to 1 / 2 in the second snapshot of the network in , and Figure 3.2(b) (because there are now the three edges - C , C - D B D - E among the same six pairs). In general, the clustering coefficient of a node ranges from 0 (when none of the node’s friends are friends with each other) to 1 (when all of the node’s friends are friends with each other), and the more strongly triadic closure is operating in the neighborhood of the node, the higher the clustering coefficient will tend to be.

64 50 CHAPTER 3. STRONG AND WEAK TIES A C B E D and - edge is a A , meaning that its removal would place A B B in Figure 3.3: The bridge distinct connected components. Bridges provide nodes with access to parts of the network that are unreachable by other means. Triadic closure is intuitively very natural, and essentially Reasons for Triadic Closure. everyone can find examples from their own experience. Moreover, experience suggests some of the basic reasons why it operates. One reason why and C are more likely to become B A friends, when they have a common friend opportunity for B and C , is simply based on the to meet: if A spends time with both B and C , then there is an increased chance that they will end up knowing each other and potentially becoming friends. A second, related reason is that in the process of forming a friendship, the fact that each of and C is friends with B (provided they are mutually aware of this) gives them a basis for trusting each other that A an arbitrary pair of unconnected people might lack. A third reason is based on the incentive A may have to bring B and C together: if A is friends with and C , then it becomes a source of latent stress in these relationships if B B and C are not friends with each other. This premise is based in theories dating back to early work in social psychology [217]; it also has empirical reflections that show up in natural but troubling ways in public-health data. For example, Bearman and Moody have found that teenage girls who have a low clustering coefficient in their network of friends are significantly more likely to contemplate suicide than those whose clustering coefficient is high [48]. 3.2 The Strength of Weak Ties So how does all this relate to Mark Granovetter’s interview subjects, telling him with such regularity that their best job leads came from acquaintances rather than close friends? In fact, triadic closure turns out to be one of the crucial ideas needed to unravel what’s going on.

65 3.2. THE STRENGTH OF WEAK TIES 51 G J K F H A C B E D A B edge is a local bridge of span 4, since the removal of this edge would Figure 3.4: The - increase the distance between A and B to 4. Bridges and Local Bridges. Let’s start by positing that information about good jobs is something that is relatively scarce; hearing about a promising job opportunity from someone suggests that they have access to a source of useful information that you don’t. Now consider this observation in the context of the simple social network drawn in Figure 3.3. The person A has four friends in this picture, but one of her friendships is qualitatively different labeled A ’s links to C from the others: D , and E connect her to a tightly-knit group of friends who , all know each other, while the link to B seems to reach into a different part of the network. We could speculate, then, that the structural peculiarity of the link to B will translate into differences in the role it plays in A A , C , ’s everyday life: while the tightly-knit group of nodes , and will all tend to be exposed to similar opinions and similar sources of information, D E ’s link to B A offers her access to things she otherwise wouldn’t necessarily hear about. To make precise the sense in which the A - B link is unusual, we introduce the following definition. We say that an edge joining two nodes A and B in a graph is a bridge if deleting the edge would cause and B to lie in two different components. In other words, this edge A is literally the only route between its endpoints, the nodes A and B . Now, if our discussion in Chapter 2 about giant components and small-world properties taught us anything, it’s that bridges are presumably extremely rare in real social networks. You may have a friend from a very different background, and it may seem that your friendship is the only thing that bridges your world and his, but one expects in reality that there will

66 52 CHAPTER 3. STRONG AND WEAK TIES W S G J K S S W S H F W W W W S S W C B A S S S S S S S S W W D E strong Figure 3.5: Each edge of the social network from Figure 3.4 is labeled here as either a ( S ) or a tie ( W ), to indicate the strength of the relationship. The labeling in the weak tie figure satisfies the Strong Triadic Closure Property at each node: if the node has strong ties to two neighbors, then these neighbors must have at least a weak tie between them. be other, hard-to-discover, multi-step paths that also span these worlds. In other words, if we were to look at Figure 3.3 as it is embedded in a larger, ambient social network, we would likely see a picture that looks like Figure 3.4. Here, the - B edge isn’t the only path that connects its two endpoints; though they may A H A B are also connected by a longer path through F not realize it, G , and and . This kind , of structure is arguably much more common than a bridge in real social networks, and we use the following definition to capture it. We say that an edge joining two nodes A and B in a graph is a local bridge if its endpoints A and B have no friends in common — in other words, if deleting the edge would increase the distance between and B to a value strictly A more than two. We say that the span of a local bridge is the distance its endpoints would be from each other if the edge were deleted [190, 407]. Thus, in Figure 3.4, the A - B edge is a local bridge with span four; we can also check that no other edge in this graph is a local bridge, since for every other edge in the graph, the endpoints would still be at distance two if the edge were deleted. Notice that the definition of a local bridge already makes an implicit connection with triadic closure, in that the two notions form conceptual opposites: an edge is a local bridge precisely when it does not form a side of any triangle in the graph. Local bridges, especially those with reasonably large span, still play roughly the same

67 3.2. THE STRENGTH OF WEAK TIES 53 role that bridges do, though in a less extreme way — they provide their endpoints with access to parts of the network, and hence sources of information, that they would otherwise be far away from. And so this is a first network context in which to interpret Granovetter’s observation about job-seeking: we might expect that if a node like A is going to get truly new information, the kind that leads to a new job, it might come unusually often (though certainly not always) from a friend connected by a local bridge. The closely-knit groups that you belong to, though they are filled with people eager to help, are also filled with people who know roughly the same things that you do. Of course, Granovetter’s interview subjects The Strong Triadic Closure Property. didn’t say, “I learned about the job from a friend connected by a local bridge.” If we believe that local bridges were overrepresented in the set of people providing job leads, how does this relate to the observation that distant acquaintances were overrepresented as well? To talk about this in any detail, we need to be able to distinguish between different levels of strength in the links of a social network. We deliberately refrain from trying to define “strength” precisely, but we mean it to align with the idea that stronger links represent closer friendship and greater frequency of interaction. In general, links can have a wide range of possible strengths, but for conceptual simplicity — and to match the friend/acquaintance dichotomy that we’re trying to explain — we’ll categorize all links in the social network as belonging to one of two types: strong ties (the stronger links, corresponding to friends), and 1 weak ties (the weaker links, corresponding to acquaintances). Once we have decided on a classification of links into strong and weak ties, we can take a social network and annotate each edge with a designation of it as either strong or weak. For example, assuming we asked the nodes in the social network of Figure 3.4 to report which of their network neighbors were close friends and which were acquaintances, we could get an annotated network as in Figure 3.5. It is useful to go back and think about triadic closure in terms of this division of edges into strong and weak ties. If we recall the arguments supporting triadic closure, based on opportunity, trust, and incentive, they all act more powerfully when the edges involved are 1 In addition to the difficulty in reducing a range of possible link strengths to a two-category strong/weak distinction, there are many other subtleties in this type of classification. For example, in the discussion here, we will take this division of links into strong and weak ties as fixed in a single snapshot of the network. In reality, of course, the strength of a particular link can vary across different times and different situations. For example, an employee of a company who is temporarily assigned to work with a new division of the company for a few months may find that her full set of available social-network links remains roughly the same, but that her links to people within the new division have been temporarily strengthened (due to the sudden close proximity and increased contact), while her links to her old division have been temporarily weakened. Similarly, a high-school student may find that links to fellow members of a particular sports team constitute strong ties while that sport is in season, but that some of these links — to the teammates he knows less well outside of the team — become weak ties in other parts of the year. Again, for our purposes, we will consider a single distinction between strong and weak ties that holds throughout the analysis.

68 54 CHAPTER 3. STRONG AND WEAK TIES strong ties than when they are weak ties. This suggests the following qualitative assumption: If a node B and C , then the B - C edge is especially likely has edges to nodes A ’s edges to B and C are both strong ties. A to form if To enable some more concrete analysis, Granovetter suggested a more formal (and somewhat more extreme version) of this, as follows. We say that a node violates the Strong Triadic Closure Property if it has strong A ties to two other nodes and C , and there is no edge at all (either a strong or B B A C . We say that a node weak tie) between satisfies the Strong Triadic and Closure Property if it does not violate it. You can check that no node in Figure 3.5 violates the Strong Triadic Closure Property, and hence all nodes satisfy the Property. On the other hand, if the A - F edge were to be a strong tie rather than a weak tie, then nodes A and F would both violate the Strong Triadic Closure Property: Node A E and F without there being an would now have strong ties to nodes G - edge, and node E would have strong ties to both A and F without there being an F A - G edge. As a further check on the definition, notice that with the labeling of edges as in Figure 3.5, node H satisfies the Strong Triadic Closure Property: H couldn’t possibly violate the Property since it only has a strong tie to one other node. Clearly the Strong Triadic Closure Property is too extreme for us to expect it hold across all nodes of a large social network. But it is a useful step as an abstraction to reality, making it possible to reason further about the structural consequences of strong and weak ties. In the same way that an introductory physics course might assume away the effects of air resistance in analyzing the flight of a ball, proposing a slightly too-powerful assumption in a network context can also lead to cleaner and conceptually more informative analysis. For now, then, let’s continue figuring out where it leads us in this case; later, we’ll return to the question of its role as a modeling assumption. Local Bridges and Weak Ties. We now have a purely local, interpersonal distinction between kinds of links — whether they are weak ties or strong ties — as well as a global, structural notion — whether they are local bridges or not. On the surface, there is no direct connection between the two notions, but in fact using triadic closure we can establish a connection, in the following claim. Claim: If a node A in a network satifies the Strong Triadic Closure Property and is involved in at least two strong ties, then any local bridge it is involved in must be a weak tie. In other words, assuming the Strong Triadic Closure Property and a sufficient number of strong ties, the local bridges in a network are necessarily weak ties.

69 3.2. THE STRENGTH OF WEAK TIES 55 Strong T riadic Closure says the B-C edge must exist, but the defi nition of a local bridge says it cannot. C S S B A Figure 3.6: If a node satifies Strong Triadic Closure and is involved in at least two strong ties, then any local bridge it is involved in must be a weak tie. The figure illustrates the A - B edge is a strong tie, then there must also be an edge between reason why: if the and B C A - B edge cannot be a local bridge. , meaning that the We’re going to justify this claim as a mathematical statement – that is, it will follow logically from the definitions we have so far, without our having to invoke any as-yet- unformalized intuitions about what social networks ought to look like. In this way, it’s a different kind of claim from our argument in Chapter 2 that the global friendship network likely contains a giant component. That was a thought experiment (albeit a very convinc- ing one), requiring us to believe various empirical statements about the network of human friendships — empirical statements that could later be confirmed or refuted by collecting data on large social networks. Here, on the other hand, we’ve constructed a small num- ber of specific mathematical definitions — particularly, local bridges and the Strong Triadic Closure Property — and we can now justify the claim directly from these. The argument is actually very short, and it proceeds by contradiction. Take some net- work, and consider a node A that satisfies the Strong Triadic Closure Property and is involved A in at least two strong ties. Now suppose is involved in a local bridge — say, to a node B — that is a strong tie. We want to argue that this is impossible, and the crux of the argument is depicted in Figure 3.6. First, since is involved in at least two strong ties, A and the edge to B is only one of them, it must have a strong tie to some other node, which we’ll call C . Now let’s ask: is there an edge connecting B and C ? Since the edge from A to - B A and B must have no friends in common, and so the B is a local bridge, C edge must not exist. But this contradicts Strong Triadic Closure, which says that since the A - B and

70 56 CHAPTER 3. STRONG AND WEAK TIES - edges are both strong ties, the B - C edge must exist. This contradiction shows that our A C initial premise, the existence of a local bridge that is a strong tie, cannot hold, finishing the argument. This argument completes the connection we’ve been looking for between the local prop- erty of tie strength and the global property of serving as a local bridge. As such, it gives us a way to think about the way in which interpersonal properties of social-network links are related to broader considerations about the network’s structure. But since the argument is based on some strong assumptions (mainly Strong Triadic Closure, since the other assump- tion is very mild), it is also worth reflecting on the role that simplifying assumptions play in a result like this. First, simplifying assumptions are useful when they lead to statements that are robust in practice, making sense as qualitative conclusions that hold in approximate forms even when the assumptions are relaxed. This is the case here: the mathematical argument can be summarized more informally and approximately as saying that in real life, a local bridge between nodes A and B tends to be a weak tie because if it weren’t, triadic closure would tend to produce short-cuts to A B that would eliminate its role as a local bridge. Again, and one is tempted to invoke the analogy to freshman physics: even if the assumptions used to derive the perfectly parabolic flight of a ball don’t hold exactly in the real world, the conclusions about flight trajectories are a very useful, conceptually tractable approximation to reality. Second, when the underlying assumptions are stated precisely, as they are here, it becomes possible to test them on real-world data. In the past few years researchers have studied the relationship of tie strength and network structure quantitatively across large populations, and have shown that the conclusions described here in fact hold in an approximate form. We describe some of this empirical research in the next section. Finally, this analysis provides a concrete framework for thinking about the initially sur- prising fact that life transitions such as a new jobs are often rooted in contact with distant acquaintances. The argument is that these are the social ties that connect us to new sources of information and new opportunities, and their conceptual “span” in the social network (the local bridge property) is directly related to their weakness as social ties. This dual role as weak connections but also valuable conduits to hard-to-reach parts of the network — this is the surprising strength of weak ties. 3.3 Tie Strength and Network Structure in Large-Scale Data The arguments connecting tie strength with structural properties of the underlying social network make intriguing theoretical predictions about the organization of social networks

71 3.3. TIE STRENGTH AND NETWORK STRUCTURE IN LARGE-SCALE DATA 57 in real life. For many years after Granovetter’s initial work, however, these predictions remained relatively untested on large social networks, due to the difficulty in finding data that reliably captured the strengths of edges in large-scale, realistic settings. This state of affairs began to change rapidly once detailed traces of digital communication became available. Such “who-talks-to-whom” data exhibits the two ingredients we need for empirical evaluation of hypotheses about weak ties: it contains the network structure of communication among pairs of people, and we can use the total time that two people spend talking to each other as a proxy for the strength of the tie — the more time spent communicating during the course of an observation period, the stronger we declare the tie to be. In one of the more comprehensive studies of this type, Onnela et al. studied the who- talks-to-whom network maintained by a cell-phone provider that covered roughly 20% of a national population [334]. The nodes correspond to cell-phone users, and there is an edge joining two nodes if they made phone calls to each other in both directions over an 18- week observation period. Because the cell phones in this population are generally used for personal communication rather than business purposes, and because the lack of a central directory means that cell-phone numbers are generally exchanged among people who already know each other, the underlying network can be viewed as a reasonable sampling of the conversations occurring within a social network representing a significant fraction of one country’s population. Moreover, the data exhibits many of the broad structural features giant component of large social networks discussed in Chapter 2, including a — a single connected component containing most (in this case 84%) of the individuals in the network. Generalizing the Notions of Weak Ties and Local Bridges. The theoretical formu- lation in the preceding section is based on two definitions that impose sharp dichotomies on the network: an edge is either a strong tie or a weak tie, and it is either a local bridge or it isn’t. For both of these definitions, it is useful to have versions that exhibit smoother gradations when we go to examine real data at a large scale. Above, we just indicated a way to do this for tie strength: we can make the strength of an edge a numerical quantity, defining it to be the total number of minutes spent on phone calls between the two ends of the edge. It is also useful to sort all the edges by tie strength, so that for a given edge we can ask what percentile it occupies this ordering of edges sorted by strength. Since a very small fraction of the edges in the cell-phone data are local bridges, it makes sense to soften this definition as well, so that we can view certain edges as being “almost” neighborhood overlap of an edge connecting A and local bridges. To do this, we define the B to be the ratio number of nodes who are neighbors of A and B both , (3.1) A or B number of nodes who are neighbors of at least one of

72 58 CHAPTER 3. STRONG AND WEAK TIES Figure 3.7: A plot of the neighborhood overlap of edges as a function of their percentile in the sorted order of all edges by tie strength. The fact that overlap increases with increasing tie strength is consistent with the theoretical predictions from Section 3.2. (Image from [334].) A or themselves (even though A is a neighbor of where in the denominator we don’t count B and B is a neighbor of A ). As an example of how this definition works, consider the edge B A - F in Figure 3.4. The denominator of the neighborhood overlap for A - F is determined by the nodes B , C , D , E , G , and J , since these are the ones that are a neighbor of at least one and of F . Of these, only C is a neighbor of both A or F , so the neighborhood overlap is A 1 / 6. The key feature of this definition is that this ratio in question is 0 precisely when the numerator is 0, and hence when the edge is a local bridge. So the notion of a local bridge is contained within this definition — local bridges are the edges of neighborhood overlap 0 — and hence we can think of edges with very small neighborhood overlap as being “almost” local bridges. (Since intuitively, edges with very small neighborhood overlap consist of nodes that travel in “social circles” having almost no one in common.) For example, this definition views the A - F edge as much closer to being a local bridge than the A - E edge is, which accords with intuition.

73 3.3. TIE STRENGTH AND NETWORK STRUCTURE IN LARGE-SCALE DATA 59 Using these defi- Empirical Results on Tie Strength and Neighborhood Overlap. nitions, we can formulate some fundamental quantitative questions based on Granovetter’s theoretical predictions. First, we can ask how the neighborhood overlap of an edge depends on its strength; the strength of weak ties predicts that neighborhood overlap should grow as tie strength grows. In fact, this is borne out extremely cleanly by the data. Figure 3.7 shows the neigh- borhood overlap of edges as a function of their percentile in the sorted order of all edges x -axis, we get edges of greater and by tie strength. Thus, as we go to the right on the greater strength, and because the curve rises in a strikingly linear fashion, we also get edges of greater and greater neighborhood overlap. The relationship between these quantities thus 2 aligns well with the theoretical prediction. The measurements underlying Figure 3.7 describe a connection between tie strength and network structure at a local level — in the neighborhoods of individual nodes. It is also interesting to consider how this type of data can be used to evaluate the more global picture suggested by the theoretical framework, that weak ties serve to link together different tightly-knit communities that each contain a large number of stronger ties. Here, Onnela et al. provided an indirect analysis to address this question, as follows. They first deleted edges from the network one at a time, starting with the strongest ties and working downward in order of tie strength. The giant component shrank steadily as they did this, its size going down gradually due to the elimination of connections among the nodes. They then tried the same thing, but starting from the weakest ties and working upward in order of tie strength. In this case, they found that the giant component shrank more rapidly, and moreover that its remnants broke apart abruptly once a critical number of weak ties had been removed. This is consistent with a picture in which the weak ties provide the more crucial connective structure for holding together disparate communities, and for keeping the global structure of the giant component intact. Ultimately, this is just a first step toward evaluating theories of tie strength on net- work data of this scale, and it illustrates some of the inherent challenges: given the size and complexity of the network, we cannot simply look at the structure and “see what’s there.” Indirect measures must generally be used, and since one knows relatively little about the meaning or significance of any particular node or edge, it remains an ongoing research challenge to draw richer and more detailed conclusions in the way that one can on small datasets. 2 It is of course interesting to note the deviation from this trend at the very right-hand edge of the plot in Figure 3.7, corresponding to the edges of greatest possible tie strength. It is not clear what causes this deviation, but it is certainly plausible that these extremely strong edges are associated with people who are using their cell-phones in some unusual fashion.

74 60 CHAPTER 3. STRONG AND WEAK TIES 3.4 Tie Strength, Social Media, and Passive Engage- ment As an increasing amount of social interaction moves on-line, the way in which we maintain and access our social networks begins to change as well. For example, as is well-known to users of social-networking tools, people maintain large explicit lists of friends in their profiles on these sites — in contrast to the ways in which such friendship circles were once much more implicit, and in fact relatively difficult for individuals even to enumerate or mentally access [244]. What effect does this have on social network structure more broadly? Understanding the changes arising from these forms of technological mediation is a challenge that was already being articulated in the early 1990s by researchers including Barry Wellman [414, 413], as the Internet began making remote interaction possible for a broad public; these issues have of course grown steadily more pervasive between then and now. Tie strength can provide an important perspective on such questions, providing a lan- guage for asking how on-line social activity is distributed across different kinds of links — and in particular, how it is distributed across links of different strengths. When we see people maintaining hundreds of friendship links on a social-networking site, we can ask how many of these correspond to strong ties that involve frequent contact, and how many of these correspond to weak ties that are activated relatively rarely. Tie Strength on Facebook. Researchers have begun to address such questions of tie strength using data from some of the most active social media sites. At Facebook, Cameron Marlow and his colleagues analyzed the friendship links reported in each user’s profile, ask- ing to what extent each link was actually used for social interaction, beyond simply being reported in the profile [286]. In other words, where are the strong ties among a user’s friends? To make this precise using the data they had available, they defined three categories of links based on usage over a one-month observation period. • A link represents reciprocal (mutual) communication , if the user both sent messages to the friend at the other end of the link, and also received messages from them during the observation period. • A link represents one-way communication if the user sent one or more messages to the friend at the other end of the link (whether or not these messages were reciprocated). • A link represents a maintained relationship if the user followed information about the friend at the other end of the link, whether or not actual communication took place; “following information” here means either clicking on content via Facebook’s News Feed service (providing information about the friend) or visiting the friend’s profile more than once.

75 3.4. TIE STRENGTH, SOCIAL MEDIA, AND PASSIVE ENGAGEMENT 61 All Friends Maintained Relationships One-way Communication Mutual Communication Figure 3.8: Four different views of a Facebook user’s network neighborhood, showing the structure of links coresponding respectively to all declared friendships, maintained relation- ships, one-way communication, and reciprocal (i.e. mutual) communication. (Image from [286].) Notice that these three categories are not mutually exclusive — indeed, the links classified as reciprocal communication always belong to the set of links classified as one-way commu- nication. This stratification of links by their use lets us understand how a large set of declared friendships on a site like Facebook translates into an actual pattern of more active social interaction, corresponding approximately to the use of stronger ties. To get a sense of the relative volumes of these different kinds of interaction through an example, Figure 3.8 shows the network neighborhood of a sample Facebook user — consisting of all his friends, and all links among his friends. The picture in the upper-left shows the set of all declared friendships in this user’s profile; the other three pictures show how the set of links becomes sparser once we consider only maintained relationships, one-way communication, or reciprocal communi-

76 62 CHAPTER 3. STRONG AND WEAK TIES Figure 3.9: The number of links corresponding to maintained relationships, one-way com- munication, and reciprocal communication as a function of the total neighborhood size for users on Facebook. (Image from [286].) cation. Moreover, as we restrict to stronger ties, certain parts of the network neighborhood thin out much faster than others. For example, in the neighborhood of the sample user in Figure 3.8, we see two distinct regions where there has been a particularly large amount of triadic closure: one in the upper part of the drawing, and one on the right-hand side of the drawing. However, when we restrict to links representing communication or a maintained relationship, we see that a lot of the links in the upper region survive, while many fewer of the links in the right-hand region do. One could conjecture that the right-hand region rep- resents a set of friends from some earlier phase of the user’s life (perhaps from high school) who declare each other as friends, but do not actively remain in contact; the upper region, on the other hand, consists of more recent friends (perhaps co-workers) for whom there is more frequent contact. We can make the relative abundance of these different types of links quantitative through the plot in Figure 3.9. On the x -axis is the total number of friends a user declares, and the curves then show the (smaller) numbers of other link types as a function of this total. There are several interesting conclusions to be drawn from this. First, it confirms that even for users who report very large numbers of friends on their profile pages (on the order of 500),

77 3.4. TIE STRENGTH, SOCIAL MEDIA, AND PASSIVE ENGAGEMENT 63 Figure 3.10: The total number of a user’s strong ties (defined by multiple directed messages) as a function of the number of followees he or she has on Twitter. (Image from [222].) the number with whom they actually communicate is generally between 10 and 20, and the number they follow even passively (e.g. by reading about them) is under 50. But beyond this observation, Marlow and his colleagues draw a further conclusion about the power of media passive engagement like Facebook to enable this kind of , in which one keeps up with friends by reading news about them even in the absence of communication. They argue that this passive network occupies an interesting middle ground between the strongest ties maintained by regular communication and the weakest ties from one’s distant past, preserved only in lists on social-networking profile pages. They write, “The stark contrast between reciprocal and passive networks shows the effect of technologies such as News Feed. If these people were required to talk on the phone to each other, we might see something like the reciprocal network, where everyone is connected to a small number of individuals. Moving to an environment where everyone is passively engaged with each other, some event, such as a new baby or engagement can propagate very quickly through this highly connected network.” Tie Strength on Twitter. Similar lines of investigation have been carried out recently on the social media site Twitter, where individual users engage in a form of micro-blogging by posting very short, 140-character public messages known as “tweets.” Twitter also includes social-network features, and these enable one to distinguish between stronger and weaker ties: each user can specify a set of other users whose messages he or she will follow, and each user can also direct messages specifically to another user. (In the latter case, the message

78 64 CHAPTER 3. STRONG AND WEAK TIES remains public for everyone to read, but it is marked with a notation indicating that it is intended for a particular user.) Thus, the former kind of interaction defines a social network based on more passive, weak ties — it is very easy for a user to follow many people’s messages without ever directly communicating with any of them. The latter kind of interaction — especially when we look at users directing multiple messages to others — corresponds to a stronger kind of direct interaction. In a style analogous to the work of Marlow et al., Huberman, Romero, and Wu analyzed the relative abundance of these two kinds of links on Twitter [222]. Specifically, for each user they considered the number of users whose messages she followed (her “followees”), and then defined her strong ties to consist of the users to whom she had directed at least two messages over the course of an observation period. Figure 3.10 shows how the number of strong ties varies as a function of the number of followees. As we saw for Facebook, even for users who maintain very large numbers of weak ties on-line, the number of strong ties remains relatively modest, in this case stabilizing at a value below 50 even for users with over 1000 followees. There is another useful way to think about the contrast between the ease of forming links and the relative scarcity of strong ties in environments like Facebook and Twitter. By definition, each strong tie requires the continuous investment of time and effort to maintain, and so even people who devote a lot of their energy to building strong ties will eventually reach a limit — imposed simply by the hours available in a day — on the number of ties that they can maintain in this way. The formation of weak ties is governed by much milder constraints — they need to be established at their outset but not necessarily maintained continuously — and so it is easier for someone to accumulate them in large numbers. We will encounter this distinction again in Chapter 13, when we consider how social networks differ at a structural level from information networks such as the World Wide Web. Understanding the effect that on-line media have on the maintenance and use of social networks is a complex problem for which the underlying research is only in its early stages. But some of these preliminary studies already highlight the ways in which networks of strong ties can still be relatively sparse even in on-line settings where weak ties abound, and how the nature of the underlying on-line medium can affect the ways in which different links are used for conveying information. 3.5 Closure, Structural Holes, and Social Capital Our discussion thus far suggests a general view of social networks in terms of tightly-knit groups and the weak ties that link them. The analysis has focused primarily on the roles that different kinds of edges of a network play in this structure — with a few edges spanning different groups while most are surrounded by dense patterns of connections.

79 3.5. CLOSURE, STRUCTURAL HOLES, AND SOCIAL CAPITAL 65 C E B A D F Figure 3.11: The contrast between densely-knit groups and boundary-spanning links is re- flected in the different positions of nodes and B in the underyling social network. A nodes There is a lot of further insight to be gained by asking about the roles that different play in this structure as well. In social networks, access to edges that span different groups is not equally distributed across all nodes: some nodes are positioned at the interface between multiple groups, with access to boundary-spanning edges, while others are positioned in the middle of a single group. What is the effect of this heterogeneity? Following the expositional lead of social-network researchers including Ron Burt [87], we can formulate an answer to this question as a story about the different experiences that nodes have in a network like the one in Figure 3.11 — particularly in the contrast between the experience of a node such as , who sits at the center of a single tightly-knit group, and node B , who sits at the interface A between several groups. Embeddedness. Let’s start with node A . Node A ’s set of network neighbors has been subject to considerable triadic closure; has a high clustering coefficient. (Recall that the A clustering coefficient is the fraction of pairs of neighbors who are themselves neighbors). To talk about the structure around A it is useful to introduce an additional definition. We define the embeddedness of an edge in a network to be the number of common neighbors the two endpoints have. Thus, for example, the A - B edge has an embeddedness of two, since A and B have the two common neighbors E and F . This definition relates to two notions from earlier in the chapter. First, the embeddedness of an edge is equal to the numerator in

80 66 CHAPTER 3. STRONG AND WEAK TIES neighborhood overlap the ratio that defines the in Equation (3.1) from Section 3.3. Second, we observe that local bridges are precisely the edges that have an embeddedness of zero — since they were defined as those edges whose endpoints have no neighbors in common. In the example shown in Figure 3.11, what stands out about A is the way in which all of his edges have significant embeddedness. A long line of research in sociology has argued that if two individuals are connected by an embedded edge, then this makes it easier for them to trust one another, and to have confidence in the integrity of the transactions (social, economic, or otherwise) that take place between them [117, 118, 193, 194, 395]. Indeed, the presence of mutual friends puts the interactions between two people “on display” in a social sense, even when they are carried out in private; in the event of misbehavior by one of the two parties to the interaction, there is the potential for social sanctions and reputational consequences from their mutual friends. As Granovetter writes, “My mortification at cheat- ing a friend of long standing may be substantial even when undiscovered. It may increase when a friend becomes aware of it. But it may become even more unbearable when our mutual friends uncover the deceit and tell one another” [194]. No similar kind of deterring threat exists for edges with zero embeddedness, since there is no one who knows both people involved in the interaction. In this respect, the interactions that B has with C and D are much riskier than the embedded interactions that A experiences. Moreover, the constraints on ’s behavior are made complicated by the fact that she is B subject to potentially contradictory norms and expectations from the different groups she associates with [116]. Structural holes. Thus far we have been discussing the advantages that accrue to node A in Figure 3.11 from the closure in his network neighborhood, and the embedded edges that result from this. But a related line of research in sociology, catalyzed by influential work of Burt [86], has argued that network positions such as node B ’s, at the ends of multiple local bridges, confer a distinct set of equally fundamental advantages. The canonical setting for this argument is the social network within an organization or company, consisting of people who are in some ways collaborating on common objectives and in other ways implicitly competing for career advancement. Note that although we may be thinking about settings in which there is a formal organizational hierarchy — encoding who reports to whom — we’re interested in the more informal network of who knows whom, and who talks to whom on a regular basis. Empirical studies of managers in large corporations has correlated an individual’s success within a company to their access to local bridges [86, 87]. At a more abstract level, the central arguments behind these studies are also supported by the network principles we have been discussing, as we now explore further. Let’s go back to the network in Figure 3.11, imagining the network to represent the interaction and collaboration among managers in a large company. In Burt’s language,

81 3.5. CLOSURE, STRUCTURAL HOLES, AND SOCIAL CAPITAL 67 B , with her multiple local bridges, spans a in the organization — the node structural hole “empty space” in the network between two sets of nodes that do not otherwise interact closely. (Unlike the term “local bridge,” which has a precise mathematical definition in terms of the underlying graph, we will keep the term “structural hole” somewhat informal in ’s position offers advantages in several dimensions B this discussion.) The argument is that A relative to ’s. The first kind of advantage, following the observations in the previous section, is an informational one: B has early access to information originating in multiple, non-interacting parts of the network. Any one person has a limited amount of energy they B can invest in maintaining contacts across the organization, and is investing her energy efficiently by reaching out to different groups rather than basing all her contacts in the same group. A second, related kind of advantage is based on the way in which standing at one end of a local bridge can be an amplifier for creativity [88]. Experience from many domains suggests that innovations often arise from the unexpected synthesis of multiple ideas, each of them on their own perhaps well-known, but well-known in distinct and unrelated bodies of expertise. B Thus, ’s position at the interface between three non-interacting groups gives her not only access to the combined information from these groups, but also the opportunity for novel ideas by combining these disparate sources of information in new ways. Finally, B ’s position in the network provides an opportunity for a kind of social “gate- keeping” — she regulates the access of both C and D to the tightly-knit group she belongs to, and she controls the ways in which her own group learns about information coming from C D ’s groups. This provides B with a source of power in the organization, and one ’s and could imagine that certain people in this situation might try to prevent triangles from form- ing around the local bridges they’re part of — for example, another edge from or D into C B B ’s gatekeeping role. ’s group would reduce This last point highlights a sense in which the interests of node B and of the organization as a whole may not be aligned. For the functioning of the organization, accelerating the flow of information between groups could be beneficial, but this building of bridges would come B ’s latent power at the boundaries of these groups. It also emphasizes that at the expense of our analysis of structural holes is primarily a static one: we look at the network at a single point in time, and consider the effects of the local bridges. How long these local bridges last before triadic closure produces short-cuts around them, and the extent to which people in an organization are consciously, strategically seeking out local bridges and trying to maintain them, is less well understood; it is a topic of ongoing research [90, 188, 252, 259]. Ultimately, then, there are trade-offs in the relative positions of A and B . B ’s position at the interface between groups means that her interactions are less embedded within a single group, and less protected by the presence of mutual network neighbors. On the other hand, this riskier position provides her with access to information residing in multiple groups, and

82 68 CHAPTER 3. STRONG AND WEAK TIES the opportunity to both regulate the flow of this information and to synthesize it in new ways. Closure and Bridging as Forms of Social Capital. All of these arguments are framed in terms of individuals and groups deriving benefits from an underlying social structure or social capital [117, 118, social network; as such, they are naturally related to the notion of 279, 342, 344]. Social capital is a term in increasingly widespread use, but it is a famously difficult one to define [138]. In Alejandro Portes’s review of the topic, he writes, “Consensus is growing in the literature that social capital stands for the ability of actors to secure benefits by virtue of membership in social networks or other social structures” [342]. The term “social capital” is designed to suggest its role as part of an array of different forms of capital, all of which serve as tangible or intangible resources that can be mobilized to accomplish tasks. James Coleman and others speak of social capital alongside physical — the implements and technologies that help perform work — and human capital capital — the skills and talents that individual people bring to a job or goal [118]. Pierre Bourdieu offers a related but distinct taxonomy, considering social capital in relation to economic capital — consisting of monetary and physical resources — and cultural capital — the accumulated resources of a culture that exist at a level beyond any one individual’s social circle, conveyed through education and other broad social institutions [17, 75]. Borgatti, Jones, and Everett [74], summarizing discussions within the sociology commu- nity, observe two important sources of variation in the use of the term “social capital.” First, social capital is sometimes viewed as a property of a group, with some groups functioning more effectively than others because of favorable properties of their social structures or net- works. Alternately, it has also been considered as a property of an individual; used in this sense, a person can have more or less social capital depending on his or her position in the underlying social structure or network. A second, related, source of terminological variation is based on whether social capital is a property that is purely intrinsic to a group — based only on the social interactions among the group’s members — or whether it is also based on the interactions of the group with the outside world. A view at this level of generality does not yet specify what kinds of network structures are the most effective for creating social capital, and our discussion earlier in this section highlights several different perspectives on the question. The writings of Coleman and oth- ers on social capital emphasize the benefits of triadic closure and embedded edges for the reasons discussed above: they enable the enforcement of norms and reputational effects, and hence can help protect the integrity of social and economic transactions. Burt, on the other hand, discusses social capital as a tension between closure and brokerage — with the former referring to Coleman’s conception and the latter referring to benefits arising from the ability to “broker” interactions at the interface between different groups, across structural holes.

83 3.6. ADVANCED MATERIAL: BETWEENNESS MEASURES AND GRAPH PARTITIONING 69 In addition to the structural distinctions between these perspectives, they also illustrate different focuses on groups versus individuals, and on the activity within a group versus its contacts with a larger population. The contrasts are also related to Robert Putnam’s dichotomy between bonding capital and bridging capital [344]; these terms, while intended informally, correspond roughly to the kinds of social capital arising respectively from con- nections within a tightly-knit group and from connections between such groups. The notion of social capital thus provides a framework for thinking about social structures as facilitators of effective action by individuals and groups, and a way of focusing discussions of the different kinds of benefits conferred by different structures. Networks are at the heart of such discussions — both in the way they produce closed groups where transactions can be trusted, and in the way they link different groups and thereby enable the fusion of different sources of information residing in these groups. 3.6 Advanced Material: Betweenness Measures and Graph Partitioning This is the first in a series of sections throughout the book labeled “Advanced Material.” Each of these sections comes at the end of a chapter, and it explores mathematically more sophisticated aspects of some of the models developed earlier in the chapter. They are strictly optional, in that nothing later in the book builds on them. Also, while these sections are technically more involved, they are written to be completely self-contained, except where specific pieces of mathematical background are needed; this necessary background is spelled out at the beginnings of the sections where it is required. In this section, we will try formulating more concrete mathematical definitions for some of the basic concepts from earlier in the chapter. The discussion in this chapter has articulated a way of thinking about networks in terms of their tightly-knit regions and the weaker ties that link them together. We have formulated precise definitions for some of the underlying concepts, such as the clustering coefficient and the definition of a local bridge. In the process, however, we have refrained from trying to precisely delineate what we mean by a “tightly-knit region,” and how to formally characterize such regions. For our purposes so far, it has been useful to be able to speak in this more general, informal way about tightly-knit regions; it helps to be flexible since the exact characterization of the notion may differ depending on the different domains in which we encounter it. But there are also settings in which having a more precise, formal definition is valuable. In particular, a formal definition can be crucial if we are faced with a network dataset and actually want to identify densely connected groups of nodes within it. This will be our focus here: describing a method that can take a network and break it down into a set of tightly-knit regions, with sparser interconnections between the regions.

84 70 CHAPTER 3. STRONG AND WEAK TIES Figure 3.12: A co-authorship network of physicists and applied mathematicians working on networks [322]. Within this professional community, more tightly-knit subgroups are evident from the network structure. We will refer to this as the problem of , and the constituent parts the graph partitioning network is broken into as the regions arising from the partitioning method. Formulating a method for graph partitioning will implicitly require working out a set of definitions for all these notions that are both mathematically tractable and also useful on real datasets. To give a sense for what we might hope to achieve from such a method, let’s consider two examples. The first, shown in Figure 3.12, depicts the co-authorships among a set of physicists and applied mathematicians working on networks [322]. Recall that we discussed co-authorship networks in Chapter 2 as a way of encoding the collaborations within a profes- sional community. It’s clear from the picture that there are tightly-knit groups within this community, and some people who sit on the boundaries of their respective groups. Indeed it resembles, at a somewhat larger scale, some of the pictures of tightly-knit groups and weak ties that we drew in schematic form earlier, in examples such as Figure 3.11. Is there a general way to pull these groups out of the data, beyond using just our visual intuition?

85 3.6. ADVANCED MATERIAL: BETWEENNESS MEASURES AND GRAPH PARTITIONING 71 27 23 15 20 10 4 16 13 31 1 1 14 30 34 6 17 12 1 9 33 21 7 3 29 5 18 22 19 2 28 25 8 24 32 26 Figure 3.13: A karate club studied by Wayne Zachary [421] — a dispute during the course of the study caused it to split into two clubs. Could the boundaries of the two clubs be predicted from the network structure? A second example, in Figure 3.13, is a picture of the social network of a karate club studied by Wayne Zachary [421] and discussed in Chapter 1: a dispute between the club president (node 34) and the instructor (node 1) led the club to split into two. Figure 3.13 shows the network structure, with the membership in the two clubs after the division indicated by the shaded and unshaded nodes. Now, a natural question is whether the structure itself contains enough information to predict the fault line. In other words, did the split occur along a weak interface between two densely connected regions? Unlike the network in Figure 3.12, or in some of the earlier examples in the chapter, the two conflicting groups here are still heavily interconnected. So to identify the division in this case, we need to look for more subtle signals in the way in which edges between the groups effectively occur at lower “density” than edges within the groups. We will see that this is in fact possible, both for the definitions we consider here as well as other definitions. A. A Method for Graph Partitioning Many different approaches have been developed for the problem of graph partitioning, and for networks with clear divisions into tightly-knit regions, there is often a wide range of methods that will prove to be effective. While these methods can differ considerably in their specifics, it is useful to identify the different general styles that motivate their designs.

86 72 CHAPTER 3. STRONG AND WEAK TIES One class of methods focuses on iden- General Approaches to Graph Partitioning. tifying and removing the “spanning links” between densely-connected regions. Once these links are removed, the network begins to fall apart into large pieces; within these pieces, further spanning links can be identified, and the process continues. We will refer to these as divisive methods of graph partitioning, since they divide the network up as they go. An alternate class of methods starts from the opposite end of the problem, focusing on the most tightly-knit parts of the network, rather than the connections at their boundaries. Such methods find nodes that are likely to belong to the same region and merge them together. Once this is done, the network consists of a large number of merged chunks, each containing the seeds of a densely-connected region; the process then looks for chunks that should be further merged together, and in this way the regions are assembled “bottom-up.” We refer to these as agglomerative methods of graph partitioning, since they glue nodes together into regions as they go. To illustrate the conceptual differences between these two kinds of approaches, let’s con- sider the simple graph in Figure 3.14(a). Intuitively, as indicated in Figure 3.14(b), there appears to be a broad separation between one region consisting of nodes 1-7, and another consisting of nodes 8-14. Within each of these regions, there is a further split: on the left into nodes 1-3 and nodes 4-6; on the right into nodes 9-11 and nodes 12-14. Note how this simple example already illustrates that the process of graph partitioning can usefully be nested : larger regions poten- viewed as producing regions in the network that are naturally tially containing several smaller, even more tightly-knit regions “nested” within them. This is of course a familiar picture from everyday life, where — for example — a separation of the gobal population into national groups can be further subdivided into sub-populations within particular local areas within countries. In fact, a number of graph partitioning methods will find the nested set of regions indi- cated in Figure 3.14(b). Divisive methods will generally proceed by breaking apart the graph first at the 7-8 edge, and subsequently at the remaining edges into nodes 7 and 8. Agglom- erative methods will arrive at the same result from the opposite direction, first merging the four triangles into clumps, and then finding that the triangles themselves can be naturally paired off. This is a good point at which to make the discussion more concrete, and to do so we focus on a particular divisive method proposed by Girvan and Newman [184, 322]. The Girvan-Newman method has been applied very widely in recent years, and to social network data in particular. Again, however, we emphasize that graph partitioning is an area in which there is an especially wide range of different approaches in use. The approach we discuss is an elegant and particular widely-used one; however, understanding which types of methods work best in different situations remains a subject of active research.

87 3.6. ADVANCED MATERIAL: BETWEENNESS MEASURES AND GRAPH PARTITIONING 73 10 1 9 3 1 1 2 8 7 6 12 4 13 5 14 (a) A sample network 10 1 9 3 1 1 2 7 8 12 6 13 4 5 14 (b) Tightly-knit regions and their nested structure Figure 3.14: In many networks, there are tightly-knit regions that are intuitively apparent, and they can even display a nested structure, with smaller regions nesting inside larger ones. The Notion of Betweenness. To motivate the design of a divisive method for graph partitioning, let’s think about some general principles that might lead us to remove the 7-8 edge first in Figure 3.14(a). A first idea, motivated by the discussion earlier in this chapter, is that since bridges and local bridges often connect weakly interacting parts of the network, we should try removing these bridges and local bridges first. This is in fact an idea along the right lines; the problem is simply that it’s not strong enough, for two reasons. First, when there are several bridges, it doesn’t tell us which to remove first. As we see in Figure 3.14(a), where there are five bridges, certain bridges can produce more reasonable splits than others. Second, there can be graphs where no edge is even a local bridge, because every edge belongs to a triangle — and yet there is still a natural division into regions. Figure 3.15 shows a simple example, where we might want to identify nodes 1-5 and nodes 7-11 as tightly-knit regions, despite

88 74 CHAPTER 3. STRONG AND WEAK TIES 2 9 6 1 4 5 8 7 1 1 3 10 Figure 3.15: A network can display tightly-knit regions even when there are no bridges or local bridges along which to separate it. the fact that there are no local bridges to remove. However, if we think more generally about what bridges and local bridges are doing, then we can arrive at a notion that forms the central ingredient of the Girvan-Newman method. Local bridges are important because they form part of the shortest path between pairs of nodes in different parts of the network — without a particular local bridge, paths between many pairs of nodes may have to be “re-routed” a longer way. We therefore define an abstract notion of “traffic” on the network, and look for the edges that carry the most of this traffic. Like crucial bridges and highway arteries, we might expect these edges to link different densely-connected regions, and hence be good candidates for removal in a divisive method. We define our notion of traffic as follows. For each pair of nodes and B in the graph A that are connected by a path, we imagine having one unit of fluid “flow” along the edges from A to B . (If A and B belong to different connected components, then no fluid flows between them.) The flow between and B divides itself evenly along all the possible shortest paths A from A to B : so if there are k shortest paths from A and B , then 1 /k units of flow pass along each one. betweenness of an edge to be the total amount of flow it carries, count- We define the ing flow between all pairs of nodes using this edge. For example, we can determine the betweenness of each edge in Figure 3.14(a) as follows. • Let’s first consider the 7-8 edge. For each node A in the left half of the graph, and each node B in the right half of the graph, their full unit of flow passes through the 7-8 edge. On the other hand, no flow passing between pairs of nodes that both lie in the same half uses this edge. As a result, the betweenness of the 7-8 edge is 7 · 7 = 49. • The 3-7 edge carries the full unit of flow from each node among 1, 2, and 3 to each

89 3.6. ADVANCED MATERIAL: BETWEENNESS MEASURES AND GRAPH PARTITIONING 75 10 10 1 1 9 9 3 3 1 1 1 1 2 2 7 7 8 8 12 6 12 6 4 13 4 13 14 14 5 5 Step 1 (b) Step 2 (a) 10 1 9 3 1 1 2 8 7 6 12 4 13 5 14 (c) Step 3 Figure 3.16: The steps of the Girvan-Newman method on the network from Figure 3.14(a). node among 4-14. Thus, the betweenness of this edge is 3 · 11 = 33. The same goes for the edges 6-7, 8-9, and 8-12. The 1-3 edge carries all the flow from 1 to every other node except 2. As a result, its • betweennness is 12. By strictly symmetric reasoning, the other edges linked from 3, 6, 9, and 12 into their respective triangles have betweenness 12 as well. • Finally, the 1-2 edge only carries flow between its endpoints, so its betweenness is 1. This also holds for the edges 4-5, 10-11, and 13-14. Thus, betweenness has picked out the 7-8 edge as the one carrying the most traffic. In fact, the idea of using betweenness to identify important edges draws on a long history in sociology, where most attribute its first explicit articulation to Linton Freeman [73, 168, 169]. Its use by sociologists has traditionally focused more on nodes than on edges, where the definition the same: the betweenness of a node is the total amount of flow that it carries, when a unit of flow between each pair of nodes is divided up evenly over shortest paths. Like edges of high betweenness, nodes of high betweenness occupy critical roles in the network

90 76 CHAPTER 3. STRONG AND WEAK TIES 2 2 9 9 6 6 1 4 5 4 5 1 8 7 7 1 1 1 1 8 3 3 10 10 Step 1 (a) (b) Step 2 2 2 9 9 6 6 5 4 4 1 5 1 8 7 1 7 8 1 1 1 3 3 10 10 (c) (d) Step 4 Step 3 Figure 3.17: The steps of the Girvan-Newman method on the network from Figure 3.15. structure — indeed, because carrying a large amount of flow suggests a position at the interface between tightly-knit groups, there are clear relationships of betweenness with our earlier discussions of nodes that span structural holes in a social network [86]. The Girvan-Newman Method: Successively Deleting Edges of High Betweenness. Edges of high betweenness are the ones that, over all pairs of nodes, carry the highest volume of traffic along shortest paths. Based on the premise that these are the most “vital” edges for connecting different regions of the network, it is natural to try removing these first. This is the crux of the Girvan-Newman method, which can now be summarized as follows. (1) Find the edge of highest betweenness — or multiple edges of highest betweenness, if there is a tie — and remove these edges from the graph. This may cause the graph to separate into multiple components. If so, this is the first level of regions in the partitioning of the graph. (2) Now recalculate all betweennesses, and again remove the edge or edges of highest be- tweenness. This may break some of the existing components into smaller components; if so, these are regions nested within the larger regions. (...) Proceed in this way as long as edges remain in graph, in each step recalculating all betweennesses and removing the edge or edges of highest betweenness. Thus, as the graph falls apart first into large pieces and then into smaller ones, the method naturally exposes a nested structure in the tightly-knit regions. In Figures 3.16 and 3.17

91 3.6. ADVANCED MATERIAL: BETWEENNESS MEASURES AND GRAPH PARTITIONING 77 we show how the method operates on the graphs from Figures 3.14(a) and 3.15 respectively. Note how smaller regions emerge from larger ones as edges are successively removed. The sequence of steps in Figure 3.17 in fact exposes some interesting points about how the method works. • When we calculate the betweennesses in the first step, the 5-7 edge carries all the flow from nodes 1-5 to nodes 7-11, for a betweenness of 25. The 5-6 edge, on the other hand, only carries flow from node 6 to each of nodes 1-5, for a betweenness of 5. (Similarly for the 6-7 edge.) • Once the 5-7 edge is deleted, however, we recalculate all the betweennesses for the second step. At this point, all 25 units of flow that used to be on this deleted edge have shifted onto the path through nodes 5, 6, and 7, and so the betweenness of the 5-6 edge (and also the 6-7 edge) has increased to 5 + 25 = 30. This is why these two edges are deleted next. In their original presentation of the method, Girvan and Newman showed its effectiveness at partitioning a number of real network datasets into intuitively reasonable sets of regions. For example, on Zachary’s karate club network in Figure 3.13, when the method is used to remove edges until the graph first separates into two pieces, the resulting partition agrees with the actual split that occurred in the club except for a single person — node 9 in the figure. In real life, node 9 went with the instructor’s club, even though the graph partitioning analysis here would predict that he would join the president’s club. Zachary’s original analysis of the karate club employed a different approach that also used the network structure. He first supplemented the network with numerical estimates of tie strength for the edges, based on his empirical study of the relationships within the karate club. He then identified a set of edges of minimum total strength whose removal would place node 1 and node 34 (the rival leaders) in different connected components, and he predicted this as the split. The approach Zachary used, deleting edges of minimum total strength so as to separate two specified nodes, is known as the problem of finding a minimum cut in a graph, and it has the been the subject of extensive research and applications [8, 164, 253]. On the karate-club network, this minimum-cut approach produced the same split as the Girvan- Newman method: it agreed with the split that actually occurred except for the outcome of node 9, an alignment of predictions that emphasizes how different approaches to graph partitioning can produce corresponding results. It is also interesting to note that Zachary traced the anomalous nature of node 9 to a fact that the network structure could not capture: at the time of the actual split, the person corresponding to node 9 was three weeks away from completing a four-year quest to obtain a black belt, which he could only do with the instructor (node 1).

92 78 CHAPTER 3. STRONG AND WEAK TIES A F B C D E C B I H F G A G K D I J J H E K A sample network (b) Breadth-first search starting at node (a) A Figure 3.18: The first step in the efficient method for computing betweenness values is to perform a breadth-first search of the network. Here the results of breadth-first from node A are shown; over the course of the method, breadth-first search is performed from each node in turn. Among the other examples discussed by Girvan and Newman, they provide a partition of the co-authorship network from Figure 3.12, with the top level of regions suggested by the different shadings of the nodes in that figure. Ultimately, it is a challenge to rigorously evaluate graph partitioning methods and to formulate ways of asserting that one is better than another — both because the goal is hard to formalize, and because different methods may be more or less effective on different kinds of networks. Moreover, a line of recent work by Leskovec et al. has argued that in real social- network data, it is much easier to separate a tightly-knit region from the rest of the network when it is relatively small, on the order of at most a few hundred nodes [275]. Studies on a range of different social and information networks suggest that beyond this size, sets of nodes become much more “inextricable” from the rest of the network, suggesting that graph partitioning approaches on this type of data may produce qualitatively different kinds of results for small networks and small regions than for large ones. This is an area of ongoing investigation. In the remainder of this section, we address a final important issue: how to actually compute the betweenness quantities that are needed in order to make the Girvan-Newman method work.

93 3.6. ADVANCED MATERIAL: BETWEENNESS MEASURES AND GRAPH PARTITIONING 79 B. Computing Betweenness Values In order to perform the Girvan-Newman method, we need a way to find the edges of highest betweenness in each step. This is done by computing all the betweennesses of all edges and then looking for the ones with the highest values. The tricky part is that the definition all the shortest paths between pairs of of betweenness involves reasoning about the set of nodes. Since there could be a very large number of such shortest paths, how can we efficiently compute betweenness without the overhead of actually listing out all such paths? This is crucial for implementing the method on a computer to work with datasets of any reasonable size. In fact, there is a clever way to compute betweennesses efficiently [77, 317], and it is based on the notion of breadth-first search from Section 2.3. We will consider the graph from the perspective of one node at a time; for each given node, we will compute how the total flow from that node to all others is distributed over the edges. If we do this for every node, then we can simply add up the flows from all of them to get the betweennesses on every edge. So let’s consider how we would determine the flow from one node to all other nodes in the graph. As an example, we’ll look at the graph in Figure 3.18(a), focusing on how the A flow from node reaches all other nodes. We do this in three high-level steps; below we explain the details of how each of these steps works. (1) Perform a breadth-first search of the graph, starting at . A A to each other node. (2) Determine the number of shortest paths from (3) Based on these numbers, determine the amount of flow from to all other nodes that A uses each edge. For the first step, recall that breadth-first search divides a graph into layers starting at a given node ( A in our case), with all the nodes in layer d having distance d from A . Moreover, the shortest paths from A X in layer d are precisely the paths that move downward to a node A one layer at a time, thereby taking exactly X from d steps. Figure 3.18(b) shows the to A in our graph, with the layers placed horizontally going result of breadth-first search from downward from A . Thus, for example, some inspection of the figure shows that there are two shortest paths (each of length two) from A to F : one using nodes A , B , and F , and the . other using nodes , C , and F A Counting Shortest Paths. Now, let’s consider the second step: determining the number of shortest paths from A to each other node. There is a remarkably clean way to do this, by working down through the layers of the breadth-first search.

94 80 CHAPTER 3. STRONG AND WEAK TIES A C B 1 1 1 E D 1 H 2 2 1 G F J 3 I 3 # shortest A-J paths = # shortest A-G paths + A-H paths # shortest # shortest A-I paths = 6 K # shortest A-F paths + # shortest A-G paths A-K paths # shortest A-I paths = # shortest + # shortest A-J paths Figure 3.19: The second step in computing betweenness values is to count the number of A to all other nodes in the network. This can be done shortest paths from a starting node by adding up counts of shortest paths, moving downward through the breadth-first search structure. I in Figure 3.18(b). All shortest-paths from A to To motivate this, consider a node like I must take their last step through either F or G , since these are the two nodes above it in the breadth-first search. (For terminological convenience, we will say that a node X is a node in the breadth-first search if X is in the layer immediately preceding Y , and above Y has an edge to Y X I , a path must first be a .) Moreover, in order to be a shortest path to shortest path to one of F or G , and then take this last step to I . It follows that the number of shortest paths from A to I is precisely the number of shortest paths from A to F , plus the number of shortest paths from to G . A We can use this as a general method to count the number of shortest paths from A to all other nodes, as depicted in Figure 3.19. Each node in the first layer is a neighbor of A , and so it has only one shortest path from A : the edge leading straight from A to it. So we give each of these nodes a count of 1. Now, as we move down through the BFS layers, we apply the reasoning discussed above to conclude that the number of shortest paths to

95 3.6. ADVANCED MATERIAL: BETWEENNESS MEASURES AND GRAPH PARTITIONING 81 A 4 2 2 2 B C 1 1 1 E 1 D 1 1 1 1 2 G 2 F 1 2 H 1 1/2 1/2 1 I 3 J 3 1/2 1/2 6 K Figure 3.20: The final step in computing betweenness values is to determine the flow values from a starting node A to all other nodes in the network. This is done by working up from the lowest layers of the breadth-first search, dividing up the flow above a node in proportion to the number of shortest paths coming into it on each edge. sum each node should be the of the number of shortest paths to all nodes directly above it in the breadth-first search. Working downward through the layers, we thus get the number of shortest paths to each node, as shown in Figure 3.19. Note that by the time we get to deeper layers, it may not be so easy to determine these number by visual inspection — for A example, to immediately list the six different shortest paths from K — but it is quite to easy when they are built up layer-by-layer in this way. Determining Flow Values. Finally, we come to the third step, computing how the flow from A to all other nodes spreads out across the edges. Here too we use the breadth-first search structure, but this time working up from the lowest layers. We first show the idea in Figure 3.20 on our running example, and then describe the general procedure. • Let’s start at the bottom with node K . A single unit of flow arrives at K , and an equal , so this unit A to K come through nodes I and J number of the shortest paths from

96 82 CHAPTER 3. STRONG AND WEAK TIES of flow is equally divided over the two incoming edges. Therefore we put a half-unit of flow on each of these edges. I is equal to the one unit • Now, working upward, the total amount of flow arriving at plus the half-unit passing through to / , for a total of 3 I 2. How actually destined for K 2 amount of flow get divided over the edges leading upward from I , to F does this 3 / G and respectively? We see from the second step that there are twice as many shortest paths from through F as through G , so twice as much of the flow should come from A . Therefore, we put one unit of the flow on F , and a half-unit of the flow on G , as F indicated in the figure. • We continue in this way for each other node, working upward through the layers of the breadth-first search. From this, it is not hard to describe the principle in general. When we get to a node X in the breadth-first search structure, working up from the bottom, we add up all the flow arriving from edges directly below X X itself. We then , plus 1 for the flow destined for X in proportion to the number of shortest divide this up over the edges leading upward from , You can check that applying this principle leads to the numbers paths coming through each. shown in Figure 3.20. We are now essentially done. We build one of these breadth-first structures from each node in the network, determine flow values from the node using this procedure, and then sum up the flow values to get the betweenness value for each edge. Notice that we are counting the flow between each pair of nodes X and Y twice: once when we do the breadth-first search from , and once when we do it from Y . So at the end we divide everything by two to X cancel out this double-counting. Finally, using these betweenness values, we can identify the edges of highest betweenness for purposes of removing them in the Girvan-Newman method. Final Observations. The method we have just described can be used to compute the betweennesses of nodes as well as edges. In fact, this is already happening in the third step: notice that we are implicitly keeping track of the amounts of flow through the nodes as well as through the edges, and this is what is needed to determine the betweennesses of the nodes. The original Girvan-Newman method described here, based on repeated removal of high- betweenness edges, is a good conceptual way to think about graph partitioning, and it works well on networks of moderate size (up to a few thousand nodes). However, for larger networks, the need to recompute betweenness values in every step becomes computationally very expensive. In view of this, a range of different alternatives have been proposed to identify similar sets of tightly-knit regions more efficiently. These include methods of approximating the betweenness [34] and related but more efficient graph partitioning approaches using

97 3.7. EXERCISES 83 divisive and agglomerative methods [35, 321]. There remains considerable interest in finding fast partitioning algorithms that can scale to very large network datasets. 3.7 Exercises 1. In 2-3 sentences, explain what triadic closure is, and how it plays a role in the formation of social networks. You can draw a schematic picture in case this is useful. 2. Consider the graph in Figure 3.21, in which each edge — except the edge connecting c — is labeled as a strong tie (S) or a weak tie (W). b and According to the theory of strong and weak ties, with the strong triadic closure as- b and c to be labeled? Give a sumption, how would you expect the edge connecting brief (1-3 sentence) explanation for your answer. W a d S S e W f W S S b c ? Figure 3.21: 3. In the social network depicted in Figure 3.22, with each edge labeled as either a strong or weak tie, which nodes satisfy the Strong Triadic Closure Property from Chapter 3, and which do not? Provide an explanation for your answer. 4. In the social network depicted in Figure 3.23 with each edge labeled as either a strong or weak tie, which two nodes violate the Strong Triadic Closure Property? Provide an explanation for your answer. 5. In the social network depicted in Figure 3.24, with each edge labeled as either a strong or weak tie, which nodes satisfy the Strong Triadic Closure Property from Chapter 3, and which do not? Provide an explanation for your answer.

98 84 CHAPTER 3. STRONG AND WEAK TIES S B C W S W S A S W S E D Figure 3.22: S C B S W S W A W S S D E Figure 3.23: A graph with a strong/weak labeling. A S S S C B W S W D E Figure 3.24:

99 Chapter 4 Networks in Their Surrounding Contexts In Chapter 3 we considered some of the typical structures that characterize social net- works, and some of the typical processes that affect the formation of links in the network. Our discussion there focused primarily on the network as an object of study in itself, relatively independent of the broader world in which it exists. However, the contexts in which a social network is embedded will generally have signif- icant effects on its structure, Each individual in a social network has a distinctive set of personal characteristics, and similarities and compatibilities among two people’s characteris- tics can strongly influence whether a link forms between them. Each individual also engages in a set of behaviors and activities that can shape the formation of links within the network. These considerations suggest what we mean by a network’s surrounding contexts : factors that exist outside the nodes and edges of a network, but which nonetheless affect how the network’s structure evolves. In this chapter we consider how such effects operate, and what they imply about the structure of social networks. Among other observations, we will find that the surrounding contexts affecting a network’s formation can, to some extent, be viewed in network terms as well — and by expanding the network to represent the contexts together with the individuals, we will see in fact that several different processes of network formation can be described in a common framework. D. Easley and J. Kleinberg. Networks, Crowds, and Markets: Reasoning about a Highly Connected World . Cambridge University Press, 2010. Draft version: June 10, 2010. 85

100 86 CHAPTER 4. NETWORKS IN THEIR SURROUNDING CONTEXTS 4.1 Homophily One of the most basic notions governing the structure of social networks is — the homophily principle that we tend to be similar to our friends. Typically, your friends don’t look like a random sample of the underlying population: viewed collectively, your friends are generally similar to you along racial and ethnic dimensions; they are similar in age; and they are also similar in characteristics that are more or less mutable, including the places they live, their occupations, their levels of affluence, and their interests, beliefs, and opinions. Clearly most of us have specific friendships that cross all these boundaries; but in aggregate, the pervasive fact is that links in a social network tend to connect people who are similar to one another. This observation has a long history; as McPherson, Smith-Lovin, and Cook note in their extensive review of research on homophily [294], the underlying idea can be found in writings of Plato (“similarity begets friendship”) and Aristotle (people “love those who are like themselves”), as well as in proverbs such as “birds of a feather flock together.” Its role in modern sociological research was catalyzed in large part by influential work of Lazarsfeld and Merton in the 1950s [269]. Homophily provides us with a first, fundamental illustration of how a network’s sur- rounding contexts can drive the formation of its links. Consider the basic contrast between a friendship that forms because two people are introduced through a common friend and a friendship that forms because two people attend the same school or work for the same company. In the first case, a new link is added for reasons that are to the network intrinsic itself; we need not look beyond the network to understand where the link came from. In the second case, the new link arises for an equally natural reason, but one that makes sense only when we look at the contextual factors beyond the network — at some of the social environments (in this case schools and companies) to which the nodes belong. Often, when we look at a network, such contexts capture some of the dominant fea- tures of its overall structure. Figure 4.1, for example, depicts the social network within a particular town’s middle school and high school (encompassing grades 7-12) [304]; in this image, produced by the study’s author James Moody, students of different races are drawn as differently-colored circles. Two dominant divisions within the network are apparent. One division is based on race (from left to right in the figure); the other, based on age and school attendance, separates students in the middle school from those in the high school (from top to bottom in the figure). There are many other structural details in this network, but the effects of these two contexts stand out when the network is viewed at a global level. Of course, there are strong interactions between intrinsic and contextual effects on the formation of any single link; they are both operating concurrently in the same network. For example, the principle of triadic closure — that triangles in the network tend to “close” as links form between friends of friends — is supported by a range of mechanisms that range from the intrinsic to the contextual. In Chapter 3 we motivated triadic closure by

101 4.1. HOMOPHILY 87 Homophily can produce a division of a social network into densely-connected, homogeneous Figure 4.1: parts that are weakly connected to each other. In this social network from a town’s middle school and high school, two such divisions in the network are apparent: one based on race (with students of different races drawn as differently colored circles), and the other based on friendships in the middle and high schools respectively [304]. B and hypothesizing intrinsic mechanisms: when individuals have a common friend A , C then there are increased opportunities and sources of trust on which to base their interactions, and A will also have incentives to facilitate their friendship. However, social contexts also provide natural bases for triadic closure: since we know that A - B and A - C friendships already exist, the principle of homophily suggests that and C are each likely to be similar B to A in a number of dimensions, and hence quite possibly similar to each other as well. As a result, based purely on this similarity, there is an elevated chance that a B - C friendship will form; and this is true even if neither of them is aware that the other one knows A . The point isn’t that any one basis for triadic closure is the “correct” one. Rather, as we take into account more and more of the factors that drive the formation of links in a social

102 88 CHAPTER 4. NETWORKS IN THEIR SURROUNDING CONTEXTS Figure 4.2: Using a numerical measure, one can determine whether small networks such as this one (with nodes divided into two types) exhibit homophily. network, it inevitably becomes difficult to attribute any individual link to a single factor. And ultimately, one expects most links to in fact arise from a combination of several factors — partly due to the effect of other nodes in the network, and partly due to the surrounding contexts. When we see striking divisions within a network like the one in Measuring Homophily. Figure 4.1, it is important to ask whether they are “genuinely” present in the network itself, and not simply an artifact of how it is drawn. To make this question concrete, we need to formulate it more precisely: given a particular characteristic of interest (like race, or age), is there a simple test we can apply to a network in order to estimate whether it exhibits homophily according to this characteristic? Since the example in Figure 4.1 is too large to inspect by hand, let’s consider this question on a smaller example where we can develop some intuition. Let’s suppose in particular that we have the friendship network of an elementary-school classroom, and we suspect that it exhibits homophily by gender: boys tend to be friends with boys, and girls tend to be friends with girls. For example, the graph in Figure 4.2 shows the friendship network of a (small) hypothetical classroom in which the three shaded nodes are girls and the six unshaded nodes are boys. If there were no cross-gender edges at all, then the question of homophily would be easy to resolve: it would be present in an extreme sense. But we expect that homophily should be a more subtle effect that is visible mainly in aggregate — as it is, for example, in the real data from Figure 4.1. Is the picture in Figure 4.2 consistent with homophily? There is a natural numerical measure of homophily that we can use to address questions

103 4.1. HOMOPHILY 89 like this [202, 319]. To motivate the measure (using the example of gender as in Figure 4.2), we first ask the following question: what would it mean for a network not to exhibit ho- mophily by gender? It would mean that the proportion of male and female friends a person has looks like the background male/female distribution in the full population. Here’s a closely related formulation of this “no-homophily” definition that is a bit easier to analyze: if we were to randomly assign each node a gender according to the gender balance in the real network, then the number of cross-gender edges should not change significantly relative to what we see in the real network. That is, in a network with no homophily, friendships are being formed as though there were random mixing across the given characteristic. fraction of all individuals are male, and Thus, suppose we have a network in which a p a q fraction of all individuals are female. Consider a given edge in this network. If we male with probability independently assign each node the gender and the gender female p 2 , then both ends of the edge will be male with probability p with probability q , and both 2 ends will be female with probability . On the other hand, if the first end of the edge is q male and the second end is female, or vice versa, then we have a cross-gender edge, so this pq . happens with probability 2 So we can summarize the test for homophily according to gender as follows: Homophily Test: If the fraction of cross-gender edges is significantly less than 2 pq , then there is evidence for homophily. In Figure 4.2, for example, 5 of the 18 edges in the graph are cross-gender. Since p / 3 = 2 q / 3 in this example, we should be comparing the fraction of cross-gender edges to and = 1 pq = 4 the quantity 2 9 = 8 / 18. In other words, with no homophily, one should expect to / see 8 cross-gender edges rather than than 5, and so this example shows some evidence of homophily. There are a few points to note here. First, the number of cross-gender edges in a random assignment of genders will deviate some amount from its expected value of 2 pq , and so to perform the test in practice one needs a working definition of “significantly less than.” Standard measures of statistical significance (quantifying the significance of a deviation below a mean) can be used for this purpose. Second, it’s also easily possible for a network to have a fraction of cross-gender edges that is significantly more than 2 pq . In such a case, we say that the network exhibits inverse homophily. The network of romantic relationships in Figure 2.7 from Chapter 2 is a clear example of this; almost all the relationships reported by the high- school students in the study involved opposite-sex partners, rather than same-sex partners, so almost all the edges are cross-gender. Finally, it’s easy to extend our homophily test to any underlying characteristic (race, ethnicity, age, native language, political orientation, and so forth). When the characteristic can only take two possible values (say, one’s voting preference in a two-candidate election), then we can draw a direct analogy to the case of two genders, and use the same formula

104 90 CHAPTER 4. NETWORKS IN THEIR SURROUNDING CONTEXTS pq . When the characteristic can take on more than two possible values, we still perform a 2 if it general version of the same calculation. For this, we say that an edge is heterogeneous connects two nodes that are different according to the characteristic in question. We then ask how the number of heterogeneous edges compares to what we’d see if we were to randomly assign values for the characteristic to all nodes in the network — using the proportions from the real data as probabilities. In this way, even a network in which the nodes are classified into many groups can be tested for homophily using the same underlying comparison to a baseline of random mixing. 4.2 Mechanisms Underlying Homophily: Selection and Social Influence The fact that people tend to have links to others who are similar to them is a statement about the structure of social networks; on its own, it does not propose an underlying mechanism by which ties among similar people are preferentially formed. In the case of immutable characteristics such as race or ethnicity, the tendency of people to form friendships with others who are like them is often termed selection , in that people are selecting friends with similar characteristics. Selection may operate at several different scales, and with different levels of intentionality. In a small group, when people choose friends who are most similar from among a clearly delineated pool of potential contacts, there is clearly active choice going on. In other cases, and at more global levels, selection can be more implicit. For example, when people live in neighborhoods, attend schools, or work for companies that are relatively homogeneous compared to the population at large, the social environment is already favoring opportunities to form friendships with others like oneself. For this discussion, we will refer to all these effects cumulatively as selection. When we consider how immutable characteristics interact with network formation, the order of events is clear: a person’s attributes are determined at birth, and they play a role in how this person’s connections are formed over the course of his or her life. With characteristics that are more mutable, on the other hand — behaviors, activities, interests, beliefs, and opinions — the feedback effects between people’s individual characteristics and their links in the social network become significantly more complex. The process of selection still operates, with individual characteristics affecting the connections that are formed. But now another process comes into play as well: people may modify their behaviors to bring them more closely into alignment with the behaviors of their friends. This process has been variously described as socialization [233] and social influence [170], since the existing social connections in a network are influencing the individual characteristics of the nodes. Social influence can be viewed as the reverse of selection: with selection, the individual characteristics drive the formation of links, while with social influence, the existing links in

105 4.2. MECHANISMS UNDERLYING HOMOPHILY: SELECTION AND SOCIAL INFLUENCE 91 1 the network serve to shape people’s (mutable) characteristics. When we look at a single snapshot The Interplay of Selection and Social Influence. of a network and see that people tend to share mutable characteristics with their friends, it can be very hard to sort out the distinct effects and relative contributions of selection and social influence. Have the people in the network adapted their behaviors to become more like their friends, or have they sought out people who were already like them? Such questions can longitudinal studies of a social network, in which the social connections be addressed using and the behaviors within a group are both tracked over a period of time. Fundamentally, this makes it possible to see the behavioral changes that occur after changes in an individual’s network connections, as opposed to the changes to the network that occur after an individual changes his or her behavior. This type of methodology has been used, for example, to study the processes that lead pairs of adolescent friends to have similar outcomes in terms of scholastic achievement and delinquent behavior such as drug use [92]. Empirical evidence confirms the intuitive fact that teenage friends are similar to each other in their behaviors, and both selection and social influence have a natural resonance in this setting: teenagers seek out social circles composed of people like them, and peer pressure causes them to conform to behavioral patterns within their social circles. What is much harder to resolve is how these two effects interact, and whether one is more strongly at work than the other. As longitudinal behavior relevant to this question became available, researchers began quantifying the relative impact of these different factors. A line of work beginning with Cohen and Kandel has suggested that while both effects are present in the data, the outsized role that earlier informal arguments had accorded to peer pressure (i.e. social influence) is actually more moderate; the effect of selection here is in fact comparable to (and sometimes greater than) the effect of social influence [114, 233]. Understanding the tension between these different forces can be important not just for identifying underlying causes, but also for reasoning about the effect of possible interventions one might attempt in the system [21, 396]. For example, once we find that illicit drug use displays homophily across a social network — with students showing a greater likelihood to use drugs when their friends do — we can ask about the effects of a program that targets certain high-school students and influences them to stop using drugs. To the extent that the observed homophily is based on some amount of social influence, such a program could have a broad impact across the social network, by causing the friends of these targeted students to stop using drugs as well. But one must be careful; if the observed homophily is arising instead almost entirely from selection effects, then the program may not reduce drug use 1 There are other cognitive effects at work as well; for example, people may systematically misperceive the characteristics of their friends as being more in alignment with their own than they really are [224]. For our discussion here, we will not focus explicitly on such effects.

106 92 CHAPTER 4. NETWORKS IN THEIR SURROUNDING CONTEXTS beyond the students it directly targets: as these students stop using drugs, they change their social circles and form new friendships with students who don’t use drugs, but the drug-using behavior of other students is not strongly affected. Another example of research addressing this subtle interplay of factors is the work of Christakis and Fowler on the effect of social networks on health-related outcomes. In one recent study, using longitudinal data covering roughly 12,000 people, they tracked obesity status and social network structure over a 32-year period [108]. They found that obese and non-obese people clustered in the network in a fashion consistent with homophily, according to the numerical measure described in Section 4.1: people tend to be more similar in obesity status to their network neighbors than in a version of the same network where obesity status is assigned randomly. The problem is then to distinguish among several hypotheses for why this clustering is present: is it (i) because of selection effects, in which people are choosing to form friendships with others of similar obesity status? (ii) because of the confounding effects of homophily according to other characteristics, in which the network structure indicates existing patterns of similarity in other dimensions that correlate with obesity status? or (iii) because changes in the obesity status of a person’s friends was exerting a (presumably behavioral) influence that affected his or her future obesity status? Statistical analysis in Christakis and Fowler’s paper argues that, even accounting for effects of types (i) and (ii), there is significant evidence for an effect of type (iii) as well: that obesity is a health condition displaying a form of social influence, with changes in your friends’ obesity status in turn having a subsequent effect on you. This suggests the intriguing prospect that obesity (and perhaps other health conditions with a strong behavioral aspect) may exhibit some amount of “contagion” in a social sense: you don’t necessarily catch it from your friends the way you catch the flu, but it nonetheless can spread through the underlying social network via the mechanism of social influence. These examples, and this general style of investigation, show how careful analysis is needed to distinguish among different factors contributing to an aggregate conclusion: even when people tend to be similar to their neighbors in a social network, it may not be clear why. The point is that an observation of homophily is often not an endpoint in itself, but rather the starting point for deeper questions — questions that address why the homophily is present, how its underlying mechanisms will affect the further evolution of the network, and how these mechanisms interact with possible outside attempts to influence the behavior of people in the network.

107 4.3. AFFILIATION 93 Literacy Anna V olunteers Karate Daniel Club Figure 4.3: An affiliation network is a bipartite graph that shows which individuals are affiliated with which groups or activities. Here, Anna participates in both of the social foci on the right, while Daniel participates in only one. 4.3 Affiliation Thus far, we have been discussing contextual factors that affect the formation of links in a network — based on similarities in characteristics of the nodes, and based on behaviors and activities that the nodes engage in. These surrounding contexts have been viewed, appropriately, as existing “outside” the network. But in fact, it’s possible to put these contexts into the network itself, by working with a larger network that contains both people and contexts as nodes. Through such a network formulation, we will get additional insight into some broad aspects of homophily, and see how the simultaneous evolution of contexts and friendships can be put on a common network footing with the notion of triadic closure from Chapter 3. In principle we could represent any context this way, but for the sake of concreteness we’ll focus on how to represent the set of activities a person takes part in, and how these affect the formation of links. We will take a very general view of the notion of an “activity” here. Being part of a particular company, organization, or neigborhood; frequenting a particular place; pursuing a particular hobby or interest — these are all activities that, when shared between two people, tend to increase the likelihood that they will interact and hence form a link in the social network [78, 161]. Adopting terminology due to Scott Feld, we’ll refer to such activities as foci — that is, “focal points” of social interaction — constituting “social, psychological, legal, or physical entit[ies] around which joint activities are organized (e.g. workplaces, voluntary organizations, hangouts, etc.)” [161]. Affiliation Networks. As a first step, we can represent the participation of a set of people in a set of foci using a graph as follows. We will have a node for each person, and a node for each focus, and we will connect person A to focus X by an edge if A participates in X .

108 CHAPTER 4. NETWORKS IN THEIR SURROUNDING CONTEXTS 94 John Doerr Amazon Shirley ilghman T Google Arthur Levinson Apple Al Gore Steve Disney Jobs Andrea Jung General Electric Susan eld Hockfi Figure 4.4: One type of affiliation network that has been widely studied is the memberships of people on corporate boards of directors [301]. A very small portion of this network (as of mid-2009) is shown here. The structural pattern of memberships can reveal subtleties in the interactions among both the board members and the companies. A very simple example of such a graph is depicted in Figure 4.3, showing two people (Anna and Daniel) and two foci (working for a literacy tutoring organization, and belonging to a karate club). The graph indicates that Anna participates in both of the foci, while Daniel participates in only one. We will refer to such a graph as an , since it represents the affiliation of affiliation network people (drawn on the left) with foci (drawn on the right) [78, 323]. More generally, affiliation networks are examples of a class of graphs called bipartite graphs . We say that a graph is bipartite if its nodes can be divided into two sets in such a way that every edge connects a node in one set to a node in the other set. (In other words, there are no edges joining a pair of nodes that belong to the same set; all edges go between the two sets.) Bipartite graphs are very useful for representing data in which the items under study come in two categories, and we want to understand how the items in one category are associated with the items in the other. In the case of affiliation networks, the two categories are the people and the foci, with each edge connecting a person to a focus that he or she participates in. Bipartite

109 4.3. AFFILIATION 95 graphs are often drawn as in Figure 4.3, with the two different sets of nodes drawn as two parallel vertical columns, and the edges crossing between the two columns. Affiliation networks are studied in a range of settings where researchers want to un- derstand the patterns of participation in structured activities. As one example, they have received considerable attention in studying the composition of boards of directors of major corporations [301]. Boards of directors are relatively small advisory groups populated by high-status individuals; and since many people serve on multiple boards, the overlaps in their participation have a complex structure. These overlaps can be naturally represented by an affiliation network; as the example in Figure 4.4 shows, there is a node for each person and a node for each board, and each edge connects a person to a board that they belong to. Affiliation networks defined by boards of directors have the potential to reveal interesting relationships on both sides of the graph. Two companies are implicitly linked by having the same person sit on both their boards; we can thus learn about possible conduits for information and influence to flow between different companies. Two people, on the other hand, are implicitly linked by serving together on a board, and so we learn about particular patterns of social interaction among some of the most powerful members of society. Of course, even the complete affiliation network of people and boards (of which Figure 4.4 is only a small piece) still misses other important contexts that these people inhabit; for example, the seven people in Figure 4.4 include the presidents of two major universities and 2 a former Vice-President of the United States. Co-Evolution of Social and Affiliation Networks. It’s clear that both social networks and affiliation networks change over time: new friendship links are formed, and people become associated with new foci. Moreover, these changes represent a kind of co-evolution that reflects the interplay between selection and social influence: if two people participate in a shared focus, this provides them with an opportunity to become friends; and if two people are friends, they can influence each other’s choice of foci. There is a natural network perspective on these ideas, which begins from a network representation that slightly extends the notion of an affiliation network. As before, we’ll have nodes for people and nodes for foci, but we now introduce two distinct kinds of edges as well. The first kind of edge functions as an edge in a social network: it connects two 2 The structure of this network changes over time as well, and sometimes in ways that reinforce the points in our present discussion. For example, the board memberships shown in Figure 4.4 are taken from the middle of 2009; by the end of 2009, Arthur Levinson had resigned from the board of directors of Google (thus removing one edge from the graph). As part of the news coverage of this resignation, the chair of the U.S. Federal Trade Commission, Jon Leibowitz, explicitly invoked the notion of overlaps in board membership, saying, “Google, Apple and Mr. Levinson should be commended for recognizing that overlapping board members between competing companies raise serious antitrust issues, and for their willingness to resolve our concerns without the need for litigation. Beyond this matter, we will continue to monitor companies that share board members and take enforcement actions where appropriate” [219].

110 96 CHAPTER 4. NETWORKS IN THEIR SURROUNDING CONTEXTS Claire Literacy Anna Bob olunteers V Karate Daniel Club Figure 4.5: A social-affiliation network shows both the friendships between people and their affiliation with different social foci. people, and indicates friendship (or alternatively some other social relation, like professional collaboration). The second kind of edge functions as an edge in an affiliation network: it connects a person to a focus, and indicates the participation of the person in the focus. We will call such a network a social-affiliation network , reflecting the fact that it simultaneously contains a social network on the people and an affiliation network on the people and foci. Figure 4.5 depicts a simple social-affiliation network. Once we have social-affiliation networks as our representation, we can appreciate that a range of different mechanisms for link formation can all be viewed as types of closure , in that they involve “closing” the third edge of a triangle in the network. In processes B and C with a common neighbor A particular, suppose we have two nodes in the network, and suppose that an edge forms between B and C . There are several interpretations for what this corresponds to, depending on whether A , B , and C are people or foci. (i) If A B , and C each represent a person, then the formation of the link between B and , C is triadic closure, just as in Chapter 3. (See Figure 4.6(a).) (ii) If B and C represent people, but A represents a focus, then this is something different: it is the tendency of two people to form a link when they have a focus in common. (See Figure 4.6(b).) This is an aspect of the more general principle of selection, forming links to others who share characteristics with you. To emphasize the analogy with focal closure [259]. triadic closure, this process has been called (iii) If A and B are people, and C is a focus, then we have the formation of a new affiliation: B takes part in a focus that her friend A is already involved in. (See Figure 4.6(c).) This is a kind of social influence, in which B ’s behavior comes into closer alignment

111 4.4. TRACKING LINK FORMATION IN ON-LINE DATA 97 B B C C person person person person A A focus person (a) (b) Focal closure Triadic closure B C person focus A person Membership closure (c) Figure 4.6: Each of triadic closure, focal closure, and membership closure corresponds to the closing of a triangle in a social-affiliation network. with that of her friend A . Continuing the analogy with triadic closure, we will refer to this kind of link formation as membership closure . Thus, three very different underlying mechanisms — reflecting triadic closure and aspects of selection and social influence — can be unified in this type of network as kinds of closure: the formation of a link in cases where the two endpoints already have a neighbor in common. Figure 4.7 shows all three kinds of closure processes at work: triadic closure leads to a new link between Anna and Claire; focal closure leads to a new link between Anna and Daniel; and membership closure leads to Bob’s affiliation with the karate club. Oversimplifying the mechanisms at work, they can be summarized in the following succinct way: (i) Bob introduces Anna to Claire. (ii) Karate introduces Anna to Daniel. (iii) Anna introduces Bob to Karate. 4.4 Tracking Link Formation in On-Line Data In this chapter and the previous one, we have identified a set of different mechanisms that lead to the formation of links in social networks. These mechansisms are good examples

112 98 CHAPTER 4. NETWORKS IN THEIR SURROUNDING CONTEXTS Claire Literacy Bob Anna V olunteers Karate Daniel Club Figure 4.7: In a social-affiliation network containing both people and foci, edges can form under the effect of several different kinds of closure processes: two people with a friend in common, two people with a focus in common, or a person joining a focus that a friend is already involved in. of social phenomena which are clearly at work in small-group settings, but which have traditionally been very hard to measure quantitatively. A natural research strategy is to try tracking these mechanisms as they operate in large populations, where an accumulation of many small effects can produce something observable in the aggregate. However, given that most of the forces responsible for link formation go largely unrecorded in everyday life, it is a challenge to select a large, clearly delineated group of people (and social foci), and accurately quantify the relative contributions that these different mechanisms make to the formation of real network links. The availability of data from large on-line settings with clear social structure has made it possible to attempt some preliminary research along these lines. As we emphasized in Chapter 2, any analysis of social processes based on such on-line datasets must come with a number of caveats. In particular, it is never a priori clear how much one can extrapolate from digital interactions to interactions that are not computer-mediated, or even from one computer-mediated setting to another. Of course, this problem of extrapolation is present whenever one studies phenomena in a model system, on-line or not, and the kinds of mea- surements these large datasets enable represent interesting first steps toward a deeper quan- titative understanding of how mechanisms of link formation operate in real life. Exploring these questions in a broader range of large datasets is an important problem, and one that will become easier as large-scale data becomes increasingly abundant. Triadic closure. With this background in mind, let’s start with some questions about triadic closure. Here’s a first, basic numerical question: how much more likely is a link to

113 4.4. TRACKING LINK FORMATION IN ON-LINE DATA 99 Grace Claire Literacy Bob Anna Esther V olunteers Karate Daniel Club Frank Figure 4.8: A larger network that contains the example from Figure 4.7. Pairs of people can have more than one friend (or more than one focus) in common; how does this increase the likelihood that an edge will form between them? form between two people in a social network if they already have a friend in common? (In other words, how much more likely is a link to form if it has the effect of closing a triangle?) Here’s a second question, along the same lines as the first: How much more likely is an edge to form between two people if they have multiple friends in common? For example, in Figure 4.8, Anna and Esther have two friends in common, while Claire and Daniel only have one friend in common. How much more likely is the formation of a link in the first of these two cases? If we go back to the arguments for why triadic closure operates in social networks, we see that they all are qualitatively strengthened as two people have more friends in common: there are more sources of opportunity and trust for the interaction, there are more people with an incentive to bring them together, and the evidence for homophily is arguably stronger. We can address these questions empirically using network data as follows. (i) We take two snapshots of the network at different times. (ii) For each k , we identify all pairs of nodes who have exactly k friends in common in the first snapshot, but who are not directly connected by an edge. (iii) We define T ( k ) to be the fraction of these pairs that have formed an edge by the time

114 100 CHAPTER 4. NETWORKS IN THEIR SURROUNDING CONTEXTS 0.006 0.005 0.004 0.003 0.002 prob. of link formation 0.001 0 10 0 2 8 6 4 Number of common friends Figure 4.9: Quantifying the effects of triadic closure in an e-mail dataset [259]. The curve determined from the data is shown in the solid black line; the dotted curves show a compar- ison to probabilities computed according to two simple baseline models in which common friends provide independent probabilities of link formation. of the second snapshot. This is our empirical estimate for the probability that a link k friends in common. will form between two people with T (iv) We plot k ) as a function of k to illustrate the effect of common friends on the ( formation of links. Note that T (0) is the rate at which link formation happens when it does not close a triangle, determine the rate at which link formation happens while the values of k ) for larger k ( T when it does close a triangle. Thus, the comparison between T (0) and these other values addresses the most basic question about the power of triadic closure. Kossinets and Watts computed this function T ( k ) using a dataset encoding the full history of e-mail communication among roughly 22,000 undergraduate and graduate students over a one-year period at a large U.S. university [259]. This is a “who-talks-to-whom” type of dataset, as we discussed in Chapter 2; from the communication traces, Kossinets and Watts constructed a network that evolved over time, joining two people by a link at a given instant if they had exchanged e-mail in each direction at some point in the past 60 days. They then determined an “average” version of T ( k ) by taking multiple pairs of snapshots: they built a curve for T ( k ) on each pair of snapshots using the procedure described above, and then

115 4.4. TRACKING LINK FORMATION IN ON-LINE DATA 101 averaged all the curves they obtained. In particular, the observations in each snapshot were one day apart, so their computation gives the average probability that two people form a link per day, as a function of the number of common friends they have. Figure 4.9 shows a plot of this curve (in the solid black line). The first thing one notices is the clear evidence for triadic closure: T (0) is very close to 0, after which the probability of link formation increases steadily as the number of common friends increases. Moreover, for much of the plot, this probability increases in a roughly linear fashion as a function of the number of common friends, with an upward bend away from a straight-line shape. The curve turns upward in a particularly pronounced way from 0 to 1 to 2 friends: having two common friends produces significantly more than twice the effect on link formation compared to having a single common friend. (The upward effect from 8 to 9 to 10 friends is also significant, but it occurs on a much smaller sub-population, since many fewer people in the data have this many friends in common without having already formed a link.) To interpret this plot more deeply, it helps to compare it to an intentionally simplified baseline model, describing what one might have expected the data to look like in the presence p , each common friend that two of triadic closure. Suppose that for some small probability people have gives them an independent probability p of forming a link each day. So if two people have k friends in common, the probability they fail to form a link on any given day is k p ) (1 : this is because each common friend fails to cause the link to form with probability − k − p , and these k trials are independent. Since (1 − p ) 1 is the probability the link fails to form on a given day, the probability that it does form, according to our simple baseline model, is k ( k ) = 1 − (1 − T ) p . baseline We plot this curve in Figure 4.9 as the upper dotted line. Given the small absolute effect of k − 1 − p ) the first common friend in the data, we also show a comparison to the curve 1 − (1 , which just shifts the simple baseline curve one unit to the right. Again, the point is not to propose this baseline as an explanatory mechanism for triadic closure, but rather to look at how the real data compares to it. Both the real curve and the baseline curve are close to linear, and hence qualitatively similar; but the fact that the real data turns upward while the baseline curve turns slightly downward indicates that the assumption of independent effects from common friends is too simple to be fully supported by the data. A still larger and more detailed study of these effects was conducted by Leskovec et al. [272], who analyzed properties of triadic closure in the on-line social networks of LinkedIn, Flickr, Del.icio.us, and Yahoo! Answers. It remains an interesting question to try under- standing the similarities and variations in triadic closure effects across social interaction in a range of different settings.

116 102 CHAPTER 4. NETWORKS IN THEIR SURROUNDING CONTEXTS 0.0005 0.0004 0.0003 0.0002 prob. of link formation 0.0001 0 1 2 3 5 0 4 number of common foci Figure 4.10: Quantifying the effects of focal closure in an e-mail dataset [259]. Again, the curve determined from the data is shown in the solid black line, while the dotted curve provides a comparison to a simple baseline. Focal and Membership Closure. Using the same approach, we can compute probabil- ities for the other kinds of closure discussed earlier — specifically, focal closure: what is the probability that two people form a link as a function of the • number of foci they are jointly affiliated with? • membership closure: what is the probability that a person becomes involved with a particular focus as a function of the number of friends who are already involved in it? As an example of the first of these kinds of closure, using Figure 4.8, Anna and Grace have one activity in common while Anna and Frank have two in common. As an example of the second, Esther has one friend who belongs to the karate club while Claire has two. How do these distinctions affect the formation of new links? For focal closure, Kossinets and Watts supplemented their university e-mail dataset with information about the class schedules for each student. In this way, each class became a focus, and two students shared a focus if they had taken a class together. They could then compute the probability of focal closure by direct analogy with their computation for triadic closure, determining the probability of link formation per day as a function of the number of shared foci. Figure 4.10 shows a plot of this function. A single shared class turns out to have roughly the same absolute effect on link formation as a single shared friend, but after this the

117 4.4. TRACKING LINK FORMATION IN ON-LINE DATA 103 Probability of joining a community when k friends are already members 0.025 0.02 0.015 probability 0.01 0.005 0 15 20 25 30 0 40 45 50 5 10 35 k Figure 4.11: Quantifying the effects of membership closure in a large online dataset: The plot shows the probability of joining a LiveJournal community as a function of the number of friends who are already members [32]. curve for focal closure behaves quite differently from the curve for triadic closure: it turns downward and appears to approximately level off, rather than turning slightly upward. Thus, subsequent shared classes after the first produce a “diminishing returns” effect. Comparing to the same kind of baseline, in which the probability of link formation with shared classes k k − (1 is 1 p ) (shown as the dotted curve in Figure 4.10), we see that the real data turns − downward more significantly than this independent model. Again, it is an interesting open question to understand how this effect generalizes to other types of shared foci, and to other domains. For membership closure, the analogous quantities have been measured in other on-line domains that possess both person-to-person interactions and person-to-focus affiliations. Figure 4.11 is based on the blogging site LiveJournal, where friendships are designated by users in their profiles, and where foci correspond to membership in user-defined communities [32]; thus the plot shows the probability of joining a community as a function of the number of friends who have already done so. Figure 4.12 shows a similar analysis for Wikipedia [122]. Here, the social-affiliation network contains a node for each Wikipedia editor who maintains a user account and user talk page on the system; and there is an edge joining two such editors if they have communicated, with one editor writing on the user talk page of the other. Each

118 104 CHAPTER 4. NETWORKS IN THEIR SURROUNDING CONTEXTS Figure 4.12: Quantifying the effects of membership closure in a large online dataset: The plot shows the probability of editing a Wikipedia articles as a function of the number of friends who have already done so [122]. Wikipedia article defines a focus — an editor is associated with a focus corresponding to a particular article if he or she has edited the article. Thus, the plot in Figure 4.12 shows the probability a person edits a Wikipedia article as a function of the number of prior editors with whom he or she has communicated. As with triadic and focal closure, the probabilities in both Figure 4.11 and 4.12 increase with the number k of common neighbors — representing friends associated with the foci. The marginal effect diminishes as the number of friends increases, but the effect of subsequent friends remains significant. Moreover, in both sources of data, there is an initial increasing effect similar to what we saw with triadic closure: in this case, the probability of joining a LiveJournal community or editing a Wikipedia article is more than twice as great when you have two connections into the focus rather than one. In other words, the connection to a second person in the focus has a particularly pronounced effect, and after this the diminishing marginal effect of connections to further people takes over. Of course, multiple effects can operate simultaneously on the formation of a single link. For example, if we consider the example in Figure 4.8, triadic closure makes a link between Bob and Daniel more likely due to their shared friendship with Anna; and focal closure also makes this link more likely due to the shared membership of Bob and Daniel in the karate club. If a link does form between them, it will not necessarily be a priori clear how to attribute it to these two distinct effects. This is also a reflection of an issue we discussed

119 4.4. TRACKING LINK FORMATION IN ON-LINE DATA 105 in Section 4.1, when describing some of the mechanisms behind triadic closure: since the principle of homophily suggests that friends tend to have many characteristics in common, the existence of a shared friend between two people is often indicative of other, possibly unobserved, sources of similarity (such as shared foci in this case) that by themselves may also make link formation more likely. Quantifying the Interplay Between Selection and Social Influence. As a final illustration of how we can use large-scale on-line data to track processes of link formation, let’s return to the question of how selection and social influence work together to produce homophily, considered in Section 4.2. We’ll make use of the Wikipedia data discussed earlier in this section, asking: how do similarities in behavior between two Wikipedia editors relate to their pattern of social interaction over time? [122] To make this question precise, we need to define both the social network and an underlying measure of behavioral similarity. As before, the social network will consist of all Wikipedia editors who maintain talk pages, and there is an edge connecting two editors if they have communicated, with one writing on the talk page of the other. An editor’s behavior will correspond to the set of articles she has edited. There are a number of natural ways to define numerical measures of similarity between two editors based on their actions; a simple one is to declare their similarity to be the value of the ratio number of articles edited by A and B both , (4.1) or B number of articles edited by A at least one of For example, if editor A has edited the Wikipedia articles on Ithaca NY and Cornell Uni- Stanford University versity has edited the articles on Cornell University and B , , and editor then their similarity under this measure is 1 / 3, since they have jointly edited one article ( Cornell ) out of three that they have edited in total ( Cornell , Ithaca , and Stanford ). Note the close similarity to the definition of used in Section 3.3; indeed, neighborhood overlap the measure in Equation (4.1) is precisely the neighborhood overlap of two editors in the bipartite affiliation network of editors and articles, consisting only of edges from editors to 3 the articles they’ve edited. Pairs of Wikipedia editors who have communicated are significantly more similar in their behavior than pairs of Wikipedia editors who have not communicated, so we have a case where homophily is clearly present. Therefore, we are set up to address the question of selec- tion and social influence: is the homophily arising because editors are forming connections with those who have edited the same articles they have (selection), or is it because editors are led to the articles of those they talk to (social influence)? 3 For technical reasons, a minor variation on this simple similarity measure is used for the results that follow. However, since this variation is more complicated to describe, and the differences are not significant for our purposes, we can think of similarity as consisting of the numerical measure just defined.

120 106 CHAPTER 4. NETWORKS IN THEIR SURROUNDING CONTEXTS Social infl uence: continued slower increase in similarity rst contact after fi Selection: rapid increase in similarity rst contact before fi Figure 4.13: The average similarity of two editors on Wikipedia, relative to the time (0) x -axis, is measured in discrete units, at which they first communicated [122]. Time, on the where each unit corresponds to a single Wikipedia action taken by either of the two editors. The curve increases both before and after the first contact at time 0, indicating that both selection and social influence play a role; the increase in similarity is steepest just before time 0. Because every action on Wikipedia is recorded and time-stamped, it is not hard to get an initial picture of this interplay, using the following method. For each pair of editors A and who have ever communicated, record their similarity over time, where “time” here B moves in discrete units, advancing by one “tick” whenever either A or B performs an action on Wikipedia (editing an article or communicating with another editor). Next, declare time 0 for the pair A - B to be the point at which they first communicated. This results in many curves showing similarity as a function of time — one for each pair of editors who ever communicated, and each curve shifted so that time is measured for each one relative to the moment of first communication. Averaging all these curves yields the single plot in Figure 4.13 — it shows the average level of similarity relative to the time of first interaction, over all pairs of editors who have ever interacted on Wikipedia [122]. There are a number of things to notice about this plot. First, similarity is clearly increas- ing both before and after the moment of first interaction, indicating that both selection and

121 4.5. A SPATIAL MODEL OF SEGREGATION 107 social influence are at work. However, the the curve is not symmetric around time 0; the period of fastest increase in similarity is clearly occurring before 0, indicating a particular role for selection: there is an especially rapid rise in similarity, on average, just before two 4 editors meet. Also note that the levels of similarity depicted in the plot are much higher than for pairs of editors who have not interacted: the dashed blue line at the bottom of the plot shows similarity over time for a random sample of non-interacting pairs; it is both far lower and also essentially constant as time moves forward. At a higher level, the plot in Figure 4.13 once again illustrates the trade-offs involved in working with large-scale on-line data. On the one hand, the curve is remarkably smooth, because so many pairs are being averaged, and so differences between selection and social influence show up that are genuine, but too subtle to be noticeable at smaller scales. On the other hand, the effect being observed is an aggregate one: it is the average of the interaction histories of many different pairs of individuals, and it does not provide more detailed insight 5 into the experience of any one particular pair. A goal for further research is clearly to find ways of formulating more complex, nuanced questions that can still be meaningfully addressed on large datasets. Overall, then, these analyses represent early attempts to quantify some of the basic mechanisms of link formation at a very large scale, using on-line data. While they are promising in revealing that the basic patterns indeed show up strongly in the data, they raise many further questions. In particular, it natural to ask whether the general shapes of the curves in Figures 4.9–4.13 are similar across different domains — including domains that are less technologically mediated — and whether these curve shapes can be explained at a simpler level by more basic underlying social mechanisms. 4.5 A Spatial Model of Segregation One of the most readily perceived effects of homophily is in the formation of ethnically and racially homogeneous neighborhoods in cities. Traveling through a metropolitan area, one finds that homophily produces a natural spatial signature; people live near others like them, and as a consequence they open shops, restaurants, and other businesses oriented toward the populations of their respective neighborhoods. The effect is also striking when superimposed on a map, as Figure 4.14 by M ̈obius and Rosenblat [302] illustrates. Their images depict the 4 To make sure that these are editors with significant histories on Wikipedia, this plot is constructed using only pairs of editors who each had at least 100 actions both before and after their first interaction with each other. 5 Because the individual histories being averaged took place at many distinct points in Wikipedia’s history, it is also natural to ask whether the aggregate effects operated differently in different phases of this history. This is a natural question for further investigation, but initial tests — based on studying these types of properties on Wikipedia datasets built from different periods — show that the main effects have remained relatively stable over time.

122 108 CHAPTER 4. NETWORKS IN THEIR SURROUNDING CONTEXTS Chicago, 1940 (b) Chicago, 1960 (a) Figure 4.14: The tendency of people to live in racially homogeneous neighborhoods produces spatial patterns of segregation that are apparent both in everyday life and when superim- posed on a map — as here, in these maps of Chicago from 1940 and 1960 [302]. In blocks colored yellow and orange the percentage of African-Americans is below 25, while in blocks colored brown and black the percentage is above 75. percentage of African-Americans per city block in Chicago for the years 1940 and 1960; in blocks colored yellow and orange the percentage is below 25, while in blocks colored brown and black the percentage is above 75. This pair of figures also shows how concentrations of different groups can intensify over time, emphasizing that this is a process with a dynamic aspect. Using the principles we’ve been considering, we now discuss how simple mechansisms based on similarity and selection can provide insight into the observed patterns and their dynamics. The Schelling Model. A famous model due to Thomas Schelling [365, 366] shows how global patterns of spatial segregation can arise from the effect of homophily operating at a local level. There are many factors that contribute to segregation in real life, but Schelling’s model focuses on an intentionally simplified mechanism to illustrate how the forces leading to segregation are remarkably robust — they can operate even when no one individual explicitly wants a segregated outcome.

123 4.5. A SPATIAL MODEL OF SEGREGATION 109 X X X X O X O O X O O X O X O X X O O O X O X X X X O X X O O X X X X X O O O O O O O O Agents occupying cells on a grid. Neighbor relations as a graph. (b) (a) Figure 4.15: In Schelling’s segregation model, agents of two different types ( X O ) occupy and cells on a grid. The neighbor relationships among the cells can be represented very simply as a graph. Agents care about whether they have at least some neighbors of the same type. The general formulation of the model is as follows. We assume that there is a population agents ; each agent is of type X or type O . We think of the of individuals, whom we’ll call two types as representing some (immutable) characteristic that can serve as the basis for homophily — for example, race, ethnicity, country of origin, or native language. The agents reside in the cells of a grid, intended as a stylized model of the two-dimensional geography of a city. As illustrated in Figure 4.15(a), we will assume that some cells of the grid contain agents while others are unpopulated. A cell’s neighbors are the cells that touch it, including diagonal contact; thus, a cell that is not on the boundary of the grid has eight neighbors. We can equivalently think of the neighbor relationships as defining a graph: the cells are the nodes, and we put an edge between two cells that are neighbors on the grid. In this view, the agents thus occupy the nodes of a graph that are arranged in this grid-like pattern, as shown in Figure 4.15(b). For ease of visualization, however, we will continue to draw things using a geometric grid, rather than a graph. The fundamental constraint driving the model is that each agent wants to have at least some other agents of its own type as neighbors. We will assume that there is a threshold t common to all agents: if an agent discovers that fewer than t of its neighbors are of the same type as itself, then it has an interest in moving to a new cell. We will call such an agent unsatisfied with its current location. For example, in Figure 4.16(a), we indicate with an asterisk all the agents that are unsatisfied in the arrangement from Figure 4.15(a), when the threshold t is equal to 3. (In Figure 4.16(a) we have also added a number after each agent. This is simply to provide each with a unique name; the key distinction is still whether each agent is of type X or type O .)

124 110 CHAPTER 4. NETWORKS IN THEIR SURROUNDING CONTEXTS X1* X2* O1* X3 O2 O3 X5 X4 O5* O4 X8 X7 O6 X6* X10 O8 X9* O7 X1 1 1* O1 O10 O9 (a) An initial configuration. O2 X3 X6 O1 X5 O4 O3 X4 X7 X8 O6 X2 X1 X9 O7 O8 X1 1 O1 1 X10 O5 O9 O10* (b) After one round of movement. Figure 4.16: After arranging agents in cells of the grid, we first determine which agents are unsatisfied , with fewer than t other agents of the same type as neighbors. In one round, each of these agents moves to a cell where they will be satisfied; this may cause other agents to become unsatisfied, in which case a new round of movement begins.

125 4.5. A SPATIAL MODEL OF SEGREGATION 111 Thus far, we have simply specified a set of agents that The Dynamics of Movement. want to move, given an underlying threshold; we now discuss how this gives the model its dynamic aspect. Agents move in a sequence of rounds : in each round, we consider the unsatisfied agents in some order, and for each one in turn, we have it move to an unoccupied cell where it will be satisfied. After this, the round of movement has come to an end, representing a fixed time period during which unsatisfied agents have changed where they live. These new locations may cause different agents to be unsatisfied, and this leads to a new round of movement. In the literature on this model, there are numerous variations in the specific details of how the movement of agents within a round is handled. For example, the agents can be scheduled to move in a random order, or in an order that sweeps downward along rows of the grid; they can move to the nearest location that will make them satisfied or to a random one. There also needs to be a way of handling situations in which an agent is scheduled to move, and there is no cell that will make it satisified. In such a case, the agent can be left where it is, or moved to a completely random cell. Research has found that the qualitative results of the model tend to be quite similar however these issues are resolved, and different investigations of the model have tended to resolve them differently. For example, Figure 4.16(b) shows the results of one round of movement, starting from the arrangement in Figure 4.16(a), when the threshold is 3. Unsatisfied agents are scheduled t to move by considering them one row at a time working downward through the grid, and each agent moves to the nearest cell that will make it satisfied. (The unique name of each agent in the figure allows us to see where it has moved in Figure 4.16(b), relative to the initial state in Figure 4.16(a).) Notice that in some concrete respects, the pattern of agents has become more “segregated” after this round of movement. For example, in Figure 4.16(a), there is only a single agent with no neighbors of the opposite type. After this first round of movement, however, there are six agents in Figure 4.16(b) with no neighbors of the opposite type. As we will see, this increasing level of segregation is the key behavior to emerge from the model. Small examples of the type in Figures 4.15 and 4.16 are helpful in Larger examples. working through the details of the model by hand; but at such small scales it is difficult to see the kinds of typical patterns that arise. For this, computer simulation is very useful. There are many on-line computer programs that make it possible to simulate the Schelling model; as with the published literature on the model, they all tend to differ slightly from each other in their specifics. Here we discuss some examples from a simulation written by Sean Luke [282], which is like the version of the model we have discussed thus far except that unsatisfied agents move to a random location. In Figure 4.17, we show the results of simulating the model on a grid with 150 rows and

126 112 CHAPTER 4. NETWORKS IN THEIR SURROUNDING CONTEXTS A simulation with threshold 3. Another simulation with threshold 3. (a) (b) t of 3, on a Figure 4.17: Two runs of a simulation of the Schelling model with a threshold , 000 agents of each type. Each cell of the grid is colored red if it is 150-by-150 grid with 10 occupied by an agent of the first type, blue if it is occupied by an agent of the second type, and black if it is empty (not occupied by any agent). 150 columns, 10 , 000 agents of each type, and 2500 empty cells. The threshold t is equal to 3, as in our earlier examples. The two images depict the results of two different runs of the simulation, with different random starting patterns of agents. In each case, the simulation reached a point (shown in the figures) at which all agents were satisfied, after roughly 50 rounds of movement. Because of the different random starts, the final arrangement of agents is different in the two cases, but the qualitative similarities reflect the fundamental consequences of the model. By seeking out locations near other agents of the same type, the model produces large homogeneous regions, interlocking with each other as they stretch across the grid. In the midst of these regions are large numbers of agents who are surrounded on all sides by other agents of the same type — and in fact at some distance from the nearest agent of the opposite type. The geometric pattern has become segregated, much as in the maps of Chicago from Figure 4.14 with which we began the section. Interpretations of the Model. We’ve now seen how the model works, what it looks like at relatively large scales, and how it produces spatially segregated outcomes. But what broader insights into homophily and segregation does it suggest? The first and most basic one is that spatial segregation is taking place even though no

127 4.5. A SPATIAL MODEL OF SEGREGATION 113 O X X X O X X X O O X X O X O O X O X O X O O O X O X O X X X X X O X O Figure 4.18: With a threshold of 3, it is possible to arrange agents in an integrated pattern: all agents are satisfied, and everyone who is not on the boundary on the grid has an equal number of neighbors of each type. individual agent is actively seeking it. Sticking to our focus on a threshold t of 3, we see that although agents want to be near others like them, their requirements are not particularly draconian. For example, an agent would be perfectly happy to be in the minority among its neighbors, with five neighbors of the opposite type and three of its own type. Nor are the requirements globally incompatible with complete integration of the population. By arranging agents in a checkerboard pattern as shown in Figure 4.18, we can make each agent satisfied, and all agents not on the boundary of the grid have exactly four neighbors of each type. This is a pattern that we can continue on as large a grid as we want. Thus, segregation is not happening because we have subtly built it into the model — agents are willing to be in the minority, and they could all be satisfied if we were only able to carefully arrange them in an integrated pattern. The problem is that from a random start, it is very hard for the collection of agents to find such integrated patterns. Much more typically, agents will attach themselves to clusters of others like themselves, and these clusters will grow as other agents follow suit. Moreover, there is a compounding effect as the rounds of movement unfold, in which agents who fall below their threshold depart for more homogeneous parts of the grid, causing previously satisfied agents to fall below their thresholds and move as well — an effect that Schelling describes as the progressive “unraveling” of more integrated regions [366]. In the long run, this process will tend to cause segregated regions to grow at the expense of more integrated ones. The overall effect is one in which the local preferences of individual agents have produced a global pattern that none of them necessarily intended.

128 114 CHAPTER 4. NETWORKS IN THEIR SURROUNDING CONTEXTS (a) After 20 steps (b) After 150 steps (c) After 350 steps (d) After 800 steps Figure 4.19: Four intermediate points in a simulation of the Schelling model with a threshold t of 4, on a 150-by-150 grid with 10 , 000 agents of each type. As the rounds of movement progress, large homogeneous regions on the grid grow at the expense of smaller, narrower regions.

129 4.5. A SPATIAL MODEL OF SEGREGATION 115 This point is ultimately at the heart of the model: although segregation in real life is amplified by a genuine desire within some fraction of the population to belong to large clusters of similar people — either to avoid people who belong to other groups, or to acquire a critical mass of members from one’s own group — we see here that such factors are not necessary for segregation to occur. The underpinnings of segregation are already present in a system where individuals simply want to avoid being in too extreme a minority in their own local area. The process operates even more powerfully when we raise the threshold t in our examples from 3 to 4. Even with a threshold of 4, nodes are willing to have an equal number of neighbors of each type; and a slightly more elaborate checkerboard example in the spirit of Figure 4.18 shows that with careful placement, the agents can be arranged so that all are satisfied and most still have a significant number of neighbors of the opposite type. But now, not only is an integrated pattern very hard to reach from a random starting arrangement — any vestiges of integration among the two types tends to collapse completely over time. As one example of this, Figure 4.19 shows four intermediate points in one run of a simulation with threshold 4 and other properties the same as before (a 150-by-150 grid with 10 , 000 agents of each type and random movement by unsatisfied agents) [282]. Figure 4.19(a) shows that after 20 rounds of movement, we have an arrangement of agents that roughly resembles what we saw with a lower threshold of 3. However, this does not last long: crucially, the long tendrils where one type interlocks with the other quickly wither and retract, leaving the more homogeneous regions shown after 150 rounds in Figure 4.19(b). This pulling-back continues, passing through a phase with a large and small region of each type after 350 rounds (Figure 4.19(c)) eventually to a point where there is only a single significant region of each type, after roughly 800 rounds (Figure 4.19(d)). Note that this is not the end of the process, since there remain agents around the edges still looking for places to move, but by this point the overall two-region layout has become very stable. Finally, we stress that this figure corresponds to just a single run of the simulation — but computational experiments show that the sequence of events it depicts, leading to almost complete separation of the two types, is very robust when the threshold is this high. Viewed at a still more general level, the Schelling model is an example of how character- istics that are fixed and unchanging (such as race or ethnicity) can become highly correlated with other characteristics that are mutable. In this case, the mutable characteristic is the decision about where to live, which over time conforms to similarities in the agents’ (im- mutable) types, producing segregation. But there are other, non-spatial manifestation of the same effect, in which beliefs and opinions become correlated across racial or ethnic lines, and for similar underlying reasons: as homophily draws people together along immutable characteristics, there is a natural tendency for mutable characteristics to change in accor- dance with the network structure.

130 116 CHAPTER 4. NETWORKS IN THEIR SURROUNDING CONTEXTS As a final point, we note that while the model is mathematically precise and self- contained, the discussion has been carried out in terms of simulations and qualitative obser- vations. This is because rigorous mathematical analysis of the Schelling model appears to be quite difficult, and is largely an open research question. For partial progress on analyzing properties of the Schelling model, see the work of Young [420], who compares properties of different arrangements in which all agents are satisfied; M ̈obius and Rosenblat [302], who perform a probabilistic analysis; and Vinkovi ́c and Kirman [401], who develop analogies to models for the mixing of two liquids and other physical phenomena. 4.6 Exercises a b c d e Figure 4.20: A social network where triadic closure may occur. 1. Consider the social network represented in Figure 4.20. Suppose that this social net- work was obtained by observing a group of people at a particular point in time and recording all their friendship relations. Now suppose that we come back at some point in the future and observe it again. According to the theories based on empirical studies of triadic closure in networks, which new edge is most likely to be present? (I.e. which pair of nodes, who do not currently have an edge connecting them, are most likely to be linked by an edge when we return to take the second observation?) Also, give a brief explanation for your answer. 2. Given a bipartite affiliation graph, showing the membership of people in different social foci, researchers sometimes create a projected graph on just the people, in which we join two people when they have a focus in common. (a) Draw what such a projected graph would look like for the example of memberships on corporate boards of directors from Figure 4.4. Here the nodes would be the

131 4.6. EXERCISES 117 seven people in the figure, and there would be an edge joining any two who serve on a board of directors together. (b) Give an example of two different affiliation networks — on the same set of people, but with different foci — so that the projected graphs from these two different affiliation networks are the same. This shows how information can be “lost” when moving from the full affiliation network to just the projected graph on the set of people. A X B C Y D Z E F Figure 4.21: An affiliation network on six people labeled A – F , and three foci labeled X , Y , and . Z 3. Consider the affiliation network in Figure 4.21, with six people labeled A – F , and three foci labeled X , Y , and Z . (a) Draw the derived network on just the six people as in Exercise 2, joining two people when they share a focus. (b) In the resulting network on people, can you identify a sense in which the triangle on the nodes A , C , and E has a qualitatively different meaning than the other triangles that appear in the network? Explain.

132 118 CHAPTER 4. NETWORKS IN THEIR SURROUNDING CONTEXTS B A E F C D Figure 4.22: A graph on people arising from an (unobserved) affiliation network. 4. Given a network showing pairs of people who share activities, we can try to reconstruct an affiliation network consistent with this data. For example, suppose that you are trying to infer the structure of a bipartite affiliation network, and by indirect observation you’ve obtained the projected network on just the set of people, constructed as in Exercise 2: there is an edge joining each pair of people who share a focus. This projected network is shown in Figure 4.22. (a) Draw an affiliation network involving these six people, together with four foci that you should define, whose projected network is the graph shown in Figure 4.22. (b) Explain why any affiliation network capable of producing the projected network in Figure 4.22 must have at least four foci.

133 Chapter 5 Positive and Negative Relationships In our discussion of networks thus far, we have generally viewed the relationships con- tained in these networks as having positive connotations — links have typically indicated such things as friendship, collaboration, sharing of information, or membership in a group. The terminology of on-line social networks reflects a largely similar view, through its em- phasis on the connections one forms with friends, fans, followers, and so forth. But in most network settings, there are also negative effects at work. Some relations are friendly, but others are antagonistic or hostile; interactions between people or groups are regularly beset by controversy, disagreement, and sometimes outright conflict. How should we reason about the mix of positive and negative relationships that take place within a network? Here we describe a rich part of social network theory that involves taking a network and annotating its links (i.e., its edges) with positive and negative signs. Positive links represent friendship while negative links represent antagonism, and an important problem in the study of social networks is to understand the tension between these two forces. The notion of structural balance that we discuss in this chapter is one of the basic frameworks for doing this. In addition to introducing some of the basics of structural balance, our discussion here serves a second, methodological purpose: it illustrates a nice connection between local and global network properties. A recurring issue in the analysis of networked systems is the way in which local effects — phenomena involving only a few nodes at a time — can have global consequences that are observable at the level of the network as a whole. Structural balance offers a way to capture one such relationship in a very clean way, and by purely mathematical analysis: we will consider a simple definition abstractly, and find that it inevitably leads to certain macroscopic properties of the network. D. Easley and J. Kleinberg. Networks, Crowds, and Markets: Reasoning about a Highly Connected World . Cambridge University Press, 2010. Draft version: June 10, 2010. 119

134 120 CHAPTER 5. POSITIVE AND NEGATIVE RELATIONSHIPS 5.1 Structural Balance We focus here on perhaps the most basic model of positive and negative relationships, since it captures the essential idea. Suppose we have a social network on a set of people, in which everyone knows everyone else — so we have an edge joining each pair of nodes. Such a clique complete graph . We then label each edge with either + or − ; network is called a , or a label indicates that its two a + label indicates that its two endpoints are friends, while a − endpoints are enemies. Note that since there’s an edge connecting each pair, we are assuming that each pair of people are either friends or enemies — no two people are indifferent to one another, or unaware of each other. Thus, the model we’re considering makes the most sense for a group of people small enough to have this level of mutual awareness (e.g. a classroom, a small company, a sports team, a fraternity or sorority), or for a setting such as international relations, in which the nodes are countries and every country has an official diplomatic 1 position toward every other. The principles underlying structural balance are based on theories in social psychology dating back to the work of Heider in the 1940s [216], and generalized and extended to the language of graphs beginning with the work of Cartwright and Harary in the 1950s [97, 126, 204]. The crucial idea is the following. If we look at any two people in the group in isolation, the edge between them can be labeled + or − ; that is, they are either friends three or enemies. But when we look at sets of people at a time, certain configurations of +’s and − ’s are socially and psychologically more plausible than others. In particular, there are four distinct ways (up to symmetry) to label the three edges among three people with +’s and − ’s; see Figure 5.1. We can distinguish among these four possibilities as follows. • Given a set of people A , B , and C , having three pluses among them (as in Figure 5.1(a)) is a very natural situation: it corresponds to three people who are mutual friends. • Having a single plus and two minuses in the relations among the there people is also very natural: it means that two of the three are friends, and they have a mutual enemy in the third. (See Figure 5.1(c).) The other two possible labelings of the triangle on A , B , and • introduce some amount C of psychological “stress” or “instability” into the relationships. A triangle with two pluses and one minus corresponds (as in Figure 5.1(b)) to a person A who is friends with each of B and C , but B and C don’t get along with each other. In this type of situation, there would be implicit forces pushing A to try to get B and C to become 1 Later, in Section 5.5, we will consider the more general setting in which not every pair of nodes is necessarily connected by an edge.

135 5.1. STRUCTURAL BALANCE 121 A A + + + + B C B C - + is friends with B , A (a) C (b) A are mutual friends: balanced. B and C , but they don’t get , and along with each other: not balanced. A A + - - - B B C C - - are mutual enemies: not bal- (c) B are friends with C and as a mutual en- (d) A , B , and C A emy: balanced. anced. Figure 5.1: Structural balance: Each labeled triangle must have 1 or 3 positive edges. friends (thus turning the B - C edge label to +); or else for A to side with one of B or C A to a − ). against the other (turning one of the edge labels out of • Similarly, there are sources of instability in a configuration where each of A , B , and C are mutual enemies (as in Figure 5.1(d)). In this case, there would be forces motivating two of the three people to “team up” against the third (turning one of the three edge labels to a +). Based on this reasoning, we will refer to triangles with one or three +’s as balanced , since they are free of these sources of instability, and we will refer to triangles with zero or two +’s as unbalanced . The argument of structural balance theorists is that because unbalanced triangles are sources of stress or psychological dissonance, people strive to minimize them in their personal relationships, and hence they will be less abundant in real social settings than

136 122 CHAPTER 5. POSITIVE AND NEGATIVE RELATIONSHIPS A A - - + - - + B B + + - - D C D C + - not balanced balanced Figure 5.2: The labeled four-node complete graph on the left is balanced; the one on the right is not. balanced triangles. Defining Structural Balance for Networks. So far we have been talking about struc- tural balance for groups of three nodes. But it is easy to create a definition that naturally generalizes this to complete graphs on an arbitrary number of nodes, with edges labeled by +’s and − ’s. balanced if every one of its triangles Specifically, we say that a labeled complete graph is is balanced — that is, if it obeys the following: every set of three nodes, if we consider the three Structural Balance Property: For edges connecting them, either all three of these edges are labeled + , or else exactly one of them is labeled + . For example, consider the two labeled four-node networks in Figure 5.2. The one on the left is balanced, since we can check that each set of three nodes satisfies the Structural Balance Property above. On the other hand, the one on the right is not balanced, since among the three nodes A,B,C , there are exactly two edges labeled +, in violation of Structural Balance. (The triangle on B,C,D also violates the condition.) Our definition of balanced networks here represents the limit of a social system that has eliminated all unbalanced triangles. As such, it is a fairly extreme definition — for example, one could instead propose a definition which only required that at least some large percentage of all triangles were balanced, allowing a few triangles to be unbalanced. But the version with all triangles balanced is a fundamental first step in thinking about this concept; and

137 5.2. CHARACTERIZING THE STRUCTURE OF BALANCED NETWORKS 123 mutual mutual friends mutual friends antagonism inside X inside Y between sets Y set X set Figure 5.3: If a complete graph can be divided into two sets of mutual friends, with complete mutual antagonism between the two sets, then it is balanced. Furthermore, this is the only way for a complete graph to be balanced. as we will see next, it turns out to have very interesting mathematical structure that in fact helps to inform the conclusions of more complicated models as well. 5.2 Characterizing the Structure of Balanced Networks At a general level, what does a balanced network (i.e. a balanced labeled complete graph) look like? Given any specific example, we can check all triangles to make sure that they each obey the balance conditions; but it would be much better to have a simple conceptual description of what a balanced network looks like in general. One way for a network to be balanced is if everyone likes each other; in this case, all triangles have three + labels. On the other hand, the left-hand side of Figure 5.2 suggests a slightly more complicated way for a network to be balanced: it consists of two groups of A,B and C,D ), with negative relations between people in different groups. This is friends ( actually true in general: suppose we have a labeled complete graph in which the nodes can be divided into two groups, X and Y , such that every pair of nodes in X like each other, every pair of nodes in like each other, and everyone in X is the enemy of everyone in Y Y . (See the schematic illustration in Figure 5.3.) You can check that such a network is balanced: a triangle contained entirely in one group or the other has three + labels, and a triangle with two people in one group and one in the other has exactly one + label. So this describes two basic ways to achieve structural balance: either everyone likes each other; or the world consists of two groups of mutual friends with complete antagonism

138 124 CHAPTER 5. POSITIVE AND NEGATIVE RELATIONSHIPS only ways to have between the groups. The surprising fact is the following: these are the , a balanced network. We formulate this fact precisely as the following Balance Theorem proved by Frank Harary in 1953 [97, 204]: Balance Theorem: If a labeled complete graph is balanced, then either all pairs of nodes are friends, or else the nodes can be divided into two groups, X and Y , such that every pair of nodes in Y like X like each other, every pair of nodes in X is the enemy of everyone in Y . each other, and everyone in The Balance Theorem is not at all an obvious fact, nor should it be initially clear why it local is true. Essentially, we’re taking a purely property, namely the Structural Balance Property, which applies to only three nodes at a time, and showing that it implies a strong global property: either everyone gets along, or the world is divided into two battling factions. We’re now going to show why this claim in fact is true. Proving the Balance Theorem. Establishing the claim requires a proof: we’re going to suppose we have an arbitrary labeled complete graph, assume only that it is balanced, and X and Y conclude that either everyone is friends, or that there are sets as described in the claim. Recall that we worked through a proof in Chapter 3 as well, when we used simple assumptions about triadic closure in a social network to conclude all local bridges in the network must be weak ties. Our proof here will be somewhat longer, but still very natural and straightforward — we use the definition of balance to directly derive the conclusion of the claim. To start, suppose we have a labeled complete graph, and all we know is that it’s balanced. We have to show that it has the structure in the claim. If it has no negative edges at all, then everyone is friends, and we’re all set. Otherwise, there is at least one negative edge, and we need to somehow come up with a division of the nodes into sets of mutual friends X and , with complete antagonism between them. The difficulty is that, knowing so little Y about the graph itself other than that it is balanced, it’s not clear how we’re supposed to identify and Y . X A — and consider things from Let’s pick any node in the network — we’ll call it ’s A perspective. Every other node is either a friend of A or an enemy of A . Thus, natural candidates to try for the sets X and Y would be to define X to be A and all its friends, and define to be all the enemies of A . This is indeed a division of all the nodes, since every Y node is either a friend or an enemy of A . Recall what we need to show in order for these two sets X and Y to satisfy the conditions of the claim: (i) Every two nodes in X are friends. (ii) Every two nodes in Y are friends.

139 5.2. CHARACTERIZING THE STRUCTURE OF BALANCED NETWORKS 125 B D ? + - ? ? A + - C E enemies of A friends of A Figure 5.4: A schematic illustration of our analysis of balanced networks. (There may be other nodes not illustrated here.) is an enemy of every node in Y . (iii) Every node in X X and . This will Let’s argue that each of these conditions is in fact true for our choice of Y X Y do satisfy the conditions of the claim, and will complete the proof. The mean that and rest of the argument, establishing (i), (ii), and (iii), is illustrated schematically in Figure 5.4. For (i), we know that is friends with every other node in X . How about two other A nodes in (let’s call them B and C ) — must they be friends? We know that A is friends X B B C , so if B with both C were enemies of each other, then A , and , and C would and form a triangle with two + labels — a violation of the balance condition. Since we know the network is balanced, this can’t happen, so it must be that B and C in fact are friends. Since B and C were the names of any two nodes in X , we have concluded that every two nodes in X are friends. Y Let’s try the same kind of argument for (ii). Consider any two nodes in (let’s call them and E ) — must they be friends? We know that A is enemies with both D and E , so if D D and were enemies of each other, then A , E , and E would form a triangle with no + labels D — a violation of the balance condition. Since we know the network is balanced, this can’t happen, so it must be that D and E in fact are friends. Since D and E were the names of any two nodes in Y Y are friends. , we have concluded that every two nodes in Finally, let’s try condition (iii). Following the style of our arguments for (i) and (ii), consider a node in X B ) and a node in Y (call it D ) — must they be enemies? We (call if , and A is friends with B and enemies with D , so if B and D were friends, then a , B know

140 126 CHAPTER 5. POSITIVE AND NEGATIVE RELATIONSHIPS would form a triangle with two + labels — a violation of the balance condition. Since D B and D in fact we know the network is balanced, this can’t happen, so it must be that B and D were the names of any node in X and any node in are enemies. Since , we have Y concluded that every such pair constitutes a pair of enemies. So, in conclusion, assuming only that the network is balanced, we have described a division of the nodes into two sets X and Y , and we have checked conditions (i), (ii), and (iii) required by the claim. This completes the proof of the Balance Theorem. 5.3 Applications of Structural Balance Structural balance has grown into a large area of study, and we’ve only described a simple but central example of the theory. In Section 5.5, we discuss two extensions to the basic theory: one to handle graphs that are not necessarily complete, and one to describe the structure of complete graphs that are “approximately balanced,” in the sense that most but not all their triangles are balanced. There has also been recent research looking at dynamic aspects of structural balance theory, modeling how the set of friendships and antagonisms in a complete graph — in other words, the labeling of the edges — might evolve over time, as the social network implicitly seeks out structural balance. Antal, Krapivsky, and Redner [20] study a model in which we − randomly for each edge); we then repeatedly start with a random labeling (choosing + or look for a triangle that is not balanced, and flip one of its labels to make it balanced. This captures a situation in which people continually reassess their likes and dislikes of others, as they strive for structural balance. The mathematics here becomes quite complicated, and turns out to resemble the mathematical models one uses for certain physical systems as they reconfigure to minimize their energy [20, 287]. In the remainder of this section, we consider two further areas in which the ideas of struc- tural balance are relevant: international relations, where the nodes are different countries; and on-line social media sites where users can express positive or negative opinions about each other. International Relations. International politics represents a setting in which it is natural to assume that a collection of nodes all have opinions (positive or negative) about one another — here the nodes are nations, and + and − labels indicate alliances or animosity. Research in political science has shown that structural balance can sometimes provide an effective explanation for the behavior of nations during various international crises. For example, Moore [306], describing the conflict over Bangladesh’s separation from Pakistan in 1972, explicitly invokes structural balance theory when he writes, “[T]he United States’s somewhat surprising support of Pakistan ... becomes less surprising when one considers that the USSR

141 5.3. APPLICATIONS OF STRUCTURAL BALANCE 127 AH GB AH GB GB AH Fr Ge Ge Fr Ge Fr Ru Ru It It It Ru (b) Three Emperors’ League 1872– (c) German-Russian Lapse 1890 Triple Alliance 1882 (a) 81 AH GB AH AH GB GB Ge Ge Fr Fr Fr Ge Ru Ru Ru It It It French-Russian Alliance 1891– British Russian Alliance 1907 (e) Entente Cordiale 1904 (d) (f) 94 Figure 5.5: The evolution of alliances in Europe, 1872-1907 (the nations GB, Fr, Ru, It, Ge, and AH are Great Britain, France, Russia, Italy, Germany, and Austria-Hungary respec- tively). Solid dark edges indicate friendship while dotted red edges indicate enmity. Note how the network slides into a balanced labeling — and into World War I. This figure and example are from Antal, Krapivsky, and Redner [20]. was China’s enemy, China was India’s foe, and India had traditionally bad relations with Pakistan. Since the U.S. was at that time improving its relations with China, it supported the enemies of China’s enemies. Further reverberations of this strange political constellation became inevitable: North Vietnam made friendly gestures toward India, Pakistan severed diplomatic relations with those countries of the Eastern Bloc which recognized Bangladesh, and China vetoed the acceptance of Bangladesh into the U.N.” Antal, Krapivsky, and Redner use the shifting alliances preceding World War I as another example of structural balance in international relations — see Figure 5.5. This also reinforces the fact that structural balance is not necessarily a good thing: since its global outcome is often two implacably opposed alliances, the search for balance in a system can sometimes be seen as a slide into a hard-to-resolve opposition between two sides.

142 128 CHAPTER 5. POSITIVE AND NEGATIVE RELATIONSHIPS A growing source for network data with both Trust, Distrust, and On-Line Ratings. positive and negative edges comes from user communities on the Web where people can express positive or negative sentiments about each other. Examples include the technology news site Slashdot, where users can designate each other as a “friend” or a “foe” [266], and on-line product-rating sites such as Epinions, where a user can express evaluations of distrust trust of other users. different products, and also express or Guha, Kumar, Raghavan, and Tomkins performed an analysis of the network of user evaluations on Epinions [201]; their work identified an interesting set of issues that show how the trust/distrust dichotomy in on-line ratings has both similarities and differences with the friend/enemy dichotomy in structural balance theory. One difference is based on a simple structural distinction: we have been considering structural balance in the context of undirected graphs, whereas user evaluations on a site like Epinions form a directed graph. That is, when a user B , we don’t necessarily know A expresses trust or distrust of a user B thinks of A , or whether B is even aware of A . what A more subtle difference between trust/distrust and friend/enemy relations becomes ap- parent when thinking about how we should expect triangles on three Epinions users to behave. Certain patterns are easy to reason about: for example, if user trusts user B , A B C , then it is natural to expect that A will trust C . Such triangles and user trusts user with three forward-pointing positive edges make sense here, by analogy with the all-positive A distrusts (undirected) triangles of structural balance theory. But what if and B dis- B trusts C : should we expect A to trust or to distrust C ? There are appealing arguments in both directions. If we think of distrust as fundamentally a kind of enemy relationship, then the arguments from structural balance theory would suggest that A C : should trust A otherwise we’d have a triangle with three negative edges. On the other hand, if ’s distrust of B expresses A ’s belief that she is more knowledgeable and competent than B — and if ’s distrust of C reflects a corresponding belief by B — then we might well expect that A B will distrust , and perhaps even more strongly than she distrusts B . C It is reasonable to expect that these two different interpretations of distrust may each apply, simply in different settings. And both might apply in the context of a single product- rating site like Epinions. For example, among users who are primarily rating best-selling books by political commentators, trust/distrust evaluations between users may become strongly aligned with agreement or disagreement in these users’ own political orientations. In such a case, if A distrusts B and B distrusts C , this may suggest that A and C are close to each other on the underlying political spectrum, and so the prediction of structural A should trust C may apply. On the other hand, among users who are balance theory that primarily rating consumer electronics products, trust/distrust evaluations may largely reflect the relative expertise of users about the products (their respective features, reliability, and so forth). In such a case, if is distrusts B and B distrusts C , we might conclude that A A

143 5.4. A WEAKER FORM OF STRUCTURAL BALANCE 129 set V mutual set W friends set X inside V mutual friends inside X mutual friends inside W mutual antagonism between all sets mutual friends mutual inside Z friends inside Y set Z Y set Figure 5.6: A complete graph is weakly balanced precisely when it can be divided into multiple sets of mutual friends, with complete mutual antagonism between each pair of sets. far more expert than C , and so should distrust C as well. Ultimately, understanding how these positive and negative relationships work is impor- tant for understanding the role they play on social Web sites where users register subjective evaluations of each other. Research is only beginning to explore these fundamental questions, including the ways in which theories of balance — as well as related theories — can be used to shed light on these issues in large-scale datasets [274]. 5.4 A Weaker Form of Structural Balance In studying models of positive and negative relationships on networks, researchers have also formulated alternate notions of structural balance, by revisiting the original assumptions we

144 130 CHAPTER 5. POSITIVE AND NEGATIVE RELATIONSHIPS used to motivate the framework. In particular, our analysis began from the claim that there are two kinds of structures on a group of three people that are inherently unbalanced: a triangle with two positive edges and one negative edge (as in Figure 5.1(b)); and a triangle with three negative edges (as in Figure 5.1(d)). In each of these cases, we argued that the relationships within the triangle contained a latent source of stress that the network might try to resolve. The underlying arguments in the two cases, however, were fundamentally different. In a triangle with two positive edges, we have the problem of a person whose two friends don’t get along; in a triangle with three negative edges, there is the possibility that two of the nodes will ally themselves against the third. James Davis and others have argued that in many settings, the first of these factors may be significantly stronger than the second [127]: we may see friends of friends trying to recon- cile their differences (resolving the lack of balance in Figure 5.1(b)), while at the same time there could be less of a force leading any two of three mutual enemies (as in Figure 5.1(d)) to become friendly. It therefore becomes natural to ask what structural properties arise when we rule out only triangles with exactly two positive edges, while allowing triangles with three negative edges to be present in the network. Characterizing Weakly Balanced Networks. More precisely, we will say that a com- plete graph, with each edge labeled by + or − , is weakly balanced if the following property holds. Weak Structural Balance Property: There is no set of three nodes such that the edges among them consist of exactly two positive edges and one negative edge. Since weak balance imposes less of a restriction on what the network can look like, we should expect to see a broader range of possible structures for weakly balanced networks — beyond what the Balance Theorem required for networks that were balanced under our original definition. And indeed, Figure 5.6 indicates a new kind of structure that can arise. Suppose that the nodes can be divided into an arbitrary number of groups (possibly more than two), so that two nodes are friends when they belong to the same group, and enemies when they belong to different groups. Then we can check that such a network is weakly balanced: in any triangle that contains at least two positive edges, all three nodes must belong to the same group. Therefore, the third edge of this triangle must be positive as well — in other words, the network contains no triangles with exactly two + edges. Just as the Balance Theorem established that all balanced networks must have a simple structure, an analogous result holds for weakly balanced networks: they must have the structure depicted in Figure 5.6, with any number of groups. Characterization of Weakly Balanced Networks: If a labeled complete graph is weakly balanced, then its nodes can be divided into groups in such a way that

145 5.4. A WEAKER FORM OF STRUCTURAL BALANCE 131 B D ? + - ? A - + E C friends of enemies of A A Figure 5.7: A schematic illustration of our analysis of weakly balanced networks. (There may be other nodes not illustrated here.) every two nodes belonging to the same group are friends, and every two nodes belonging to different groups are enemies. The fact that this characterization is true in fact provided another early motivation for studying weak structural balance. The Cartwright-Harary notion of balance predicted only dichotomies (or mutual consensus) as its basic social structure, and thus did not provide a model for reasoning about situations in which a network is divided into more than two factions. Weak structural balance makes this possible, since weakly balanced complete graphs can contain any number of opposed groups of mutual friends [127]. Proving the Characterization. It is not hard to give a proof for this characteriza- tion, following the structure of our proof for the Balance Theorem, and making appropriate changes where necessary. Starting with a weakly balanced complete graph, the characteriza- tion requires that we produce a division of its nodes into groups of mutual friends, such that all relations between nodes in different groups are negative. Here is how we will construct this division. First, we pick any node A , and we consider the set consisting of A and all its friends. Let’s call this set of nodes X . We’d like to make X our first group, and for this to work, we need to establish two things: (i) All of A ’s friends are friends with each other. (This way, we have indeed produced a group of mutual friends).

146 132 CHAPTER 5. POSITIVE AND NEGATIVE RELATIONSHIPS A and all his friends are enemies with everyone else in the graph. (This way, the people (ii) in this group will be enemies with everyone in other groups, however we divide up the rest of the graph.) Fortunately, ideas that we already used inside the proof of the Balance Theorem can be adapted to our new setting here to establish (i) and (ii). The idea is shown in Figure 5.7. and C who are both friends with First, for (i), let’s consider two nodes . If B and C B A A , B , and were enemies of each other, then the triangle on nodes would have exactly two C + labels, which would violate weak structural balance. So and C must indeed be friends B with each other. For (ii), we know that is enemies with all nodes in the graph outside X , since the A X group A ’s friends. How about an edge between a node B in X is defined to include all of and a node D outside X ? If B and D were friends, then the triangle on nodes A , B , and D would have exactly two + labels — again, a violation of weak structural balance. So B and must be enemies. D X — consisting of A and all his Since properties (i) and (ii) hold, we can remove the set friends — from the graph and declare it to be the first group. We now have a smaller complete graph that is still weakly balanced; we find a second group in this graph, and proceed to remove groups in this way until all the nodes have been assigned to a group. Since each group consists of mutual friends (by property (i)), and each group has only negative relations with everyone outside the group (by property (ii)), this proves the characterization. It is interesting to reflect on this proof in relation to the proof of the Balance Theorem — in particular, the contrast reflected by the small differences between Figures 5.4 and 5.7. In proving the Balance Theorem, we had to reason about the sign of the edge between D and E , to show that the enemies of the set X themselves formed a set Y of mutual friends. In characterizing weakly balanced complete graphs, on the other hand, we made no attempt to D - E edge, because weak balance imposes no condition on it: two enemies reason about the of A can be either friends or enemies. As a result, the set of enemies in Figure 5.7 might not be a set of mutual friends when only weak balance holds; it might consist of multiple groups of mutual friends, and as we extract these groups one by one over the course of the proof, we recover the multi-faction structure illustrated schematically in Figure 5.6. 5.5 Advanced Material: Generalizing the Definition of Structural Balance In this section, we consider more general ways of formulating the idea of structural balance in a network. In particular, our definition of structural balance thus far is fairly demanding in two respects:

147 5.5. ADVANCED MATERIAL: GENERALIZING THE DEFINITION OF STRUCTURAL BALANCE 133 1 + + + 2 3 - - + - 4 5 6 + - - - 7 8 - - 9 10 1 1 + - + + - 14 + 12 13 - - 15 Figure 5.8: In graphs that are not complete, we can still define notions of structural balance when the edges that are present have positive or negative signs indicating friend or enemy relations. 1. It applies only to complete graphs: we require that each person know and have an opinion (positive or negative) on everyone else. What if only some pairs of people know each other? 2. The Balance Theorem, showing that structural balance implies a global division of the world into two factions [97, 204], only applies to the case in which every triangle is balanced. Can we relax this to say that if most triangles are balanced, then the world can be approximately divided into two factions? In the two parts of this section, we discuss a pair of results that address these questions. The first is based on a graph-theoretic analysis involving the notion of breadth-first search from Chapter 2, while the second is typical of a style of proof known as a “counting argument.” Throughout this section, we will focus on the original definition of structural balance from Sections 5.1 and 5.2, rather than the weaker version from Section 5.4.

148 134 CHAPTER 5. POSITIVE AND NEGATIVE RELATIONSHIPS 1 1 - - + + - 5 5 2 2 - - - - + + - + + + 4 3 4 3 (a) (b) Filling in the missing edges to A graph with signed edges. achieve balance. Y 1 X - + 5 2 - + + 3 4 Dividing the graph into two sets. (c) Figure 5.9: There are two equivalent ways to define structural balance for general (non-complete) graphs. One definition asks whether it is possible to fill in the remaining edges so as to produce a signed complete graph that is balanced. The other definition asks whether it is possible to divide the nodes into two sets X and Y so that all edges inside X and inside Y are positive, and all edges between X and Y are negative. A. Structural Balance in Arbitrary (Non-Complete) Networks First, let’s consider the case of a social network that is not necessarily complete — that is, there are only edges between certain pairs of nodes, but each of these edges is still labeled with + or − . So now there are three possible relations between each pair of nodes: a positive edge, indicating friendship; a negative edge, indicating enmity; or the absence of an edge, indicating that the two endpoints do not know each other. Figure 5.8 depicts an example of such a signed network. Defining Balance for General Networks. Drawing on what we’ve learned from the special case of complete graphs, what would be a good definition of balance for this more general kind of structure? The Balance Theorem suggests that we can view structural balance

149 5.5. ADVANCED MATERIAL: GENERALIZING THE DEFINITION OF STRUCTURAL BALANCE 135 local view, as a condition on each triangle of the network; in either of two equivalent ways: a view, as a requirement that the world be divided into two mutually opposed sets or a global of friends. Each of these suggests a way of defining structure balance for general signed graphs. 1. One option would be to treat balance for non-complete networks as a problem of filling in “missing values.” Suppose we imagine, as a thought experiment, that all people in the group in fact do know and have an opinion on each other; the graph under consideration is not complete only because we have failed to observe the relations between some of the pairs. We could then say that the graph is balanced if it possible to fill in all the missing labeled edges in such a way that the resulting signed complete graph is balanced. In other words, a (non-complete) graph is balanced if it can be “completed” by adding edges to form a signed complete graph that is balanced. For example, Figure 5.9(a) shows a graph with signed edges, and Figure 5.9(b) shows how the remaining edges can be “filled in” to produce a balanced complete graph: we declare the missing edge between nodes 3 and 5 to be positive, and the remaining missing edges to be negative, and one can check that this causes all triangles to be balanced. 2. Alternately, we could take a more global view, viewing structural balance as implying a division of the network into two mutually opposed sets of friends. With this in mind, we could define a signed graph to be balanced if it is possible to divide the nodes into X two sets Y , such that any edge with both ends inside X or both ends inside Y and is positive, and any edge with one end in X and the other in Y is negative. That is, people in X are all mutual friends to the extent that they know each other; the same Y to the extent X are all enemies of people in Y is true for people in ; and people in that they know each other. Continuing the example from Figure 5.9(a), in Figure 5.9(c) we show how to divide this graph into two sets with the desired properties. This example hints at a principle that is true in general: these two ways of defining balance are equivalent. An arbitrary signed graph is balanced under the first definition if and only if it is balanced under the second definition. This is actually not hard to see. If a signed graph is balanced under the first definition, then after filling in all the missing edges appropriately, we have a signed complete graph to which we can apply the Balance Theorem. This gives us a division of the network into two X and Y that satisfies the properties of the second definition. On the other hand, if sets a signed graph is balanced under the second definition, then after finding a division of the nodes into sets , and fill in and Y , we can fill in positive edges inside X and inside Y X

150 136 CHAPTER 5. POSITIVE AND NEGATIVE RELATIONSHIPS X 1 - + X X 5 2 label as X or Y - - + 4 3 Y Y Figure 5.10: If a signed graph contains a cycle with an odd number of negative edges, then it is not balanced. Indeed, if we pick one of the nodes and try to place it in X , then following the set of friend/enemy relations around the cycle will produce a conflict by the time we get to the starting node. negative edges between X and Y , and then we can check that all triangles will be balanced. So this gives a “filling-in” that satisfies the first definition. The fact that the two definitions are equivalent suggests a certain “naturalness” to the definition, since there are fundamentally different ways to arrive at it. It also lets us use either definition, depending on which is more convenient in a given situation. As the example in Figure 5.9 suggests, the second definition is generally more useful to work with — it tends to be much easier to think about dividing the nodes into two sets than to reason about filling in edges and checking triangles. Characterizing Balance for General Networks. Conceptually, however, there is some- thing not fully satisfying about either definition: the definitions themselves do not provide much insight into how to easily check that a graph is balanced. There are, after all, lots of ways to choose signs for the missing edges, or to choose ways of splitting the nodes into sets X and Y . And if a graph is not balanced, so that there is no way to do these things suc- cessfully, what could you show someone to convince them of this fact? To take just a small example to suggest some of the difficulties, it may not be obvious from a quick inspection of Figure 5.8 that this is not a balanced graph — or that if we change the edge connecting nodes 2 and 4 to be positive instead of negative, it becomes a balanced graph. In fact, however, all these problems can be remedied if we explore the consequences of

151 5.5. ADVANCED MATERIAL: GENERALIZING THE DEFINITION OF STRUCTURAL BALANCE 137 the definitions a little further. What we will show is a simple characterization of balance in general signed graphs, also due to Harary [97, 204]; and the proof of this characterization also provides an easy method for checking whether a graph is balanced. The characterization is based on considering the following question: what prevents a graph from being balanced? Figure 5.10 shows a graph that is not balanced (obtained from Figure 5.9(a) and changing the sign of the edge from node 4 to node 5). It also illustrates a reason why it’s not balanced, as follows. If we start at node 1 and try to divide the nodes and Y into sets X , then our choices are forced at every step. Suppose we initially decide X . (For the first node, it doesn’t matter, by symmetry.) Then that node 1 should belong to X since node 2 is friends with node 1, it too must belong to . Node 3, an enemy of 2, must therefore belong to ; hence node 4, a friend of 3, must belong to Y as well; and node Y X 5, an enemy of 4, must belong to . The problem is that if we continue this reasoning one step further, then node 1, an enemy of 5, should belong to Y — but we had already decided at the outset to put it into X . We had no freedom of choice during this process — so this shows that there is no way to divide the nodes in sets X and Y so as to satisfy the mutual-friend/mutual-enemy conditions of structural balance, and hence the signed graph in Figure 5.10 is not balanced. The reasoning in the previous paragraph sounds elaborate, but in fact it followed a simple principle: we were walking around a cycle, and every time we crossed a negative edge, we had to change the set into which we were putting nodes. The difficulty was that getting back around to node 1 required crossing an odd number of negative edges, and so our original decision to put node 1 into clashed with the eventual conclusion that node 1 ought to be X in Y . This principle applies in general: if the graph contains a cycle with an odd number of negative edges, then this implies the graph is not balanced. Indeed, if we start at any node A in the cycle and place it in one of the two sets, and then we walk around the cycle placing the other nodes where they must go, the identity of the set where we’re placing nodes switches an odd number of times as we go around the cycle. Thus we end up with the “wrong set” by the time we make it back to A . A cycle with an odd number of negative edges is thus a very simple-to-understand reason why a graph is not balanced: you can show someone such a cycle and immediately convince them that the graph is not balanced. For example, the cycle back in Figure 5.8 consisting of nodes, 2, 3, 6, 11, 13, 12, 9, 4 contains five negative edges, thus supplying a succinct reason why this graph is not balanced. But are there other, more complex reasons why a graph is not balanced? In fact, though it may seem initially surprising, cycles with an odd number of negative edges are the only obstacles to balance. This is the crux of the following claim [97, 204]. Claim: A signed graph is balanced if and only if it contains no cycle with an odd

152 138 CHAPTER 5. POSITIVE AND NEGATIVE RELATIONSHIPS 1 + + + 2 3 - - + - 6 4 5 + - - 8 7 - - - 10 1 9 1 1 1 + - - + + 14 + 12 13 - - 15 Figure 5.11: To determine if a signed graph is balanced, the first step is to consider only the positive edges, find the connected components using just these edges, and declare each of these components to be a supernode . In any balanced division of the graph into X and Y , all nodes in the same supernode will have to go into the same set. number of negative edges. We now show how to prove this claim; this is done by designing a method that analyzes the graph and either finds a division into the desired sets and Y , or else finds a cycle with an X odd number of negative edges. Proving the Characterization: Identifying Supernodes. Let’s recall what we’re try- ing to do: find a division of the nodes into sets and Y so that all edges inside X and Y are X positive, and all edges crossing between X and Y are negative. We will call a partitioning into sets and Y with these properties a balanced division . We now describe a procedure X that searches for a balanced division of the nodes into sets X and Y ; either it succeeds, or it stops with a cycle containing an odd number of negative edges. Since these are the only two possible outcomes for the procedure, this will give a proof of the claim. The procedure works in two main steps: the first step is to convert the graph to a reduced one in which there are only negative edges, and the second step is to solve the problem on this reduced graph. The first step works as follows. Notice that whenever two nodes are

153 5.5. ADVANCED MATERIAL: GENERALIZING THE DEFINITION OF STRUCTURAL BALANCE 139 A + - + B + + + + and Figure 5.12: Suppose a negative edge connects two nodes that belong to the same A B A supernode. Since there is also a path consisting entirely of positive edges that connects and through the inside of the supernode, putting this negative edge together with the B all-positive path produces a cycle with an odd number of negative edges. X Y in a connected by a positive edge, they must belong to the same one of the sets or balanced division. So we begin by considering what the connected components of the graph would be if we were to only consider positive edges. These components can be viewed as a set of contiguous “blobs” in the overall graph, as shown in Figure 5.11. We will refer to each of these blobs as a supernode : each supernode is connected internally via positive edges, and the only edges going between two different supernodes are negative. (If there were a positive edge linking two different supernodes, we should have combined them together into a single supernode.) Now, if any supernode contains a negative edge between some pair of nodes A B , and then we already have a cycle with an odd number of negative edges, as illustrated in the example of Figure 5.12. Consider the path of positive edges that connects and B inside A A and B . the supernode, and then close off a cycle by including the negative edge joining This cycle has only a single negative edge, linking A and B , and so it shows that the graph is not balanced. If there are no negative edges inside any of the supernodes, then there is no “internal” problem with declaring each supernode to belong entirely to one of X or Y . So the problem is now how to assign a single label “ X ” or “ Y ” to each supernode, in such a way that these choices are all consistent with each other Since the decision-making is now at the level of supernodes, we create a new version of the problem in which there is a node for each

154 140 CHAPTER 5. POSITIVE AND NEGATIVE RELATIONSHIPS Figure 5.13: The second step in determining whether a signed graph is balanced is to look for a labeling of the supernodes so that adjacent supernodes (which necessarily contain mutual enemies) get opposite labels. For this purpose, we can ignore the original nodes of the graph reduced graph whose nodes are the supernodes of the original graph. and consider a supernode, and an edge joining two supernodes if there is an edge in the original that connects the two supernodes. Figure 5.13 shows how this works for the example of Figure 5.11: we essentially forget about the individual nodes inside the supernodes, and build a new graph at the level of the large “blobs.” Of course, having done so, we can draw the graph in a less blob-like way, as in Figure 5.14. We now enter the second step of the procedure, using this reduced graph whose nodes are the supernodes of the original graph. Re- Proving the Characterization: Breadth-First Search of the Reduced Graph. call that only negative edges go between supernodes (since a positive edge between two su- pernodes would have merged them together into a single one). As a result, our reduced graph has only negative edges. The remainder of the procedure will produce one of two possible outcomes. 1. The first possible outcome is to label each node in the reduced graph as either X or Y , in such a way that every edge has endpoints with opposite labels. From this we

155 5.5. ADVANCED MATERIAL: GENERALIZING THE DEFINITION OF STRUCTURAL BALANCE 141 A - - E B - - - F - D C - - G Figure 5.14: A more standard drawing of the reduced graph from the previous figure. A negative cycle is visually apparent in this drawing. can create a balanced division of the original graph, by labeling each node the way its supernode is labeled in the reduced graph. 2. The second possible outcome will be to find a cycle in the reduced graph that has an odd number of edges. We can then convert this to a (potentially longer) cycle in the original graph with an odd number of negative edges: the cycle in the reduced graph connects supernodes, and corresponds to a set of negative edges in the original graph. We can simply “stitch together” these negative edges using paths consisting entirely of positive edges that go through the insides of the supernodes. This will be a path containing an odd number of negative edges in the original graph. For example, the odd-length cycle in Figure 5.14 through nodes A through can be E realized in the original graph as the darkened negative edges shown in Figure 5.15. This can then be turned into a cycle in the original graph by including paths through the supernodes – in this example using the additional nods 3 and 12. In fact, this version of the problem when there are only negative edges is known in graph theory as the problem of determining whether a graph is : whether its nodes can be bipartite divided into two groups (in this case X and Y ) so that each edge goes from one group to the other. We saw bipartite graphs when we considered affiliation networks in Chapter 4, but there the fact that the graphs were bipartite was apparent from the ready-made division of the nodes into people and social foci. Here, on the other hand, we are handed a graph “in

156 142 CHAPTER 5. POSITIVE AND NEGATIVE RELATIONSHIPS 1 + + + 3 2 - - + - 6 5 4 + - - 8 7 - - - 1 1 1 10 9 1 + - - + + 14 + 12 13 - - 15 Figure 5.15: Having found a negative cycle through the supernodes, we can then turn this into a cycle in the original graph by filling in paths of positive edges through the inside of the supernodes. The resulting cycle has an odd number of negative edges. the wild,” with no pre-specified division into two sets, and we want to know if it is possible to identify such a division. We now show a way to do this using the idea of breadth-first search from Chapter 2, resulting either in the division we seek, or in a cycle of odd length. We simply perform breadth-first search starting from any “root” node in the graph, producing a set of layers at increasing distances from this root. Figure 5.16 shows how this is done for the reduced graph in Figure 5.14, with node G as the starting root node. Now, because edges cannot jump over successive layers in breadth-first search, each edge either connects two nodes in adjacent layers or it connects two nodes in the same layer. If all edges are of the first type, then we can find the desired division of nodes into sets X and Y : we simply declare all nodes in even-numbered layers to belong to X , and all nodes in odd-numbered layers to belong to Y . Since edges only go between adjacent layers, all edges have one end in and the other end in Y , as desired. X Otherwise, there is an edge connecting two nodes that belong to the same layer. Let’s call them A and B (as they are in Figure 5.16). For each of these two nodes, there is a path that descends layer-by-layer from the root to it. Consider the last node that is common to these two paths — let’s call this node D (as it is in Figure 5.16). The D - A path and the

157 5.5. ADVANCED MATERIAL: GENERALIZING THE DEFINITION OF STRUCTURAL BALANCE 143 G D F An odd cycle is C E formed from two equal-length paths leading to an edge inside a single layer . B A Figure 5.16: When we perform a breadth-first search of the reduced graph, there is either an edge connecting two nodes in the same layer or there isn’t. If there isn’t, then we can produce the desired division into and Y by putting alternate layers in different sets. If X there is such an edge (such as the edge joining and B in the figure), then we can take two A paths of the same length leading to the two ends of the edge, which together with the edge itself forms an odd cycle. D - B path have the same length k , so a cycle created from the two of these plus the A - B edge must have length 2 + 1: an odd number. This is the odd cycle we seek. k And this completes the proof. To recap: if all edges in the reduced graph connect nodes in adjacent layers of the breadth-first search, then we have a way to label the nodes in the reduced graph as into X and Y , which in turn provides a balanced division of the nodes in the original graph into X and Y . In this case, we’ve established that the graph is balanced. Otherwise, there is an edge connecting two nodes in the same layer of the breadth-first search, in which case we produce an odd cycle in the reduced graph as in Figure 5.16. In this case, we can convert into this to a cycle in the original graph containing an odd number of negative edges, as in Figure 5.15. Since these are the only two possibilities, this proves the claim.

158 144 CHAPTER 5. POSITIVE AND NEGATIVE RELATIONSHIPS B. Approximately Balanced Networks We now return to the case in which the graph is complete, so that every node has a positive or negative relation with every other node, and we think about a different way of generalizing the characterization of structural balance. First let’s write down the original Balance Theorem again, with some additional format- ting to make its logical structure clear. Claim: If all triangles in a labeled complete graph are balanced, then either (a) all pairs of nodes are friends, or else and Y , such that (b) the nodes can be divided into two groups, X like each other, X (i) every pair of nodes in (ii) every pair of nodes in Y like each other, and X is the enemy of everyone in Y (iii) everyone in . The conditions of this theorem are fairly extreme, in that we require every single triangle to be balanced. What if we only know that most triangles are balanced? It turns out that the conditions of the theorem can be relaxed in a very natural way, allowing us to prove statements like the following one. We phrase it so that the wording remains completely parallel to that of the Balance Theorem. Claim: If at least 99 . 9% of all triangles in a labeled complete graph are balanced, then either (a) there is a set consisting of at least of the nodes in which at least 90% 90% of all pairs are friends, or else (b) the nodes can be divided into two groups, X and Y , such that (i) at least 90% of the pairs in X like each other, (ii) at least 90% Y like each other, and of the pairs in are 90% X (iii) at least Y of the pairs with one end in and the other end in enemies. This is a true statement, though the choice of numbers is very specific. Here is a more general statement that includes both the Balance Theorem and the preceding claim as special cases. √ 1 3 δ . If at least , and define = ε ≤ ε be any number such that Claim: Let ε < 0 8 1 − ε of all triangles in a labeled complete graph are balanced, then either (a) there is a set consisting of at least 1 − δ of the nodes in which at least 1 − δ of all pairs are friends, or else

159 5.5. ADVANCED MATERIAL: GENERALIZING THE DEFINITION OF STRUCTURAL BALANCE 145 X (b) the nodes can be divided into two groups, and Y , such that δ X like each other, − of the pairs in 1 (i) at least (ii) at least of the pairs in Y like each other, and 1 − δ − δ of the pairs with one end in X and the other end in Y are (iii) at least 1 enemies. ε = 0, and the other claim above is Notice that the Balance Theorem is the case in which √ 3 . 001 (since in this latter case, δ the case in which ε = ε = . 1). = We now prove this last claim. The proof is self-contained, but it is most easily read with some prior experience in what is sometimes called the analysis of “permutations and combinations” — counting the number of ways to choose particular subsets of larger sets. The proof loosely follows the style of the proof we used for the Balance Theorem: we and to be the friends and enemies, respectively, of a designated will define the two sets Y X node A will give us the A . Things are trickier here, however, because not all choices of structure we need — in particular, if a node is personally involved in too many unbalanced triangles, then splitting the graph into its friends and enemies may give a very disordered structure. Consequently, the proof consists of two steps. We first find a “good” node that is not involved in too many unbalanced triangles. We then show that if we divide the graph into the friends and enemies of this good node, we have the desired properties. Warm-Up: Counting Edges and Triangles. Before launching into the proof itself, let’s consider some basic counting questions that will show up as ingredients in the proof. Recall that we have a complete graph, with an (undirected) edge joining each pair of nodes. If is the number of nodes in the graph, how many edges are there? We can count this N quantity as follows. There are N possible ways to choose one of the two endpoints, and then N − 1 possible ways to choose a different node as the other endpoint, for a total of N ( N − 1) possible ways to choose the two endpoints in succession. If we write down a list of all these possible pairs of endpoints, then an edge with endpoints and B will appear twice on the A AB BA . In general, each edge will appear twice on the list, and so list: once as and once as N ( N − 1) / 2. the total number of edges is A very similar argument lets us count the total number of triangles in the graph. Specif- ically, there are ways to pick the first corner, then N − N 1 ways to pick a different node as the second corner, and then N − 2 ways to pick a third corner different from the first two. This yields a total of N ( N − 1)( N − 2) sequences of three corners. If we write down this 2) sequences, then a triangle with corners list of N − 1)( N − ( A , B , and C will appear six N times: as ABC,ACB,BAC,BCA,CAB, and CBA . In general, each triangle will appear six times in this list, and so the total number of triangles is 1)( ( N − N N − 2) . 6

160 146 CHAPTER 5. POSITIVE AND NEGATIVE RELATIONSHIPS The First Step: Finding a “Good” Node. Now let’s move on to the first step of the proof, which is to find a node that isn’t involved in too many unbalanced triangles. ε fraction of triangles are unbalanced, and the Since we are assuming that at most an − − total number of triangles in the graph is N N 2) / 6, it follows that the total number ( N 1)( ( of unbalanced triangles is at most − 1)( N − 2) / 6. Suppose we define the weight of a εN N node to be the number of unbalanced triangles that it is a part of; thus, a node of low weight will be precisely what we’re seeking — a node that is in relatively few unbalanced triangles. One way to count the total weight of all nodes would be to list — for each node — the unbalanced triangles that it belongs to, and then look at the length of all these lists combined. In these combined lists, each triangle will appear three times — once in the list for each of its corners — and so the total weight of all nodes is exactly three times the number of unbalanced triangles. As a result, the total weight of all nodes is at most εN ( N − 1)( N − 2) / 6 = εN ( N − 1)( N − 2) / 2. 3 N N average weight of a node is at most ε ( N − 1)( nodes, so the − 2) / 2. It’s There are not possible for all nodes to have weights that are strictly above the average, so there is at least one node whose weight is equal to the average or below it. Let’s pick one such node and 2 A . This will be our “good” node: a node whose weight is at most ε ( N − 1)( N − 2) / 2. call it 2 2 Since ( N − 2) < N N , this good node is in at most εN 1)( / 2 triangles, and because the − algebra is a bit simpler with this slightly larger quantity, we will use it in the rest of the analysis. By analogy The Second Step: Splitting the Graph According to the Good Node. X consisting with the proof of the Balance Theorem, we divide the graph into two sets: a set A , as illustrated in Y consisting of all the enemies of A of and all its friends, and a set A Figure 5.17. Now, using the definition of unbalanced triangles, and the fact that node is not involved in too many of them, we can argue that there are relatively few negative edges inside each of X and Y , and relatively few positive edges between them. Specifically, this works as follows. • Each negative edge connecting two nodes in X creates a distinct unbalanced triangle 2 involving node εN A / 2 unbalanced triangles involving A , . Since there are at most 2 εN / 2 negative edges inside X . there are at most • A closely analogous argument applies to Y : Each negative edge connecting two nodes in creates a distinct unbalanced triangle involving node A , and so there are at most Y 2 Y / 2 negative edges inside εN . 2 This is a very common trick in counting arguments, referred to as the pigeonhole principle : to compute the average value of a set of objects, and then argue that there must be at least one node that is equal to the average or below. (Also, of course, there must be at least at least one object that is equal to the average or above, although this observation isn’t useful for our purposes here.)

161 5.5. ADVANCED MATERIAL: GENERALIZING THE DEFINITION OF STRUCTURAL BALANCE 147 mainly negative D B - + mainly mainly A positive positive - + E C A A enemies of A friends of a good node Figure 5.17: The characterization of approximately balanced complete graphs follows from an analysis similar to the proof of the original Balance Theorem. However, we have to be more careful in dividing the graph by first finding a “good” node that isn’t involved in too many unbalanced triangles. X and the other • And finally, an analogous argument applies to edges with one end in . Each such edge that is positive creates a distinct unbalanced triangle in- Y end in 2 εN A volving 2 positive edges with one end in X and the , and so there are at most / Y . other end in X We now consider several possible cases, depending on the sizes of the sets . Essen- and Y or consists of almost the entire graph, then we show that alternative X Y tially, if either of (a) in the claim holds. Otherwise, if each of Y contain a non-negligible number of X and nodes, then we show that alternative (b) in the claim holds. We’re also going to assume, to N is even and that the quantity δN is a whole number, make the calculations simpler, that although this is not in fact necessary for the proof. To start, let be the number of nodes in X and y be the number of nodes in Y . Suppose x √ 1 1 1 3 ε , it follows that δ < . Since ε < first that . Now, and δ = x ≥ (1 − δ ) N , and so x > N 2 2 8 recall our earlier counting argument that gave a formula for the number of edges in a complete ( has x nodes, so it has graph, in terms of its number of nodes. In this case, X x − 1) / 2 x 1 1 1 1 2 2 8. / N + 1)( x > N N ) / 2 ≥ ( edges. Since N N ) , this number of edges is at least ( / 2 = 2 2 2 2 2 2 negative edges inside / There are at most X , and so the fraction of negative edges εN X is at most inside 2 εN / 2 3 = 4 ε = 4 δ < δ, 2 N / 8

162 148 CHAPTER 5. POSITIVE AND NEGATIVE RELATIONSHIPS 1 3 . We thus conclude that if X contains at least ε where we use the facts that and δ < = δ 2 − N nodes, then it is a set containing at least a 1 − δ fraction of the nodes in which at ) δ (1 of all pairs are friends, satisfying part (a) in the conclusion of the claim. δ − least 1 contains at least (1 − δ ) N The same argument can be applied if Y nodes. Thus we are X Y contain strictly fewer than (1 − δ ) N and left with the case in which both , and in this case we will show that part (b) in the conclusion of the claim holds. First, of all the edges with one end in X and the other in Y , what fraction are positive? The total number of edges X and the other end in Y can be counted as follows: there are x ways to with one end in X choose the end in y ways to choose the end in Y , for a total of xy such edges. , and then , and they add up to x are less than (1 − δ ) N y N , this product xy Now, since each of and 2 2 − δ ) N = δ (1 − is at least ( ) N δN ≥ δN )(1 / 2 , where the last inequality follows from the δ 1 2 . There are at most εN δ < / 2 positive edges with one end in X and the other fact that 2 in Y , so as a fraction of the total this is at most 2 εN / 2 ε 2 < δ. δ = = 2 δ 2 δN / Finally, what fraction of edges inside each of X and Y are negative? Let’s calculate X ; the argument for Y is exactly the same. There are x ( x − 1) / 2 edges inside X this for x > δN , this total number of edges is at least in total, and since we’re in the case where 2 2 2 2 ) / 2 ≥ ( δN ) εN / 2 = δ ( N δN / 2. There are at most + 1)( δN / 2 negative edges inside X , so as a fraction of the total this is at most 2 ε / 2 εN δ. = = 2 2 2 δ / N 2 δ X and Y satisfies all the requirements in conclusion (b) Thus, the division of nodes into sets of the claim, and so the proof is complete. As a final comment on the claim and its proof, one might feel that the difference between √ 3 ε in the assumption of the claim and 1 − 1 − ε is a bit excessive: as we saw above, when ε . 001, this means we need to assume that 99 . 9% of all triangles are balanced in order to = get sets with a 90% density of edges having the correct sign. But in fact, it is possible to construct examples showing that this relationship between ε and δ is in fact essentially the best one can do. In short, the claim provides the kind of approximate version of the Balance Theorem that we wanted at a qualitative level, but we need to assume a fairly small fraction of unbalanced triangles in order to be able to start drawing strong conclusions. 5.6 Exercises 1. Suppose that a team of anthropologists is studying a set of three small villages that neighbor one another. Each village has 30 people, consisting of 2-3 extended families.

163 5.6. EXERCISES 149 Everyone in each village knows all the people in their own village, as well as the people in the other villages. When the anthropologists build the social network on the people in all three villages taken together, they find that each person is friends with all the other people in their own village, and enemies with everyone in the two other villages. This gives them a network on 90 people (i.e., 30 in each village), with positive and negative signs on its edges. According to the definitions in this chapter, is this network on 90 people balanced? Give a brief explanation for your answer. 2. Consider the network shown in Figure 5.18: there is an edge between each pair of nodes, with five of the edges corresponding to positive relationships, and the other five of the edges corresponding to negative relationships. A + + B C - - - + - - + E + D Figure 5.18: A network with five positive edges and five negative edges. Each edge in this network participates in three triangles: one formed by each of the additional nodes who is not already an endpoint of the edge. (For example, the - B A edge participates in a triangle on A , B , and C , a triangle on A , B , and D , and a . We can list triangles for the other edges in a similar way.) triangle on , B , and E A For each edge, how many of the triangles it participates in are balanced, and how many are unbalanced. (Notice that because of the symmetry of the network, the answer will be the same for each positive edge, and also for each negative edge; so it is enough to consider this for one of the positive edges and one of the negative edges.) 3. When we think about structural balance, we can ask what happens when a new node

164 150 CHAPTER 5. POSITIVE AND NEGATIVE RELATIONSHIPS tries to join a network in which there is existing friendship and hostility. In Fig- ures 5.19–5.22, each pair of nodes is either friendly or hostile, as indicated by the + or − label on each edge. A + + C B + Figure 5.19: A 3-node social network in which all pairs of nodes know each other, and all pairs of nodes are friendly toward each other. D D + + + - - - A A + + + + B C C B + + (a) joins the network by becom- D (b) D joins the network by becom- ing friends with all nodes. ing enemies with all nodes. Figure 5.20: There are two distinct ways in which node D can join the social network from Figure 5.19 without becoming involved in any unbalanced triangles. First, consider the 3-node social network in Figure 5.19, in which all pairs of nodes know each other, and all pairs of nodes are friendly toward each other. Now, a fourth node wants to join this network, and establish either positive or negative relations D with each existing node A , B , and C . It wants to do this in such a way that it doesn’t become involved in any unbalanced triangles. (I.e. so that after adding D and the labeled edges from , there are no unbalanced triangles that contain D .) Is this D possible? In fact, in this example, there are two ways for D to accomplish this, as indicated in Figure 5.20. First, D can become friends with all existing nodes; in this way, all the

165 5.6. EXERCISES 151 triangles containing it have three positive edges, and so are balanced. Alternately, it can become enemies with all existing nodes; in this way, each triangle containing it has exactly one positive edge, and again these triangles would be balanced. to join without becoming involved in any D So for this network, it was possible for unbalanced triangles. However, the same is not necessarily possible for other networks. We now consider this kind of question for some other networks. A - - B C - Figure 5.21: All three nodes are mutual enemies. (a) Consider the 3-node social network in Figure 5.21, in which all pairs of nodes know each other, and each pair is either friendly or hostile as indicated by the − + or D wants to join this network, and label on each edge. A fourth node A , establish either positive or negative relations with each existing node , and B C . Can node D do this in such a way that it doesn’t become involved in any unbalanced triangles? • If there is a way for D to do this, say how many different such ways there are, and give an explanation. (That is, how many different possible labelings of the edges out of have the property that all triangles containing D are D balanced?) • D to do this, give an explanation why not. If there is no such way for (In this and the subsequent questions, it possible to work out an answer by rea- soning about the new node’s options without having to check all possibilities.) (b) Same question, but for a different network. Consider the 3-node social network in Figure 5.22, in which all pairs of nodes know each other, and each pair is either friendly or hostile as indicated by the + or − label on each edge. A fourth node D wants to join this network, and establish either positive or negative relations with each existing node A , B , and C . Can node D do this in such a way that it doesn’t become involved in any unbalanced triangles? • If there is a way for D to do this, say how many different such ways there are, and give an explanation. (That is, how many different possible labelings

166 152 CHAPTER 5. POSITIVE AND NEGATIVE RELATIONSHIPS A + + B C - A is friends with nodes B and C , who are enemies with each other. Figure 5.22: Node D of the edges out of D are have the property that all triangles containing balanced?) If there is no such way for D to do this, give an explanation why not. • (c) Using what you’ve worked out in Questions 2 and 3, consider the following ques- tion. Take any labeled complete graph — on any number of nodes — that is not balanced; i.e. it contains at least one unbalanced triangle. (Recall that a labeled complete graph is a graph in which there is an edge between each pair of nodes, − X wants to join this and each edge is labeled with either + or .) A new node network, by attaching to each node using a positive or negative edge. When, if X to do this in such a way that it does not become involved ever, is it possible for in any unbalanced triangles? Give an explanation for your answer. (Hint: Think about any unbalanced triangle in the network, and how X must attach to the nodes in it.) 4. Together with some anthropologists, you’re studying a sparsely populated region of a rain forest, where 50 farmers live along a 50-mile-long stretch of river. Each farmer lives on a tract of land that occupies a 1-mile stretch of the river bank, so their tracts exactly divide up the 50 miles of river bank that they collectively cover. (The numbers are chosen to be simple and to make the story easy to describe.) The farmers all know each other, and after interviewing them, you’ve discovered that each farmer is friends with all the other farmers that live at most 20 miles from him or her, and is enemies with all the farmers that live more than 20 miles from him or her. You build the signed complete graph corresponding to this social network, and you wonder whether it satisfies the Structural Balance property. This is the question: is the network structurally balanced or not? Provide an explanation for your answer.

167 Part II Game Theory 153

168

169 Chapter 6 Games In the opening chapter of the book, we emphasized that the “connectedness” of a complex social, natural, or technological system really means two things: first, an underlying structure of interconnecting links; and second, an interdependence in the behaviors of the individuals who inhabit the system, so that the outcome for any one depends at least implicitly on the combined behaviors of all. The first issue – network structure – was addressed in the first part of the book using graph theory. In this second part of the book, we study interconnectedness game theory at the level of behavior, developing basic models for this in the language of . Game theory is designed to address situations in which the outcome of a person’s decision depends not just on how they choose among several options, but also on the choices made by the people they are interacting with. Game-theoretic ideas arise in many contexts. Some contexts are literally games; for example, choosing how to target a soccer penalty kick and choosing how to defend against it can be modeled using game theory. Other settings are not usually called games, but can be analyzed with the same tools. Examples include the pricing of a new product when other firms have similar new products; deciding how to bid in an auction; choosing a route on the Internet or through a transportation network; deciding whether to adopt an aggressive or a passive stance in international relations; or choosing whether to use performance-enhancing drugs in a professional sport. In these examples, each decision-maker’s outcome depends on the decisions made by others. This introduces a strategic element that game theory is designed to analyze. As we will see later in Chapter 7, game-theoretic ideas are also relevant to settings where no one is overtly making decisions. Evolutionary biology provides perhaps the most striking example. A basic principle is that mutations are more likely to succeed in a population when they improve the fitness of the organisms that carry the mutation. But often, this fitness cannot be assessed in isolation; rather, it depends on what all the other (non-mutant) D. Easley and J. Kleinberg. Networks, Crowds, and Markets: Reasoning about a Highly Connected World . Cambridge University Press, 2010. Draft version: June 10, 2010. 155

170 156 CHAPTER 6. GAMES organisms are doing, and how the mutant’s behavior interacts with the non-mutants’ be- haviors. In such situations, reasoning about the success or failure of the mutation involves game-theoretic definitions, and in fact very closely resembles the process of reasoning about decisions that intelligent actors make. Similar kinds of reasoning have been applied to the success or failure of new cultural practices and conventions — it depends on the existing patterns of behavior into which they are introduced. This indicates that the ideas of game theory are broader than just a model of how people reason about their interactions with oth- ers; game theory more generally addresses the question of which behaviors tend to sustain themselves when carried out in a larger population. Game-theoretic ideas will appear in many places throughout the book. Chapters 8 and 9 describe two initial and fundamental applications: to network traffic, where travel time depends on the routing decisions of others; and to auctions, where the success of a bidder depends on how the other bidders behave. There will be many further examples later in the book, including the ways in which prices are set in markets and the ways in which people choose to adopt new ideas in situations where adoption decisions are affected by what others are doing. As a first step, then, we begin with a discussion of the basic ideas behind game theory. For now, this will involve descriptions of situations in which people interact with one an- other, initially without an accompanying graph structure. Once these ideas are in place, we will bring graphs back into the picture in subsequent chapters, and begin to consider how structure and behavior can be studied simultaneously. 6.1 What is a Game? Game theory is concerned with situations in which decision-makers interact with one another, and in which the happiness of each participant with the outcome depends not just on his or her own decisions but on the decisions made by everyone. To help make the definitions concrete, it’s useful to start with an example. A First Example. Suppose that you’re a college student, and you have two large pieces of work due the next day: an exam, and a presentation. You need to decide whether to study for the exam, or to prepare for the presentation. For simplicity, and to make the example as clean as possible, we’ll impose a few assumptions. First, we’ll assume you can either study for the exam or prepare for the presentation, but not both. Second, we’ll assume you have an accurate estimate of the expected grade you’ll get under the outcomes of different decisions. The outcome of the exam is easy to predict: if you study, then your expected grade is a 92, while if you don’t study, then your expected grade is an 80. The presentation is a bit more complicated to think about. For the presentation, you’re

171 6.1. WHAT IS A GAME? 157 doing it jointly with a partner. If both you and your partner prepare for the presentation, then the presentation will go extremely well, and your expected joint grade is a 100. If just one of you prepares (and the other doesn’t), you’ll get an expected joint grade of 92; and if neither of you prepares, your expected joint grade is 84. The challenge in reasoning about this is that your partner also has the same exam the next day, and we’ll assume that he has the same expected outcome for it: 92 if he studies, and 80 if he doesn’t. He also has to choose between studying for the exam and preparing for the presentation. We’ll assume that neither of you is able to contact the other, so you can’t jointly discuss what to do; each of you needs to make a decision independently, knowing that the other will also be making a decision. Both of you are interested in maximizing the average grade you get, and we can use the discussion above to work out how this average grade is determined by the way the two of you invest your efforts: • If both of you prepare for the presentation, you’ll both get 100 on the presentation and 80 on the exam, for an average of 90. • If both of you study for the exam, you’ll both get 92 on the exam and 84 on the presentation, for an average of 88. • If one of you studies for the exam while the other prepares for the presentation, the result is as follows. The one who prepares for the presentation gets a 92 on the presentation but only – an 80 on the exam, for an average of 86. – On the other hand, the one who studies for the exam still gets a 92 on the presentation — since it’s a joint grade, this person benefits from the fact that one of the two of you prepared for it. This person also get a 92 on the exam, through studying, and so gets an average of 92. There’s a simple tabular way to summarize all these outcomes, as follows. We represent your two choices — to prepare for the presentation, or to study for the exam — as the rows of a 2 × 2 table. We represent your partner’s two choices as the columns. So each box in this table represents a decision by each of you. In each box, we record the average grade you each receive: first yours, then your partner’s. Writing all this down, we have the table shown in Figure 6.1. This describes the set-up of the situation; now you need to figure out what to do: prepare for the presentation, or study for the exam? Clearly, your average grade depends not just on which of these two options you choose, but also on what your partner decides. Therefore, as part of your decision, you have to reason about what your partner is likely to do. Thinking

172 158 CHAPTER 6. GAMES Your Partner Presentation Exam 90 , Presentation 86 , 92 90 You 92 , Exam 88 , 88 86 Figure 6.1: Exam or Presentation? about the strategic consequences of your own actions, where you need to consider the effect of decisions by others, is precisely the kind of reasoning that game theory is designed to facilitate. So before moving on to the actual outcome of this exam-or-presentation scenario, it is useful to introduce some of the basic definitions of game theory, and then continue the discussion in this language. Basic Ingredients of a Game. The situation we’ve just described is an example of a game . For our purposes, a game is any situation with the following three aspects. (i) There is a set of participants, whom we call the players . In our example, you and your partner are the two players. (ii) Each player has a set of options for how to behave; we will refer to these as the player’s strategies . In the example, you and your partner each have two possible possible strategies: to prepare for the presentation, or to study for the exam. payoff that can depend on the (iii) For each choice of strategies, each player receives a strategies selected by everyone. The payoffs will generally be numbers, with each player preferring larger payoffs to smaller payoffs. In our current example, the payoff to each player is the average grade he or she gets on the exam and the presentation. We will generally write the payoffs in a as in Figure 6.1. payoff matrix Our interest is in reasoning about how players will behave in a given game. For now we focus on games with only two players, but the ideas apply equally well to games with any number of players. Also, we will focus on simple, one-shot games: games in which the players simultaneously and independently choose their actions, and they do so only once. In Section 6.10 at the end of this chapter, we discuss how to reinterpret the theory to deal with dynamic games, in which actions can be played sequentially over time. 6.2 Reasoning about Behavior in a Game Once we write down the description of a game, consisting of the players, the strategies, and the payoffs, we can ask how the players are likely to behave — that is, how they will go about selecting strategies.

173 6.2. REASONING ABOUT BEHAVIOR IN A GAME 159 In order to make this question tractable, we will make a Underlying Assumptions. few assumptions. First, we assume everything that a player cares about is summarized in the player’s payoffs. In the Exam-or-Presentation Game described in Section 6.1, this means that the two players are solely concerned with maximizing their own average grade. However, nothing in the framework of game theory requires that players care only about personal rewards. For example, a player who is altruistic may care about both his or her own benefits, and the other player’s benefit. If so, then the payoffs should reflect this; once the payoffs have been defined, they should constitute a complete description of each player’s happiness with each of the possible outcomes of the game. We also assume that each player knows everything about the structure of the game. To begin with, this means that each player knows his or her own list of possible strategies. It seems reasonable in many settings to assume that each player also knows who the other player is (in a two-player game), the strategies available to this other player, and what his or her payoff will be for any choice of strategies. In the Exam-or-Presentation Game, this corresponds to the assumption that you realize you and your partner are each faced with the choice of studying for the exam or preparing for the presentation, and you have an accurate estimate of the expected outcome under different courses of action. There is considerable research on how to analyze games in which the players have much less knowledge about the underlying structure, and in fact John Harsanyi shared the 1994 Nobel Prize in Economics for his work on games with incomplete information [208]. Finally, we suppose that each individual chooses a strategy to maximize her own payoff, given her beliefs about the strategy used by the other player. This model of individual behavior, which is usually called rationality , actually combines two ideas. The first idea is that each player wants to maximize her own payoff. Since the individual’s payoff is defined to be whatever the individual cares about, this hypothesis seems reasonable. The second idea is that each player actually succeeds in selecting the optimal strategy. In simple settings, and for games played by experienced players, this too seems reasonable. In complex games, or for games played by inexperienced players, it is surely less reasonable. It is interesting to consider players who make mistakes and learn from the play of the game. There is an extensive literature which analyzes problems of this sort [175], but we will not consider these issues here. Reasoning about Behavior in the Exam-or-Presentation Game. Let’s go back to the Exam-or-Presentation Game and ask how we should expect you and your partner — the two players in the game — to behave. We first focus on this from your point of view. (The reasoning for your partner will be symmetric, since the game looks the same from his point of view.) It would be easier to decide what to do if you could predict what your partner would do, but to begin with, let’s

174 160 CHAPTER 6. GAMES consider what you should do for each possible choice of strategy by your partner. • First, if you knew your partner was going to study for the exam, then you would get a payoff of 88 by also studying, and a payoff of only 86 by preparing for the presentation. So in this case, you should study for the exam. • On the other hand, if you knew that your partner was going to prepare for the pre- sentation, then you’d get a payoff of 90 by also preparing for the presentation, but a payoff of 92 by studying for the exam. So in this case too, you should study for the exam. This approach of considering each of your partner’s options separately turns out to be a very useful way of analyzing the present situation: it reveals that no matter what your partner does, you should study for the exam. When a player has a strategy that is strictly better than all other options regardless of what the other player does, we will refer to it as a strictly dominant strategy . When a player has a strictly dominant strategy, we should expect that they will definitely play it. In the Exam-or-Presentation Game, studying for the exam is also a strictly dominant strategy for your partner (by the same reasoning), and so we should expect that the outcome will be for both of you to study, each getting an average grade of 88. So this game has a very clean analysis, and it’s easy to see how to end up with a prediction for the outcome. Despite this, there’s something striking about the conclusion. If you and your partner could somehow agree that you would both prepare for the presentation, you would each get an average grade of 90 — in other words, you would each be better off. But despite the fact that you both understand this, this payoff of 90 cannot be achieved by rational play. The reasoning above makes it clear why not: even if you were to personally commit to preparing for the presentation — hoping to achieve the outcome where you both get 90 — and even if your partner knew you were doing this, your partner would still have an incentive to study for the exam so as to achieve a still-higher payoff of 92 for himself. This result depends on our assumption that the payoffs truly reflect everything each player values in the outcome — in this case, that you and your partner only care about maximizing your own average grade. If, for example, you cared about the grade that your partner received as well, then the payoffs in this game would look different, and the outcome could be different. Similarly, if you cared about the fact that your partner will be angry at you for not preparing for the joint presentation, then this too should be incorporated into the payoffs, again potentially affecting the results. But with the payoffs as they are, we are left with the interesting situation where there is an outcome that is better for both of you — an average grade of 90 each — and yet it cannot be achieved by rational play of the game.

175 6.2. REASONING ABOUT BEHAVIOR IN A GAME 161 The outcome of the Exam-or-Presentation A Related Story: The Prisoner’s Dilemma. Game is closely related to one of the most famous examples in the development of game the- . Here is how this example works. ory, the Prisoner’s Dilemma Suppose that two suspects have been apprehended by the police and are being interro- gated in separate rooms. The police strongly suspect that these two individuals are respon- sible for a robbery, but there is not enough evidence to convict either of them of the robbery. However, they both resisted arrest and can be charged with that lesser crime, which would carry a one-year sentence. Each of the suspects is told the following story. “If you confess, and your partner doesn’t confess, then you will be released and your partner will be charged with the crime. Your confession will be sufficient to convict him of the robbery and he will be sent to prison for 10 years. If you both confess, then we don’t need either of you to testify against the other, and you will both be convicted of the robbery. (Although in this case your sentence will be less — 4 years only — because of your guilty plea.) Finally, if neither of you confesses, then we can’t convict either of you of the robbery, so we will charge each of you with resisting arrest. Your partner is being offered the same deal. Do you want to confess?” To formalize this story as a game we need to identify the players, the possible strategies, and the payoffs. The two suspects are the players, and each has to choose between two possi- NC ( C ) or Not-Confess ble strategies — Confess ). Finally, the payoffs can be summarized ( from the story above as in Figure 6.2. (Note that the payoffs are all 0 or less, since there are no good outcomes for the suspects, only different gradations of bad outcomes.) Suspect 2 NC C NC 0 − 1 , − 1 − 10 , Suspect 1 C , − 10 − 4 0 − 4 , Figure 6.2: Prisoner’s Dilemma As in the Exam-or-Presentation Game, we can consider how one of the suspects — say Suspect 1 — should reason about his options. • If Suspect 2 were going to confess, then Suspect 1 would receive a payoff of − 4 by confessing and a payoff of 10 by not confessing. So in this case, Suspect 1 should − confess. • If Suspect 2 were not going to confess, then Suspect 1 would receive a payoff of 0 by confessing and a payoff of − 1 by not confessing. So in this case too, Suspect 1 should confess. So confessing is a strictly dominant strategy — it is the best choice regardless of what the other player chooses. As a result, we should expect both suspects to confess, each getting a

176 162 CHAPTER 6. GAMES − 4. payoff of We therefore have the same striking phenomenon as in the Exam-or-Presentation Game: there is an outcome that the suspects know to be better for both of them — in which they both choose not to confess — but under rational play of the game there is no way for them to achieve this outcome. Instead, they end up with an outcome that is worse for both of them. And here too, it is important that the payoffs reflect everything about the outcome of the game; if, for example, the suspects could credibly threaten each other with retribution for confessing, thereby making confessing a less desirable option, then this would affect the payoffs and potentially the outcome. Interpretations of the Prisoner’s Dilemma. The Prisoner’s Dilemma has been the subject of a huge amount of literature since its introduction in the early 1950s [343, 346], since it serves as a highly streamlined depiction of the difficulty in establishing cooperation in the face of individual self-interest. While no model this simple can precisely capture complex scenarios in the real world, the Prisoner’s Dilemma has been used as an interpretive framework for many different real-world situations. For example, the use of performance-enhancing drugs in professional sports has been modeled as a case of the Prisoner’s Dilemma game [210, 367]. Here the athletes are the players, and the two possible strategies are to use performance-enhancing drugs or not. If you use drugs while your opponent doesn’t, you’ll get an advantage in the competition, but you’ll suffer long-term harm (and may get caught). If we consider a sport where it is difficult to detect the use of such drugs, and we assume athletes in such a sport view the downside as a smaller factor than the benefits in competition, we can capture the situation with numerical payoffs that might look as follows. (The numbers are arbitrary here; we are only interested in their relative sizes.) Athlete 2 Use Drugs Don’t Use Drugs Don’t Use Drugs 3 , 3 1 , 4 Athlete 1 Use Drugs 4 , 1 2 , 2 Figure 6.3: Performance-Enhancing Drugs Here, the best outcome (with a payoff of 4) is to use drugs when your opponent doesn’t, since then you maximize your chances of winning. However, the payoff to both using drugs (2) is worse than the payoff to both not using drugs (3), since in both cases you’re evenly matched, but in the former case you’re also causing harm to yourself. We can now see that using drugs is a strictly dominant strategy, and so we have a situation where the players use drugs even though they understand that there’s a better outcome for both of them. More generally, situations of this type are often referred to as arms races , in which

177 6.3. BEST RESPONSES AND DOMINANT STRATEGIES 163 two competitors use an increasingly dangerous arsenal of weapons simply to remain evenly matched. In the example above, the performance-enhancing drugs play the role of the weapons, but the Prisoner’s Dilemma has also been used to interpret literal arms races between opposing nations, where the weapons correspond to the nations’ military arsenals. To wrap up our discussion of the Prisoner’s Dilemma, we should note that it only arises when the payoffs are aligned in a certain way — as we will see in the remainder of the chapter, there are many situations where the structure of the game and the resulting behavior looks very different. Indeed, even simple changes to a game can change it from an instance of the Prisoner’s Dilemma to something more benign. For example, returning to the Exam- or-Presentation Game, suppose that we keep everything the same as before, except that we make the exam much easier, so that you’ll get a 100 on it if you study, and a 96 if you don’t. Then we can check that the payoff matrix now becomes Your Partner Presentation Exam Presentation 98 96 , 94 , 98 You 96 92 94 92 , Exam , Figure 6.4: Exam-or-Presentation Game with an easier exam. Furthermore, we can check that with these new payoffs, preparing for the presentation now becomes a strictly dominant strategy; so we can expect that both players will play this strategy, and both will benefit from this decision. The downsides of the previous scenario no longer appear: like other dangerous phenomena, the Prisoner’s Dilemma only manifests itself when the conditions are right. 6.3 Best Responses and Dominant Strategies In reasoning about the games in the previous section, we used two fundamental concepts that will be central to our discussion of game theory. As such, it is useful to define them carefully here, and then to look further at some of their implications. The first concept is the idea of a : it is the best choice of one player, given a best response belief about what the other player will do. For instance, in the Exam-or-Presentation Game, we determined your best choice in response to each possible choice of your partner. We can make this precise with a bit of notation, as follows. If is a strategy chosen by S Player 1, and T is a strategy chosen by Player 2, then there is an entry in the payoff matrix corresponding to the pair of chosen strategies ( S,T ). We will write P ) to denote the ( S,T 1 P ( S,T ) to denote the payoff to payoff to Player 1 as a result of this pair of strategies, and 2 Player 2 as a result of this pair of strategies. Now, we say that a strategy S for Player 1 is a best response to a strategy T for Player 2 if S produces at least as good a payoff as any

178 164 CHAPTER 6. GAMES : other strategy paired with T ′ S,T ) ≥ P ) ( P ( ,T S 1 1 ′ for all other strategies S of Player 1. Naturally, there is a completely symmetric definition for Player 2, which we won’t write down here. (In what follows, we’ll present the definitions from Player 1’s point of view, but there are direct analogues for Player 2 in each case.) Notice that this definition allows for multiple different strategies of Player 1 to be tied as T . This can make it difficult to predict which of these multiple the best response to strategy different strategies Player 1 will use. We can emphasize that one choice is uniquely the best S of Player 1 is a strict best response to a strategy T for Player 2 by saying that a strategy S produces a strictly higher payoff than any other strategy paired with T : if ′ ) ( S,T ) > P ,T ( S P 1 1 ′ S of Player 1. When a player has a strict best response to T , this is for all other strategies T . clearly the strategy she should play when faced with The second concept, which was central to our analysis in the previous section, is that of a strictly dominant strategy. We can formulate its definition in terms of best responses as follows. • We say that a dominant strategy for Player 1 is a strategy that is a best response to every strategy of Player 2. • We say that a strictly dominant strategy for Player 1 is a strategy that is a strict best response to every strategy of Player 2. In the previous section, we made the observation that if a player has a strictly dominant strategy, then we can expect him or her to use it. The notion of a dominant strategy is slightly weaker, since it can be tied as the best option against some opposing strategies. As a result, a player could potentially have multiple dominant strategies, in which case it may not be obvious which one should be played. The analysis of the Prisoner’s Dilemma was facilitated by the fact that both players had strictly dominant strategies, and so it was easy to reason about what was likely to happen. But most settings won’t be this clear-cut, and we now begin to look at games which lack strictly dominant strategies. A Game in Which Only One Player Has a Strictly Dominant Strategy. As a first step, let’s consider a setting in which one player has a strictly dominant strategy and the other one doesn’t. As a concrete example, we consider the following story. Suppose there are two firms that are each planning to produce and market a new product; these two products will directly compete with each other. Let’s imagine that the population

179 6.3. BEST RESPONSES AND DOMINANT STRATEGIES 165 of consumers can be cleanly divided into two market segments: people who would only buy a low-priced version of the product, and people who would only buy an upscale version. Let’s also assume that the profit any firm makes on a sale of either a low price or an upscale product is the same. So to keep track of profits it’s good enough to keep track of sales. Each firm wants to maximize its profit, or equivalently its sales, and in order to do this it has to decide whether its new product will be low-priced or upscale. So this game has two players — Firm 1 and Firm 2 — and each has two possible strategies: to produce a low-priced product or an upscale one. To determine the payoffs, here is how the firms expect the sales to work out. • First, people who would prefer a low-priced version account for 60% of the population, and people who would prefer an upscale version account for 40% of the population. Firm 1 is the much more popular brand, and so when the two firms directly compete • in a market segment, Firm 1 gets 80% of the sales and Firm 2 gets 20% of the sales. (If a firm is the only one to produce a product for a given market segment, it gets all the sales.) Based on this, we can determine payoffs for different choices of strategies as follows. If the two firms market to different market segments, they each get all the sales in that • . segment. So the one that targets the low-priced segment gets a payoff 60 and the one that targets the upscale segment gets . 40. If both firms target the low-priced segment, then Firm 1 gets 80% of it, for a payoff of • 48, and Firm 2 gets 20% of it, for a payoff of . 12. . • Analogously, if both firms target the upscale segment, then Firm 1 gets a payoff of ( . 8)( . 4) = . 32 and Firm 2 gets a payoff of ( . 2)( . 4) = . 08. This can be summarized in the following payoff matrix. Firm 2 Low-Priced Upscale Low-Priced . 48 ,. 12 . 60 ,. 40 Firm 1 Upscale . 40 ,. 60 . 32 ,. 08 Figure 6.5: Marketing Strategy Low-Priced Notice that in this game, Firm 1 has a strictly dominant strategy: for Firm 1, is a strict best response to each strategy of Firm 2. On the other hand, Firm 2 does not have a dominant strategy: Low-Priced is its best response when Firm 1 plays Upscale , and Upscale is its best response when Firm 1 plays Low-Priced .

180 166 CHAPTER 6. GAMES Still, it is not hard to make a prediction about the outcome of this game. Since Firm 1 has a strictly dominant strategy in Low-Priced , we can expect it will play it. Now, what should Firm 2 do? If Firm 2 knows Firm 1’s payoffs, and knows that Firm 1 wants to maximize Low-Priced . Then, since profits, then Firm 2 can confidently predict that Firm 1 will play is the strict best response by Firm 2 to Low-Priced , we can predict that Firm 2 will Upscale play . So our overall prediction of play in this marketing game is Low-Priced by Firm Upscale 1 and Upscale by Firm 2, resulting in payoffs of . 60 and . 40 respectively. Note that although we’re describing the reasoning in two steps — first the strictly dom- inant strategy of Firm 1, and then the best response of Firm 2 — this is still in the context of a game where the players move simultaneously: both firms are developing their marketing strategies concurrently and in secret. It is simply that the reasoning about strategies natu- rally follows this two-step logic, resulting in a prediction about how the simultaneous play will occur. It’s also interesting to note the intuitive message of this prediction. Firm 1 is so strong that it can proceed without regard to Firm 2’s decision; given this, Firm 2’s best strategy is to stay safely out of the way of Firm 1. Finally, we should also note how the Marketing Strategy Game makes use of the knowl- edge we assume players have about the game being played and about each other. In particu- lar, we assume that each player knows the entire payoff matrix. And in reasoning about this specific game, it is important that Firm 2 knows that Firm 1 wants to maximize profits, and that Firm 2 knows that Firm 1 knows its own profits. In general, we will assume that the players have common knowledge of the game: they know the structure of the game, they know that each of them know the structure of the game, they know that each of them know that each of them know, and so on. While we will not need the full technical content of common knowledge in anything we do here, it is an underlying assumption and a topic of research in the game theory literature [28]. As mentioned earlier, it is still possible to analyze games in situations where common knowledge does not hold, but the analysis becomes more complex [208]. It’s also worth noting that the assumption of common knowledge is a bit stronger than we need for reasoning about simple games such as the Prisoner’s Dilemma, where strictly dominant strategies for each player imply a particular course of action regardless of what the other player is doing. 6.4 Nash Equilibrium When neither player in a two-player game has a strictly dominant strategy, we need some other way of predicting what is likely to happen. In this section, we develop methods for doing this; the result will be a useful framework for analyzing games in general.

181 6.4. NASH EQUILIBRIUM 167 To frame the question, it helps to think about a An Example: A Three-Client Game. simple example of a game that lacks strictly dominant strategies. Like our previous example, it will be a marketing game played between two firms; however, it has a slightly more intricate set-up. Suppose there are two firms that each hope to do business with one of three large , or B , and C . Each firm has three possible strategies: whether to approach A , clients, A , B . The results of their two decisions will work out as follows. C • If the two firms approach the same client, then the client will give half its business to each. • Firm 1 is too small to attract business on its own, so if it approaches one client while Firm 2 approaches a different one, then Firm 1 gets a payoff of 0. If Firm 2 approaches client B or C on its own, it will get their full business. However, • is a larger client, and will only do business with the firms if both approach A . A Because A • is a larger client, doing business with it is worth 8 (and hence 4 to each firm or C B if it’s split), while doing business with is worth 2 (and hence 1 to each firm if it’s split). From this description, we can work out the following payoff matrix. Firm 2 A B C , A , 4 0 4 2 0 , 2 Firm 1 B 0 , 0 1 , 1 0 , 2 C 0 0 0 , 2 1 , 1 , Figure 6.6: Three-Client Game If we study how the payoffs in this game work, we see that neither firm has a dominant strategy. Indeed, each strategy by each firm is a strict best response to some strategy by the other firm. For Firm 1, A is a strict best response to strategy A by Firm 2, B is a strict best response to B , and C is a strict best response to C . For Firm 2, A is a strict best response to strategy by Firm 1, C is a strict best response to B , and B is a strict best response to A C . So how should we reason about the outcome of play in this game? Defining Nash Equilibrium. In 1950, John Nash proposed a simple but powerful prin- ciple for reasoning about behavior in general games [313, 314], and its underlying premise is the following: even when there are no dominant strategies, we should expect players to use strategies that are best responses to each other. More precisely, suppose that Player 1 chooses a strategy S and Player 2 chooses a strategy T . We say that this pair of strategies

182 168 CHAPTER 6. GAMES T S,T if S is a best response to T , and Nash equilibrium is a best response to S . ( ) is a This is not a concept that can be derived purely from rationality on the part of the players; instead, it is an concept. The idea is that if the players choose strategies that equilibrium are best responses to each other, then no player has an incentive to deviate to an alternative strategy — so the system is in a kind of equilibrium state, with no force pushing it toward a different outcome. Nash shared the 1994 Nobel Prize in Economics for his development and analysis of this idea. To understand the idea of Nash equilibrium, we should first ask why a pair of strategies that are not best responses to each other would not constitute an equilibrium. The answer is that the players cannot both believe that these strategies will be actually used in the game, as they know that at least one player would have an incentive to deviate to another strategy. So Nash equilibrium can be thought of as an equilibrium in beliefs. If each player believes that the other player will actually play a strategy that is part of a Nash equilibrium, then she is willing to play her part of the Nash equilibrium. Let’s consider the Three-Client Game from the perspective of Nash equilibrium. If Firm 1 chooses A and Firm 2 chooses A , then we can check that Firm 1 is playing a best response to Firm 2’s strategy, and Firm 2 is playing a best response to Firm 1’s strategy. Hence, the pair of strategies ( ) forms a Nash equilibrium. Moreover, we can check that this is the A,A 1 only Nash equilibrium. No other pair of strategies are best responses to each other. This discussion also suggests two ways to find Nash equilibria. The first is to simply check all pairs of strategies, and ask for each one of them whether the individual strategies are best responses to each other. The second is to compute each player’s best response(s) to each strategy of the other player, and then find strategies that are mutual best responses. 6.5 Multiple Equilibria: Coordination Games For a game with a single Nash equilibrium, such as the Three-Client Game in the previ- ous section, it seems reasonable to predict that the players will play the strategies in this equilibrium: under any other play of the game, at least one player will not be using a best response to what the other is doing. Some natural games, however, can have more than one Nash equilibrium, and in this case it becomes difficult to predict how rational players will actually behave in the game. We consider some fundamental examples of this problem here. A Coordination Game. Coordination A simple but central example is the following Game , which we can motivate through the following story. Suppose you and a partner are 1 In this discussion, each player only has three available strategies: A , B , or C . Later in this we will introduce the possibility of more complex strategies in which players can randomize over their available options. With this more complex formulation of possible strategies, we will find additional equilibria for the Three-Client Game.

183 6.5. MULTIPLE EQUILIBRIA: COORDINATION GAMES 169 each preparing slides for a joint project presentation; you can’t reach your partner by phone, and need to start working on the slides now. You have to decide whether to prepare your half of the slides in PowerPoint or in Apple’s Keynote software. Either would be fine, but it will be much easier to merge your slides together with your partner’s if you use the same software. So we have a game in which you and your partner are the two players, choosing Power- Point or choosing Keynote form the two strategies, and the payoffs are as shown in Figure 6.7. Your Partner PowerPoint Keynote PowerPoint 1 0 , 0 , 1 You Keynote 0 , 0 1 , 1 Figure 6.7: Coordination Game This is called a Coordination Game because the two players’ shared goal is really to coordinate on the same strategy. There are many settings in which coordination games arise. For example, two manufacturing companies that work together extensively need to decide whether to configure their machinery in metric units of measurement or English units of measurement; two platoons in the same army need to decide whether to attack an enemy’s left flank or right flank; two people trying to find each other in a crowded mall need to decide whether to wait at the north end of the mall or at the south end. In each case, either choice can be fine, provided that both participants make the same choice. (Power- The underlying difficulty is that the game has two Nash equilibria — i.e., and (Keynote,Keynote) in our example from Figure 6.7. If the players Point,PowerPoint) fail to coordinate on one of the Nash equilibria, perhaps because one player expects Power- Point to be played and the other expects Keynote, then they receive low payoffs. So what do the players do? This remains a subject of considerable discussion and research, but some proposals have focal received attention in the literature. Thomas Schelling [364] introduced the idea of a point as a way to resolve this difficulty. He noted that in some games there are natural reasons (possibly outside the payoff structure of the game) that cause the players to focus on one of the Nash equilibria. For example, suppose two drivers are approaching each other at night on an undivided country road. Each driver has to decide whether to move over to the left or the right. If the drivers coordinate — making the same choice of side — then they pass each other, but if they fail to coordinate, then they get a severely low payoff due to the resulting collision. Fortunately, social convention can help the drivers decide what to do in this case: if this game is being played in the U.S., convention strongly suggests that they should move to the right, while if the game is being played in England, convention strongly suggests that they should move to the left. In other words, social conventions, while often

184 170 CHAPTER 6. GAMES arbitrary, can sometimes be useful in helping people coordinate among multiple equilibria. Variants on the Basic Coordination Game. One can enrich the structure of our basic Coordination Game to capture a number of related issues surrounding the problem of mul- tiple equilibria. To take a simple extension of our previous example, suppose that both you and your project partner each prefer Keynote to PowerPoint. You still want to coordinate, but you now view the two alternatives as unequal. This gives us the payoff matrix for an Unbalanced Coordination Game , shown in Figure 6.8. Your Partner PowerPoint Keynote PowerPoint 1 , 1 0 , 0 You 0 0 2 , 2 Keynote , Figure 6.8: Unbalanced Coordination Game Notice that and (Keynote,Keynote) are still both Nash equi- (PowerPoint,PowerPoint) libria for this game, despite the fact that one of them gives higher payoffs to both players. (The point is that if you believe your partner will choose PowerPoint, you still should choose PowerPoint as well.) Here, Schelling’s theory of focal points suggests that we can use a feature intrinsic to the game — rather than an arbitrary social convention — to make a prediction about which equilibrium will be chosen by the players. That is, we can predict that when the players have to choose, they will select strategies so as to reach the equilib- rium that gives higher payoffs to both of them. (To take another example, consider the two people trying to meet at a crowded mall. If the north end of the mall has a bookstore they both like, while the south end consists of a loading dock, the natural focal point would be the equilibrium in which they both choose the north end.) Things get more complicated if you and your partner don’t agree on which software you prefer, as shown in the payoff matrix of Figure 6.9. Your Partner PowerPoint Keynote 0 PowerPoint 1 , 2 0 , You 0 , 0 2 , 1 Keynote Figure 6.9: Battle of the Sexes In this case, the two equilibria still correspond to the two different ways of coordinating, but your payoff is higher in the (Keynote,Keynote) equilibrium, while your partner’s payoff is higher in the (PowerPoint,PowerPoint) equilibrium. This game is traditionally called the Battle of the Sexes , because of the following motivating story. A husband and wife want to see a movie together, and they need to choose between a romantic comedy and an action

185 6.5. MULTIPLE EQUILIBRIA: COORDINATION GAMES 171 (Romance,Romance) equilibrium movie. They want to coordinate on their choice, but the equilibrium gives a higher gives a higher payoff to one of them while the (Action,Action) payoff to the other. In Battle of the Sexes, it can be hard to predict the equilibrium that will be played using either the payoff structure or some purely external social convention. Rather, it helps to know something about conventions that exist between the two players themselves, suggesting how they resolve disagreements when they prefer different ways of coordinating. It’s worth mentioning one final variation on the basic Coordination Game, which has Stag Hunt Game [374]; the name is motivated attracted attention in recent years. This is the by the following story from writings of Rousseau. Suppose that two people are out hunting; if they work together, they can catch a stag (which would be the highest-payoff outcome), but on their own each can catch a hare. The tricky part is that if one hunter tries to catch a stag on his own, he will get nothing, while the other one can still catch a hare. Thus, the Hunt Stag and Hunt Hare hunters are the two players, their strategies are , and the payoffs are as shown in Figure 6.10. Hunter 2 Hunt Stag Hunt Hare Hunt Stag 3 4 , 4 0 , Hunter 1 Hunt Hare , 0 3 , 3 3 Figure 6.10: Stag Hunt This is quite similar to the Unbalanced Coordination Game, except that if the two players miscoordinate, the one who was trying for the higher-payoff outcome gets penalized more than the one who was trying for the lower-payoff outcome. (In fact, the one trying for the lower-payoff outcome doesn’t get penalized at all.) As a result, the challenge in reasoning about which equilibrium will be chosen is based on the trade-off between the high payoff of one and the low downside of miscoordination from the other. It has been argued that the Stag Hunt Game captures some of the intuitive challenges that are also raised by the Prisoner’s Dilemma. The structures are clearly different, since the Prisoner’s Dilemma has strictly dominant strategies; both, however, have the property that players can benefit if they cooperate with each other, but risk suffering if they try cooperating while their partner doesn’t. Another way to see some of the similarities between the two games is to notice that if we go back to the original Exam-or-Presentation Game and make one small change, then we end up changing it from an instance of Prisoner’s Dilemma to something closely resembling Stag Hunt. Specifically, suppose that we keep the grade outcomes the same as in Section 6.1, except that we require both you and your partner to prepare for the presentation in order to have any chance of a better grade. That is, if you both prepare, you both get a 100 on the presentation, but if at most one of you prepares, you

186 172 CHAPTER 6. GAMES both get the base grade of 84. With this change, the payoffs for the Exam-or-Presentation Game become what is shown in Figure 6.11. Your Partner Presentation Exam 90 , 90 82 , 88 Presentation You 88 , 82 88 Exam 88 , Figure 6.11: Exam-or-Presentation Game (Stag Hunt version) We now have a structure that closely resembles the Stag Hunt Game: coordinating on (Presentation,Presentation) (Exam,Exam) are both equilibria, but if you attempt to go or for the higher-payoff equilibrium, you risk getting a low grade if your partner opts to study for the exam. 6.6 Multiple Equilibria: The Hawk-Dove Game Multiple Nash equilibria also arise in a different but equally fundamental kind of game, in which the players engage in a kind of “anti-coordination” activity. Probably the most basic form of such a game is the Hawk-Dove Game , which is motivated by the following story. Suppose two animals are engaged in a contest to decide how a piece of food will be divided between them. Each animal can choose to behave aggressively (the Hawk strategy) or passively (the Dove strategy). If the two animals both behave passively, they divide the food evenly, and each get a payoff of 3. If one behaves aggressively while the other behaves passively, then the aggressor gets most of the food, obtaining a payoff of 5, while the passive one only gets a payoff of 1. But if both animals behave aggressively, then they destroy the food (and possibly injure each other), each getting a payoff of 0. Thus we have the payoff matrix in Figure 6.12. Animal 2 D H 3 , 3 1 , 5 D Animal 1 H 5 , 1 0 , 0 Figure 6.12: Hawk-Dove Game ). Without knowing more about the D,H ) and ( H,D This game has two Nash equilibria: ( animals we cannot predict which of these equilibria will be played. So as in the coordination games we looked at earlier, the concept of Nash equilibrium helps to narrow down the set of reasonable predictions, but it does not provide a unique prediction. The Hawk-Dove game has been studied in many contexts. For example, suppose we sub- stitute two countries for the two animals, and suppose that the countries are simultaneously

187 6.7. MIXED STRATEGIES 173 choosing whether to be aggressive or passive in their foreign policy. Each country hopes to gain through being aggressive, but if both act aggressively they risk actually going to war, which would be disastrous for both. So in equilibrium, we can expect that one will be aggressive and one will be passive, but we can’t predict who will follow which strategy. Again we would need to know more about the countries to predict which equilibrium will be played. Hawk-Dove is another example of a game that can arise from a small change to the payoffs in the Exam-or-Presentation Game. Let’s again recall the set-up from the opening section, and now vary things so that if neither you nor your partner prepares for the presentation, you will get a very low joint grade of 60. (If one or both of you prepare, the grades for the presentation are the same as before.) If we compute the average grades you get for different choices of strategies in this version of the game, we have the payoffs in Figure 6.13. Your Partner Exam Presentation Presentation 90 , 90 86 , 92 You Exam 92 , 86 76 , 76 Figure 6.13: Exam or Presentation? (Hawk-Dove version) In this version of the game, there are two equilibria: and (Exam, (Presentation, Exam) . Essentially, one of you must behave passively and prepare for the presentation, Presentation) while the other achieves the higher payoff by studying for the exam. If you both try to avoid the role of the passive player, you end up with very low payoffs, but we cannot predict from the structure of the game alone who will play this passive role. The Hawk-Dove game is also known by a number of other names in the game theory literature. For example, it is frequently referred to as the game of Chicken , to evoke the image of two teenagers racing their cars toward each other, daring each other to be the one to swerve out of the way. The two strategies here are and Don’t Swerve : the one who Swerve swerves first suffers humiliation from his friends, but if neither swerves, then both suffer an actual collision. 6.7 Mixed Strategies In the previous two sections, we have been discussing games whose conceptual complexity comes from the existence of multiple equilibria. However, there are also games which have no Nash equilibria at all. For such games, we will make predictions about players’ behavior by enlarging the set of strategies to include the possibility of randomization; once players are allowed to behave randomly, one of John Nash’s main results establishes that equilibria always exist [313, 314].

188 174 CHAPTER 6. GAMES Probably the simplest class of games to expose this phenomenon are what might be called “attack-defense” games. In such games, one player behaves as the attacker, while the other behaves as the defender. The attacker can use one of two strategies — let’s call them A — while the defender’s two strategies are “defend against ” or “defend against B .” B and A If the defender defends against the attack the attacker is using, then the defender gets the higher payoff; but if the defender defends against the wrong attack, then the attacker gets the higher payoff. Matching Pennies. A simple attack-defense game is called Matching Pennies , and is based on a game in which two people each hold a penny, and simultaneously choose whether to show heads ( ) or tails ( T ) on their penny. Player 1 loses his penny to player 2 if they H match, and wins player 2’s penny if they don’t match. This produces a payoff matrix as shown in Figure 6.14. Player 2 H T , − 1 H +1 +1 , − 1 Player 1 T +1 , − 1 − 1 , +1 Figure 6.14: Matching Pennies Matching pennies is a simple example of a large class of interesting games with the property that the payoffs of the players sum to zero in every outcome. Such games are called zero-sum games , and many attack-defense games — and more generally, games where the players’ interests are in direct conflict — have this structure. Games like Matching Pennies have in fact been used as metaphorical descriptions of decisions made in combat; for example, the Allied landing in Europe on June 6, 1944 — one of the pivotal moments in World War II — involved a decision by the Allies whether to cross the English Channel at Normandy or at Calais, and a corresponding decision by the German army whether to mass its defensive forces at Normandy or Calais. This has an attack-defense structure that closely resembles the Matching Pennies game [123]. The first thing to notice about Matching Pennies is that there is no pair of strategies that are best responses to each other. To see this, observe that for any pair of strategies, one of the players gets a payoff of − 1, and this player would improve his or her payoff to +1 by switching strategies. So for any pair of strategies, one of the players wants to switch 2 what they’re doing. 2 Incidentally, although it’s not crucial for the discussion here, it’s interesting to note that the Three- Client Game used as an example in Section 6.4 can be viewed intuitively as a kind of hybrid of the Matching Pennies Game and the Stag Hunt Game. If we look just at how the two players evaluate the options of approaching Clients B and C , we have Matching Pennies: Firm 1 wants to match, while Firm 2 wants to not match. However, if they coordinate on approaching Client A , then they both get even higher payoffs –

189 6.7. MIXED STRATEGIES 175 H or , then This means that if we treat each player as simply having the two strategies T there is no Nash equilibrium for this game. This is not so surprising if we consider how Matching Pennies works. A pair of strategies, one for each player, forms a Nash equilibrium if even given knowledge of each other’s strategies, neither player would have an incentive to switch to an alternate strategy. But in Matching Pennies, if Player 1 knows that Player 2 is or T going to play a particular choice of H , then Player 1 can exploit this by choosing the opposite and receiving a payoff of +1. Analogous reasoning holds for Player 2. When we think intuitively about how games of this type are played in real life, we see that players generally try to make it difficult for their opponents to predict what they will play. This suggests that in our modeling of a game like Matching Pennies, we shouldn’t treat H or T , but as ways of randomizing one’s behavior between the strategies as simply and H T . We now see how to build this into a model for the play of this kind of game. The simplest way to introduce randomized behavior is to say that Mixed Strategies. H or T directly, but rather is choosing a probability with each player is not actually choosing which she will play . So in this model, the possible strategies for Player 1 are numbers H p between 0 and 1; a given number p means that Player 1 is committing to play H with probability p , and T with probability 1 − p . Similarly, the possible strategies for Player 2 are numbers q H . between 0 and 1, representing the probability that Player 2 will play Since a game consists of a set of players, strategies, and payoffs, we should notice that by allowing randomization, we have actually changed the game. It no longer consists of two strategies by each player, but instead a set of strategies corresponding to the interval of numbers between 0 and 1. We will refer to these as mixed strategies , since they involve H and “mixing” between the options . Notice that the set of mixed strategies still includes T the original two options of committing to definitely play H or T ; these two choices correspond to selecting probabilities of 1 or 0 respectively, and we will refer to them as the two pure strategies in the game. To make things more informal notationally, we will sometimes refer p = 1 by Player 1 equivalently as the “pure strategy H to the choice of ”, and similarly for p = 0 and q = 1 or 0. Payoffs from Mixed Strategies. With this new set of strategies, we also need to deter- mine the new set of payoffs. The subtlety in defining payoffs is that they are now random quantities: each player will get +1 with some probability, and will get − 1 with the remain- ing probability. When payoffs were numbers it was obvious how to rank them: bigger was better. Now that payoffs are random, it is not immediately obvious how to rank them: we want a principled way to say that one random outcome is better than another. To think about this issue, let’s start by considering Matching Pennies from Player 1’s analogously to the two hunters coordinating to hunt stag.

190 176 CHAPTER 6. GAMES point of view, and focus first on how she evaluates her two pure strategies of definitely playing H . Suppose that Player 2 chooses the strategy q ; that is, he commits or definitely playing T with probability . Then if Player 1 chooses pure and T with probability 1 − q H to playing q , she receives a payoff of − 1 with probability q (since the two pennies match with strategy H , in which event she loses), and she receives a payoff of +1 with probability 1 − q probability q − q (since the two pennies don’t match with probability 1 ). Alternatively, if Player 1 chooses pure strategy , she receives +1 with probability q , and − 1 with probability (1 − q ). So even T if Player 1 uses a pure strategy, her payoffs can still be random due to the randomization employed by Player 2. How should we decide which of or T is more appealing to Player H 1 in this case? In order to rank random payoffs numerically, we will attach a number to each distribution that represents how attractive this distribution is to the player. Once we have done this, we can then rank outcomes according to their associated number. The number we will use for expected value of the payoff. So for example, if Player 1 chooses the pure this purpose is the H while Player 2 chooses a probability of q , as above, then the expected payoff to strategy Player 1 is ( − 1)( q ) + (1)(1 − q ) = 1 − 2 q. Similarly, if Player 1 chooses the pure strategy T q , while Player 2 chooses a probability of then the expected payoff to Player 1 is 1 ) + ( − 1)(1 − q (1)( q − q . ) = 2 We will assume players are seeking to maximize the expected payoff they get from a choice of mixed strategies. Although the expectation is a natural quantity, it is a subtle question whether maximizing expectation is a reasonable modeling assumption about the behavior of players. By now, however, there is a well-established foundation for the assumption that players rank distributions over payoffs according to their expected values [288, 363, 398], and so we will follow it here. We have now defined the mixed-strategy version of the Matching Pennies game: strategies are probabilities of playing H , and payoffs are the expectations of the payoffs from the four ). We can now ask whether there is a Nash pure outcomes ( ) , ( H,T ) , ( T,H ), and ( T,T H,H equilibrium for this richer version of the game. We define a Nash equilibrium for the mixed- Equilibrium with Mixed Strategies. strategy version just as we did for the pure-strategy version: it is a pair of strategies (now probabilities) so that each is a best response to the other. First, let’s observe that no pure strategy can be part of a Nash equilibrium. This is equivalent to the reasoning we did at the outset of this section. Suppose, for example, that the pure strategy H (i.e. probability p = 1) by Player 1 were part of a Nash equilibrium.

191 6.7. MIXED STRATEGIES 177 H as well (since Player 2 Then Player 2’s unique best response would be the pure strategy by Player 1 is not a best response to H H gets +1 whenever he matches). But by Player 2, so in fact this couldn’t be a Nash equilibrium. Analogous reasoning applies to the other possible pure strategies by the two players. So we reach the natural conclusion that in any Nash equilibrium, both players must be using probabilities that are strictly between 0 and 1. q used by Player Next, let’s ask what Player 1’s best response should be to the strategy in 2. Above, we determined that the expected payoff to Player 1 from the pure strategy H this case is 1 − 2 q, while the expected payoff to Player 1 from the pure strategy T is q − 2 . 1 Now, here’s the key point: if 1 2 q 6 = 2 q − 1, then one of the pure strategies H or T is − q in fact the unique best response by Player 1 to a play of by Player 2. This is simply − 2 q because one of 1 q − 1 is larger in this case, and so there is no point for Player 1 or 2 to put any probability on her weaker pure strategy. But we already established that pure strategies cannot be part of any Nash equilibrium for Matching Pennies, and because pure strategies are the best responses whenever 1 − 2 q 6 = 2 q − 1, probabilities that make these two expectations unequal cannot be part of a Nash equilibrium either. So we’ve concluded that in any Nash equilibrium for the mixed-strategy version of Match- ing Pennies, we must have 1 2 q = 2 q − 1 , − q or in other words, / 2. The situation is symmetric when we consider things from Player = 1 2’s point of view, and evaluate the payoffs from a play of probability p by Player 1. We conclude from this that in any Nash equilibrium, we must also have p = 1 / 2. Thus, the pair of strategies p = 1 / 2 and q = 1 / 2 is the only possibility for a Nash equilibrium. We can check that this pair of strategies in fact do form best responses to each other. As a result, this is the unique Nash equilibrium for the mixed-strategy version of Matching Pennies. Interpreting the Mixed-Strategy Equilibrium for Matching Pennies. Having de- rived the Nash equilibrium for this game, it’s useful to think about what it means, and how we can apply this reasoning to games in general. First, let’s picture a concrete setting in which two people actually sit down to play Matching Pennies, and each of them actually commits to behaving randomly according to probabilities p and q respectively. If Player 1 believes that Player 2 will play H strictly more than half the time, then she should definitely play T — in which case Player 2 should not

192 178 CHAPTER 6. GAMES H more than half the time. The symmetric reasoning applies if Player 1 believes be playing strictly more than half the time. In neither case would we have that Player 2 will play T a Nash equilibrium. So the point is that the choice of = 1 / 2 by Player 2 makes Player 1 q between playing H or T : the strategy indifferent = 1 / 2 is effectively “non-exploitable” by q Player 1. This was in fact our original intuition for introducing randomization: each player wants their behavior to be unpredictable to the other, so that their behavior can’t be taken advantage of. We should note that the fact that both probabilities turned out to be 1 2 is / a result of the highly symmetric structure of Matching Pennies; as we will see in subsequent examples in the next section, when the payoffs are less symmetric, the Nash equilibrium can consist of unequal probabilities. This notion of indifference is a general principle behind the computation of mixed-strategy equilibria in two-player, two-strategy games when there are no equilibria involving pure strategies: each player should randomize so as to make the other player indifferent between their two alternatives. This way, neither player’s behavior can be exploited by a pure strategy, and the two choices of probabilities are best responses to each other. And although we won’t pursue the details of it here, a generalization of this principle applies to games with any finite number of players and any finite number of strategies: Nash’s main mathematical every result accompanying his definition of equilibrium was to prove that such game has at least one mixed-strategy equilibrium [313, 314]. It’s also worth thinking about how to interpret mixed-strategy equilibria in real-world situations. There are in fact several possible interpretations that are appropriate in different situations: • Sometimes, particularly when the participants are genuinely playing a sport or game, the players may be actively randomizing their actions [107, 337, 405]: a tennis player may be randomly deciding whether to serve the ball up the center or out to the side of the court; a card-player may be randomly deciding whether to bluff or not; two children may be randomizing among rock, paper, and scissors in the perennial elementary-school contest of the same name. We will look at examples of this in the next section. • Sometimes the mixed strategies are better viewed as proportions within a population. Suppose for example that two species of animals, in the process of foraging for food, regularly engage in one-on-one attack-defense games with the structure of Matching Pennies. Here, a single member of the first species always plays the role of attacker, and a single member of the second species always plays the role of defender. Let’s suppose that each individual animal is genetically hard-wired to always play H or always play T ; and suppose further that the population of each species consists half of animals hard-wired to play H , and half of animals hard-wired to play T . Then with this population mixture, H -animals in each species do exactly as well on average, over

193 6.8. MIXED STRATEGIES: EXAMPLES AND EMPIRICAL ANALYSIS 179 T many random interactions, as -animals. Hence the population as a whole is in a kind of mixed equilibrium, even though each individual is playing a pure strategy. This story suggests an important link with evolutionary biology, which has in fact been developed through a long line of research [375, 376]; this topic will be our focus in Chapter 7. • Maybe the most subtle interpretation is based on recalling, from Section 6.4, that Nash equilibrium is often best thought of as an equilibrium in beliefs. If each player believes that her partner will play according to a particular Nash equilibrium, then she too will want to play according to it. In the case of Matching Pennies, with its unique mixed equilibrium, this means that it is enough for you to expect that when you meet an arbitrary person, they will play their side of Matching Pennies with a probability of 1 / 2. In this case, playing a probability of 1 / 2 makes sense for you too, and hence this choice of probabilities is self-reinforcing — it is in equilibrium — across the entire population. 6.8 Mixed Strategies: Examples and Empirical Anal- ysis Because mixed-strategy equilibrium is a subtle concept, it’s useful to think about it through further examples. We will focus on two main examples, both drawn from the realm of sports, and both with attack-defense structures. The first is stylized and partly metaphorical, while the second represents a striking empirical test of whether people in high-stakes situations actually follow the predictions of mixed-strategy equilibrium. We conclude the section with a general discussion of how to identify all the equilibria of a two-player, two-strategy game. The Run-Pass Game. First, let’s consider a streamlined version of the problem faced by two American football teams as they plan their next play in a football game. The offense can choose either to run or to pass, and the defense can choose either to defend against the run or to defend against the pass. Here is how the payoffs work. • If the defense correctly matches the offense’s play, then the offense gains 0 yards. • If the offense runs while the defense defends against the pass, the offense gains 5 yards. • If the offense passes while the defense defends against the run, the offense gains 10 yards. Hence we have the payoff matrix shown in Figure 6.15. (If you don’t know the rules of American football, you can follow the discussion simply by taking the payoff matrix as self-contained. Intuitively, the point is simply that we have

194 180 CHAPTER 6. GAMES Defense Defend Pass Defend Run 0 0 10 , − 10 Pass , Offense , − 5 Run , 0 5 0 Figure 6.15: Run-Pass Game an attack-defense game with two players named “offense” and “defense” respectively, and where the attacker has a stronger option (pass) and a weaker option (run).) Just as in Matching Pennies, it’s easy to check that there is no Nash equilibrium where either player uses a pure strategy: both have to make their behavior unpredictable by ran- domizing. So let’s work out a mixed-strategy equilibrium for this game: let p be the prob- q be the probability that the defense defends against ability that the offense passes, and let the pass. (We know from Nash’s result that at least one mixed-strategy equilibrium must exist, but not what the actual values of and q should be.) p We use the principle that a mixed equilibrium arises when the probabilities used by each player makes his opponent indifferent between his two options. First, suppose the defense chooses a probability of q for defending against the pass. • Then the expected payoff to the offense from passing is q ) + (10)(1 − q ) = 10 (0)( 10 q, − while the expected payoff to the offense from running is (5)( q ) + (0)(1 − q ) = 5 q. To make the offense indifferent between its two strategies, we need to set 10 − 10 q = 5 q , and hence q / 3. = 2 Next, suppose the offense chooses a probability of for passing. Then the expected • p payoff to the defense from defending against the pass is p ) + ( − 5)(1 (0)( p ) = 5 p − 5 , − with the expected payoff to the defense from defending against the run is ( − 10)( p ) + (0)(1 − p ) = − 10 p. To make the defense indifferent between its two strategies, we need to set 5 − 5 = − 10 p , p and hence p = 1 / 3. Thus, the only possible probability values that can appear in a mixed-strategy equilibrium are p = 1 / 3 for the offense, and q = 2 / 3 for the defense, and this in fact forms an equilibrium.

195 6.8. MIXED STRATEGIES: EXAMPLES AND EMPIRICAL ANALYSIS 181 / 3, and the Notice also that the expected payoff to the offense with these probabilities is 10 10 3. Also, in contrast to Matching Pennies, corresponding expected payoff to the defense is − / notice that because of the asymmetric structure of the payoffs here, the probabilities that appear in the mixed-strategy equilibrium are unbalanced as well. Strategic Interpretation of the Run-Pass Game. There are several things to notice about this equilibrium. First, the strategic implications of the equilibrium probabilities are intriguing and a bit subtle. Specifically, although passing is the offense’s more powerful p = 1 / 3 on passing. This weapon, it uses it less than half the time: it places only probability initially seems counter-intuitive: why not spend more time using your more powerful option? But the calculation that gave us the equilibrium probabilities also supplies the answer to this question. If the offense placed any higher probability on passing, then the defense’s best response would be to always defend against the pass, and the offense would actually do worse in expectation. p p = 1 / 2. In this case, the We can see how this works by trying a larger value for , like / 2, defense will always defend against the pass, and so the offense’s expected payoff will be 5 since it gains 5 half the time and 0 the other half the time: (1 / 2)(0) + (1 / 2)(5) = 5 / 2 . Above, we saw that with the equilibrium probabilities, the offense has an expected payoff of 10 / > 5 / 2. Moreover, because p = 1 / 3 makes the defense indifferent between its two 3 3 strategies, an offense that uses / 3 is guaranteed to get 10 / = 1 > 5 / 2 no matter what the p defense does. One way to think about the real power of passing as a strategy is to notice that in equilibrium, the defense is defending against the pass 2 3 of the time, even though the / / offense is using it only 1 threat of passing is helping the 3 of the time. So somehow the offense, even though it uses it relatively rarely. This example clearly over-simplifies the strategic issues at work in American football: there are many more than just two strategies, and teams are concerned with more than just their yardage on the very next play. Nevertheless, this type of analysis has been applied quantitatively to statistics from American football, verifying some of the main qualitative conclusions at a broad level — that teams generally run more than they pass, and that the expected yardage gained per play from running is close to the expected yardage gained per play from passing for most teams [82, 84, 355]. The Penalty-Kick Game. The complexity of American football makes it hard to cast it truly accurately as a two-person, two-strategy game. We now focus on a different setting, also from professional sports, where such a formalization can be done much more exactly — the modeling of penalty kicks in soccer as a two-player game.

196 182 CHAPTER 6. GAMES In 2002, Ignacio Palacios-Huerta undertook a large study of penalty kicks from the per- spective of game theory [337], and we focus on his analysis here. As he observed, penalty kicks capture the ingredients of two-player, two-strategy games remarkably faithfully. The kicker can aim the ball to the left or the right of the goal, and the goalie can dive to either 3 The ball moves to the goal fast enough that the decisions of the the left or right as well. kicker and goalie are effectively being made simultaneously; and based on these decisions the kicker is likely to score or not. Indeed, the structure of the game is very much like Matching Pennies: if the goalie dives in the direction where the ball is aimed, he has a good chance of blocking it; if the goalie dives in the wrong direction, it is very likely to go in the goal. Based on an analysis of roughly 1400 penalty kicks in professional soccer, Palacios-Huerta determined the empirical probability of scoring for each of the four basic outcomes: whether the kicker aims left or right, and whether the goalie dives left or right. This led to a payoff matrix as shown in Figure 6.16. Goalie L R L 0 . 58 , − 0 . 58 95 0 95 , − 0 . . Kicker 70 0 93 , − 0 . 93 0 R . , − 0 . 70 . Figure 6.16: The Penalty-Kick Games (from empirical data [337]). There are a few contrasts to note in relation to the basic Matching Pennies Game. First, a kicker has a reasonably good chance of scoring even when the goalie dives in the correct direction (although a correct choice by the goalie still greatly reduces this probability). Second, kickers are generally right-footed, and so their chance of scoring is not completely 4 symmetric between aiming left and aiming right. Despite these caveats, the basic premise of Matching Pennies is still present here: there is no equilibrium in pure strategies, and so we need to consider how players should random- ize their behavior in playing this game. Using the principle of indifference as in previous q is the probability that a goalie chooses examples, we see that if , we need to set q so as L to make the kicker indifferent between his two options: ( . 58)( q ) + ( . 95)(1 − q ) = ( . 93)( q ) + ( . 70)(1 − q ) . Solving for , we get q = . 42. We can do the analogous calculation to obtain the value of p q that makes the goalie indifferent, obtaining p = . 39. The striking punchline to this study is that in the dataset of real penalty kicks, the . 42 fraction of the time (matching the prediction to two decimal places), goalies dive left a 3 Kicks up the center, and decisions by the goalie to remain in the center, are very rare, and can be ignored in a simple version of the analysis. 4 For purposes of the analysis, we take all the left-footed kickers in the data and apply a left-right reflection to all their actions, so that R always denotes the “natural side” for each kicker.

197 6.8. MIXED STRATEGIES: EXAMPLES AND EMPIRICAL ANALYSIS 183 . 40 fraction of the time (coming within 01 of the prediction). It is and the kickers aim left a . particularly nice to find the theory’s predictions borne out in a setting such as professional soccer, since the two-player game under study is being played by experts, and the outcome is important enough to the participants that they are investing significant attention to their choice of strategies. Finding all Nash Equilibria. To conclude our discussion of mixed-strategy equilibria, we consider the general question of how to find all Nash equilibria of a two-player, two-strategy game. First, it is important to note that a game may have both pure-strategy and mixed- strategy equilibria. As a result, one should first check all four pure outcomes (given by pairs of pure strategies) to see which, if any, form equilibria. Then, to check whether there are any mixed-strategy equilibria, we need to see whether there are mixing probabilities p and q that are best responses to each other. If there is a mixed-strategy equilibrium, then we can determine Player 2’s strategy ( ) from the requirement that Player 1 randomizes. q Player 1 will only randomize if his pure strategies have equal expected payoff. This equality q of expected payoffs for Player 1 gives us one equation which we can solve to determine . The same process gives an equation to solve for determining Player 2’s strategy p . If both p and q of the obtained values are strictly between 0 and 1, and are thus legitimate mixed strategies, then we have a mixed-strategy equilibrium. Thus far, our examples of mixed-strategy equilibria have been restricted to games with an attack-defense structure, and so we have not seen an example exhibiting both pure and mixed equilibria. However, it is not hard to find such examples: in particular, Coordination and Hawk-Dove games with two pure equilibria will have a third mixed equilibrium in which each player randomizes. As an example, let’s consider the Unbalanced Coordination Game from Section 6.5: Your Partner PowerPoint Keynote PowerPoint 1 , 1 0 , 0 You 2 Keynote 0 2 , , 0 Figure 6.17: Unbalanced Coordination Game Suppose that you place a probability of p strictly between 0 and 1 on PowerPoint, and your partner places a probability of q strictly between 0 and 1 on PowerPoint. Then you’ll be indifferent between PowerPoint and Keynote if ) q ) + (0)(1 − q ) = (0)( q ) + (2)(1 − q (1)( , or in other words, if q = 2 / 3. Since the situation is symmetric from your partner’s point of view, we also get p = 2 / 3. Thus, in addition to the two pure equilibria, we also get an

198 184 CHAPTER 6. GAMES / 3. Note that unlike equilibrium in which each of you chooses PowerPoint with probability 2 the two pure equilibria, this mixed equilibrium comes with a positive probability that the two of you will miscoordinate; but this is still an equilibrium, since if you truly believe that your partner is choosing PowerPoint with probability 2 / / 3, 3 and Keynote with probability 1 then you’ll be indifferent between the two options, and will get the same expected payoff however you choose. 6.9 Pareto-Optimality and Social Optimality In a Nash equilibrium, each player’s strategy is a best response to the other player’s strategies. In other words, the players are optimizing individually. But this doesn’t mean that, as a group, the players will necessarily reach an outcome that is in any sense good. The Exam-or- Presentation Game from the opening section, and related games like the Prisoner’s Dilemma, serve as examples of this. (We redraw the payoff matrix for the basic Exam-or-Presentation Game in Figure 6.18.) Your Partner Presentation Exam , Presentation 90 , 90 86 92 You Exam , 86 88 , 92 88 Figure 6.18: Exam or Presentation? It is interesting to classify outcomes in a game not just by their strategic or equilibrium properties, but also by whether they are “good for society.” In order to reason about this latter issue, we first need a way of making it precise. There are two useful candidates for such a definition, as we now discuss. Pareto-Optimality. The first definition is Pareto-optimality , named after the Italian economist Vilfredo Pareto who worked in the late 1800’s and early 1900’s. Pareto-optimal if there is no A choice of strategies — one by each player — is other choice of strategies in which all players receive payoffs at least as high, and at least one player receives a strictly higher payoff. To see the intuitive appeal of Pareto-optimality, let’s consider a choice of strategies that is not Pareto-optimal. In this case, there’s an alternate choice of strategies that makes at least one player better off without harming any player. In basically any reasonable sense, this alternate choice is superior to what’s currently being played. If the players could jointly agree on what to do, and make this agreement binding, then surely they would prefer to move to this superior choice of strategies.

199 6.9. PARETO-OPTIMALITY AND SOCIAL OPTIMALITY 185 The motivation here relies crucially on the idea that the players can construct a binding agreement to actually play the superior choice of strategies: if this alternate choice is not a Nash equilibrium, then absent a binding agreement, at least one player would want to switch to a different strategy. As an illustration of why this is a crucial point, consider the outcomes in the Exam-or-Presentation Game. The outcome in which you and your partner both study for the exam is not Pareto-optimal, since the outcome in which you both prepare for the presentation is strictly better for both of you. This is the central difficulty at the heart of this example, now phrased in terms of Pareto-optimality. It shows that even though you and your partner realize there is a superior solution, there is no way to maintain it without a binding agreement between the two of you. In this example, the two outcomes in which exactly one of you prepares for the pre- sentation are also Pareto-optimal. In this case, although one of you is doing badly, there is no alternate choice of strategies in which everyone is doing at least as well. So in fact, the Exam-or-Presentation Game – and the Prisoner’s Dilemma — are examples of games in which the only outcome that is not Pareto-optimal is the one corresponding to the unique Nash equilibrium. Social Optimality. A stronger condition that is even simpler to state is social optimality. A choice of strategies — one by each player — is a (or social welfare maximizer socially optimal ) if it maximizes the sum of the players’ payoffs. In the Exam-or-Presentation Game, the social optimum is achieved by the outcome in which both you and your partner prepare for the presentation, which produces a combined payoff of 90 + 90 = 180. Of course, this definition is only appropriate to the extent that it makes sense to add the payoffs of different players together — it’s not always clear that we can meaningfully combine my satisfaction with an outcome and your satisfaction by simply adding them up. Outcomes that are socially optimal must also be Pareto-optimal: if such an outcome weren’t Pareto-optimal, there would be a different outcome in which all payoffs were at least as large, and one was larger — and this would be an outcome with a larger sum of payoffs. On the other hand, a Pareto-optimal outcome need not be socially optimal. For example, the Exam-or-Presentation Game has three outcomes that are Pareto-optimal, but only one of these is the social optimum. Finally, of course, it’s not the case that Nash equilibria are at odds with the goal of social optimality in every game. For example, in the version of the Exam-or-Presentation Game with an easier exam, yielding the payoff matrix that we saw earlier in Figure 6.4, the unique Nash equilibrium is also the unique social optimum.

200 186 CHAPTER 6. GAMES 6.10 Advanced Material: Dominated Strategies and Dynamic Games In this final section, we consider two further issues that arise in the analysis of games. First, dominated strategies we study the role of in reasoning about behavior in a game, and find that the analysis of dominated strategies can provide a way to make predictions about play based on rationality, even when no player has a dominant strategy. Second, we discuss how to reinterpret the strategies and payoffs in a game to deal with situations in which play actually occurs sequentially through time. Before doing this, however, we begin with a formal definition for games that have more than two players. A. Multi-Player Games A (multi-player) game consists, as in the two-player case, of a set of players, a set of strategies for each player, and a payoff to each player for each possible outcome. players named 1 , Specifically, suppose that a game has ,...,n . Each player has a set n 2 outcome joint strategy ) of the game is a choice of a strategy for of possible strategies. An (or i payoff function P that maps outcomes of the game each player. Finally, each player has a i i : that is, for each outcome consisting of strategies ( S to a numerical payoff for ,S ), ,...,S n 1 2 ,...,S S ,S there is a payoff ( ) to player i . P i n 1 2 S Now, we can say that a strategy is a best response by Player i to a choice of strategies i ( S ,S ,...,S ,S ,...,S ) by all the other players if − 1 2 i +1 1 n i ′ P S ,...,S ,S ,S ,...,S P ,S ,...,S ,S ,S ,S S ( ,...,S ( ) ≥ ) n +1 i 1 i 2 1 i − i − i i 2 i +1 1 n 1 i ′ available to player i . for all other possible strategies S i if each ,S ,...,S Finally, an outcome consisting of strategies ( ) is a Nash equilibrium S 2 1 n strategy it contains is a best response to all the others. B. Dominated Strategies and their Role in Strategic Reasoning In Sections 6.2 and 6.3, we discussed (strictly) dominant strategies — strategies that are a (strict) best response to every possible choice of strategies by the other players. Clearly if a player has a strictly dominant strategy then this is the strategy she should employ. But we also saw that even for two-player, two-strategy games, it is common to have no dominant strategies. This holds even more strongly for larger games: although dominant and strictly dominant strategies can exist in games with many players and many strategies, they are rare.

201 6.10. ADVANCED MATERIAL: DOMINATED STRATEGIES AND DYNAMIC GAMES 187 B C E F A D Figure 6.19: In the Facility Location Game, each player has strictly dominated strategies but no dominant strategy. However, even if a player does not have a dominant strategy, she may still have strategies that are dominated by other strategies. In this section, we consider the role that such dominated strategies play in reasoning about behavior in games. We begin with a formal definition: a strategy is strictly dominated if there is some other strategy available to the same player that produces a strictly higher payoff in response to choice of strategies by the other players. In the notation we’ve just developed, strategy every ′ i is strictly dominated if there is another strategy S S for player for player i such that i i ′ ) ( S ,...,S ,S ,S ,...,S ,S ,...,S ,S ,S ( S ,S P > P ,...,S ) 2 n i +1 1 1 2 − i − 1 i i i i +1 1 n i i for all choices of strategies ( S ,S ,...,S ,S ,...,S ) by the other players. − 1 2 i +1 1 n i Now, in the two-player, two-strategy games we’ve been considering thus far, a strategy is strictly dominated precisely when the other strategy available to the same player is strictly dominant. In this context, it wouldn’t make sense to study strictly dominated strategies as a separate concept on their own. However, if a player has many strategies, then it’s possible for a strategy to be strictly dominated without any strategy being dominant. In such cases, we will find that strictly dominated strategies can play a very useful role in reasoning about play in a game. In particular, we will see that there are cases in which there are no dominant strategies, but where the outcome of the game can still be uniquely predicted using the structure of the dominated strategies. In this way, reasoning based on dominated strategies forms an intriguing intermediate approach between dominant strategies and Nash equilibrium: on the one hand, it can be more powerful than reasoning based solely on dominant strategies; but on the other hand, it still relies only on the premise that players seek to maximize payoffs, and doesn’t require the introduction of an equilibrium notion. To see how this works, it’s useful to introduce the approach in the context of a basic example. Example: The Facility Location Game. Our example is a game in which two firms compete through their choice of locations. Suppose that two firms are each planning to open a store in one of six towns located along six consecutive exits on a highway. We can represent the arrangement of these towns using a six-node graph as in Figure 6.19.

202 188 CHAPTER 6. GAMES Now, based on leasing agreements, Firm 1 has the option of opening its store in any of towns C , or E , while Firm 2 has the option of opening its store in any of towns B , D , A , . These decisions will be executed simultaneously. Once the two stores are opened, F or customers from the towns will go to the store that is closer to them. So for example, if Firm C and Firm 2 opens its store in town , then the store in town B will 1 open its store in town B A and B , while the store in town C attract customers from C , will attract customers from D E , and F . If we assume that the towns contain an equal number of customers, and that , payoffs are directly proportional to the number of customers, this would result in a payoff of 4 for Firm 1 and 2 for Firm 2, since Firm 1 claims customers from 4 towns while Firm 2 claims customers from the remaining 2 towns. Reasoning in this way about the number of towns claimed by each store, based on proximity to their locations, we get the payoff matrix shown in Figure 6.20. Firm 2 B D F , 1 3 5 2 A 4 3 , , Firm 1 C 4 , 2 3 , 3 4 , 2 E 3 3 2 , 4 5 , 1 , Figure 6.20: Facility Location Game We refer to this as a . The competitive location of facilities is a Facility Location Game topic that has been the subject of considerable study in operations research and other areas [135]. Moreover, closely related models have been used when the entities being “located” are not stores along a one-dimensional highway but the positions of political candidates along a one-dimensional ideological spectrum — here too, choosing a certain position relative to one’s electoral opponent can attract certain voters while alienating others [350]. We will return to issues related to political competition, though in a slightly different direction, in Chapter 23. We can verify that neither player has a dominant strategy in this game: for example, if Firm 1 locates at node A , then the strict best response of Firm 2 is B , while if Firm 1 locates at node E D . The situation is symmetric , then the strict best response of Firm 2 is if we interchange the roles of the two firms (and read the graph from the other direction). Dominated Strategies in the Facility Location Game. We can make progress in reasoning about the behavior of the two players in the Facility Location Game by thinking about their dominated strategies. First, notice that A is a strictly dominated strategy for Firm 1: in any situation where Firm 1 has the option of choosing A , it would receive a strictly higher payoff by choosing . Similarly, F is a strictly dominated strategy for Firm C 2: in any situation where Firm 1 has the option of choosing F , it would receive a strictly higher payoff by choosing D .

203 6.10. ADVANCED MATERIAL: DOMINATED STRATEGIES AND DYNAMIC GAMES 189 It is never in a player’s interest to use a strictly dominated strategy, since it should always be replaced by a strategy that does better. So Firm 1 isn’t going to use strategy A . Moreover, since Firm 2 knows the structure of the game, including Firm 1’s payoffs, Firm A 2 knows that Firm 1 won’t use strategy . It can be effectively eliminated from the game. The same reasoning shows that F can be eliminated from the game. We now have a smaller instance of the Facility Location Game, involving only the four nodes C , D , and E , and the payoff matrix shown in Figure 6.21. B , Firm 2 B D 4 , 2 3 , 3 C Firm 1 3 3 E 2 , 4 , Figure 6.21: Smaller Facility Location Game Now something interesting happens. The strategies B and E weren’t previously strictly dominated: they were useful in case the other player used A or F respectively. But with A and eliminated, the strategies B and E now are strictly dominated — so by the same F reasoning, both players know they won’t be used, and so we can eliminate them from the game. This gives us the even smaller game shown in Figure 6.22. Firm 2 D Firm 1 C , 3 3 Figure 6.22: Even smaller Facility Location Game At this point, there is a very clear prediction for the play of the game: Firm 1 will play C , and Firm 2 will play D . And the reasoning that led to this is clear: after repeatedly removing strategies that were (or became) strictly dominated, we were left with only a single plausible option for each player. The process that led us to this reduced game is called the iterative deletion of strictly dominated strategies , and we will shortly describe it in its full generality. Before doing this, however, it’s worth making some observations about the example of the Facility Location Game. First, the pair of strategies ( C,D ) is indeed the unique Nash equilibrium in the game, and when we discuss the iterated deletion of strictly dominated strategies in general, we will see that it is an effective way to search for Nash equilibria. But beyond this, it is also an effective way to justify the Nash equilibria that one finds. When we first introduced Nash equilibrium, we observed that it couldn’t be derived purely from an assumption of rationality on the part of the players; rather, we had to assume further that play of the game would be found at an equilibrium from which neither player had an incentive to deviate. On the

204 190 CHAPTER 6. GAMES other hand, when a unique Nash equilibrium emerges from the iterated deletion of strictly dominated strategies, it is in fact a prediction made purely based on the assumptions of the players’ rationality and their knowledge of the game, since all the steps that led to it were based simply on removing strategies that were strictly inferior to others from the perspective of payoff-maximization. A final observation is that iterated deletion can in principle be carried out for a very large number of steps, and the Facility Location Game illustrates this. Suppose that instead of a path of length six, we had a path of length 1000, with the options for the two firms still strictly alternating along this path (constituting 500 possible strategies for each player). Then it would be still be the case that only the outer two nodes would be strictly dominated; after their removal, we’d have a path of length 998 in which the two new outer nodes had now become strictly dominated. We can continue removing nodes in this way, and after 499 th st steps of such reasoning, we’ll have a game in which only the 500 and 501 nodes have survived as strategies. This is the unique Nash equilibrium for the game, and this unique prediction can be justified by a very long sequence of deletions of dominated strategies. It’s also interesting how this prediction is intuitively natural, and one that is often seen in real life: two competing stores staking out positions next to each other near the center of the population, or two political candidates gravitating toward the ideological middle ground as they compete for voters in a general election. In each case, this move toward the center is the unique way to maximize the territory that you can claim at the expense of your competitor. Iterated Deletion of Dominated Strategies: The General Principle. In general, iterated deletion of strictly for a game with an arbitrary number of players, the process of dominated strategies proceeds as follows. We start with any n -player game, find all the strictly dominated strategies, and delete • them. • We then consider the reduced game in which these strategies have been removed. In this reduced game there may be strategies that are now strictly dominated, despite not having been strictly dominated in the full game. We find these strategies and delete them. • We continue this process, repeatedly finding and removing strictly dominated strategies until none can be found. An important general fact is that the set of Nash equilibria of the original game coincides with the set of Nash equilibria for the final reduced game, consisting only of strategies that survive iterated deletion. To prove this fact, it is enough to show that the set of Nash equilibria does not change when we perform one round of deleting strictly dominated

205 6.10. ADVANCED MATERIAL: DOMINATED STRATEGIES AND DYNAMIC GAMES 191 strategies; if this is true, then we have established that the Nash equilibria continue to remain unchanged through an arbitrary finite sequence of deletions. To prove that the set of Nash equilibria remains the same through one round of deletion, we need to show two things. First, any Nash equilibrium of the original game is a Nash equilibrium of the reduced game. To see this, note that otherwise there would be a Nash S equilibrium of the original game involving a strategy that was deleted. But in this case, ′ . Hence S S is strictly dominated by some other strategy S cannot be part of a Nash equilibrium of the original game: it is not a best response to the strategies of the other ′ that dominates it is a better response. This establishes that S players, since the strategy no Nash equilibrium of the original game can be removed by the deletion process. Second, we need to show that any Nash equilibrium of the reduced game is also a Nash equilibrium of the original game. In order for this not to be the case, there would have to be a Nash ′ S equilibrium ,S E ,...,S = ( ) of the reduced game, and a strategy S that was deleted n 1 2 i has an incentive to deviate from its strategy S i in from the original game, such that player i ′ ′ was deleted because it was strictly dominated by at . But strategy S S to the strategy E i i ′′ that strictly dominated it and least one other strategy; we can therefore find a strategy S i ′′ ′′ i to S S is still was not deleted. Then player S also has an incentive to deviate from , and i i i present in the reduced game, contradicting our assumption that E is a Nash equilibrium of the reduced game. This establishes that the game we end up with, after iterated deletion of strictly domi- nated strategies, still has all the Nash equilibria of the original game. Hence, this process can be a powerful way to restrict the search for Nash equilibria. Moreover, although we described the process as operating in rounds, with all currently strictly dominated strategies being removed in each round, this is not essential. One can show that eliminating strictly dominated strategies in any order will result in the same set of surviving strategies. Weakly Dominated Strategies. It is also natural to ask about notions that are slightly weaker than our definition of strictly dominated strategies. One fundamental definition in this spirit is that of a weakly dominated strategy. We say that a strategy is weakly dominated if there is another strategy that does at least as well no matter what the other players do, and does strictly better against some joint strategy of the other players. In our notation S for player from earlier, we say that a strategy i is weakly dominated if there is another i ′ strategy S for player i such that i ′ ) ( S ,...,S ,S ,S ,...,S ,S ,...,S ,S ,S P S ,S ( P ,...,S ≥ ) i n i 1 1 2 − i − 1 i i 2 i +1 1 n i +1 i for all choices of strategies ( S ,S ,...,S ,S ,...,S ) by the other players, and 1 2 i +1 1 n i − ′ ( ,S ,...,S P ,S S ) ,S ,S ,S ,...,S ,S ) > P ,...,S ( S ,...,S i i 1 1 − 2 +1 1 i − 1 n i i i +1 i n 2 i ) by the other players. S ,...,S ,S ,S ,...,S for at least one choice of strategies ( i − 1 2 i +1 1 n

206 192 CHAPTER 6. GAMES For strictly dominated strategies, the argument for deleting them was compelling: they are never best responses. For weakly dominated strategies, the issue is more subtle. Such strategies could be best responses to some joint strategy by the other players. So a rational player could play a weakly dominated strategy, and in fact Nash equilibria can involve weakly dominated strategies. There are simple examples that make this clear even in two-player, two-strategy games. Consider for example a version of the Stag Hunt Game in which the payoff from successfully catching a stag is the same as the payoff from catching a hare: Hunter 2 Hunt Stag Hunt Hare Hunt Stag 3 3 3 0 , , Hunter 1 Hunt Hare 3 , 0 3 , 3 Figure 6.23: Stag Hunt: A version with a weakly dominated strategy In this case, Hunt Stag is a weakly dominated strategy, since each player always does at least as well, and sometimes strictly better, by playing Hunt Hare . Nevertheless, the Hunt Stag outcome in which both players choose is a Nash equilibrium, since each is playing a best response to the other’s strategy. Thus, deleting weakly dominated strategies is not in general a safe thing to do, if one wants to preserve the essential structure of the game: such deletion operations can destroy Nash equilibria. Of course, it might seem reasonable to suppose that a player should not play an equilib- rium involving a weakly dominated strategy (such as ( Hunt Stag, Hunt Stag )) if he had any uncertainty about what the other players would do — after all, why not use an alternate strategy that is at least as good in every eventuality? But Nash equilibrium does not take into account this idea of uncertainty about the behavior of others, and hence has no way to rule out such outcomes. In the next chapter, we will discuss an alternate equilibrium concept known as evolutionary stability that in fact does eliminate weakly dominated strategies in a principled way. The relationship between Nash equilibrium, evolutionary stability and weakly dominated strategies is considered in the exercises at the end of Chapter 7. C. Dynamic Games Our focus in this chapter has been on games in which all players choose their strategies simultaneously, and then receive payoffs based on this joint decision. Of course, actual simultaneity is not crucial for the model, but it has been central to our discussions so far that each player is choosing a strategy without knowledge of the actual choices made by the other players. Many games, however, are played over time: some player or set of players moves first, other players observe the choice(s) made, and then they respond, perhaps according to a

207 6.10. ADVANCED MATERIAL: DOMINATED STRATEGIES AND DYNAMIC GAMES 193 Player 1 B A Player 2 A B A B 8 12 6 4 6 12 2 4 Figure 6.24: A simple game in extensive form. predetermined order of governing who moves when. Such games are called dynamic games , and there are many basic examples: board games and card games in which players alternate turns; negotiations, which usually involve a sequence of offers and counter-offers; and bidding in an auction or pricing competing goods where participants must make decisions over time. Here we’ll discuss an adaptation of the theory of games that incorporates this dynamic aspect. Normal and Extensive Forms of a Game. To begin with, specifying a dynamic game is going to require a new kind of notation. Thus far, we’ve worked with something called the normal-form representation of a game; this specifies the list of players, their possible strategies, and the payoffs arising from every possible (simultaneous) choice of strategies by the players. (For two-player games, the payoff matrices we’ve seen in this chapter encode the normal-form representation of a game in a compact way.) To describe a dynamic game, we’re going to need a richer representation; we need to be able to specify who moves when, what each player knows at any opportunity they have to move, what they can do when it is their turn to move, and what the payoffs are at the end of the game. We refer to this specification as the extensive-form representation of the game.

208 194 CHAPTER 6. GAMES Let’s start with a very simple example of a dynamic game, so that we can discuss what its extensive-form representation looks like. As we’ll see, this game is simple enough that it avoids some of the subtleties that arise in the analysis of dynamic games, but it is useful as a first illustration, and we’ll proceed to a more complex second example afterward. In our first example, we imagine that there are two firms — Firm 1 and Firm 2 — each of whom is trying to decide whether to focus its advertising and marketing on two possible A or B . Firm 1 gets to choose first. If Firm 2 follows Firm 1 into the same regions, named region, then Firm 1’s “first-mover advantage” gives it 2 / 3 of the profit obtainable from the / 3. If Firm 2 moves into the other region, market in that region, while Firm 2 will only get 1 then each firm gets all the profit obtainable in their respective region. Finally, Region A has B twice as large a market as Region A is equal to 12, : the total profit obtainable in region while in Region , it’s 6. B We write the extensive-form representation as a “game tree,” depicted in Figure 6.24. This tree is designed to be read downward from the top. The top node represents Firm 1’s A or B . initial move, and the two edges descending from this node represent its two options Based on which branch is taken, this leads to a node representing Firm 2’s subsequent move. A or B , again represented by edges descending from the Firm 2 can then also choose option node. This leads to a terminal node representing the end of play in the game; each terminal node is labeled with the payoffs to the two players. Thus, a specific play — determined by a sequence of choices by Firm 1 and Firm 2 — corresponds to a path from the top node in the tree down to some terminal node. First Firm 1 chooses or B , then Firm 2 chooses A or B , and then the two players receive their A payoffs. In a more general model of dynamic games, each node could contain an annotation saying what information about the previous moves is known to the player currently making a move; however, for our purposes here, we will focus on the case in which each player knows the complete history of past moves when they go to make their current move. Reasoning about Behavior in a Dynamic Game. As with simultaneous-move games, we’d like to make predictions for what players will do in dynamic games. One way is to reason from the game tree. In our current example, we can start by considering how Firm 2 will behave after each of the two possible opening moves by Firm 1. If Firm 1 chooses A , then Firm 2 maximizes its payoff by choosing B , while if Firm 1 chooses B , then Firm 2 maximizes its payoff by choosing . Now let’s consider Firm 1’s opening move, given A what we’ve just concluded about Firm 2’s subsequent behavior. If Firm 1 chooses A , then it expects Firm 2 to choose B , yielding a payoff of 12 for Firm 1. If Firm 1 chooses B , then it expects Firm 2 to choose A , yielding a payoff of 6 for Firm 1. Since we expect the firms to try maximizing their payoffs, we predict that Firm 1 should choose A , after which Firm 2 should choose B .

209 6.10. ADVANCED MATERIAL: DOMINATED STRATEGIES AND DYNAMIC GAMES 195 This is a useful way to analyze dynamic games. We start one step above the terminal nodes, where the last player to move has complete control over the outcome of the payoffs. This lets us predict what the last player will do in all cases. Having established this, we then move one more level up the game tree, using these predictions to reason about what the player one move earlier will do. We continue in this way up the tree, eventually making predictions for play all the way up to the top node. A different style of analysis exploits an interesting connection between normal and exten- sive forms, allowing us to write a normal-form representation for a dynamic game as follows. Suppose that, before the game is played, each player makes up a plan for how to play the entire game, covering every possible eventuality. This plan will serve as the player’s strategy. One way to think about such strategies, and a useful way to be sure that they include a complete description of every possibility, is to imagine that each player has to provide all of the information needed to write a computer program which will actually play the game in their place. A or B For the game in Figure 6.24, Firm 1 only has two possible strategies: . Since Firm 2 moves after observing what Firm 1 did, and Firm 2 has two possible choices for each of the two options by Firm 1, Firm 2 has four possible plans for playing the game. They can be written as contingencies, specifying what Firm 2 will do in response to each possible move by Firm 1: ( A if A , A if B ), ( A if A , B if B ), ( B if A , A if B ), and ( B if A , B if B ), or in abbreviated form as ( , AB ), ( AA , BB ), ( BA , AA ), and ( BA , BB ), AB If each player chooses a complete plan for playing the game as its strategy, then we can determine the payoffs directly from this pair of chosen strategies via a payoff matrix. Firm 2 AA,AB AA,BB BA,AB BA,BB 6 A 8 , 4 8 , 4 12 , 6 12 , Firm 1 12 6 , 12 4 , 2 6 , B 4 , 2 Figure 6.25: Conversion to normal form. Because the plans describe everything about how a player will behave, we have managed to describe this dynamic game in normal form: each player chooses a strategy (consisting of a complete plan) in advance, and from this joint choice of strategies, we can determine payoffs. We will see later that there are some important subtleties in using this interpretation of the underlying dynamic game, and in particular the translation from extensive to normal form will sometimes not preserve the full structure implicit in the game. But the translation is a

210 196 CHAPTER 6. GAMES useful tool for analysis, and the subtle lack of fidelity that can arise in the translation is in itself a revealing notion to develop and explore. With this in mind, we first finish our simple example, where the translation will work perfectly, and then move on to a second example where the complications begin to arise. For the normal-form payoff matrix corresponding to our first example, the payoff matrix has eight cells, while the extensive-form representation only has four terminal nodes with payoffs. This occurs because each terminal node can be reached with two different pairs of strategies, with each pair forming a cell of the payoff matrix. Both pairs of strategies dictate the same actions in the path of the game tree which actually occurs, but describe different hypothetical actions in other unrealized paths. For example, the payoffs in the entries for A, ( ( )) and for ( A, ( AA,BB )) are the same because both strategy combinations lead AA,AB to the same terminal node. In both cases Firm 2 chooses A in response to what Firm 1 actually does; Firm 2’s plan for what to do in the event Firm 1 chose B is not realized by the actual play. Now, using the normal-form representation, we can quickly see that for Firm 1, strategy A is strictly dominant. Firm 2 does not have a strictly dominant strategy, but it should play a best response to Firm 1, which would be either ( BA,AB ) or ( BA,BB ). Notice that this prediction of play by Firm 1 and Firm 2 based on the normal-form representation is the same as our prediction based on direct analysis of the game tree, where we reasoned upward A B . from the terminal nodes: Firm 1 will play , and in response Firm 2 will play A More Complex Example: The Market Entry Game. In our first dynamic game, reasoning based on the extensive and normal form representations led to essentially identical conclusions. As games get larger, extensive forms are representationally more streamlined than normal forms for dynamic games, but if this were the only distinction, it would be hard to argue that dynamic games truly add much to the overall theory of games. In fact, however, the dynamic aspect leads to new subtleties, and this can be exposed by considering a case in which the translation from extensive form to normal form ends up obscuring some of the structure that is implicit in the dynamic game. For this, we consider a second example of a dynamic game, also played between two competing firms. We call this the Market Entry Game, and it’s motivated by the following scenario. Consider a region where Firm 2 is currently the only serious participant in a given line of business, and Firm 1 is considering whether to enter the market. • The first move in this game is made by Firm 1, who must decide whether to stay out of the market or enter it. • If Firm 1 chooses to stay out, then the game ends, with Firm 1 getting a payoff of 0 and Firm 2 keeping the payoff from the entire market.

211 6.10. ADVANCED MATERIAL: DOMINATED STRATEGIES AND DYNAMIC GAMES 197 Player 1 Stay Out Enter Player 2 0 Cooperate Retaliate 2 1 -1 1 -1 Figure 6.26: Extensive-form representation of the Market Entry Game. • If Firm 1 chooses to enter, then the game continues to a second move by Firm 2, who must choose whether to cooperate and divide the market evenly with Firm 1, or retaliate and engage in a price war. If Firm 2 cooperates, then each firm gets a payoff corresponding to half the market. – – If Firm 2 retaliates, then each firm gets a negative payoff. Choosing numerical payoffs to fill in this story, we can write the extensive-form representation of the Market Entry Game as in Figure 6.26. Subtle Distinctions Between Extensive and Normal Form Representations. Let’s take the two ways we developed to analyze our previous dynamic game and apply them here. First, we can work our way up the game tree starting at the terminal nodes, as follows. If Firm 1 chooses to enter the market, then Firm 2 achieves a higher payoff by cooperating than by retaliating, so we should predict cooperation in the event the game reaches this point. Given this, when Firm 1 goes to make its first move, it can expect a payoff of 0 by staying out, and a payoff of 1 by entering, so it should choose to enter the market. We can therefore predict that Firm 1 will enter the market, and then Firm 2 will cooperate.

212 198 CHAPTER 6. GAMES Now let’s consider the normal-form representation. Firm 1’s possible plans for playing the game are just to choose Stay Out ( E ). Firm 2’s possible plans are to choose S ) or Enter ( retaliation in the event of entry, or cooperation in the event of entry. We’ll denote these two R plans by C respectively. This gives us the payoff matrix in Figure 6.27. and Firm 2 R C 2 S , 2 0 , 0 Firm 1 − 1 , − E 1 , 1 1 Figure 6.27: Normal Form of the Market Entry Game Here’s the surprise: when we look at this game in normal form, we discover two distinct (pure-strategy) Nash equilibria: ( E,C S,R ). The first of these corresponds to the ), and ( prediction for play that we obtained by analyzing the extensive-form representation. What does the second one correspond to? To answer this, it helps to recall our view of the normal-form representation as capturing the idea that each player commits in advance to a computer program that will play the game S,R ) corresponds to an outcome in which in its place. Viewed this way, the equilibrium ( Firm 2 commits in advance to a computer program that will automatically retaliate in the event that Firm 1 enters the market. Firm 1, meanwhile, commits to a program that stays out of the market. Given this pair of choices, neither firm has an incentive to change the computer program they’re using: for example, if Firm 1 were to switch to a program that entered the market, it would trigger retaliation by the program that Firm 2 is using. This contrast between the prediction from the extensive and normal forms highlights some important points. First, it shows that the premise behind our translation from extensive to normal form — that each player commits ahead of time to a complete plan for playing the game — is not really equivalent to our initial premise in defining dynamic games — namely, that each player makes an optimal decision at each intermediate point in the game, based on what has already happened up to that point. Firm 2’s decision to retaliate on entry highlights this clearly. If Firm 2 can truly pre-commit to this plan, then the equilibrium ( S,R ) makes sense, since Firm 1 will not want to provoke the retaliation that is encoded in Firm 2’s plan. But if we take the dynamic game as originally defined in extensive form, then pre-commitment to a plan is not part of the model: rather, Firm 2 only gets to evaluate its decision to cooperate or retaliate once Firm 1 has already entered the market, and at that point its payoff is better if it cooperates. Given this, Firm 1 can predict that it is safe to enter. In game theory, the standard model for dynamic games in extensive form assumes that players will seek to maximize their payoff at any intermediate stage of play that can be reached in the game. In this interpretation, there is a unique prediction for play in our

213 6.10. ADVANCED MATERIAL: DOMINATED STRATEGIES AND DYNAMIC GAMES 199 E,C ) in normal form. However, the Market Entry Game, corresponding to the equilibrium ( ) are not simply notational or representational; issues surrounding the other equilibrium ( S,R they are deeper than this. For any given scenario, it is really a question of what we believe is being modeled by the underlying dynamic game in extensive form. It is a question of whether we are in a setting where a player can irrevocably pre-commit to a certain plan, to the extent that other players will believe the commitment as a credible threat — or not. Further, the Market Entry Game shows how the ability to commit to a particular course of action — when possible — can in fact be a valuable thing for an individual player, even if that course of action would be bad for everyone if it were actually carried out. In particular, if Firm 2 could make Firm 1 believe that it really would retaliate in the event of entry, then Firm 1 would choose to stay out, resulting in a higher payoff for Firm 2. In practice, this suggests particular courses of action that Firm 2 could take before the game even starts. For example, suppose that before Firm 1 had decided whether to enter the market, Firm 2 were to publically advertise an offer to beat any competitor’s price by 10%. This would be a safe thing to do as long as Firm 2 is the only serious participant in the market, but it becomes dangerous to both firms if Firm 1 actually enters. The fact that the plan has been publically announced means that it would be very costly (reputationally, and possibly legally) for Firm 2 to back away from it. In this way, the announcement can serve as a way of switching the underlying model from one in which Firm 2’s threat to retaliate is not credible to one in which Firm 2 can actually pre-commit to a plan for retaliation. Relationship to Weakly Dominated Strategies. In discussing these distinctions, it is also interesting to note the role that weakly dominated strategies play here. Notice that in the normal-form representation in Figure 6.27, the strategy R for Firm 2 is weakly dominated, S (since then Firm and for a simple reason: it yields the same payoff if Firm 1 chooses 2 doesn’t actually get to move), and it yields a lower payoff if Firm 1 chooses . So E our translation from extensive form to normal form for dynamic games provides another reason to be careful about predictions of play in a normal-form game that rely on weakly dominated strategies: if the structure in fact arises from a dynamic game in extensive form, then information about the dynamic game that is lost in the translation to normal form could potentially be sufficient to eliminate such equilibria. However, we can’t simply fix up the translation by eliminating weakly dominated strate- gies. We saw earlier that iterated deletion of strictly dominated strategies can be done in any order: all orders yield the same final result. But this is not true for the iterated deletion of weakly dominated strategies. To see this, suppose we vary the Market Entry Game slightly so that the payoff from the joint strategy ( E,C ) is (0 , 0). (In this version, both firms know they will fail to gain a positive payoff even if Firm 2 cooperates on entry, although they still don’t do as badly as when Firm 2 retaliates.) R is a weakly dominated strategy as before,

214 200 CHAPTER 6. GAMES E . ( and S produce the same payoff for Firm 1 when Firm 2 chooses C , and but now so is E R .) S produces a strictly higher payoff when Firm 2 chooses In this version of the game, there are now three (pure-strategy) Nash equilibria: ( S,C ), E,C ), and ( S,R ( R , then we are left ). If we first eliminate the weakly dominated strategy with ( ) and ( E,C ) as equilibria. Alternately, if we first eliminate the weakly dominated S,C E , then we are left with ( strategy ) and ( S,R ) as equilibria. In both cases, no further S,C elimination of weakly dominated strategies is possible, so the order of deletion affects the final set of equilibria. We can ask which of these equilibria actually make sense as predictions of play in this game. If this normal form actually arose from the dynamic version of the Market Entry Game, then C is still the only reasonable strategy for Firm 2, while Firm 1 could now play either S E . or The analysis framework we developed for most of this chapter is based Final Comments. on games in normal form. One approach to analyzing dynamic games in extensive form is to first find all Nash equilibria of the translation to normal form, treating each of these as a candidate prediction of play in the dynamic game, and then go back to the extensive-form version to see which of these make sense as actual predictions. There is an alternate theory that works directly with the extensive-form representation. The simplest technique used in this theory is the style of analysis we employed to analyze an extensive-form representation from the terminal nodes upward. But there are more complex components to the theory as well, allowing for richer structure such as the possibility that players at any given point have only partial information about the history of play up to that point. While we will not go further into this theory here, it is developed in a number of books on game theory and microeconomic theory [263, 288, 336, 398]. 6.11 Exercises 1. Say whether the following claim is true or false, and provide a brief (1-3 sentence) explanation for your answer. Claim: If player A in a two-person game has a dominant strategy s , then A A plays s and there is a pure strategy Nash equilibrium in which player A player B plays a best response to s . A 2. Consider the following statement: In a Nash equilibrium of a two-player game each player is playing an optimal strategy, so the two player’s strategies are social-welfare maximizing.

215 6.11. EXERCISES 201 Is this statement correct or incorrect? If you think it is correct, give a brief (1-3 sentence) explanation for why. If you think it is incorrect, give an example of a game discussed in Chapter 6 that shows it to be incorrect (you do not need to spell out all the details of the game, provided you make it clear what you are referring to), together with a brief (1-3 sentence) explanation. 3. Find all pure strategy Nash equilibria in the game below. In the payoff matrix below the rows correspond to player A’s strategies and the columns correspond to player B’s strategies. The first entry in each box is player A’s payoff and the second entry is player B’s payoff. Player B L R U , 2 3 , 2 1 Player A 2 D 4 0 , 2 , 4. Consider the two-player game with players, strategies and payoffs described in the following game matrix. Player B L M R 6 0 , 3 t , 2 1 , 1 Player A m 2 , 3 0 , 1 7 , 0 b 5 3 4 , 2 3 , 1 , Figure 6.28: Payoff Matrix (a) Does either player have a dominant strategy? Explain briefly (1-3 sentences). (b) Find all pure strategy Nash equilibria for this game. 5. Consider the following two-player game in which each player has three strategies. Player B L M R 6 U 1 , 1 2 , 3 1 , Player A M , 4 5 3 5 2 , 2 , D 1 , 10 4 , 7 0 , 4 Find all the (pure strategy) Nash equilibria for this game. 6. In this question we will consider several two-player games. In each payoff matrix below the rows correspond to player A’s strategies and the columns correspond to player B’s

216 202 CHAPTER 6. GAMES strategies. The first entry in each box is player A’s payoff and the second entry is player B’s payoff. (a) Find all pure (non-randomized) strategy Nash equilibria for the game described by the payoff matrix below. Player B L R 2 , 15 4 , 20 U Player A 6 , 6 10 , 8 D Find all pure (non-randomized) strategy Nash equilibria for the game described (b) by the payoff matrix below. Player B L R U 3 , 5 4 , 3 Player A 6 D 1 1 , , 2 (c) Find all Nash equilibria for the game described by the payoff matrix below. Player B L R 1 , 1 4 , 2 U Player A 3 2 3 D , 2 , [Hint: This game has a both pure strategy equilibria and a mixed strategy equilibrium. To find the mixed strategy equilibrium let the probability that player A uses strategy U be p and the probability that player B uses strategy L be q . As we learned in our analysis of matching pennies, if a player uses a mixed strategy (one that is not really just some pure strategy played with probability one) then the player must be indifferent between two pure strategies. That is the strategies must have equal expected payoffs. q p is not 0 or 1 then it must be the case that q +4(1 − q ) = 3 q +2(1 − So, for example, if ) as these are the expected payoffs to player A from U and D when player B uses probability q .] 7. In this question we will consider several two-player games. In each payoff matrix below the rows correspond to player A’s strategies and the columns correspond to player B’s strategies. The first entry in each box is player A’s payoff and the second entry is player B’s payoff. (a) Find all Nash equilibria for the game described by the payoff matrix below.

217 6.11. EXERCISES 203 Player B L R 1 1 3 , 2 U , Player A D 3 4 , 4 0 , Player B L R U 5 , 10 0 , 6 Player A 4 4 2 , 2 D , (b) Find all Nash equilibria for the game described by the payoff matrix below (include an explanation for your answer). [Hint: This game has a mixed strategy equilibrium. To find the equilibrium let the probability that player A uses strategy U be p and the probability that player B uses strategy L be q . As we learned in our analysis of matching pennies, if a player uses a mixed strategy (one that is not really just some pure strategy played with probability one) then the player must be indifferent between two pure strategies. That is, the strategies must have equal expected payoffs. So, for example, if p is not 0 or 1 then it q q − q ) = 4 q + 2(1 must be the case that 5 + 0(1 ) as these are the expected payoffs − to player A from U and D when player B uses probability q .] 8. Consider the two-player game described by the payoff matrix below. Player B L R 0 U 1 , 1 0 , Player A 0 , 0 4 , 4 D (a) Find all pure-strategy Nash equilibria for this game. (b) This game also has a mixed-strategy Nash equilibrium; find the probabilities the players use in this equilibrium, together with an explanation for your answer. (c) Keeping in mind Schelling’s focal point idea from Chapter 6, what equilibrium do you think is the best prediction of how the game will be played? Explain. 9. For each of the following two player games find all Nash equilibria. In each payoff matrix below the rows correspond to player A’s strategies and the columns correspond to player B’s strategies. The first entry in each box is player A’s payoff and the second entry is player B’s payoff.

218 204 CHAPTER 6. GAMES Player B L R (a) , 4 U , 5 5 8 Player A , 3 4 , 8 D 3 Player B R L (b) U 0 , 0 − 1 , 1 Player A 2 − D 1 2 , − 1 , 10. In the payoff matrix below the rows correspond to player A’s strategies and the columns correspond to player B’s strategies. The first entry in each box is player A’s payoff and the second entry is player B’s payoff. Player B L R 2 U 3 , 3 1 , Player A D , 1 3 , 0 2 (a) Find all pure strategy Nash equilibria of this game. (b) Notice from the payoff matrix above that Player A’s payoff from the pair of U,L ) is 3. Can you change player A’s payoff from this pair of strate- strategies ( no gies to some non-negative number in such a way that the resulting game has pure-strategy Nash equilibrium? Give a brief (1-3 sentence) explanation for your answer. (Note that in answering this question, you should only change Player A’s payoff for this one pair of strategies ( U,L ). In particular, leave the rest of the structure of the game unchanged: the players, their strategies, the payoff from strategies other than ( U,L ), and B’s payoff from ( U,L ).) (c) Now let’s go back to the original payoff matrix from part (a) and ask an analogous question about player B. So we’re back to the payoff matrix in which players A U,L ). and B each get a payoff of 3 from the pair of strategies ( Can you change player B’s payoff from the pair of strategies ( ) to some non- U,L negative number in such a way that the resulting game has no pure-strategy Nash equilibrium? Give a brief (1-3 sentence) explanation for your answer. (Again, in answering this question, you should only change Player B’s payoff for this one pair of strategies ( U,L ). In particular, leave the rest of the structure of the game unchanged: the players, their strategies, the payoff from strategies other than ( U,L ), and A’s payoff from ( U,L ).)

219 6.11. EXERCISES 205 11. In the text we’ve discussed dominant strategies and noted that if a player has a domi- nant strategy we would expect it to be used. The opposite of a dominant strategy is a strategy that is dominated. The definition of dominated is: ′ ∗ with the property that dominated if player i has another strategy s is A strategy s i i ∗ ′ no matter what the other players in than from s ’s payoff is greater from i player s i i the game do. We do not expect a player to use a strategy that is dominated and this can help in finding Nash equilibria. Here is an example of this idea. In this game, M is a dominated strategy (it is dominated by R) and player B will not use it. Player B L M R , U 2 4 2 2 1 3 , , Player A D 1 , 2 3 , 3 2 , 4 So in analyzing the game we can delete M and look at the remaining game Player B L R U 2 , 4 3 , 2 Player A 1 , 2 2 , 4 D Now player A has a dominant strategy (U) and it is easy to see that the Nash equilib- rium of the 2-by-2 game is (U,L). You can check the original game to see that (U,L) is a Nash equilibrium. Of course, using this procedure requires that we know that a 5 dominated strategy cannot be used in Nash equilibrium. Consider any two player game which has at least one (pure strategy) Nash equilibrium. Explain why the strategies used in an equilibrium of this game will not be dominated strategies. 12. In Chapter 6 we discussed dominant strategies and noted that if a player has a dominant strategy we would expect it to be used. The opposite of a dominant strategy is a strategy that is dominated. There are several possible notions of what it means for a strategy to be dominated. In this problem we will focus on weak domination. ∗ ′ if player with the property is weakly dominated A strategy i has another strategy s s i i that: 5 This is actually true for any number of players. It would also help to know that if we iteratively remove dominated strategies (in any order) and analyze the reduced games we still find the Nash equilibria of the original game. This is also true, but it is a bit more complicated.

220 206 CHAPTER 6. GAMES ′ i ’s payoff from is at least as large s (a) No matter what the other player does, player i ∗ , and s as the payoff from i ′ is (b) There is some strategy for the other player so that player i ’s payoff from s i ∗ . strictly greater than the payoff from s i It seems unlikely that a player would use a weakly dominated strategy, but these (a) strategies can occur in a Nash equilibrium. Find all pure (non-randomized) Nash equilibria for the game below. Do any of them use weakly dominated strategies? Player B L R U 1 , 1 1 , 1 Player A D 0 , 0 2 , 1 (b) One way to reason about the weakly dominated strategies that you should have found in answering the question above is to consider the following sequential game. Suppose that the players actually move sequentially, but the player to move second does not know what the player moving first chose. Player A moves first, and if he chooses U, then player B’s choice does not matter. Effectively the game is over if A chooses U as no matter what B does the payoff is (1 , 1). If player A chooses D, then , 0) if B chooses L or (2 , player B’s move matters, and the payoff is (0 1) if B chooses R. [Note that as B does not observe A’s move the simultaneous move game with payoff matrix above is equivalent to this sequential move game.] In this game how would you expect the players to behave? Explain your reasoning. [The players are not allowed to change the game. They play it once just as it is given above. You may reason from the payoff matrix or the story behind the game, but if you use the story remember that B does not observe A’s move until after the game is over.] 13. Here we consider a game with three players, named 1, 2 and 3. To define the game we need to specify the sets of strategies available to each player; also, when each of the three players chooses a strategy, this gives a triple of strategies, and we need to specify the payoff each player receives from any possible triple of strategies played. Let’s suppose that player 1’s strategy set is { U,D } , players 2’s strategy set is { L,R } and player 3’s strategy set is { l,r } . One way to specify the payoffs would be to write down every possible triple of strategies, and the payoffs for each. A different but equivalent way to interpret triples of strategies, which makes it easier to specify the payoffs, is to imagine that player 3 chooses which of two distinct two-player games players 1 and 2 will play. If 3 chooses l then the payoff matrix is

221 6.11. EXERCISES 207 l : Payoff Matrix Player B L R 4 , 4 , U 0 , 0 , 1 4 Player A 0 , D , 1 2 , 1 , 0 2 where the first entry in each cell is the payoff to player 1, the second entry is the payoff to player 2 and the third entry is the payoff to player 3. If 3 chooses r then the payoff matrix is Payoff Matrix : r Player B L R , U 2 , 0 , 0 1 , 1 1 Player A D , 1 , 1 2 , 2 , 2 1 So, for example, if player 1 chooses U R and player 3 chooses r the , player 2 chooses payoffs are 1 for each player. (a) First suppose the players all move simultaneously. That is, players 1 and 2 do not observe which game player 3 has selected until after they each chose a strategy. Find all of the (pure strategy) Nash equilibria for this game. (b) Now suppose that player 3 gets to move first and that players 1 and 2 observe player 3’s move before they decide how to play. That is, if player 3 chooses the strategy r r and they both know that then players 1 and 2 play the game defined by payoff matrix they are playing this game. Similarly, if player 3 chooses the strategy l then players 1 and 2 play the game defined by payoff matrix l and they both know that they are playing this game. Let’s also suppose that if players 1 and 2 play the game defined by payoff matrix r they play a (pure strategy) Nash equilibrium for that game; and similarly, if players l they play a (pure strategy) Nash 1 and 2 play the game defined by payoff matrix equilibrium for that game. Finally, let’s suppose that player 3 understands that this is how players 1 and 2 will behave. What do you expect player 3 to do and why? What triple of strategies would you expect to see played? Is this list of strategies a Nash equilibrium of the simultaneous move game between the three players? 14. Consider the two-player game with players, strategies and payoffs described in the following game matrix.

222 208 CHAPTER 6. GAMES Player 2 L R U 1 , 1 4 , 0 Player 1 4 , 0 D , 3 3 (a) Find all of the Nash equilibria of this game. In the mixed strategy equilibrium you found in part (a), you should notice that (b) player 1 plays strategy U more often than strategy D. One of your friends remarks that your answer to part (a) must be wrong because clearly for player 1 strategy D is a more attractive strategy than strategy U. Both U and D give player 1 a payoff of 4 on the off-diagonal elements of the payoff matrix, but D gives player 1 a payoff of 3 on the diagonal while U only gives player 1 a payoff of 1 on the diagonal. Explain what is wrong with this reasoning. 15. Two identical firms — let’s call them firm 1 and firm 2 — must decide simultaneously and independently whether to enter a new market and what product to produce if they do enter the market. Each firm, if it enters, can develop and produce either product A or product B. If both firms enter and produce product A they each lose ten million dollars. If both firms enter and both produce product B, they each make a profit of five million dollars. If both enter and one produces A while the other produces B, then they each make a profit of ten million dollars. Any firm that does not enter makes a profit of zero. Finally, if one firm does not enter and the other firm produces A it makes a profit of fifteen million dollars, while if the single entering firm produces B it makes a profit of thirty million dollars. You are the manager of firm 1 and you have to choose a strategy for your firm. (a) Set this situation up as a game with two players, firms 1 and 2, and three strategies for each firm: produce A, produce B or do not enter. (b) One of your employees argues that you should enter the market (although he is not sure what product you should produce) because no matter what firm 2 does, entering and producing product B is better than not entering. Evaluate this argument. (c) Another employee agrees with the person in part (b) and argues that as strategy A could result in a loss (if the other firm also produces A) you should enter and produce B. If both firms reason this way, and thus enter and produce product B, will their play of the game form a Nash equilibrium? Explain. (d) Find all the pure strategy Nash equilibria of this game. (e) Another employee of your firm suggests merging the two firms and deciding co- operatively on strategies so as to maximize the sum of profits. Ignoring whether this merger would be allowed by the regulators do you think its a good idea? Explain.

223 Chapter 7 Evolutionary Game Theory In Chapter 6, we developed the basic ideas of game theory, in which individual players make decisions, and the payoff to each player depends on the decisions made by all. As we saw there, a key question in game theory is to reason about the behavior we should expect to see when players take part in a given game. The discussion in Chapter 6 was based on considering how players simultaneously reason about what the other players may do. In this chapter, on the other hand, we explore the evolutionary game theory , which shows that the basic ideas of game theory can be notion of applied even to situations in which no individual is overtly reasoning, or even making explicit decisions. Rather, game-theoretic analysis will be applied to settings in which individuals can exhibit different forms of behavior (including those that may not be the result of conscious choices), and we will consider which forms of behavior have the ability to persist in the population, and which forms of behavior have a tendency to be driven out by others. As its name suggests, this approach has been applied most widely in the area of evolu- tionary biology, the domain in which the idea was first articulated by John Maynard Smith and G. R. Price [375, 376]. Evolutionary biology is based on the idea that an organism’s genes largely determine its observable characteristics, and hence its in a given envi- fitness ronment. Organisms that are more fit will tend to produce more offspring, causing genes that provide greater fitness to increase their representation in the population. In this way, fitter genes tend to win over time, because they provide higher rates of reproduction. The key insight of evolutionary game theory is that many behaviors involve the interaction of multiple organisms in a population, and the success of any one of these organisms depends on how its behavior interacts with that of others. So the fitness of an individual organism can’t be measured in isolation; rather it has to be evaluated in the context of the full population in which it lives. This opens the door to a natural game-theoretic analogy: D. Easley and J. Kleinberg. Networks, Crowds, and Markets: Reasoning about a Highly Connected World . Cambridge University Press, 2010. Draft version: June 10, 2010. 209

224 210 CHAPTER 7. EVOLUTIONARY GAME THEORY an organism’s genetically-determined characteristics and behaviors are like its strategy in a game, its fitness is like its payoff, and this payoff depends on the strategies (characteristics) of the organisms with which it interacts. Written this way, it is hard to tell in advance whether this will turn out to be a superficial analogy or a deep one, but in fact the connections turn out to run very deeply: game-theoretic ideas like equilibrium will prove to be a useful way to make predictions about the results of evolution on a population. 7.1 Fitness as a Result of Interaction To make this concrete, we now describe a first simple example of how game-theoretic ideas can be applied in evolutionary settings. This example will be designed for ease of explanation rather than perfect fidelity to the underlying biology; but after this we will discuss examples where the phenomenon at the heart of the example has been empirically observed in a variety of natural settings. For the example, let’s consider a particular species of beetle, and suppose that each beetle’s fitness in a given environment is determined largely by the extent to which it can find food and use the nutrients from the food effectively. Now, suppose a particular mutation is introduced into the population, causing beetles with the mutation to grow a significantly larger body size. Thus, we now have two distinct kinds of beetles in the population — small ones and large ones. It is actually difficult for the large beetles to maintain the metabolic requirements of their larger body size — it requires diverting more nutrients from the food they eat — and so this has a negative effect on fitness. If this were the full story, we’d conclude that the large-body-size mutation is fitness- decreasing, and so it will likely be driven out of the population over time, through multiple generations. But in fact, there’s more to the story, as we’ll now see. Interaction Among Organisms. The beetles in this population compete with each other for food – when they come upon a food source, there’s crowding among the beetles as they each try to get as much of the food as they can. And, not surprisingly, the beetles with large body sizes are more effective at claiming an above-average share of the food. Let’s assume for simplicity that food competition in this population involves two beetles interacting with each other at any given point in time. (This will make the ideas easier to describe, but the principles we develop can also be applied to interactions among many individuals simultaneously.) When two beetles compete for some food, we have the following possible outcomes. • When beetles of the same size compete, they get equal shares of the food. • When a large beetle competes with a small beetle, the large beetle gets the majority of the food.

225 7.2. EVOLUTIONARILY STABLE STRATEGIES 211 In all cases, large beetles experience less of a fitness benefit from a given quantity of • food, since some of it is diverted into maintaining their expensive metabolism. Thus, the fitness that each beetle gets from a given food-related interaction can be thought of as a numerical payoff in a two-player game between a first beetle and a second beetle, as follows. The first beetle plays one of the two strategies Small Large , depending or on its body size, and the second beetle plays one of these two strategies as well. Based on the two strategies used, the payoffs to the beetles are described by Figure 7.1. Beetle 2 Large Small Small 5 , 5 1 , 8 Beetle 1 Large 8 , 1 3 , 3 Figure 7.1: The Body-Size Game Notice how the numerical payoffs satisfy the principles just outlined: when two small beetles meet, they share the fitness from the food source equally; large beetles do well at the expense of small beetles; but large beetles cannot extract the full amount of fitness from the food source. (In this payoff matrix, the reduced fitness when two large beetles meet is particularly pronounced, since a large beetle has to expend extra energy in competing with another large beetle.) This payoff matrix is a nice way to summarize what happens when two beetles meet, but compared with the game in Chapter 6, there’s something fundamentally different in what’s being described here. The beetles in this game aren’t asking themselves, “What do I want my body size to be in this interaction?” Rather, each is genetically hard-wired to play one of these two strategies through its whole lifetime. Given this important difference, the idea of choosing strategies — which was central to our formulation of game theory — is missing from the biological side of the analogy. As a result, in place of the idea of Nash equilibrium — which was based fundamentally on the relative benefit of changing one’s own personal strategy — we will need to think about strategy changes that operate over longer time scales, taking place as shifts in a population under evolutionary forces. We develop the fundamental definitions for this in the next section. 7.2 Evolutionarily Stable Strategies In Chapter 6, the notion of Nash equilibrium was central in reasoning about the outcome of a game. In a Nash equilibrium for a two-player game, neither player has an incentive to deviate from the strategy they are currently using — the equilibrium is a choice of strategies that tends to persist once the players are using it. The analogous notion for evolutionary

226 212 CHAPTER 7. EVOLUTIONARY GAME THEORY evolutionarily stable strategy — a genetically-determined strategy settings will be that of an that tends to persist once it is prevalent in a population. We formulate this as follows. Suppose, in our example, that each beetles is repeatedly paired off with other beetles in food competitions over the course of its lifetime. We will assume the population is large enough that no two particular beetles have a significant probability of interacting with each other repeatedly. A beetle’s overall fitness will be equal to the average fitness it experiences from each of its many pairwise interactions with others, and this overall fitness determines its reproductive success — the number of offspring that carry its genes (and hence its strategy) into the next generation. if, when the whole In this setting, we say that a given strategy is evolutionarily stable population is using this strategy, any small group of invaders using a different strategy will eventually die off over multiple generations. (We can think of these invaders either as migrants who move to join the population, or as mutants who were born with the new behavior directly into the population.) We capture this idea in terms of numerical payoffs by saying that when the whole population is using a strategy S , then a small group of T should have strictly lower fitness than the users invaders using any alternate strategy S of the majority strategy . Since fitness translates into reproductive success, evolutionary principles posit that strictly lower fitness is the condition that causes a sub-population (like the users of strategy ) to shrink over time, through multiple generations, and eventually T die off with high probability. More formally, we will phrase the basic definitions as follows. We say the fitness of an organism in a population is the expected payoff it receives • from an interaction with a random member of the population. We say that a strategy T invades a strategy S at level x , for some small positive • number x , if an x fraction of the underlying population uses T and a 1 − x fraction of the underlying population uses S . Finally, we say that a strategy is evolutionarily stable if there is a (small) positive • S y number T invades S at any level x < y , the fitness such that when any other strategy of an organism playing S is strictly greater than the fitness of an organism playing T . Evolutionarily Stable Strategies in our First Example. Let’s see what happens when we apply this definition to our example involving beetles competing for food. We will first check whether the strategy Small is evolutionarily stable, and then we will do the same for the strategy Large . Following the definition, let’s suppose that for some small positive number x , a 1 − x fraction of the population uses Small and an x fraction of the population uses Large . (This

227 7.2. EVOLUTIONARILY STABLE STRATEGIES 213 is what the picture would look like just after a small invader population of large beetles arrives.) • What is the expected payoff to a small beetle in a random interaction in this popula- x − , it meets another small beetle, receiving a payoff of 5, tion? With probability 1 while with probability x , it meets a large beetle, receiving a payoff of 1. Therefore its expected payoff is x ) + 1 5(1 x = 5 − 4 x. − · What is the expected payoff to a large beetle in a random interaction in this population? • With probability 1 x , it meets a small beetle, receiving a payoff of 8, while with − probability , it meets another large beetle, receiving a payoff of 3. Therefore its x expected payoff is x. x ) + 3 · x = 8 8(1 5 − − It’s easy to check that for small enough values of x (and even for reasonably large ones in this case), the expected fitness of large beetles in this population exceeds the expected fitness of small beetles. Therefore Small is not evolutionarily stable. Now let’s check whether Large is evolutionarily stable. For this, we suppose that for some very small positive number x − x fraction of the population uses Large and an x fraction , a 1 Small . of the population uses What is the expected payoff to a large beetle in a random interaction in this population? • With probability 1 − , it meets another large beetle, receiving a payoff of 3, while with x probability x , it meets a small beetle, receiving a payoff of 8. Therefore its expected payoff is 3(1 − x ) + 8 · x = 3 + 5 x. • What is the expected payoff to a small beetle in a random interaction in this popu- lation? With probability 1 − x , it meets a large beetle, receiving a payoff of 1, while with probability x , it meets another small beetle, receiving a payoff of 5. Therefore its expected payoff is (1 − x ) + 5 · x = 1 + 4 x. In this case, the expected fitness of large beetles in this population exceeds the expected fitness of small beetles, and so Large is evolutionarily stable.

228 214 CHAPTER 7. EVOLUTIONARY GAME THEORY Intuitively, this Interpreting the Evolutionarily Stable Strategy in our Example. analysis can be summarized by saying that if a few large beetles are introduced into a population consisting of small beetles, then the large beetles do extremely well — since they rarely meet each other, they get most of the food in almost every competition they experience. As a result, the population of small beetles cannot drive out the large ones, and so Small is not evolutionarily stable. On the other hand, in a population of large beetles, a few small beetles will do very badly, losing almost every competition for food. As a result, the population of large beetles resists Large is evolutionarily stable. the invasion of small beetles, and so Therefore, if we know that the large-body-size mutation is possible, we should expect to see populations of large beetles in the wild, rather than populations of small ones. In this way, our notion of evolutionary stability has predicted a strategy for the population — as we predicted outcomes for games among rational players in Chapter 6, but by different means. What’s striking about this particular predicted outcome, though, is the fact that the fitness of each organism in a population of small beetles is 5, which is larger than the fitness of each organism in a population of large beetles. In fact, the game between small and large beetles has precisely the structure of a Prisoner’s Dilemma game; the motivating scenario based on competition for food makes it clear that the beetles are engaged in an arms race, like the game from Chapter 6 in which two competing athletes need to decide whether to use performance-enhancing drugs. There it was a dominant strategy to use drugs, even though both athletes understand that they are better off in an outcome where neither of them uses drugs — it’s simply that this mutually better joint outcome is not sustainable. In the present case, the beetles individually don’t understand anything, nor could they change their body sizes even if they wanted to. Nevertheless, evolutionary forces over multiple generations are achieving a completely analogous effect, as the large beetles benefit at the expense of the small ones. Later in this chapter, we will see that this similarity in the conclusions of two different styles of analysis is in fact part of a broader principle. Here is a different way to summarize the striking feature of our example: Starting from a population of small beetles, evolution by natural selection is causing the fitness of the organisms to decrease over time. This might seem troubling initially, since we think of natural selection as being fitness-increasing. But in fact, it’s not hard to reconcile what’s happening with this general principle of natural selection. Natural selection increases the fitness of individual organisms in a fixed environment — if the environment changes to become more hostile to the organisms, then clearly this could cause their fitness to go down. This is what is happening to the population of beetles. Each beetle’s environment includes all the other beetles, since these other beetles determine its success in food competitions; therefore the increasing fraction of large beetles can be viewed, in a sense, as a shift to an environment that is more hostile for everyone.

229 7.2. EVOLUTIONARILY STABLE STRATEGIES 215 Biologists have offered recent evi- Empirical Evidence for Evolutionary Arms Races. dence for the presence of evolutionary games in nature with the Prisoner’s-Dilemma structure we’ve just seen. It is very difficult to truly determine payoffs in any real-world setting, and so all of these studies are the subject of ongoing investigation and debate. For our purposes in this discussion, they are perhaps most usefully phrased as deliberately streamlined examples, illustrating how game-theoretic reasoning can help provide qualitative insight into different forms of biological interaction. It has been argued that the heights of trees can obey Prisoner’s-Dilemma payoffs [156, 226]. If two neighboring trees both grow short, then they share the sunlight equally. They also share the sunlight equally if they both grow tall, but in this case their payoffs are each lower because they have to invest a lot of resources in achieving the additional height. The trouble is that if one tree is short while its neighbor is tall, then the tall tree gets most of the sunlight. As a result, we can easily end up with payoffs just like the Body-Size Game among beetles, with the trees’ evolutionary strategies and Tall serving as analogues to Short the beetles’ strategies and Large . Of course, the real situation is more complex than Small this, since genetic variation among trees can lead to a wide range of different heights and hence a range of different strategies (rather than just two strategies labeled Short and Tall ). Within this continuum, Prisoner’s-Dilemma payoffs can only apply to a certain range of tree heights: there is some height beyond which further height-increasing mutations no longer provide the same payoff structure, because the additional sunlight is more than offset by the fitness downside of sustaining an enormous height. Similar kinds of competition take place in the root systems of plants [181]. Suppose you grow two soybean plants at opposite ends of a large pot of soil; then their root systems will each fill out the available soil and intermingle with each other as they try to claim as many resources as they can. In doing so, they divide the resources in the soil equally. Now, suppose that instead you partition the same quantity of soil using a wall down the middle, so that the two plants are on opposite sides of the wall. Then each still gets half the resources present in the soil, but each invests less of its energy in producing roots and consequently has greater reproductive success through seed production. This observation has implications for the following simplified evolutionary game involving root systems. Imagine that instead of a wall, we had two kinds of root-development strategies available to soybean plants: Conserve , where a plant’s roots only grow into its own share of the soil, and , where the roots grow everywhere they can reach. Then we again have Explore the scenario and payoffs from the Body-Size Game, with the same conclusion: all plants are better off in a population where everyone plays Conserve , but only Explore is evolutionarily stable. As a third example, there was recent excitement over the discovery that virus populations can also play an evolutionary version of the Prisoner’s Dilemma [326, 392]. Turner and Chao

230 216 CHAPTER 7. EVOLUTIONARY GAME THEORY studied a virus called Phage Φ6, which infects bacteria and manufactures products needed for its own replication. A mutational variant of this virus called Phage ΦH2 is also able to replicate in bacterial hosts, though less effectively on its own. However, ΦH2 is able to take advantage of chemical products produced by Φ6, which gives ΦH2 a fitness advantage when it is in the presence of Φ6. This turns out to yield the structure of the Prisoner’s Dilemma: viruses have the two evolutionary strategies Φ6 and ΦH2; viruses in a pure Φ6 population all do better than viruses in a pure ΦH2 population; and regardless of what the other viruses are doing, you (as a virus) are better off playing ΦH2. Thus only ΦH2 is evolutionarily stable. The virus system under study was so simple that Turner and Chao were able to infer an actual payoff matrix based on measuring the relative rates at which the two viral variants were able to replicate under different conditions. Using an estimation procedure derived from these measurements, they obtained the payoffs in Figure 7.2. The payoffs are re-scaled 1 00. , 1 . 00 . so that the upper-left box has the value 1 Virus 2 Φ6 ΦH2 , Φ6 1 . 00 , 1 . 00 0 . 65 99 1 . Virus 1 ΦH2 . 99 , 0 . 65 1 . 83 , 0 . 83 0 Figure 7.2: The Virus Game Whereas our earlier examples had an underlying story very much like the use of performance- enhancing drugs, this game among phages is actually reminiscent of a different story that also motivates the Prisoner’s Dilemma payoff structure: the scenario behind the Exam-or- Presentation game with which we began Chapter 6. There, two college students would both be better off if they jointly prepared for a presentation, but the payoffs led them to each think selfishly and study for an exam instead. What the Virus Game here shows is that shirking a shared responsibility isn’t just something that rational decision-makers do; evolutionary forces can induce viruses to play this strategy as well. 7.3 A General Description of Evolutionarily Stable Strate- gies The connections between evolutionary games and games played by rational participants are suggestive enough that it makes sense to understand how the relationship works in general. We will focus here, as we have thus far, on two-player two-strategy games. We will also 1 It should be noted that even in a system this simple, there are many other biological factors at work, and hence this payoff matrix is still just an approximation to the performance of Φ6 and ΦH2 populations under real experimental and natural conditions. Other factors appear to affect these populations, including the density of the population and the potential presence of additional mutant forms of the virus [393].

231 7.3. A GENERAL DESCRIPTION OF EVOLUTIONARILY STABLE STRATEGIES 217 restrict our attention to symmetric games, as in the previous sections of this chapter, where the roles of the two players are interchangeable. The payoff matrix for a completely general two-player, two-strategy game that is sym- metric can be written as in Figure 7.3. Organism 2 S T a,a b,c S Organism 1 d,d T c,b Figure 7.3: General Symmetric Game Let’s check how to write the condition that S is evolutionarily stable in terms of the four a , b , c , and d . As before, we start by supposing that for some very small positive variables x , a 1 − x fraction of the population uses S and an number fraction of the population uses x T . What is the expected payoff to an organism playing in a random interaction in this • S population? With probability 1 − x , it meets another player of S , receiving a payoff of , while with probability x , it meets a player of T a b . Therefore , receiving a payoff of its expected payoff is (1 − x ) + a bx. • What is the expected payoff to an organism playing T in a random interaction in this population? With probability 1 − x , it meets a player of S , receiving a payoff of c , while with probability x T , receiving a payoff of d . Therefore its , it meets another player of expected payoff is c − x ) + dx. (1 S is evolutionarily stable if for all sufficiently small values of x > 0, the Therefore, inequality a − x ) + bx > c (1 − x ) + dx (1 holds. As x goes to 0, the left-hand side becomes a and the right-hand side becomes c . Hence, if a > c x is sufficiently small, while if a < c , then the left-hand side is larger once then the left-hand side is smaller once is sufficiently small. Finally, if a = c , then the x left-hand side is larger precisely when b > d . Therefore, we have a simple way to express the condition that is evolutionarily stable: S In a two-player, two-strategy, symmetric game, S is evolutionarily stable precisely when either (i) . , or (ii) a = c and b > d a > c

232 218 CHAPTER 7. EVOLUTIONARY GAME THEORY It is easy to see the intuition behind our calculations that translates into this condition, as follows. S to be evolutionarily stable, the payoff to using strategy S against • First, in order for T against S . Otherwise, an S must be at least as large as the payoff to using strategy would have a higher fitness than the rest of population, and the invader who uses T fraction of the population who are invaders would have a good probability of growing over time. Second, if S and T are equally good responses to S • S to be evolu- , then in order for tionarily stable, players of must do better in their interactions with T than players S T T would do as well as against the S of do with each other. Otherwise, players of S , and at least as well against the part of the population as players of part of the T population, so their overall fitness would be at least as good as the fitness of players of S . 7.4 Relationship Between Evolutionary and Nash Equi- libria Using our general way of characterizing evolutionarily stable strategies, we can now under- stand how they relate to Nash equilibria. If we go back to the General Symmetric Game from the previous section, we can write down the condition for ( ) (i.e. the choice of S S,S S,S S is a best by both players) to be a Nash equilibrium. ( ) is a Nash equilibrium when S by the other player: this translates into the simple condition response to the choice of ≥ c. a If we compare this to the condition for S to be evolutionarily stable, (i) a > c , or (ii) a = c and b > d, we immediately get the conclusion that S is evolutionarily stable, then ( S,S ) is a Nash equilibrium. If strategy We can also see that the other direction does not hold: it is possible to have a game where ( S,S ) is a Nash equilibrium, but S is not evolutionarily stable. The difference in the b < d two conditions above tells us how to construct such a game: we should have = c and a . To get a sense for where such a game might come from, let’s recall the Stag Hunt Game from Chapter 6. Here, each player can hunt stag or hunt hare; hunting hare successfully just requires your own effort, while hunting the more valuable stag requires that you both do so. This produces payoffs as shown in Figure 7.4.

233 7.4. RELATIONSHIP BETWEEN EVOLUTIONARY AND NASH EQUILIBRIA 219 Hunter 2 Hunt Stag Hunt Hare 4 4 0 , 3 Hunt Stag , Hunter 1 Hunt Hare 0 3 , 3 3 , Figure 7.4: Stag Hunt Hunt Stag and Hunt Hare are both evolutionarily stable, as we In this game, as written, a can check from the conditions on b , c , and d . (To check the condition for Hunt Hare , we , simply need to interchange the rows and columns of the payoff matrix, to put Hunt Hare in the first row and first column.) However, suppose we make up a modification of the Stag Hunt Game, by shifting the payoffs as follows. In this new version, when the players mis-coordinate, so that one hunts stag while the other hunts hare, then the hare-hunter gets an extra benefit due to the lack of competition for hare. In this way, we get a payoff matrix as in Figure 7.5. Hunter 2 Hunt Stag Hunt Hare 0 Hunt Stag 4 , 4 4 , Hunter 1 Hunt Hare , 0 3 , 3 4 Figure 7.5: Stag Hunt: A version with added benefit from hunting hare alone In this case, the choice of strategies is still a Nash equilibrium: (Hunt Stag, Hunt Stag) if each player expects the other to hunt stag, then hunting stag is a best response. But is not an evolutionarily stable strategy for this version of the game, because (in Hunt Stag a the notation from our General Symmetric Game) we have c and b < d . Informally, = the problem is that a hare-hunter and a stag-hunter do equally well when each is paired with a stag-hunter; but hare-hunters do better than stag-hunters when each is paired with a hare-hunter. There is also a relationship between evolutionarily stable strategies and the concept of a strict Nash equilibrium . We say that a choice of strategies is a strict Nash equilibrium if each player is using the unique best response to what the other player is doing. We can check that for symmetric two-player, two-strategy games, the condition for ( S,S ) to be a strict Nash equilibrium is that . So we see that in fact these different notions of equilibrium a > c naturally refine each other. The concept of an evolutionarily stable strategy can be viewed as a refinement of the concept of a Nash equilibrium: the set of evolutionarily stable strategies S S for which ( S,S ) is a Nash equilibrium. Similarly, the is a subset of the set of strategies concept of a strict Nash equilibrium (when the players use the same strategy) is a refinement of evolutionary stability: if ( S,S ) is a strict Nash equilibrium, then S is evolutionarily stable.

234 220 CHAPTER 7. EVOLUTIONARY GAME THEORY It is intriguing that, despite the extremely close similarities between the conclusions of evolutionary stability and Nash equilibrium, they are built on very different underlying stories. In a Nash equilibrium, we consider players choosing mutual best responses to each other’s strategy. This equilibrium concept places great demands on the ability of the players to choose optimally and to coordinate on strategies that are best responses to each other. Evolutionary stability, on the other hand, supposes no intelligence or coordination on the part of the players. Instead, strategies are viewed as being hard-wired into the players, perhaps because their behavior is encoded in their genes. According to this concept, strategies which are more successful in producing offspring are selected for. Although this evolutionary approach to analyzing games originated in biology, it can be applied in many other contexts. For example, suppose a large group of people are being matched repeatedly over time to play the General Symmetric Game from Figure 7.3. Now the payoffs should be interpreted as reflecting the welfare of the players, and not their number of offspring. If any player can look back at how others have played and can observe their payoffs, then imitation of the strategies that have been most successful may induce an evolutionary dynamic. Alternatively, if a player can observe his own past successes and failures then his learning may induce an evolutionary dynamic. In either case, strategies that have done relatively well in the past will tend to be used by more people in the future. This can lead to the same behavior that underlies the concept of evolutionarily stable strategies, and hence can promote the play of such strategies. 7.5 Evolutionarily Stable Mixed Strategies As a further step in developing an evolutionary theory of games, we now consider how to handle cases in which no strategy is evolutionarily stable. In fact, it is not hard to see how this can happen, even in two-player games that have 2 pure-strategy Nash equilibria. Perhaps the most natural example is the Hawk-Dove Game from Chapter 6, and we use this to introduce the basic ideas of this section. Recall that in the Hawk-Dove Game, two animals compete for a piece of food; an animal that plays the Hawk ( H ) behaves aggressively, while an animal that plays the strategy strategy ( D ) Dove behaves passively. If one animal is aggressive while the other is passive, then the aggressive animal benefits by getting most of the food; but if both animals are aggressive, then they risk destroying the food and injuring each other. This leads to a payoff matrix as shown in Figure 7.6. In Chapter 6, we considered this game in contexts where the two players were making choices about how to behave. Now let’s consider the same game in a setting where each 2 Recall that a player is using a pure strategy if she always plays a particular one of the strategies in the game, as opposed to a mixed strategy in which she chooses at random from among several possible strategies.

235 7.5. EVOLUTIONARILY STABLE MIXED STRATEGIES 221 Animal 2 D H D 3 , 3 1 , 5 Animal 1 5 , 1 0 H 0 , Figure 7.6: Hawk-Dove Game animal is genetically hard-wired to play a particular strategy. How does it look from this perspective, when we consider evolutionary stability? Neither D nor is a best response to itself, and so using the general principles from H the last two sections, we see that neither is evolutionarily stable. Intuitively, a hawk will do very well in a population consisting of doves — but in a population of all hawks, a dove will actually do better by staying out of the way while the hawks fight with each other. As a two-player game in which players are actually choosing strategies, the Hawk-Dove Game has two pure Nash equilibria: ( D,H H,D ). But this doesn’t directly help us ) and ( identify an evolutionarily stable strategy, since thus far our definition of evolutionary stability has been restricted to populations in which (almost) all members play the same pure strategy. To reason about what will happen in the Hawk-Dove Game under evolutionary forces, we need to generalize the notion of evolutionary stability by allowing some notion of “mixing” between strategies. Defining Mixed Strategies in Evolutionary Game Theory. There are at least two natural ways to introduce the idea of mixing into the evolutionary framework. First, it could be that each individual is hard-wired to play a pure strategy, but some portion of the population plays one strategy while the rest of the population plays another. If the fitness of individuals in each part of the population is the same, and if invaders eventually die off, then this could be considered to exhibit a kind of evolutionary stability. Second, it could be that each individual is hard-wired to play a particular mixed strategy — that is, they are genetically configured to choose randomly from among certain options with certain probabilities. If invaders using any other mixed strategy eventually die off, then this too could be considered a kind of evolutionary stability. We will see that for our purposes here, these two concepts are actually equivalent to each other, and we will focus initially on the second idea, in which individuals use mixed strategies. Essentially, we will find that in situations like the Hawk-Dove game, the individuals or the population as a whole must display a mixture of the two behaviors in order to have any chance of being stable against invasion by other forms of behavior. The definition of an evolutionarily stable mixed strategy is in fact completely parallel to the definition of evolutionary stability we have seen thus far — it is simply that we now greatly enlarge the set of possible strategies, so that each strategy corresponds to a particular

236 222 CHAPTER 7. EVOLUTIONARY GAME THEORY randomized choice over pure strategies. Specifically, let’s consider the General Symmetric Game from Figure 7.3. A mixed strat- between 0 and 1, indicating that the organism plays S egy here corresponds to a probability p and plays with probability 1 − p . As in our discussion of mixed strategies p T with probability or T by simply from Chapter 6, this includes the possibility of playing the pure strategies S = 1 or p = 0. When Organism 1 uses the mixed strategy p and Organism 2 uses p setting the mixed strategy q , the expected payoff to Organism 1 can be computed as follows. There pq of an ( X,X ) pairing, yielding is a probability for the first player; there is a probability a p − q ) of an ( X,Y ) pairing, yielding b for the first player; there is a probability (1 − p ) q of (1 q a ( c for the first player; and there is a probability (1 − p )(1 − ) pairing, yielding ) of a Y,X ( Y,Y ) pairing, yielding d for the first player. So the expected payoff for this first player is V ( p,q ) = pqa + p (1 − q ) b + (1 − p ) qc + (1 − p )(1 − q ) d. As before, the fitness of an organism is its expected payoff in an interaction with a random member of the population. We can now give the precise definition of an evolutionarily stable mixed strategy. is an evolutionarily stable mixed strategy if In the General Symmetric Game, p such that when any other mixed strategy y q there is a (small) positive number at any level x < y , the fitness of an organism playing invades is strictly p p q . greater than the fitness of an organism playing This is just like our previous definition of evolutionarily stable (pure) strategies, except that we allow the strategy to be mixed, and we allow the invaders to use a mixed strategy. p p = 0 is evolutionarily stable under An evolutionarily stable mixed strategy with = 1 or our original definition for pure strategies as well. However, note the subtle point that even S were an evolutionarily stable strategy under our previous definition, it is not necessarily if p = 1. The problem an evolutionarily stable mixed strategy under this new definition with is that it is possible to construct games in which no pure strategy can successfully invade a population playing S , but a mixed strategy can. As a result, it will be important to be clear in any discussion of evolutionary stability on what kinds of behavior an invader can employ. Directly from the definition, we can write the condition for p to be an evolutionarily stable mixed strategy as follows: for some y x < y , the following inequality holds and any for all mixed strategies 6 = p : q (1 − x ) V ( p,p ) + xV ( p,q ) > (1 − x ) V ( q,p ) + xV ( q,q ) . (7.1) This inequality also makes it clear that there is a relationship between mixed Nash equilibria and evolutionarily stable mixed strategies, and this relationship parallels the one we saw earlier for pure strategies. In particular, if p is an evolutionarily stable mixed strategy,

237 7.5. EVOLUTIONARILY STABLE MIXED STRATEGIES 223 V ( ) ≥ V ( q,p ), and so p is a best response to p . As a result, the pair then we must have p,p ) is a mixed Nash equilibrium. However, because of the strict inequality of strategies ( p,p p being in Equation (7.1), it is possible for ( p,p ) to be a mixed Nash equilibrium without evolutionarily stable. So again, evolutionary stability serves as a refinement of the concept of mixed Nash equilibrium. Now let’s see Evolutionarily Stable Mixed Strategies in the Hawk-Dove Game. how to apply these ideas to the Hawk-Dove Game. First, since any evolutionarily stable mixed strategy must correspond to a mixed Nash equilibrium of the game, this gives us a way to search for possible evolutionarily stable strategies: we first work out the mixed Nash equilibria for the Hawk-Dove, and then we check if they are evolutionarily stable. As we saw in Chapter 6, in order for ( ) to be a mixed Nash equilibrium, it must make p,p the two players indifferent between their two pure strategies. When the other player is using , the expected payoff from playing D is 3 p + (1 the strategy p ) = 1 + 2 p , while the expected p − H is 5 p . Setting these two quantities equal (to capture the indifference payoff from playing p = 1 / 3. So (1 / 3 , 1 between the two strategies), we get 3) is a mixed Nash equilibrium. In / this case, both pure strategies, as well as any mixture between them, produce an expected / 3 when played against the strategy = 1 / 3. payoff of 5 p = 1 3 is evolutionarily stable, we must check Inequality (7.1) p / Now, to see whether invades at a small level x . Here is a first observation when some other mixed strategy q ) is a mixed Nash equilibrium p,p that makes evaluating this inequality a bit easier. Since ( that uses both pure strategies, we have just argued that all mixed strategies q have the same p . As a result, we have V ( payoff when played against ) = V ( q,p ) for all q . Subtracting p,p these terms from the left and right of Inequality (7.1), and then dividing by , we get the x following inequality to check: . p,q ) > V ( V ) ( (7.2) q,q The point is that since ( p,p ) is a mixed equilibrium, the strategy p can’t be a strict best response to itself — all other mixed strategies are just as good against it. Therefore, in order for p to be evolutionarily stable, it must be a strictly better response to every other mixed than is to itself. That is what will cause it to have higher fitness when q invades. strategy q q ( p,q ) > V ( q,q In fact, it is true that q 6 = p , and we can check V ) for all mixed strategies p = 1 / 3, we have this as follows. Using the fact that ( p,q ) = 1 / 3 · q · 3 + 1 / 3(1 − q ) · 1 + 2 / 3 · q · 5 = 4 q + 1 / 3 V while 2 2 q,q ) = q V · 3 + q (1 − q ) · 1 + (1 − q ) · q · 5 = 6 q − 3 q ( . Now we have 1 1 2 2 2 (3 . + 1) = q 6 1) − − q (9 q q,q V ( + 1 q 2 − q ) = 3 / ( p,q V − ) 3 = 3 3

238 224 CHAPTER 7. EVOLUTIONARY GAME THEORY V ( ) − V ( q,q ) shows that it is a perfect square, and so it is positive This last way of writing p,q ) 6 q 3. This is just what we want for showing that V ( p,q = 1 > V ( q,q ) whenever whenever / 6 = p , and so it follows that p is indeed an evolutionarily stable mixed strategy. q The kind of mixed equi- Interpretations of Evolutionarily Stable Mixed Strategies. librium that we see here in the Hawk-Dove Game is typical of biological situations in which organisms must break the symmetry between two distinct behaviors, when consistently adopting just one of these behaviors is evolutionarily unsustainable. We can interpret the result of this example in two possible ways. First, all participants in the population may actually be mixing over the two possible pure strategies with the given probability. In this case, all members of the population are genetically the same, but whenever two of them are matched up to play, any combination of D H could potentially and be played. We know the empirical frequency with which any pair of strategies will be played, but not what any two animals will actually do. A second interpretation is that the mixture / is taking place at the population level: it could be that 1 3 of the animals are hard-wired to always play , and 2 / 3 are hard-wired to always play H D . In this case, no individual is actually mixing, but as long as it is not possible to tell in advance which animal will play D and which will play H , the interaction of two randomly selected animals results in the same frequency of outcomes that we see when each animal is actually mixing. Notice also that in this case, the fitnesses of both kinds of animals are the same, since both D H are and p / 3. Thus, these two different interpretation of the best responses to the mixed strategy = 1 evolutionarily stable mixed strategy lead to the same calculations, and the same observed behavior in the population. There are a number of other settings in which this type of mixing between pure strategies has been discussed in biology. A common scenario is that there is an undesirable, fitness- lowering role in a population of organisms — but if some organisms don’t choose to play this role, then everyone suffers considerably. For example, let’s think back to the Virus Game in Figure 7.2 and suppose (purely hypothetically, for the sake of this example) that the payoff . 50 , 0 when both viruses use the strategy ΦH2 were (0 50), as shown in Figure 7.7. . Virus 2 Φ6 ΦH2 00 Φ6 1 . 99 , 1 . 00 0 . 65 , 1 . Virus 1 . 1 . 99 , 0 . 65 0 ΦH2 50 , 0 . 50 Figure 7.7: The Virus Game: Hypothetical payoffs with stronger fitness penalties to ΦH2. In this event, rather than having a Prisoner’s Dilemma type of payoff structure, we’d have a Hawk-Dove payoff structure: having both viruses play ΦH2 is sufficiently bad that one of them needs to play the role of Φ6. The two pure equilibria of the resulting two-player game

239 7.6. EXERCISES 225 — viewed as a game among rational players, rather than a biological interaction — would be (Φ6,ΦH2) and (ΦH2,Φ6). In a virus population we’d expect to find an evolutionarily stable mixed strategy in which both kinds of virus behavior were observed. This example, like the examples from our earlier discussion of the Hawk-Dove Game in Section 6.6, suggests the delicate boundary that exists between Prisoner’s Dilemma and Hawk-Dove. In both cases, a player can choose to be “helpful” to the other player or “selfish”. In Prisoner’s Dilemma, however, the payoff penalties from selfishness are mild enough that selfishness by both players is the unique equilibrium — while in Hawk-Dove, selfishness is sufficiently harmful that at least one player should try to avoid it. There has been research into how this boundary between the two games manifests itself in other biological settings as well. One example is the implicit game played by female lions in defending their territory [218, 327]. When two female lions encounter an attacker on the edge of their territory, each can choose to play the strategy , in which she Confront confronts the attacker, or Lag , in which she lags behind and tries to let the other lion confront the attacker first. If you’re one of the lions, and your fellow defender chooses the strategy Confront , then you get a higher payoff by choosing Lag , since you’re less likely to get injured. What’s harder to determine in empirical studies is what a lion’s best response should be to a play of Lag by her partner. Choosing Confront risks injury, but joining your partner in Lag risks a successful assault on the territory by the attacker. Understanding which is the best response is important for understanding whether this game is more like Prisoner’s Dilemma or Hawk-Dove, and what the evolutionary consequences might be for the observed behavior within a lion population. In this, as in many examples from evolutionary game theory, it is beyond the power of current empirical studies to work out detailed fitness values for particular strategies. How- ever, even in situations where exact payoffs are not known, the evolutionary framework can provide an illuminating perspective on the interactions between different forms of behav- ior in an underlying population, and how these interactions shape the composition of the population. 7.6 Exercises 1. In the payoff matrix below the rows correspond to player A’s strategies and the columns correspond to player B’s strategies. The first entry in each box is player A’s payoff and the second entry is player B’s payoff. Player B x y 0 x 2 , 2 0 , Player A y 0 , 0 1 , 1

240 226 CHAPTER 7. EVOLUTIONARY GAME THEORY (a) Find all pure strategy Nash equilibria. (b) Find all Evolutionarily Stable strategies. Give a brief explanation for your answer. (c) Briefly explain how the sets of predicted outcomes relate to each other. 2. In the payoff matrix below the rows correspond to player A’s strategies and the columns correspond to player B’s strategies. The first entry in each box is player A’s payoff and the second entry is player B’s payoff. Player B x y 4 x 4 3 , 5 , Player A y , 3 5 , 5 5 (a) Find all pure strategy Nash equilibria. (b) Find all Evolutionarily Stable strategies. Give a brief explanation for your answer. (c) Briefly explain how the answers in parts (2a) and (2b) relate to each other. 3. In this problem we will consider the relationship between Nash equilibria and evolu- tionarily stable strategies for games with a strictly dominant strategy. First, let’s define strictly dominant . In a two-player game, strategy, X is said to be a what we mean by strictly dominant strategy for a player i if, no matter what strategy the other player j uses, player i X is strictly greater than his payoff from any ’s payoff from using strategy a,b,c, are non-negative d other strategy. Consider the following game in which and numbers. Player B X Y X a,a b,c Player A Y c,b d,d Suppose that strategy X is a strictly dominant strategy for each player, i.e. and a > c b > d . (a) Find all of the pure strategy Nash equilibria of this game. (b) Find all of the evolutionarily stable strategies of this game. (c) How would your answers to parts (a) and (b) change if we change the assumption on payoffs to: a > c and b = d ?

241 7.6. EXERCISES 227 Player B X Y 1 , 1 2 ,x X Player A Y 3 , 3 x, 2 x can be 0, 1, or 2. 4. Consider following the two-player, symmetric game where For each of the possible values of x find all (pure strategy) Nash equilibria and all (a) evolutionarily stable strategies. (b) Your answers to part (a) should suggest that the difference between the predictions of evolutionary stability and Nash equilibrium arises when a Nash equilibrium uses a ∗ s i weakly dominated strategy is weakly dominated if player . We say that a strategy i ′ with the property that: has another strategy s i ′ is at least as large ’s payoff from s (a) No matter what the other player does, player i i ∗ as the payoff from s , and i ′ (b) There is some strategy for the other player so that player ’s payoff from s is i i ∗ s . strictly greater than the payoff from i Now, consider the following claim that makes a connection between evolutionarily stable strategies and weakly dominated strategies. Claim: Suppose that in the game below, ( X,X ) is a Nash equilibrium and that strategy is weakly dominated. Then X is not an evolutionarily stable X strategy. Player B X Y X a,a b,c Player A Y c,b d,d Explain why this claim is true. (You do not need to write a formal proof; a careful explanation is fine.)

242 228 CHAPTER 7. EVOLUTIONARY GAME THEORY

243 Chapter 8 Modeling Network Traffic using Game Theory Among the initial examples in our discussion of game theory in Chapter 6, we noted that traveling through a transportation network, or sending packets through the Internet, involves fundamentally game-theoretic reasoning: rather than simply choosing a route in isolation, individuals need to evaluate routes in the presence of the congestion resulting from the decisions made by themselves and everyone else. In this chapter, we develop models for network traffic using the game-theoretic ideas we’ve developed thus far. In the process of Braess’s Paradox [76] doing this, we will discover a rather unexpected result — known as — which shows that adding capacity to a network can sometimes actually slow down the traffic. 8.1 Traffic at Equilibrium Let’s begin by developing a model of a transportation network and how it responds to traffic congestion; with this in place, we can then introduce the game-theoretic aspects of the problem. We represent a transportation network by a directed graph: we consider the edges to be highways, and the nodes to be exits where you can get on or off a particular highway. There are two particular nodes, which we’ll call A and B , and we’ll assume everyone wants to drive is an exit from to B . For example, we can imagine that A is an exit in the suburbs, B A downtown, and we’re looking at a large collection of morning commuters. Finally, each edge has a designated travel time that depends on the amount of traffic it contains. To make this concrete, consider the graph in Figure 8.1. The label on each edge gives the D. Easley and J. Kleinberg. Networks, Crowds, and Markets: Reasoning about a Highly Connected World . Cambridge University Press, 2010. Draft version: June 10, 2010. 229

244 230 CHAPTER 8. MODELING NETWORK TRAFFIC USING GAME THEORY C 45 x/100 A B 45 x/100 D Figure 8.1: A highway network, with each edge labeled by its travel time (in minutes) when cars using it. When 4000 cars need to get from there are to B , they divide evenly over x A the two routes at equilibrium, and the travel time is 65 minutes. travel time (in minutes) when there are x cars using the edge. In this simplified example, A - D and C - B edges are insensitive to congestion: each takes 45 minutes to traverse the B A C and D - - regardless of the number of cars traveling on them. On the other hand, the x/ 100 minutes to traverse edges are highly sensitive to congestion: for each one, it takes 1 x cars using the edge. when there are Now, suppose that 4000 cars want to get from A B as part of the morning commute. to C There are two possible routes that each car can choose: the upper route through , or the D . For example, if each car takes the upper route (through C lower route through ), then the total travel time for everyone is 85 minutes, since 4000 / 100 + 45 = 85. The same is true if everyone takes the lower route. On the other hand, if the cars divide up evenly between the two routes, so that each carries 2000 cars, then the total travel time for people on both routes is 2000 / 100 + 45 = 65. Equilibrium traffic. So what do we expect will happen? The traffic model we’ve described is really a game in which the players correspond to the drivers, and each player’s possible strategies consist of the possible routes from A to B . In our example, this means that each player only has two strategies; but in larger networks, there could be many strategies for each player. The payoff for a player is the negative of his or her travel time (we use the negative since large travel times are bad). 1 The travel times here are simplified to make the reasoning clearer: in any real application, each road would have both some minimum travel time, and some sensitivity to the number of cars x that are using it. However, the analysis here adapts directly to more intricate functions specifying the travel times on edges.

245 8.2. BRAESS’S PARADOX 231 This all fits very naturally into the framework we’ve been using. One thing to notice, of course, is that up to now we have focused primarily on games with two players, whereas the current game will generally have an enormous number of players (4000 in our example). But this poses no direct problem for applying any of the ideas we’ve developed. A game can have any number of players, each of whom can have any number of available strategies, and the payoff to each player depends on the strategies chosen by all. A Nash equilibrium is still a list of strategies, one for each player, so that each player’s strategy is a best response to all the others. The notions of dominant strategies, mixed strategies and Nash equilibrium with mixed strategies all have direct parallels with their definitions for two-player games. In this traffic game, there is generally not a dominant strategy; for example, in Figure 8.1 either route has the potential to be the best choice for a player if all the other players are using the other route. The game does have Nash equilibria, however: as we will see next, any list of strategies in which the drivers balance themselves evenly between the two routes (2000 on each) is a Nash equilibrium, and these are the only Nash equilibria. Why does equal balance yield a Nash equilibrium, and why do all Nash equilibria have equal balance? To answer the first question, we just observe that with an even balance between the two routes, no driver has an incentive to switch over to the other route. For the second question, consider a list of strategies in which drivers use the upper route and the x − x drivers use the lower route. Then if remaining 4000 is not equal to 2000, the two routes x will have unequal travel times, and any driver on the slower route would have an incentive to switch to the faster one. Hence any list of strategies in which x is not equal to 2000 cannot be a Nash equilibrium; and any list of strategies in which x = 2000 is a Nash equilibrium. 8.2 Braess’s Paradox In Figure 8.1, everything works out very cleanly: self-interested behavior by all drivers causes them — at equilibrium — to balance perfectly between the available routes. But with only a small change to the network, we can quickly find ourselves in truly counterintuitive territory. The change is as follows: suppose that the city government decides to build a new, very fast highway from C D , as indicated in Figure 8.2. To keep things simple, we’ll model its to travel time as 0, regardless of the number of cars on it, although the resulting effect would happen even with more realistic (but small) travel times. It would stand to reason that people’s travel time from A to B ought to get better after this edge from C to D is added. Does it? Here’s the surprise: there is a unique Nash equilibrium in this new highway network, but it leads to a worse travel time for everyone. At equilibrium, every driver uses the route through both C and D ; and as a result, the travel time for every driver is 80 (since 4000 / 100 + 0 + 4000 / 100 = 80). To see why this is an equilibrium, note that no driver can

246 232 CHAPTER 8. MODELING NETWORK TRAFFIC USING GAME THEORY C 45 x/100 0 A B 45 x/100 D Figure 8.2: The highway network from the previous figure, after a very fast edge has been C to D added from . Although the highway system has been “upgraded,” the travel time at equilibrium is now 80 minutes, since all cars use the route through and D . C benefit by changing their route: with traffic snaking through and D the way it is, any C other route would now take 85 minutes. And to see why it’s the only equilibrium, you can check that the creation of the edge from C to D has in fact made the route through C and D a dominant strategy for all drivers: regardless of the current traffic pattern, you gain by C D . switching your route to go through and C In other words, once the fast highway from D is built, the route through C and to D acts like a “vortex” that draws all drivers into it — to the detriment of all. In the new network there is no way, given individually self-interested behavior by the drivers, to get back to the even-balance solution that was better for everyone. This phenomenon — that adding resources to a transportation network can sometimes hurt performance at equilibrium — was first articulated by Dietrich Braess in 1968 [76], and it has become known as Braess’s Paradox. Like many counterintuitive anomalies, it needs the right combination of conditions to actually pop up in real life; but it has been observed empirically in real transportation networks — including in Seoul, Korea, where the destruction of a six-lane highway to build a public park actually improved travel time into and out of the city (even though traffic volume stayed roughly the same before and after the change) [37]. Some reflections on Braess’s paradox. Having now seen how Braess’s paradox works, we can also appreciate that there is actually nothing really “paradoxical” about it. There are many settings in which adding a new strategy to a game makes things worse for everyone. For example, the Prisoner’s Dilemma from Chapter 6 can be used to illustrate this point: if

247 8.3. ADVANCED MATERIAL: THE SOCIAL COST OF TRAFFIC AT EQUILIBRIUM 233 the only strategy for each player were Not-Confess (an admittedly very simple game), then both players would be better off compared with the game where Confess is added as an option. (Indeed, that’s why the police offer Confess as an option in the first place.) Still, it’s reasonable to view the analogous phenomenon at the heart of the Braess Paradox as more paradoxical, at an intuitive level. We all have an informal sense that “upgrading” a network has to be a good thing, and so it is surprising when it turns out to make things worse. The example in this section is actually the starting point for a large body of work on game-theoretic analysis of network traffic. For example, we could ask how bad Braess’s Paradox can be for networks in general: how much larger can the equilibrium travel time be after the addition of an edge, relative to what it was before? Suppose in particular that we allow the graph to be arbitrary, and we assume that the travel time on each edge depends in a linear way on the number of cars traversing it — that is, all travel times across edges have the form + b , where each of a and b is either 0 or a positive number. In this case, elegant ax ́ results of Tim Roughgarden and Eva Tardos can be used to show that if we add edges to a network with an equilibrium pattern of traffic, there is always an equilibrium in the new network whose travel time is no more than 4 / 3 times as large [18, 353]. Moreover, 4 / 3 is the factor increase that we’d get in the example from Figures 8.1 and 8.2, if we replace the two travel times of 45 with 40. (In that case, the travel time at equilibrium would jump from 60 C D .) So the Roughgarden-Tardos result shows that to 80 when we add the edge from to this simple example is as bad as the Braess Paradox can get, in a quantitative sense, when edges respond linearly to traffic. (When edges can respond non-linearly, things can be much worse.) There are many other types of questions that can be pursued as well. For example, we could think about ways of designing networks to prevent bad equilibria from arising, or to avoid bad equilibria through the judicious use of tolls on certain parts of the network. Many of these extensions, as well as others, are discussed by Tim Roughgarden in his book on game-theoretic models of network traffic [352]. 8.3 Advanced Material: The Social Cost of Traffic at Equilibrium The Braess Paradox is one aspect of a larger phenomenon, which is that network traffic at equilibrium may not be socially optimal. In this section, we try to quantify how far from optimal traffic can be at equilibrium. We would like our analysis to apply to any network, and so we introduce the following general definitions. The network can be any directed graph. There is a set of drivers, and different drivers may have different starting points and destinations. Now, each edge e has

248 234 CHAPTER 8. MODELING NETWORK TRAFFIC USING GAME THEORY C C T T (x)=5 (x)=x CB AC 5 x 0 (x)=0 T CD A A B B 5 x D D (x)=5 T (x)=x T AD DB . x Travel times written as explicit functions of (a) Travel times written as annotations on the (b) edges. Figure 8.3: A network annotated with the travel-time function on each edge. travel-time function T ), which gives the time it takes all drivers to cross the edge when ( a x e there are x drivers using it. These travel times are simply the functions that we drew as labels inside the edges in the figures in Section 8.1. We will assume that all travel-time functions for some choice of numbers are linear in the amount of traffic, so that ( x ) = a and x + b a T e e e e b that are either positive or zero. For example, in Figure 8.3 we draw another network on e which Braess’s Paradox arises, with the travel-time functions scaled down to involve smaller numbers. The version of the drawing in Figure 8.3(a) has the travel-time functions explicitly written out, while the version of the drawing in Figure 8.3(b) has the travel-time functions written as labels inside the edges. Finally, we say that a traffic pattern is simply a choice of a path by each driver, and the social cost of a given traffic pattern is the sum of the travel times incurred by all drivers when they use this traffic pattern. For example, Figure 8.4 shows two different traffic patterns on the network from Figure 8.3, when there are four drivers, each with starting node A and destination node B . The first of these traffic patterns, in Figure 8.4(a), achieves the minimum possible social cost — each driver requires 7 units of time to get to their destination, and so the social cost is 28. We will refer to such a traffic pattern, which achieves the minimum possible social cost, as socially optimal . (There are other traffic patterns on this network that also achieve a social cost of 28; that is, there are multiple traffic patterns for this network that are socially optimal.) Note that socially optimal traffic patterns are simply the social welfare maximizers of this traffic game, since the sum of the drivers’ payoffs is the negative of the social cost. The second traffic pattern, Figure 8.4(b), is the unique Nash equilibrium, and it has a larger social cost of 32. The main two questions we consider in the remainder of this chapter are the following. First, in any network (with linear travel-time functions), is there always an equilibrium traffic

249 8.3. ADVANCED MATERIAL: THE SOCIAL COST OF TRAFFIC AT EQUILIBRIUM 235 C C 5 x 5 x 0 A B 0 A B 5 x 5 x D D (a) The social optimum. The Nash equilibrium. (b) Figure 8.4: A version of Braess’s Paradox: In the socially optimal traffic pattern (on the left), the social cost is 28, while in the unique Nash equilibrium (on the right), the social cost is 32. pattern? We have seen examples in Chapter 6 of games where equilibria do not exist using pure strategies, and it is not a priori clear that they should always exist for the traffic game we’ve defined here. However, we will find in fact that equilibria always do exist. The second main question is whether there always exists an equilibrium traffic pattern whose social cost is not much more than the social optimum. We will find that this is in fact the case: we will show a result due to Roughgarden and Tardos that there is always an equilibrium whose 2 social cost is at most that of the optimum [353]. twice A. How to Find a Traffic Pattern at Equilibrium We will prove that an equilibrium exists by analyzing the following procedure that explicitly searches for one. The procedure starts from any traffic pattern. If it is an equilibrium, we are done. Otherwise, there is at least one driver whose best response, given what everyone else is doing, is some alternate path providing a strictly lower travel time. We pick one such driver and have him switch to this alternate path. We now have a new traffic pattern and we again check whether it is an equilibrium — if it isn’t, then we have some driver switch to his best response, and we continue in this fashion. This procedure is called best-response dynamics , since it dynamically reconfigures the 2 In fact, stronger results of Roughgarden and Tardos, supplemented by subsequent results of Anshelevich et al. [18], establish that in fact every equilibrium traffic pattern has social cost at most 4 / 3 times the optimum. (One can show that this implies their result on the Braess Paradox cited in the previous section — that with linear travel times, adding edges can’t make things worse by a factor of more than 4 / 3.) However, since it is harder to prove the bound of 4 / 3, we limit ourselves here to proving the easier but weaker factor of 2 between the social optimum and some equilibrium traffic pattern.

250 236 CHAPTER 8. MODELING NETWORK TRAFFIC USING GAME THEORY players’ strategies by constantly having some player perform his or her best response to the current situation. If the procedure ever stops, in a state where everyone is in fact playing their best response to the current situation, then we have an equilibrium. So the key is to show that in any instance of our traffic game, best-response dynamics must eventually stop at an equilibrium. But why should it? Certainly for games that lack an equilibrium, best-response dynamics will run forever: for example, in the Matching Pennies game from Chapter 6, when only pure strategies are allowed, best-response dynamics will simply consist of the two players endlessly H and T switching their strategies between . It seems plausible that for some network, this could happen in the traffic game as well: one at a time, drivers shift their routes to ones that are better for them, thus increasing the delay for another driver who then switches and continues the cascade. In fact, however, this cannot happen in the traffic game. We now show that best-response dynamics must always terminate in an equilibrium, thus proving not only that equilibria exist but also that they can be reached by a simple process in which drivers constantly update what they’re doing according to best responses. Analyzing Best-Response Dynamics Via Potential Energy. How should we go about proving that best-response dynamics must come to a halt? When you have a process that runs according to some set of instructions like, “Do the following ten things and then stop,” it’s generally obvious that it will eventually come to an end: the process essentially comes with its own guarantee of termination. But we have a process that runs according to a different kind of rule, one that says, “Keep doing something until a particular condition happens to hold.” In this case, there is no a priori reason to believe it will ever stop. In such cases, a useful analysis technique is to define some kind of progress measure that tracks the process as it operates, and to show that eventually enough “progress” will be made that the process must stop. For the traffic game, it’s natural to think of the social cost of the current traffic pattern as a possible progress measure, but in fact the social cost is not so useful for this purpose. Some best-response updates by drivers can make the social cost better (for example, if a driver leaves a congested road for a relatively empty one), but others can make it worse (as in the sequence of best-response updates that shifts the traffic pattern from the social optimum to the inferior equilibrium in the Braess Paradox). So in general, as best-response dynamics runs, the social cost of the current traffic pattern can oscillate between going up and going down, and it’s not clear how this is related to our progress toward an equilibrium. Instead, we’re going to define an alternate quantity that initially seems a bit mysterious. However, we will see that it has the property that it strictly decreases with each best-response update, and so it can be used to track the progress of best-response dynamics [303]. We will

251 8.3. ADVANCED MATERIAL: THE SOCIAL COST OF TRAFFIC AT EQUILIBRIUM 237 potential energy refer to this quantity as the of a traffic pattern. The potential energy of a traffic pattern is defined edge-by-edge, as follows. If an edge e currently has x drivers on it, then we define the potential energy of this edge to be e ) = T . (1) + T ) (2) + ··· + Energy( x ( T e e e If an edge has no drivers on it, its potential energy is defined to be 0. The potential energy of a traffic pattern is then simply the sum of the potential energies of all the edges, with their current number of drivers in this traffic pattern. In Figure 8.5, we show the potential energy of each edge for the five traffic patterns that best-response dynamics produces as it moves from the social optimum to the unique equilibrium in the Braess-Paradox network from Figure 8.4. e with x drivers is not the total travel time Notice that the potential energy of an edge experienced by the drivers that cross it. Since there are x drivers each experiencing a travel x time of ( x ), their total travel time is xT ), which is a different number. The potential ( T e e energy, instead, is a sort of “cumulative” quantity in which we imagine drivers crossing the edge one by one, and each driver only “feels” the delay caused by himself and the drivers crossing the edge in front of him. Of course, the potential energy is only useful for our purpose if it lets us analyze the progress of best-response dynamics. We show how to do this next. Proving that Best-Response Dynamics Comes to an End. Our main claim is the following: each step of best-response dynamics causes the potential energy of the current traffic pattern to strictly decrease. Proving this will be enough to show that best-response dynamics must come to an end, for the following reason. The potential energy can only take a finite number of possible values — one for each possible traffic pattern. If it is strictly decreasing with each step of best-response dynamics, this means that it is “consuming” this finite supply of possible values, since it can never revisit a value once it drops below it. So best-response dynamics must come to a stop by the time the potential energy reaches its minimum possible value (if not sooner). And once best-response dynamics comes to a stop, we must be at an equilibrium — for otherwise, the dynamics would have a way to continue. Thus, showing that the potential energy strictly decreases in every step of best-response dynamics is enough to show the existence of an equilibrium traffic pattern. As an example, let’s return to the sequence of best-response steps from Figure 8.5. Al- though the social cost is rising through the five traffic patterns (increasing from 28 to 32), the potential energy decreases strictly in each step (in the sequence 26, 24, 23, 21, 20). In fact, it is easy to track the change in potential energy through this sequence as follows. From one traffic pattern to the next, the only change is that one driver abandons his current path and switches to a new one. Suppose we really view this switch as a two-step process: first the

252 238 CHAPTER 8. MODELING NETWORK TRAFFIC USING GAME THEORY energy = 5+5 energy = 1+2 energy = 5 energy = 1+2 C C 5 5 x x 0 0 B A A B 5 5 x x D D energy = 1+2 energy = 5+5 energy = 5+5 energy = 1+2+3 (b) After one step of best-response dynamics. (Po- The initial traffic pattern. (Potential energy is (a) 26.) tential energy is 24.) energy = 0 C energy = 1+2 energy = 1+2+3 C energy = 0 5 x 5 x 0 B A 0 B A 5 x 5 x D D energy = 1+2+3+4 energy = 5+5 energy = 5 energy = 1+2+3+4 (d) After three steps. (Potential energy is 21.) (c) After two steps. (Potential energy is 23.) energy = 1+2+3+4 C energy = 0 5 x 0 A B 5 x D energy = 0 energy = 1+2+3+4 (e) After four steps: Equilibrium is reached. (Potential energy is 20.) Figure 8.5: We can track the progress of best-response dynamics in the traffic game by watching how the potential energy changes.

253 239 8.3. ADVANCED MATERIAL: THE SOCIAL COST OF TRAFFIC AT EQUILIBRIUM energy = 1+2 energy = 5+5 energy = 1 energy = 5 C C 5 5 x x 0 0 B A A B 5 5 x x D D energy = 1+2 energy = 5+5 energy = 5+5 energy = 1+2 The potential energy of a traffic pattern not in Potential energy is released when a driver aban- (a) (b) equilibrium. dons their current path. energy = 5 energy = 1+2 C 5 x 0 A B 5 x D energy = 5+5 energy = 1+2+3 (c) Potential energy is put back into the system when the driver chooses a new path. Figure 8.6: When a driver abandons one path in favor of another, the change in potential energy is exactly the improvement in the driver’s travel time. drivers abandons his current path, temporarily leaving the system; then, the driver returns to the system by adopting a new path. This first step releases potential energy as the driver leaves the system, and the second step adds potential energy as he re-joins. What’s the net change? For example, the transition from Figure 8.5(a) to 8.5(b) occurs because one driver aban- dons the upper path and adopts the zigzag path. As shown in Figure 8.6, abandoning the upper path releases 2 + 5 = 7 units of potential energy, while adopting the zigzag path puts 2 + 0 + 3 units of potential energy back into the system. The resulting change is a decrease of 2. Notice that the decrease of 7 is simply the travel time the driver was experiencing on the path he abandoned, and the subsequent increase of 5 is the travel time the driver now experiences on the path he has adopted. This relationship is in fact true for any network and

254 240 CHAPTER 8. MODELING NETWORK TRAFFIC USING GAME THEORY any best response by a driver, and it holds for a simple reason. Specifically, the potential energy of edge x drivers is e with − (1) + T (2) + ··· + T , ( x T 1) + T ) ( x e e e e and when one of these drivers leaves it drops to (1) + T T . (2) + ··· + T 1) ( x − e e e e x T is ( Hence the change in potential energy on edge ), exactly the travel time that the e e . Summing this over all edges used by the driver, we see that driver was experiencing on the potential energy released when a driver abandons his current path is exactly equal to the travel time the driver was experiencing. By the same reasoning, when a driver adopts a new path, the potential energy on each edge e he joins increases from T (2) + ··· + T T ( x ) (1) + e e e to , T (2) + ··· + T ( x ) + T T ( x + 1) (1) + e e e e and the increase of T ( x + 1) is exactly the new travel time the driver experiences on this e the potential energy added to the system when a driver adopts a new path is edge. Hence, exactly equal to the travel time the driver now experiences. It follows when a driver switches paths, the net change in potential energy is simply his new travel time minus his old travel time. But in best-response dynamics, a driver only changes paths when it causes his travel time to decrease — so the change in potential energy is negative for any best-response move. This establishes what we wanted to show: that the potential energy in the system strictly decreases throughout best-response dynamics. As argued above, since the potential energy cannot decrease forever, best-response dynamics must therefore eventually come to an end, at a traffic pattern in equilibrium. B. Comparing Equilibrium Traffic to the Social Optimum Having shown that an equilibrium traffic pattern always exists, we now consider how its travel time compares to that of a socially optimal traffic pattern. We will see that the potential energy we’ve defined is very useful for making this comparison. The basic idea is to establish a relationship between the potential energy of an edge and the total travel time of all drivers crossing the edge. Once we do this, we will sum these two quantities over all the edges to compare travel times at equiibrium and at social optimality.

255 8.3. ADVANCED MATERIAL: THE SOCIAL COST OF TRAFFIC AT EQUILIBRIUM 241 (x) T e (3) T e ... T (2) e (1) T e Figure 8.7: The potential energy is the area under the shaded rectangles; it is always at least half the total travel time, which is the area inside the enclosing rectangle. Relating Potential Energy to Travel Time for a Single Edge. We denote the po- tential energy of an edge by Energy( e ), and we recall that when there are x drivers, this potential energy is defined by e Energy( T . (1) + T ) (2) + ··· + T x ( ) = e e e x drivers experiences a travel time of On the other hand, each of the ( x ), and so the total T e travel time experienced by all drivers on the edge is Total-Travel-Time( e ) = xT . ( x ) e For purposes of comparison with the potential energy, it is useful to write this as follows: . ··· ) = e ( x ) + T ) ( Total-Travel-Time( ) + T + T x ( x e e e ︷︷ ︸ ︸ terms x Since the potential energy and the total travel time each have x terms, but the terms in the latter expression are at least as large as the terms in the former, we have Energy( e ) ≤ Total-Travel-Time( e ) . Figure 8.7 shows how the potential energy and the total travel time compare when T e is a linear function: the total travel time is the shaded area under the horizontal line with

256 242 CHAPTER 8. MODELING NETWORK TRAFFIC USING GAME THEORY -value ( x ), while the potential energy is the total area under all the unit-width rectangles y T e ,...,T (1) ,T of heights (2) T is a linear ( x ). As this figure makes clear geometrically, since T e e e e function, we have 1 (1) + T T (2) + ··· + T x ( x ) ≥ ) ( xT . e e e e 2 x ( x ) = a Alternately, we can see this by a bit of simple algebra, recalling that : + b T e e e x (1) + T b (2) + ··· + T ) + ( x ) = a x (1 + 2 + ··· + T e e e e e a x ( x + 1) e = x + b e 2 ( ) ( x + 1) a e + b x = e 2 1 ≥ x ( a ) x + b e e 2 1 = ) ( x xT . e 2 In terms of energies and total travel times, this says 1 ) ≥ Energy( . · Total-Travel-Time( e ) e 2 So the conclusion is that the potential energy of an edge is never far from the total travel time: it is sandwiched between the total travel time and half the total travel time. Relating the Travel Time at Equilibrium and Social Optimality. We now use this relationship between potential energy and total travel to relate the equilibrium and socially optimal traffic patterns. Let be a traffic pattern; we define Energy( Z ) to be the total potential energy of all Z . We write Social-Cost( edges when drivers follow the traffic pattern ) to denote the Z Z social cost of the traffic pattern; recall that this is the sum of the travel times experienced by all drivers. Equivalently, summing the social cost edge-by-edge, Social-Cost( Z ) is the sum of the total travel times on all the edges. So applying our relationships between potential energy and travel time on an edge-by-edge basis, we see that the same relationships govern the potential energy and social cost of a traffic pattern: 1 · Social-Cost( Z ) ≤ . Z ) ≤ Social-Cost( Z ) Energy( 2 Now, suppose that we start from a socially optimal traffic pattern Z , and we then allow ′ best-response dynamics to run until they stop at an equilibrium traffic pattern Z . The social cost may start increasing as we run best-response dynamics, but the potential energy can only go down — and since the social cost can never be more than twice the potential energy, this shrinking potential energy keeps the social cost from ever getting more than twice as

257 8.4. EXERCISES 243 high as where it started. This shows that the social cost of the equilibrium we reach is at most twice the cost of the social optimum we started with — hence there is an equilibrium with at most twice the socially optimal cost, as we wanted to show. Let’s write this argument out in terms of the inequalities on energies and social costs. First, we saw in the previous section that the potential energy decreases as best-response ′ to , and so Z dynamics moves from Z ′ . ) Energy( Z ) ≤ Energy( Z Second, the quantitative relationships between energies and social cost say that ′ ′ ) ≤ 2 · Energy( Z Z ) Social-Cost( and ) ) ≤ Social-Cost( Z Z . Energy( Now we just chain these inequalities together, concluding that ′ ′ Z ) ≤ 2 · Energy( Social-Cost( Z ) ≤ 2 · Energy( Z ) ≤ 2 · Social-Cost( Z ) . Note that this really is the same argument that we made in words in the previous paragraph: the potential energy decreases during best-response dynamics, and this decrease prevents the social cost from every increasing by more than a factor of two. Thus, tracking potential energy is not only useful for showing that best-response dynamics must reach an equilibrium; by relating this potential energy to the social cost, we can use it to put a bound on the social cost of the equilibrium that is reached. 8.4 Exercises 1. There are 1000 cars which must travel from town A to town B. There are two possible routes that each car can take: the upper route through town C or the lower route x be the number of cars traveling on the edge AC and let y be through town D. Let the number of cars traveling on the edge DB. The directed graph in Figure 8.8 indicates that travel time per car on edge AC is x/ 100 if x cars use edge AC, and similarly the travel time per car on edge DB is 100 if y cars use edge DB. The travel time per y/ car on each of edges CB and AD is 12 regardless of the number of cars on these edges. Each driver wants to select a route to minimize his travel time. The drivers make simultaneous choices. (a) Find Nash equilibrium values of x and y . (b) Now the government builds a new (one-way) road from town C to town D. The new road adds the path ACDB to the network. This new road from C to D has a travel

258 244 CHAPTER 8. MODELING NETWORK TRAFFIC USING GAME THEORY C x/100 12 A B 12 y/100 D Figure 8.8: Traffic Network. time of 0 per car regardless of the number of cars that use it. Find a Nash equilibrium x and for the game played on the new network. What are the equilibrium values of ? y What happens to total cost-of-travel (the sum of total travel times for the 1000 cars) as a result of the availability of the new road? (c) Suppose now that conditions on edges CB and AD are improved so that the travel times on each edge are reduced to 5. The road from C to D that was constructed in part (b) is still available. Find a Nash equilibrium for the game played on the network with the smaller travel times for CB and AD. What are the equilibrium values of x and y ? What is the total cost-of-travel? What would happen to the total cost-of-travel if the government closed the road from C to D? 2. There are two cities A and B joined by two routes. There are 80 travelers who begin in city A and must travel to city B. There are two routes between A and B. Route I begins with a highway leaving city A, this highway takes one hour of travel time regardless of how many travelers use it, and ends with a local street leading into city B. This local street near city B requires a travel time in minutes equal to 10 plus the number of travelers who use the street. Route II begins with a local street leaving city A, which requires a travel time in minutes equal to 10 plus the number of travelers who use this street, and ends with a highway into city B which requires one hour of travel time regardless of the number of travelers who use this highway. (a) Draw the network described above and label the edges with the travel time needed to move along the edge. Let x be the number of travelers who use Route I. The network should be a directed graph as all roads are one-way. (b) Travelers simultaneously chose which route to use. Find the Nash equilibrium value of x . (c) Now the government builds a new (two-way) road connecting the nodes where local streets and highways meet. This adds two new routes. One new route consists of the

259 8.4. EXERCISES 245 C 3.1 x/100 B A 3.1 y/100 D Figure 8.9: Traffic Network local street leaving city A (on Route II), the new road and the local street into city B (on Route I). The second new route consists of the highway leaving city A (on Route I), the new road and the highway leading into city B (on Route II). The new road is very short and takes no travel time. Find the new Nash equilibrium. (Hint: There is an equilibrium in which no one chooses to use the second new route described above. ) What happens to total travel time as a result of the availability of the new road? (d) (e) If you can assign travelers to routes, then in fact it’s possible to reduce total travel time relative to what it was before the new road was built. That is, the total travel time of the population can be reduced (below that in the original Nash equilibrium from part (b)) by assigning travelers to routes. There are many assignments of routes that will accomplish this. Find one. Explain why your reassignment reduces total travel time. (Hint: Remember that travel on the new road can go in either direction. You do not need find the total travel time minimizing assignment of travelers. One approach to this question is to start with the Nash equilibrium from part (b) and look ) for a way to assign some travelers to different routes so as to reduce total travel time. 3. There are 300 cars which must travel from city A to city B. There are two possible routes that each car can take. The upper route through city C or the lower route through city D. Let be the number of cars traveling on the edge AC and let y be the x number of cars traveling on the edge DB. The directed graph in Figure 8.9 indicates that total travel time per car along the upper route is ( x/ 100) + 3 . 1 if x cars use the upper route, and similarly the total travel time per car along the lower route is 3 . 1 + ( y/ 100) if y cars take the lower route. Each driver wants to select a route to

260 246 CHAPTER 8. MODELING NETWORK TRAFFIC USING GAME THEORY minimize his total travel time. The drivers make simultaneous choices. (a) and y . Find Nash equilibrium values of x (b) Now the government builds a new (one-way) road from city A to city B. The new route has a travel time of 5 per car regardless of the number of cars that use it. Draw the new network and label the edges with the cost-of-travel needed to move along the edge. The network should be a directed graph as all roads are one-way. Find a Nash equilibrium for the game played on the new network. What happens to total cost-of- travel (the sum of total travel times for the 300 cars) as a result of the availability of the new road? (c) Now the government closes the direct route between city A and city B and builds a new one-way road which links city C to city D. This new road between C and D is very short and has a travel time of 0 regardless of the number of cars that use it. Draw the new network and label the edges with the cost-of-travel needed to move along the edge. The network should be a directed graph as all roads are one-way. Find a Nash equilibrium for the game played on the new network. What happens to total cost-of-travel as a result of the availability of the new road? (d) The government is unhappy with the outcome in part (c) and decides to reopen the road directly linking city A and city B (the road that was built in part (b) and closed in part (c)). The route between C and D that was constructed in part (c) remains open. This road still has a travel time of 5 per car regardless of the number of cars that use it. Draw the new network and label the edges with the cost-of-travel needed to move along the edge. The network should be a directed graph as all roads are one-way. Find a Nash equilibrium for the game played on the new network. What happens to total cost-of-travel as a result of re-opening the direct route between A and B? 4. There are two cities, A and B, joined by two routes, I and II. All roads are one-way roads. There are 100 travelers who begin in city A and must travel to city B. Route I links city A to city B through city C. This route begins with a road linking city A to . city C which has a cost-of-travel for each traveler equal to 0 x/ 200, where x is the 5 + number of travelers on this road. Route I ends with a highway from city C to city B which has a cost-of-travel for each traveler of 1 regardless of the number of travelers who use it. Route II links city A to city B through city D. This route begins with a highway linking city A to city D which has a cost-of-travel for each traveler of 1 regardless of the number of travelers who use it. Route II ends with a road linking city D to city B which has a cost-of-travel for each traveler equal to 0 . 5 + y/ 200, where y is the number of travelers on this road. These costs of travel are the value that travelers put on the time lost due to travel plus the cost of gasoline for the trip. Currently there are no tolls on these roads. So the

261 8.4. EXERCISES 247 government collects no revenue from travel on them. (a) Draw the network described above and label the edges with the cost-of-travel needed to move along the edge. The network should be a directed graph as all roads are one-way. (b) Travelers simultaneously chose which route to use. Find Nash equilibrium values x and of . y (c) Now the government builds a new (one-way) road from city C to city D. The new road is very short and has 0 cost-of-travel. Find a Nash equilibrium for the game played on the new network. (d) What happens to total cost-of-travel as a result of the availability of the new road? (e) The government is unhappy with the outcome in part (c) and decides to impose a toll on users of the road from city A to city C and to simultaneously subsidize users of . 125 to each user, and thus the highway from city A to city D. They charge a toll of 0 increase the cost-of-travel by this amount, for users of the road from city A to city C. They also subsidize travel, and thus reduce the cost-of-travel by this amount, for each user of the highway from city A to city D by 0 . 125. Find a new Nash equilibrium. [If you are curious about how a subsidy could work you can think of it as a negative toll. In this economy all tolls are collected electronically, much as New York State attempts to do with its E-ZPass system. So a subsidy just reduces the total amount that highway users owe.] (f ) As you will observe in solving part (e) the toll and subsidy in part (e) were designed so that there is a Nash equilibrium in which the amount the government collects from the toll just equals the amount it loses on the subsidy. So the government is breaking even on this policy. What happens to total cost-of-travel between parts (c) and (e)? Can you explain why this occurs? Can you think of any break-even tolls and subsidies that could be placed on the roads from city C to city B, and from city D to city B, that would lower the total cost-of-travel even more?

262 248 CHAPTER 8. MODELING NETWORK TRAFFIC USING GAME THEORY

263 Chapter 9 Auctions In Chapter 8, we considered a first extended application of game-theoretic ideas, in our analysis of traffic flow through a network. Here we consider a second major application — the behavior of buyers and sellers in an auction. An auction is a kind of economic activity that has been brought into many people’s everyday lives by the Internet, through sites such as eBay. But auctions also have a long history that spans many different domains. For example, the U.S. government uses auctions to sell Treasury bills and timber and oil leases; Christie’s and Sotheby’s use them to sell art; and Morrell & Co. and the Chicago Wine Company use them to sell wine. Auctions will also play an important and recurring role in the book, since the simplified form of buyer-seller interaction they embody is closely related to more complex forms of economic interaction as well. In particular, when we think in the next part of the book about markets in which multiple buyers and sellers are connected by an underlying network structure, we’ll use ideas initially developed in this chapter for understanding simpler auction formats. Similarly, in Chapter 15, we’ll study a more complex kind of auction in the context of a Web search application, analyzing the ways in which search companies like Google, Yahoo!, and Microsoft use an auction format to sell advertising rights for keywords. 9.1 Types of Auctions In this chapter we focus on different simple types of auctions, and how they promote different kinds of behavior among bidders. We’ll consider the case of a seller auctioning one item to a set of buyers. We could symmetrically think of a situation in which a buyer is trying to purchase a single item, and runs an auction among a set of multiple sellers, each of whom is able to provide the item. Such procurement auctions are frequently run by governments to D. Easley and J. Kleinberg. Networks, Crowds, and Markets: Reasoning about a Highly Connected World . Cambridge University Press, 2010. Draft version: June 10, 2010. 249

264 250 CHAPTER 9. AUCTIONS purchase goods. But here we’ll focus on the case in which the seller runs the auction. There are many different ways of defining auctions that are much more complex than what we consider here. The subsequent chapters will generalize our analysis to the case in which there are multiple goods being sold, and the buyers assign different values to these goods. Other variations, which fall outside the scope of the book, include auctions in which goods are sold sequentially over time. These more complex variations can also be analyzed using extensions of the ideas we’ll talk about here, and there is a large literature in economics that considers auctions at this broad level of generality [256, 292]. The underlying assumption we make when modeling auctions is that each bidder has an intrinsic value for the item being auctioned; she is willing to purchase the item for a price up to this value, but not for any higher price. We will also refer to this intrinsic value as the bidder’s for the item. There are four main types of auctions when a single item is true value being sold (and many variants of these types). 1. Ascending-bid auctions , also called English auctions . These auctions are carried out interactively in real time, with bidders present either physically or electronically. The seller gradually raises the price, bidders drop out until finally only one bidder remains, and that bidder wins the object at this final price. Oral auctions in which bidders shout out prices, or submit them electronically, are forms of ascending-bid auctions. 2. Descending-bid auctions Dutch auctions . This is also an interactive auction , also called format, in which the seller gradually lowers the price from some high initial value until the first moment when some bidder accepts and pays the current price. These auctions are called Dutch auctions because flowers have long been sold in the Netherlands using this procedure. 3. First-price sealed-bid auctions . In this kind of auction, bidders submit simultaneous “sealed bids” to the seller. The terminology comes from the original format for such auctions, in which bids were written down and provided in sealed envelopes to the seller, who would then open them all together. The highest bidder wins the object and pays the value of her bid. 4. Second-price sealed-bid auctions , also called Vickrey auctions . Bidders submit simul- taneous sealed bids to the sellers; the highest bidder wins the object and pays the value of the second-highest bid. These auctions are called Vickrey auctions in honor of William Vickrey, who wrote the first game-theoretic analysis of auctions (including the second-price auction [400]). Vickery won the Nobel Memorial Prize in Economics in 1996 for this body of work.

265 9.2. WHEN ARE AUCTIONS APPROPRIATE? 251 9.2 When are Auctions Appropriate? Auctions are generally used by sellers in situations where they do not have a good estimate of the buyers’ true values for an item, and where buyers do not know each other’s values. In this case, as we will see, some of the main auction formats can be used to elicit bids from buyers that reveal these values. Known Values. To motivate the setting in which buyers’ true values are unknown, let’s start by considering the case in which the seller and buyers know each other’s values for an item, and argue that an auction is unnecessary in this scenario. In particular, suppose that a seller is trying to sell an item that he values at x , and suppose that the maximum value y . In this case, we say there is a held by a potential buyer of the item is some larger number surplus y − x that can be generated by the sale of the item: it can go from someone who of values it less ( x ) to someone who values it more ( y ). If the seller knows the true values that the potential buyers assign to the item, then he can simply announce that the item is for sale at a fixed price just below y , and that he will not accept any lower price. In this case, the buyer with value y will buy the item, and the full value of the surplus will go to the seller. In other words, the seller has no need for an auction in this case: he gets as much as he could reasonably expect just by announcing the right price. Notice that there is an asymmetry in the formulation of this example: we gave the seller the ability to commit to the mechanism that was used for selling the object. This ability of the seller to “tie his hands” by committing to a fixed price is in fact very valuable to him: assuming the buyers believe this commitment, the item is sold for a price just below y , and the seller makes all the surplus. In contrast, consider what would happen if we gave the buyer with maximum value y the ability to commit to the mechanism. In this case, this buyer could announce that she is willing to purchase the item for a price just above the larger of x and the values held by all other buyers. With this announcement, the seller would still be willing to sell — since the price would be above — but now at least some x of the surplus would go to the buyer. As with the seller’s commitment, this commitment by the buyer also requires knowledge of everyone else’s values. These examples show how commitment to a mechanism can shift the power in the trans- action in favor of the seller or the buyer. One can also imagine more complex scenarios in which the seller and buyers know each other’s values, but neither has the power to unilater- ally commit to a mechanism. In this case, one may see some kind of bargaining take place over the price; we discuss the topic of bargaining further in Chapter 12. As we will discover in the current chapter, the issue of commitment is also crucial in the context of auctions — specifically, it is important that a seller be able to reliably commit in advance to a given auction format.

266 252 CHAPTER 9. AUCTIONS Thus far we’ve been discussing how sellers and buyers might interact Unknown Values. when everyone knows each other’s true values for the item. Beginning in the next section, we’ll see how auctions come into play when the participants do not know each other’s values. For most of this chapter we will restrict our attention to the case in which the buyers have independent, private values for the item. That is, each buyer knows how much she values the item, she does not know how much others value it, and her value for it does not depend on others’ values. For example, the buyers could be interested in consuming the item, with their values reflecting how much they each would enjoy it. common Later we will also consider the polar opposite of this setting — the case of . Suppose that an item is being auctioned, and instead of consuming the item, each values buyer plans to resell the item if she gets it. In this case (assuming the buyers will do a comparably good job of reselling it), the item has an unknown but common value regardless of who acquires it: it is equal to how much revenue this future reselling of the item will generate. Buyers’ estimates of this revenue may differ if they have some private information about the common value, and so their valuations of the item may differ. In this setting, the value each buyer assigns to the object would be affected by knowledge of the other buyers’ valuations, since the buyers could use this knowledge to further refine their estimates of the common value. 9.3 Relationships between Different Auction Formats Our main goal will be to consider how bidders behave in different types of auctions. We begin in this section with some simple, informal observations that relate behavior in interactive auctions (ascending-bid and descending-bid auctions, which play out in real time) with behavior in sealed-bid auctions. These observations can be made mathematically rigorous, but for the discussion here we will stick to an informal description. Descending-Bid and First-Price Auctions. First, consider a descending-bid auction. Here, as the seller is lowering the price from its high initial starting point, no bidder says anything until finally someone actually accepts the bid and pays the current price. Bidders therefore learn nothing while the auction is running, other than the fact that no one has yet accepted the current price. For each bidder i , there’s a first price b at which she’ll be i b . So with this view, the process willing to break the silence and accept the item at price i is equivalent to a sealed-bid first-price auction: this price b ’s bid; plays the role of bidder i i the item goes to the bidder with the highest bid value; and this bidder pays the value of her bid in exchange for the item.

267 9.3. RELATIONSHIPS BETWEEN DIFFERENT AUCTION FORMATS 253 Now let’s think about an ascending-bid Ascending-Bid and Second-Price Auctions. auction, in which bidders gradually drop out as the seller steadily raises the price. The winner of the auction is the last bidder remaining, and she pays the price at which the 1 second-to-last bidder drops out. Suppose that you’re a bidder in such an auction; let’s consider how long you should stay in the auction before dropping out. First, does it ever make sense to stay in the auction after the price reaches your true value? No: by staying in, you either lose and get nothing, or else you win and have to pay more than your value for the item. Second, does it ever make sense to drop out before the price reaches your true value for the item? Again, no: if you drop out early (before your true value is reached), then you get nothing, when by staying in you might win the item at a price below your true value. So this informal argument indicates that you should stay in an ascending-bid auction up to the exact moment at which the price reaches your true value. If we think of each bidder i ’s “drop-out price” as her bid b , this says that people should use their true values as their i bids. Moreover, with this definition of bids, the rule for determining the outcome of an ascending- bid auction can be reformulated as follows. The person with the highest bid is the one who stays in the longest, thus winning the item, and she pays the price at which the second-to- last person dropped out — in other words, she pays the bid of this second-to-last person. Thus, the item goes to the highest bidder at a price equal to the second-highest bid. This is precisely the rule used in the sealed-bid second-price auction, with the difference being that the ascending-bid auction involves real-time interaction between the buyers and seller, while the sealed-bid version takes place purely through sealed bids that the seller opens and evaluates. But the close similarity in rules helps to motivate the initially counter-intuitive pricing rule for the second-price auction: it can be viewed as a simulation, using sealed bids, of an ascending-bid auction. Moreover, the fact that bidders want to remain in an ascending-bid auction up to exactly the point at which their true value is reached provides the intuition for what will be our main result in the next section: after formulating the sealed-bid second-price auction in terms of game theory, we will find that bidding one’s true value is a dominant strategy. 1 It’s conceptually simplest to think of three things happening simultaneously at the end of an ascending- bid auction: (i) the second-to-last bidder drops out; (ii) the last remaining bidder sees that she is alone and stops agreeing to any higher prices; and (iii) the seller awards the item to this last remaining bidder at the current price. Of course, in practice we might well expect that there is some very small increment by which the bid is raised in each step, and that the last remaining bidder actually wins only after one more raising of the bid by this tiny increment. But keeping track of this small increment makes for a more cumbersome analysis without changing the underlying ideas, and so we will assume that the auction ends at precisely the moment when the second-highest bidder drops out.

268 254 CHAPTER 9. AUCTIONS In the next two sections we will consider the two main Comparing Auction Formats. formats for sealed-bid auctions in more detail. Before doing this, it’s worth making two points. First, the discussion in this section shows that when we analyze bidder behav- ior in sealed-bid auctions, we’re also learning about their interactive analogues — with the descending-bid auction as the analogue of the sealed-bid first-price auction, and the ascending-bid auction as the analogue of the sealed-bid second-price auction. Second, a purely superficial comparison of the first-price and second-price sealed-bid auctions might suggest that the seller would get more money for the item if he ran a first- price auction: after all, he’ll get paid the highest bid rather than the second-highest bid. It may seem strange that in a second-price auction, the seller is intentionally undercharging the bidders. But such reasoning ignores one of the main messages from our study of game theory — that when you make up rules to govern people’s behavior, you have to assume that they’ll adapt their behavior in light of the rules. Here, the point is that bidders in a first-price auction will tend to bid lower than they do in a second-price auction, and in fact this lowering of bids will tend to offset what would otherwise look like a difference in the size of the winning bid. This consideration will come up as a central issue at various points later in the chapter. 9.4 Second-Price Auctions The sealed-bid second-price auction is particularly interesting, and there are a number of examples of it in widespread use. The auction form used on eBay is essentially a second-price auction. The pricing mechanism that search engines use to sell keyword-based advertising is a generalization of the second-price auction, as we will see in Chapter 15. One of the most important results in auction theory is the fact we mentioned toward the end of the previous section: with independent, private values, bidding your true value is a dominant strategy in a second price sealed-bid auction. That is, the best choice of bid is exactly what the object is worth to you. To see why this is true, we Formulating the Second-Price Auction as a Game. set things up using the language of game theory, defining the auction in terms of players, strategies, and payoffs. The bidders will correspond to the players. Let v be bidder i ’s true i value for the object. Bidder i ’s strategy is an amount b to bid as a function of her true i value b . In a second-price sealed-bid auction, the payoff to bidder i with value v and bid v i i i is defined as follows. If b is the winning bid, and is not the winning bid, then the payoff to i is 0 . If b i i some other b b is the second-place bid, then the payoff to i . v − is i j j

269 9.4. SECOND-PRICE AUCTIONS 255 ' Alternate bid b i Raised bid af fects outcome only if is in between. highest other bid b j If so, i wins but pays more than value. ruthful bid b T = v i i Lowered bid af fects outcome only if highest other bid b is in between. k '' Alternate bid b If so, i loses when it was possible i f to win with non-negative payof i deviates from a truthful bid in a second-price auction, the payoff is Figure 9.1: If bidder only affected if the change in bid changes the win/loss outcome. To make this completely well-defined, we need to handle the possibility of ties: what do we do if two people submit the same bid, and it’s tied for the largest? One way to handle this is to assume that there is a fixed ordering on the bidders that is agreed on in advance, and if a set of bidders ties for the numerically largest bid, then the winning bid is the one submitted by the bidder in this set that comes first in this order. Our formulation of the payoffs works with this more refined definition of “winning bid” and “second-place bid.” (And note that in the case of a tie, the winning bidder receives the item but pays the full value of her own bid, for a payoff of zero, since in the event of a tie the first-place and second-place bids are equal.) There is one further point worth noting about our formulation of auctions in the language of game theory. When we defined games in Chapter 6, we assumed that each player knew the payoffs of all players in the game. Here this isn’t the case, since the bidders don’t know each other’s values, and so strictly speaking we need to use a slight generalization of the notions

270 256 CHAPTER 9. AUCTIONS from Chapter 6 to handle this lack of knowledge. For our analysis here, however, since we are focusing on dominant strategies in which a player has an optimal strategy regardless of the other players’ behavior, we will be able to disregard this subtlety. The precise statement of our claim about Truthful Bidding in Second-Price Auctions. second-price auctions is as follows. Claim: In a sealed-bid second-price auction, it is a dominant strategy for each to choose a bid b bidder v . i = i i bids b To prove this claim, we need to show that if bidder = v i , then no deviation from i i this bid would improve her payoff, regardless of what strategy everyone else is using. There are two cases to consider: deviations in which i raises her bid, and deviations in which i lowers her bid. The key point in both cases is that the value of i ’s bid only affects whether wins or loses, but never affects how much i pays in the event that she wins — the amount i paid is determined entirely by the other bids, and in particular by the largest among the other bids. Since all other bids remain the same when changes her bid, a change to i ’s bid i only affects her payoff if it changes her win/loss outcome. This argument is summarized in Figure 9.1. With this in mind, let’s consider the two cases. First, suppose that instead of bidding ′ , bidder i chooses a bid b ’s payoff if i > v v . This only affects bidder i would lose with bid i i i ′ . In order for this to happen, the highest other bid b v but would win with bid b must be j i i ′ ≤ b − v . In this case, the payoff to from deviating would be at most 0, i b between b and i j i i ′ does not improve i ’s payoff. and so this deviation to bid b i ′′ < v chooses a bid b v Next, suppose that instead of bidding i . This only affects , bidder i i i ′′ ’s payoff if i would win with bid v bidder but would lose with bid b i . So before deviating, i i ′′ ’s i . In this case, v was between v was the winning bid, and the second-place bid and b b i i k i v − payoff before deviating was ≥ 0, and after deviating it is 0 (since i loses), so again b i k this deviation does not improve i ’s payoff. This completes the argument that truthful bidding is a dominant strategy in a sealed- bid second-price auction. The heart of the argument is the fact noted at the outset: in a second-price auction, your bid determines whether you win or lose, but not how much you pay in the event that you win. Therefore, you need to evaluate changes to your bid in light of this. This also further highlights the parallels to the ascending-bid auction. There too, the analogue of your bid — i.e. the point up to which you’re willing to stay in the auction — determines whether you’ll stay in long enough to win; but the amount you pay in the event that you win is determined by the point at which the second-place bidder drops out. The fact that truthfulness is a dominant strategy also makes second-price auctions con- ceptually very clean. Because truthful bidding is a dominant strategy, it is the best thing to do regardless of what the other bidders are doing. So in a second-price auction, it makes

271 9.5. FIRST-PRICE AUCTIONS AND OTHER FORMATS 257 sense to bid your true value even if other bidders are overbidding, underbidding, colluding, or behaving in other unpredictable ways. In other words, truthful bidding is a good idea even if the competing bidders in the auction don’t know that they ought to be bidding truthfully as well. We now turn to first-price auctions, where we’ll find that the situation is much more complex. In particular, each bidder now has to reason about the behavior of her competitors in order to arrive at an optimal choice for her own bid. 9.5 First-Price Auctions and Other Formats In a sealed-bid first-price auction, the value of your bid not only affects whether you win but also how much you pay. As a result, most of the reasoning from the previous section has to be redone, and the conclusions are now different. To begin with, we can set up the first-price auction as a game in essentially the same way that we did for second-price auctions. As before, bidders are players, and each bidder’s strategy is an amount to bid as a function of her true value. The payoff to bidder with i v value and bid b is simply the following. i i If b is the winning bid, is not the winning bid, then the payoff to i is 0 . If b i i i b v . − then the payoff to is i i The first thing we notice is that bidding your true value is no longer a dominant strategy. By bidding your true value, you would get a payoff of 0 if you lose (as usual), and you would also get a payoff of 0 if you win, since you’d pay exactly what it was worth to you. As a result, the optimal way to bid in a first-price auction is to “shade” your bid slightly downward, so that if you win you will get a positive payoff. Determining how much to shade your bid involves balancing a trade-off between two opposing forces. If you bid too close to your true value, then your payoff won’t be very large in the event that you win. But if you bid too far below your true value, so as to increase your payoff in the event of winning, then you reduce your chance of being the high bid and hence your chance of winning at all. Finding the optimal trade-off between these two factors is a complex problem that de- pends on knowledge of the other bidders and their distribution of possible values. For example, it is intuitively natural that your bid should be higher — i.e. shaded less, closer to your true value — in a first-price auction with many competing bidders than in a first- price auction with only a few competing bidders (keeping other properties of the bidders the same). This is simply because with a large pool of other bidders, the highest competing bid is likely to be larger, and hence you need to bid higher to get above this and be the highest bid. We will discuss how to determine the optimal bid for a first-price auction in Section 9.7.

272 258 CHAPTER 9. AUCTIONS There are other sealed-bid auction formats that arise in different set- All-pay auctions. all-pay auction : tings. One that initially seems counter-intuitive in its formulation is the each bidder submits a bid; the highest bidder receives the item; and all bidders pay their bids, regardless of whether they win or lose. That is, the payoffs are now as follows. b is the winning bid, is not the winning bid, then the payoff to i is − b If . If b i i i i v − b . then the payoff to is i i Games with this type of payoff arise in a number of situations, usually where the notion of “bidding” is implicit. Political lobbying can be modeled in this way: each side must spend money on lobbying, but only the successful side receives anything of value for this expenditure. While it is not true that the side spending more on lobbying always wins, there is a clear analogy between the amount spent on lobbying and a bid, with all parties paying their bid regardless of whether they win or lose. One can picture similar considerations arising in settings such as design competitions, where competing architectural firms spend money on preliminary designs to try to win a contract from a client. This money must be spent before the client makes a decision. The determination of an optimal bid in an all-pay auction shares a number of qualitative features with the reasoning in a first-price auction: in general you want to bid below your true value, and you must balance the trade-off between bidding high (increasing your probability of winning) and bidding low (decreasing your expenditure if you lose and increasing your payoff if you win). In general, the fact that everyone must pay in this auction format means that bids will typically be shaded much lower than in a first-price auction. The framework we develop for determining optimal bids in first-price auctions will also apply to all-pay auctions, as we will see in Section 9.7. 9.6 Common Values and The Winner’s Curse Thus far, we have assumed that bidders’ values for the item being auctioned are independent: each bidder knows her own value for the item, and is not concerned with how much it is worth to anyone else. This makes sense in a lot of situations, but it clearly doesn’t apply to a setting in which the bidders intend to resell the object. In this case, there is a common eventual value for the object — the amount it will generate on resale — but it is not necessarily i may have some private information about the common value, leading known. Each bidder to an estimate v of this value. Individual bidder estimates will typically be slightly wrong, i and they will also typically not be independent of each other. One possible model for such estimates is to suppose that the true value is v , and that each bidder i ’s estimate v is defined i ’s = v + x v , where x is a random number with a mean of 0, representing the error in i by i i i estimate.

273 9.6. COMMON VALUES AND THE WINNER’S CURSE 259 Auctions with common values introduce new sources of complexity. To see this, let’s start by supposing that an item with a common value is sold using a second-price auction. i to bid v ? In fact, it’s not. To get a sense for why Is it still a dominant strategy for bidder i v + x . Suppose there are many bidders, this is, we can use the model with random errors i and that each bids her estimate of the true value. Then from the result of the auction, the winning bidder not only receives the object, she also learns something about her estimate of the common value — that it was the highest of all the estimates. So in particular, her estimate is more likely to be an over-estimate of the common value than an under-estimate. Moreover, with many bidders, the second-place bid — which is what she paid — is also likely to be an over-estimate. As a result she will likely lose money on the resale relative to what she paid. This is known as the winner’s curse , and it is a phenomenon that has a rich history in the study of auctions. Richard Thaler’s review of this history [387] notes that the winner’s curse appears to have been first articulated by researchers in the petroleum industry [95]. In this domain, firms bid on oil-drilling rights for tracts of land that have a common value, equal to the value of the oil contained in the tract. The winner’s curse has also been studied in the context of competitive contract offers to baseball free agents [98] — with the unknown 2 common value corresponding to the future performance of the baseball player being courted. Rational bidders should take the winner’s curse into account in deciding on their bids: a bidder should bid her best estimate of the value of the object conditional on both her private v and on winning the object at her bid. That is, it must be the case that at an estimate i optimal bid, it is better to win the object than not to win it. This means in a common-value auction, bidders will shade their bids downward even when the second-price format is used; with the first-price format, bids will be reduced even further. Determining the optimal bid is fairly complex, and we will not pursue the details of it here. It is also worth noting that in practice, the winner’s curse can lead to outright losses on the part of the winning bidder [387], since in a large pool of bidders, anyone who in fact makes an error and overbids is more likely to be the winner of the auction. 2 In these cases as well as others, one could argue that the model of common values is not entirely accurate. One oil company could in principle be more successful than another at extracting oil from a tract of land; and a baseball free agent may flourish if he joins one team but fail if he joins another. But common values are a reasonable approximation to both settings, as to any case where the purpose of bidding is to obtain an item that has some intrinsic but unknown future value. Moreover, the reasoning behind the winner’s curse arises even when the item being auctioned has related but non-identical values to the different bidders.

274 260 CHAPTER 9. AUCTIONS 9.7 Advanced Material: Bidding Strategies in First- Price and All-Pay Auctions In the previous two sections we offered some intuition about the way to bid in first-price auctions and in all-pay auctions, but we did not derive optimal bids. We now develop models of bidder behavior under which we can derive equilibrium bidding strategies in these auctions. We then explore how optimal behavior varies depending on the number of bidders and on the distribution of values. Finally, we analyze how much revenue the seller can expect to obtain from various auctions. The analysis in this section will use elementary calculus and probability theory. A. Equilibrium Bidding in First-Price Auctions As the basis for the model, we want to capture a setting in which bidders know how many competitors they have, and they have partial information about their competitors’ values for the item. However, they do not know their competitors’ values exactly. Let’s start with a simple case first, and then move on to a more general formulation. In the simple case, suppose that there are two bidders, each with a private value that is 3 independently and uniformly distributed between 0 and 1. This information is common strategy knowledge among the two bidders. A s ( v ) = b that maps for a bidder is a function her true value to a non-negative bid b . We will make the following simple assumptions v about the strategies the bidders are using: s ( · ) is a strictly increasing, differentiable function; so in particular, if two bidders have (i) different values, then they will produce different bids. (ii) s ( v ) ≤ v for all v : bidders can shade their bids down, but they will never bid above their true values. Notice that since bids are always non-negative, this also means that s (0) = 0. These two assumptions permit a wide range of strategies. For example, the strategy of bidding your true value is represented by the function s v ) = v , while the strategy of shading ( your bid downward to by a factor of c < 1 times your true value is represented by s ( v ) = cv . 2 are also allowed, although we will see that in ( v ) = v More complex strategies such as s first-price auctions they are not optimal. The two assumptions help us narrow the search for equilibrium strategies. The second of our assumptions only rules out strategies (based on overbidding) that are non-optimal. 3 The fact that the 0 and 1 are the lowest and highest possible values is not crucial; by shifting and re-scaling these quantities, we could equally well consider values that are uniformly distributed between any other pair of endpoints.

275 9.7. ADVANCED MATERIAL: BIDDING STRATEGIES IN FIRST-PRICE AND ALL-PAY AUCTIONS 261 The first assumption restricts the scope of possible equilibrium strategies, but it makes the analysis easier while still allowing us to study the important issues. Finally, since the two bidders are identical in all ways except the actual value they draw from the distribution, we will narrow the search for equilibria in one further way: we will s ). ( consider the case in which the two bidders follow the same strategy · Let’s consider what such Equilibrium with two bidders: The Revelation Principle. an equilibrium strategy should look like. First, assumption (i) says that the bidder with the higher value will also produce the higher bid. If bidder has a value of v , the probability that i i ’s competitor in the interval [0 , 1] is exactly v . Therefore, this is higher than the value of i i − will win the auction with probability . If i does win, i receives a payoff of v ). v s ( v i i i i Putting all this together, we see that i ’s expected payoff is ( g (9.1) ) = v . ( v )) − s ( v v i i i i s · ) to be an equilibrium strategy? It means that for each Now, what does it mean for ( i , there is no incentive for i to deviate from strategy s ( · ) if i ’s competitor is also using bidder s strategy · ). It’s not immediately clear how to analyze deviations to an arbitrary strategy ( satisfying assumptions (i) and (ii) above. Fortunately, there is an elegant device that lets us reason about deviations as follows: rather then actually switching to a different strategy, bidder can implement her deviation by keeping the strategy s ( · ) but supplying a different i “true value” to it. Here is how this works. First, if i ’s competitor is also using strategy s ( · ), then i should never announce a bid above s i can win with bid s (1) and get a higher payoff (1), since s (1). So in any possible deviation by i , the bid she will b > s (1) than with any bid with bid (0) = 0 and s (1). Therefore, for the purposes of the auction, actually report will lie between s she can simulate her deviation to an alternate strategy by first pretending that her true value ′ ′ v instead of . This is · , and then applying the existing function s ( rather than ) to v v v is i i i i a special case of a much broader idea known as the [124, 207, 310]; for Revelation Principle our purposes, we can think of it as saying that deviations in the bidding strategy function i can instead be viewed as deviations in the “true value” that bidder supplies to her current strategy s ( · ). With this in mind, we can write the condition that i does not want to deviate from strategy ( · ) as follows: s v (9.2) ( v )) − s ( v v )) ≥ v ( v ( − s i i i i for all possible alternate “true values” v between 0 and 1 that bidder i might want to supply to the function s ( · ). Is there a function that satisfies this property? In fact, it is not hard to check that ), the left-hand ( v ) = s 2 satisfies it. To see why, notice that with this choice of s ( · v/

276 262 CHAPTER 9. AUCTIONS 2 v v − v 2) = / side of Inequality (9.2) becomes v ( 2 while the right-hand side becomes / i i i i 2 − − v/ 2) = vv v ( v v / 2. Collecting all the terms on the left, the inequality becomes simply i i 1 2 2 − 2 vv ( + v v , ) ≥ 0 i i 2 1 2 ( v − v . ) which holds because the left-hand side is the square i 2 Thus, the conclusion in this case is quite simple to state. If two bidders know they are competing against each other, and know that each has a private value drawn uniformly at , 1], then it is an equilibrium for each to shade their bid down by random from the interval [0 a factor of 2. Bidding half your true value is optimal behavior if the other bidder is doing this as well. Note that unlike the case of the second-price auction, we have not identified a dominant strategy, only an equilibrium. In solving for a bidder’s optimal strategy we used each bidder’s expectation about her competitor’s bidding strategy. In an equilibrium, these expectations are correct. But if other bidders for some reason use non-equilibrium strategies, then any bidder should optimally respond and potentially also play some other bidding strategy. In our discussion of the equilibrium s ( v ) = v/ 2, Deriving the two-bidder equilibrium. s ( · ), and then checked that it satisfied we initially conjectured the form of the function s Inequality (9.2). But this approach does not suggest how to discover a function ) to use ( · as a conjecture. An alternate approach is to derive ( · ) directly by reasoning about the condition in s s ( · ) to satisfy Inequality (9.2), Inequality (9.2). Here is how we can do this. In order for v it must have the property that for any true value , the expected payoff function g ( v ) = i ′ ′ ( − s ( v )) is maximized by setting v = v v . Therefore, v g should satisfy g v ( v ) = 0, where i i i i g ( · ) with respect to v . Since is the first derivative of ′ ′ ( v ) = v g − s ( v ) − vs ) ( v i by the Product Rule for derivatives, we see that ( · ) must solve the differential equation s ( ) v s i ′ − ( s v ) = 1 i v i for all v 2. in the interval [0 , 1]. This differential equation is solved by the function s ( v / ) = v i i i Equilibrium with Many Bidders. Now let’s suppose that there are n bidders, where n can be larger than two. To start with, we’ll continue to assume that each bidder i draws her true value v independently and uniformly at random from the interval between 0 and 1. i Much of the reasoning for the case of two bidders still works here, although the basic formula for the expected payoff changes. Specifically, assumption (i) still implies that the

277 9.7. ADVANCED MATERIAL: BIDDING STRATEGIES IN FIRST-PRICE AND ALL-PAY AUCTIONS 263 bidder with the highest true value will produce the highest bid and hence win the auction. For a given bidder v i , what is the probability that her bid is the highest? This with true value i v requires each other bidder to have a value below ; since the values are chosen independently, i n − 1 v . Therefore, bidder i ’s expected payoff is this event has a probability of i − n 1 ) = v v ( G ( v (9.3) − s ( v . )) i i i i The condition for s ( · ) to be an equilibrium strategy remains the same as it was in the case of two bidders. Using the Revelation Principle, we view a deviation from the bidding v to the function strategy as supplying a “fake” value ( · ); given this, we require that the s true value v produces an expected payoff at least as high as the payoff from any deviation: i − 1 n n − 1 v − s ( v v )) ≥ v ( (9.4) ( v v )) − s ( i i i i for all between 0 and 1. v s ( · From this, we can derive the form of the bidding function ) using the differential- equation approach that worked for two bidders. The expected payoff function G ( v ) = ′ n − 1 ( v − s ( v )) must be maximized by setting v = v ) = 0 . Setting the derivative G v ( v i i i , we get and applying the Product Rule to differentiate G 1 − n n − 2 2 ′ − n − ( − 1) v − ) = 0 v s ( v v ) n v 1) − n ( s ( v i i i i ′ 2 − n for all n between 0 and 1. Dividing through by ( v and solving for s v ( v 1) ), we get − i i the equivalent but typographically simpler equation ) ( ) v ( s i ′ (9.5) 1 − ( n ) = ( 1) v s − i v i for all v between 0 and 1. This differential equation is solved by the function i ) ( − n 1 v . ) = v ( s i i n So if each bidder shades her bid down by a factor of ( n − 1) /n , then this is optimal behavior given what everyone else is doing. Notice that when = 2 this is our two-bidder n strategy. The form of this strategy highlights an important principle that we discussed in Section 9.5 about strategic bidding in first-price auctions: as the number of bidders increases, you generally have to bid more “aggressively,” shading your bid down less, in order to win. For the simple case of values drawn independently from the uniform distribution, our analysis here quantifies exactly how this increased aggressiveness should depend on the number of bidders n .

278 264 CHAPTER 9. AUCTIONS In addition to considering larger numbers of bidders, we can General Distributions. also relax the assumption that bidders’ values are drawn from the uniform distribution on an interval. Suppose that each bidder has her value drawn from a probability distribution over the cumulative non-negative real numbers. We can represent the probability distribution by its ( · ): for any x , the value F ( x ) is the probability that a number drawn distribution function F . We will assume that from the distribution is at most is a differentiable function. x F Most of the earlier analysis continues to hold at a general level. The probability that a i bidder v wins the auction is the probability that no other bidder has a with true value i − n 1 ) ( F larger value, so it is equal to . Therefore, the expected payoff to v is v i i n − 1 v ( . ) ( F )) − s ( v v i i i i does not want to deviate from this strategy becomes Then, the requirement that bidder − 1 n − 1 n F ( ( v )) − s ( v v )) ≥ F ( v ) v (9.6) ) ( v ( − s i i i i between 0 and 1. v for all Finally, this equilibrium condition can be used to write a differential equation just as before, using the fact that the function of v on the right-hand side of Inequality (9.6) should = v be maximized when . We apply the Product Rule, and also the Chain Rule for v i F ( · ) derivatives, keeping in mind that the derivative of the cumulative distribution function f ( · ) for the distribution. Proceeding by analogy with the is the probability density function analysis for the uniform distribution, we get the differential equation ( ) ( s ) v ( ) ( v ) v v − f f i i i i ′ 1) ) = ( s v ( − n . (9.7) i ) ( F v i Notice that for the uniform distribution on the interval [0 , 1], the cumulative distribution function is F ( v ) = v and the density is f ( v ) = 1, which applied to Equation (9.7) gives us back Equation (9.5). Finding an explicit solution to Equation (9.7) isn’t possible unless we have an explicit form for the distribution of values, but it provides a framework for taking arbitrary distributions and solving for equilibrium bidding strategies. B. Seller Revenue Now that we’ve analyzed bidding strategies for first-price auctions, we can return to an issue that came up at the end of Section 9.3: how to compare the revenue a seller should expect to make in first-price and second-price auctions. There are two competing forces at work here. On the one hand, in a second-price auction, the seller explicitly commits to collecting less money, since he only charges the second-highest

279 9.7. ADVANCED MATERIAL: BIDDING STRATEGIES IN FIRST-PRICE AND ALL-PAY AUCTIONS 265 bid. On the other hand, in a first-price auction, the bidders reduce their bids, which also reduces what the seller can collect. To understand how these opposing factors trade off against each other, suppose we have bidders with values drawn independently from the uniform distribution on the interval n , 1]. Since the seller’s revenue will be based on the values of the highest and second-highest [0 bids, which in turn depend on the highest and second-highest values, we need to know the 4 expectations of these quantities. Computing these expectations is complicated, but the form of the answer is very simple. Here is the basic statement: numbers are drawn independently from the uniform distribution on the Suppose n [0 , 1] and then sorted from smallest to largest. The expected value of the interval k th . k number in the position on this list is + 1 n Now, if the seller runs a second-price auction, and the bidders follow their dominant strategies and bid truthfully, the seller’s expected revenue will be the expectation of the second-highest value. Since this will be the value in position n − 1 in the sorted order of n the n − 1) / ( n + 1), by the random values from smallest to largest, the expected value is ( formula just described. On the other hand, if the seller runs a first-price auction, then in equilibrium we expect the winning bidder to submit a bid that is ( n − 1) /n times her true value. Her true value has an expectation of + 1) (since it is the largest of n numbers n n/ ( drawn independently from the unit interval), and so the seller’s expected revenue is ( )( ) n − 1 1 n − n = . + 1 n n + 1 n The two auctions provide exactly the same expected revenue to the seller! As far as seller revenue is concerned, this calculation is in a sense Revenue Equivalence. the tip of the iceberg: it is a reflection of a much broader and deeper principle known in the auction literature as revenue equivalence [256, 288, 311]. Roughly speaking, revenue equivalence asserts that a seller’s revenue will be the same across a broad class of auctions and arbitrary independent distributions of bidder values, when bidders follow equilibrium strategies. A formalization and proof of the revenue equivalence principle can be found in [256]. From the discussion here, it is easy to see how the ability to commit to a selling mechanism is valuable for a seller. Consider, for example, a seller using a second-price auction. If the bidders bid truthfully and the seller does not sell the object as promised, then the seller knows the bidders’ values and can bargain with them from this advantaged position. At worst, the seller should be able to sell the object to the bidder with the highest value at a 4 In the language of probability theory, these are known as the expectations of the order statistics .

280 266 CHAPTER 9. AUCTIONS price equal to the second highest value. (The bidder with the highest value knows that if she turns down the trade at this price, then the bidder with the second-highest value will take it.) But the seller may be able to do better than this in the negotiation, and so overall the bidders lose relative to the originally promised second-price auction. If bidders suspect that this scenario may occur with some probability, then they may no longer find it optimal to bid truthfully in the auction, and so it is not clear what the seller receives. In our discussion of how a seller should choose an auction format, we Reserve Prices. have implicitly assumed that the seller must sell the object. Let’s briefly consider how the seller’s expected revenue changes if he has the option of holding onto the item and choosing not to sell it. To be able to reason about the seller’s payoff in the event that this happens, ≥ 0, which is thus the payoff he gets from u let’s assume that the seller values the item at keeping the item rather than selling it. It’s clear that if 0, then the seller should not use a simple first-price or second-price u > auction. In either case, the winning bid might be less than , and the seller would not want u to sell the object. If the seller refuses to sell after having specified a first-price or second-price auction, then we are back in the case of a seller who might break his initial commitment to the format. reserve price of Instead, it is better for the seller to announce a before running the r auction. With a reserve price, the item is sold to the highest bidder if the highest bid is above r ; otherwise, the item is not sold. In a first-price auction with a reserve price, the winning bidder (if there is one) still pays her bid. In a second-price auction with a reserve price, the winning bidder (if there is one) pays the maximum of the second-place bid and r the reserve price . As we will see, it is in fact useful for the seller to declare a reserve price even if his value for the item is u = 0. Let’s consider how to reason about the optimal value for the reserve price in the case of a second-price auction. First, it is not hard to go back over the argument that truthful bidding is a dominant strategy in second-price auctions and check that it still holds in the presence of a reserve price. Essentially, it is as if the seller were another “simulated” bidder who always bids r ; and since truthful bidding is optimal regardless of how other bidders behave, the presence of this additional simulated bidder has no effect on how any of the real bidders should behave. Now, what value should the seller choose for the reserve price? If the item is worth u to the seller, then clearly he should set ≥ u . But in fact the reserve price that maximizes the r seller’s expected revenue is strictly greater than u . To see why this is true, let’s first consider a very simple case: a second-price auction with a single bidder, whose value is uniformly distributed on [0 , 1], and a seller whose value for the item is u = 0. With only one bidder, the second-price auction with no reserve price will sell the item to the bidder at a price of

281 9.7. ADVANCED MATERIAL: BIDDING STRATEGIES IN FIRST-PRICE AND ALL-PAY AUCTIONS 267 r > 0. In this case, with 0. On the other hand, suppose the seller sets a reserve price of r − , and the object will be sold to the bidder at , the bidder’s value is above probability 1 r r r , and so the seller keeps the . With probability , the bidder’s value is below r a price of item, receiving a payoff of (1 − r ), and u = 0. Therefore, the seller’s expected revenue is r = 1 / 2. If the seller’s value u this is maximized at r is greater than zero, then his expected r − r ) + ru (since he receives a payoff of u when the item is not sold), and this payoff is (1 r = (1 + ) / 2. So with a single bidder, the optimal reserve price is maximized by setting u is halfway between the value of the object to the seller and the maximum possible bidder value. With more intricate analyses, one can similarly determine the optimal reserve price for a second-price auction with multiple bidders, as well as for a first-price auction with equilibrium bidding strategies of the form we derived earlier. C. Equilibrium Bidding in All-Pay Auctions The style of analysis we’ve been using for first-price auctions can be adapted without much difficulty to other formats as well. Here we will show how this works for the analysis of all-pay auctions: recall from Section 9.5 that this is an auction format — designed to model activities such as lobbying — where the highest bidder wins the item but everyone pays their bid. We will keep the general framework we used for first-price auctions earlier in this section, bidders, each with a value drawn independently and uniformly at random from with n s ( · ) mapping values to bids, so that between 0 and 1. As before, we want to find a function ( using ) is optimal if all other bidders are using it. s · i has a negative term if i does With an all-pay auction, the expected payoff for bidder not win. The formula is now n − 1 1 − n ( v ( v )) + (1 − v v s − , )( − s ( v )) i i i i i where the first term corresponds to the payoff in the event that i wins, and the second term corresponds to the payoff in the event that i loses. As before, we can think of a deviation s to the function ( · from this bidding strategy as supplying a fake value s ( · ) is an v ); so if equilibrium choice of strategies by the bidders, then − n − 1 1 n 1 1 n n − − v )) + (1 − v v v (9.8) ( )( − s ( v s )) ≥ v ( v )) ( v − − s ( − )) + (1 − v s ( v )( i i i i i i for all in the interval [0 , 1]. v s ( v ) that is paid regardless of the Notice that the expected payoff consists of a fixed cost v in the event that i wins. Canceling the common terms win/loss outcome, plus a value of i in Inequality (9.8), we can rewrite it as n 1 − n v v ( . ) ≥ v s − ) v v − s ( (9.9) i i i

282 268 CHAPTER 9. AUCTIONS n − 1 v ( v ) = v in the interval [0 1]. Now, writing the right-hand side as a function g v for all − , i maximizes the function ), we can view Inequality (9.9) as requiring that v = v s v g ( · ). The ( i ′ ) = 0 then gives us a differential equation that specifies ( v ) quite resulting equation g ( · s i simply: n − 1 ′ n ( 1) v v s ) = ( − , i i ) ( − n 1 n v ) = v ( s and hence i n th v )) reduces < 1, raising it to the n ( power (as specified by the function s Since · i it exponentially in the number of bidders. This shows that bidders will shade their bids downward significantly as the number of bidders in an all-pay auction increases. We can also work out the seller’s expected revenue. The seller collects money from everyone in an all-pay auction; on the other hand, the bidders all submit low bids. The expected value of a single bidder’s contribution to seller revenue is simply ( ) )( ( ) ∫ ∫ 1 1 n − 1 1 1 − n n ) v ( dv s = dv = . v + 1 n n n 0 0 Since the seller collects this much in expectation from each bidder, the seller’s overall ex- pected revenue is ( ) )( 1 − n 1 1 − n . n = n + 1 n n + 1 This is exactly the same as the seller’s expected revenue in the first-price and second-price auctions with the same assumptions about bidder values. Again, this is a reflection of the much broader revenue equivalence principle [256, 288], which includes all-pay auctions in the general set of auction formats it covers. 9.8 Exercises 1. In this question we will consider an auction in which there is one seller who wants to sell one unit of a good and a group of bidders who are each interested in purchasing the good. The seller will run a sealed-bid, second-price auction. Your firm will bid in the auction, but it does not know for sure how many other bidders will participate in the auction. There will be either two or three other bidders in addition to your firm. All bidders have independent, private values for the good. Your firm’s value for the good is c . What bid should your firm submit, and how does it depend on the number of other bidders who show up? Give a brief (1-3 sentence) explanation for your answer. 2. In this problem we will ask how the number of bidders in a second-price, sealed-bid auction affects how much the seller can expect to receive for his object. Assume that there are two bidders who have independent, private values v which are either 1 or 3. i

283 9.8. EXERCISES 269 / For each bidder, the probabilities of 1 and 3 are both 1 2. (If there is a tie at a bid of for the highest bid the winner is selected at random from among the highest bidders x x and the price is .) / Show that the seller’s expected revenue is 6 (a) 4. (b) Now let’s suppose that there are three bidders who have independent, private values which are either 1 or 3. For each bidder, the probabilities of 1 and 3 are both v i / 2. What is the seller’s expected revenue in this case? 1 (c) Briefly explain why changing the number of bidders affects the seller’s expected revenue. 3. In this problem we will ask how much a seller can expect to receive for his object in a second-price, sealed-bid auction. Assume that all bidders have independent, private v / which are either 0 or 1. The probability of 0 and 1 are both 1 values 2. i Suppose there are two bidders. Then there are four possible pairs of their values (a) v 4. Show ,v / ): (0 , 0) , (1 , 0) , ( , 1) , and (1 , 1). Each pair of values has probability 1 (0 2 1 that the seller’s expected revenue is 1 / 4. (Assume that if there is a tie at a bid of x for the highest bid the winner is selected at random from among the highest bidders and the price is x .) (b) What is the seller’s expected revenue if there are three bidders? This suggests a conjecture that as the number of bidders increases the seller’s (c) expected revenue also increases. In the example we are considering the seller’s expected revenue actually converges to 1 as the number of bidders grows. Explain why this should occur. You do not need to write a proof; an intuitive explanation is fine. 4. A seller will run a second-price, sealed-bid auction for an object. There are two bidders, a , who have independent, private values v and which are either 0 or 1. For both b i v bidders the probabilities of = 0 and v 2. Both bidders understand = 1 are each 1 / i i the auction, but bidder b sometimes makes a mistake about his value for the object. Half of the time his value is 1 and he is aware that it is 1; the other half of the time his value is 0 but occasionally he mistakenly believes that his value is 1. Let’s suppose 1 b ’s value is 0 he acts as if it is 1 with probability that when and as if it is 0 with 2 1 1 . So in effect bidder b sees value 0 with probability probability and value 1 with 2 4 3 probability . Bidder a never makes mistakes about his value for the object, but he 4 is aware of the mistakes that bidder b makes. Both bidders bid optimally given their perceptions of the value of the object. Assume that if there is a tie at a bid of x for the highest bid the winner is selected at random from among the highest bidders and the price is x .

284 270 CHAPTER 9. AUCTIONS a ? Explain briefly (a) Is bidding his true value still a dominant strategy for bidder (b) What is the seller’s expected revenue? Explain briefly. 5. Consider a second-price, sealed-bid auction with one seller who has one unit of the object which he values at , 2 who have values of v s and v and two buyers 1 for the 2 1 object. The values are all independent, private values. Suppose that both ,v s,v 2 1 s , but they do not know buyers know that the seller will submit his own sealed bid of the value of . Is it optimal for the buyers to bid truthfully; that is should they each s bid their true value? Give an explanation for your answer. 6. In this question we will consider the effect of collusion between bidders in a second- price, sealed-bid auction. There is one seller who will sell one object using a second- price sealed-bid auction. The bidders have independent, private values drawn from a distribution on [0 , 1]. If a bidder with value v gets the object at price p , his payoff is v − ; if a bidder does not get the object his payoff is 0. We will consider the possibility p of collusion between two bidders who know each others’ value for the object. Suppose that the objective of these two colluding bidders is to choose their two bids as to maximize the sum of their payoffs. The bidders can submit any bids they like as long as the bids are in [0 , 1]. Let’s first consider the case in which there are only two bidders. What two bids (a) should they submit? Explain. (b) Now suppose that there is a third bidder who is not part of the collusion. Does the existence of this bidder change the optimal bids for the two bidders who are colluding? Explain. 7. A seller announces that he will sell a case of rare wine using a sealed-bid, second-price auction. A group of I individuals plan to bid on this case of wine. Each bidder is interested in the wine for his or her personal consumption; the bidders’ consumption values for the wine may differ, but they don’t plan to resell the wine. So we will view their values for the wine as independent, private values (as in Chapter 9). You are one of these bidders; in particular, you are bidder number i and your value for the wine is v . i How should you bid in each of the following situations? In each case, provide an explanation for your answer; a formal proof is not necessary. (a) You know that a group of the bidders will collude on bids. This group will chose one bidder to submit a “real bid” of v and the others will all submit bids of 0. You are not a member of this collusive group and you cannot collude with any other bidder.

285 9.8. EXERCISES 271 You, and all of the other bidders, have just learned that this seller will collect (b) bids, but won’t actually sell the wine according to the rules of a second-price auction. Instead, after collecting the bids the seller will tell all of the bidders that some other fictional bidder actually submitted the highest bid and so won the auction. This bidder, of course, doesn’t exist so the seller will still have the wine after the auction is over. The seller plans to privately contact the highest actual bidder and tell him or her that the fictional high bidder defaulted (he didn’t buy the wine after all) and that this bidder can buy the wine for the price he or she bid in the auction. You cannot collude with any bidder. [You do not need to derive an optimal bidding strategy. It is enough to explain whether your bid would differ from your value and if so in what direction.] 8. In this problem we will ask how irrational behavior on the part of one bidder affects optimal behavior for the other bidders in an auction. In this auction the seller has one unit of the good which will be sold using a second-price, sealed-bid auction. Assume v v , that there are three bidders who have independent, private values for the good, 1 2 , 1]. v , which are uniformly distributed on the interval [0 3 (a) Suppose first that all bidders behave rationally; that is they submit optimal bids. Which bidder (in terms of values) wins the auction and how much does this bidder pay (again in terms of the bidder’s values)? Suppose now that bidder 3 irrationally bids more than his true value for the (b) object; in particular, bidder 3’s bid is ( v + 1) / 2. All other bidders know that bidder 3 3 is irrational in this way, although they do not know bidder 3’s actual value for the object. How does this affect the behavior of the other bidders? (c) What effect does bidder 3’s irrational behavior have on the expected payoffs of v which bidder 1 does and v bidder 1? Here the expectation is over the values of 3 2 not know. You do not need to provide an explicit solution or write a proof for your answer; an intuitive explanation of the effect is fine. [Remember a bidder’s payoff is the bidder’s value for the object minus the price, if the bidder wins the auction; or 0, if the bidder does not win the auction.] 9. In this problem we will ask how much a seller can expect to receive for his object in a second-price, sealed-bid auction. Assume that there are two bidders who have independent, private values v which are either 1 or 2. For each bidder, the probabilities i 2. Assume that if there is a tie at a bid of v for the = 1 and v x = 2 are each 1 / of i i highest bid the winner is selected at random from among the highest bidders and the price is x . We also assume that the value of the object to the seller is 0. (a) Show that the seller’s expected revenue is 5 / 4.

286 272 CHAPTER 9. AUCTIONS R with 1 2: that (b) Now let’s suppose that the seller sets a reserve price of < R < , and the price R is, the object is sold to the highest bidder if her bid is at least R . If no bid is this bidder pays is the maximum of the second highest bid and , then the object is not sold, and the seller receives 0 revenue. Suppose at least R . What is the seller’s expected revenue as a function of R that all bidders know R ? (c) Using the previous part, show that a seller who wants to maximize expected R revenue would never set a reserve price, . 5. , that is more than 1 and less than 1 10. In this problem we will examine a second-price, sealed-bid auction. Assume that there are two bidders who have independent, private values which are either 1 or 7. For v i v 2. So there are four each bidder, the probabilities of v / = 7 are each 1 = 1 and i i possible pairs of the bidders’ values ( v 7). Each pair ,v , ): (1 , 1) , (1 , 7) , (7 , 1) , and (7 2 1 / of values has probability 1 4. Assume that if there is a tie at a bid of x for the highest bid the winner is selected at x random from among the highest bidders and the price is . (a) For each pair of values, what bid will each bidder submit, what price will the winning bidder pay, and how much profit (the difference between the winning bidder’s value and price he pays) will the winning bidder earn? (b) Now let’s examine how much revenue the seller can expect to earn and how much profit the bidders can expect to make in the second price auction. Both revenue and profit depend on the values, so let’s calculate the average of each of these numbers across all four of the possible pairs of values. [Note that in doing this we are computing each bidder’s expected profit before the bidder knows his value for the object.] What is the seller’s expected revenue in the second price auction? What is the expected profit for each bidder? (c) The seller now decides to charge an entry fee of 1. Any bidder who wants to participate in the auction must pay this fee to the seller before bidding begins and, in fact, this fee is imposed before each bidder knows his or her own value for the object. The bidders know only the distribution of values and that anyone who pays the fee will be allowed to participate in a second price auction for the object. This adds a new first stage to the game in which bidders decide simultaneously whether to pay the fee and enter the auction, or to not pay the fee and stay out of the auction. This first stage is then followed by a second stage in which anyone who pays the fee participates in the auction. We will assume that after the first stage is over both potential bidders learn their own value for the object (but not the other potential bidder’s value for the

287 9.8. EXERCISES 273 object) and that they both learn whether or not the other potential bidder decided to enter the auction. Let’s assume that any potential bidder who does not participate in the auction has a profit of 0, if no one chooses to participate then the seller keeps the object and does not run an auction, if only one bidder chooses to participate in the auction then the seller runs a second price auction with only this one bidder (and treats the second highest bid as 0), and finally if both bidders participate the second price auction is the one you solved in part (a). Is there an equilibrium in which each bidder pays the fee and participates in the auction? Give an explanation for your answer. 11. In this question we will examine a second-price, sealed-bid auction for a single item. We’ll consider a case in which true values for the item may differ across bidders, and it requires extensive research by a bidder to determine her own true value for an item — maybe this is because the bidder needs to determine her ability to extract value from the item after purchasing it (and this ability may differ from bidder to bidder). There are three bidders. Bidders 1 and 2 have values v and v , each of which is a ran- 2 1 dom number independently and uniformly distributed on the interval [0 , 1]. Through having performed the requisite level of research, bidders 1 and 2 know their own values for the item, v , respectively, but they do not know each other’s value for item. and v 2 1 Bidder 3 has not performed enough research to know his own true value for the item. He does know that he and bidder 2 are extremely similar, and therefore that his true value v is exactly equal to the true value v of bidder 2. The problem is that bidder 2 3 3 does not know this value v ). (nor does he know v 1 2 (a) How should bidder 1 bid in this auction? How should bidder 2 bid? (b) How should bidder 3 behave in this auction? Provide an explanation for your answer; a formal proof is not necessary.

288 274 CHAPTER 9. AUCTIONS

289 Part III Markets and Strategic Interaction in Networks 275

290

291 Chapter 10 Matching Markets We have now seen a number of ways of thinking both about network structure and about the behavior of agents as they interact with each other. A few of our examples have brought these together directly — such as the issue of traffic in a network, including Braess’s Paradox — and in the next few chapters we explore this convergence of network structure and strategic interaction more fully, and in a range of different settings. First, we think about markets as a prime example of network-structured interaction between many agents. When we consider markets creating opportunities for interaction among buyers and sellers, there is an implicit network encoding the access between these buyers and sellers. In fact, there are a number of ways of using networks to model interactions among market participants, and we will discuss several of these models. Later, in Chapter 12 on network exchange theory , we will discuss how market-style interactions become a metaphor for the broad notion of social exchange , in which the social dynamics within a group can be modeled by the power imbalances of the interactions within the group’s social network. 10.1 Bipartite Graphs and Perfect Matchings Matching markets form the first class of models we consider, as the focus of the current chapter. Matching markets have a long history of study in economics, operations research, and other areas because they embody, in a very clean and stylized way, a number of basic principles: the way in which people may have different preferences for different kinds of goods, the way in which prices can decentralize the allocation of goods to people, and the way in which such prices can in fact lead to allocations that are socially optimal. We will introduce these various ingredients gradually, by progressing through a succession of increasingly rich models. We begin with a setting in which goods will be allocated to people D. Easley and J. Kleinberg. Networks, Crowds, and Markets: Reasoning about a Highly Connected World . Cambridge University Press, 2010. Draft version: June 10, 2010. 277

292 278 CHAPTER 10. MATCHING MARKETS V Room1 ikram V ikram Room1 endy W endy Room2 W Room2 Xin Room3 Room3 Xin oram Y Room4 oram Room4 Y Room5 Zoe Room5 Zoe Bipartite Graph (b) (a) A Perfect Matching Figure 10.1: (a) An example of a bipartite graph. (b) A perfect matching in this graph, indicated via the dark edges. based on preferences, and these preferences will be expressed in network form, but there is no explicit buying, selling, or price-setting. This first setting will also be a crucial component of the more complex ones that follow. Bipartite Graphs. The model we start with is called the bipartite matching problem , and we can motivate it via the following scenario. Suppose that the administrators of a college dormitory are assigning rooms to returning students for a new academic year; each room is designed for a single student, and each student is asked to list several acceptable options for the room they’d like to get. Students can have different preferences over rooms; some people might want larger rooms, quieter rooms, sunnier rooms, and so forth — and so the lists provided by the students may overlap in complex ways. We can model the lists provided by the students using a graph, as follows. There is a node for each student, a node for each room, and an edge connecting a student to a room if the student has listed the room as an acceptable option. Figure 10.1(a) shows an example with five students and five rooms (indicating, for instance, that the student named Vikram has listed each of Rooms 1, 2, and 3 as acceptable options, while the student named Wendy only listed Room 1). This type of graph is bipartite , an important property that we saw earlier, in a different

293 10.1. BIPARTITE GRAPHS AND PERFECT MATCHINGS 279 context, in talking about affiliation networks in Chapter 4. In a bipartite graph the nodes are divided into two categories, and each edge connects a node in one category to a node in the other category. In this case, the two categories are students and rooms. Just as bipartite graphs were useful in Chapter 4 to represent the participation of people in different activities, here they are useful for modeling situations in which individuals or objects of one type are being assigned or matched up with individuals or objects of another type. As in Chapter 4, we will generally draw bipartite graphs as in Figure 10.1(a), with the two different categories of nodes drawn as two parallel vertical columns, and the edges crossing between the two columns. Let’s return to the task that the college dorm administrators were Perfect Matchings. trying to solve: assigning each student a room that they’d be happy to accept. This task has a natural interpretation in terms of the graph we’ve just drawn: since the edges represent acceptable options for students, we want to assign a distinct room to each student, so that each student is assigned a room to which he or she is connected by an edge. Figure 10.1(b) shows such an assignment, with the darkened edges indicating who gets which room. We will refer to such an assignment as a perfect matching : When there are an equal number of nodes on each side of a bipartite graph, a perfect matching is an assignment of nodes on the left to nodes on the right, in such a way that (i) each node is connected by an edge to the node it is assigned to, and (ii) no two nodes on the left are assigned to the same node on the right. As this picture makes clear, there is an equivalent way to think about perfect matchings in terms of the edges that form the assignment: a perfect matching can also be viewed as a choice of edges in the bipartite graph so that each node is the endpoint of exactly one of the chosen edges. Constricted Sets. If a bipartite graph has a perfect matching, it’s easy to demonstrate this: you just indicate the edges that form the perfect matching. But what if a bipartite graph has no perfect matching? What could you show someone to convince them that there isn’t one? At first glance, this is not clear; one naturally worries that the only way to convince someone that there is no perfect matching is to plow through all the possibilities and show that no pairing works. But in fact there is a clean way to demonstrate that no perfect matching exists, based on the idea illustrated in Figure 10.2. First, Figure 10.2(a) shows a bipartite graph that contains no perfect matching. In Figure 10.2(b) we show a succinct reason why there is no perfect matching in this graph: the set consisting of Vikram, Wendy,

294 280 CHAPTER 10. MATCHING MARKETS ikram V Room1 Room1 V ikram endy Room2 W Room2 endy W Xin Room3 Xin Room3 oram Y Room4 oram Y Room4 Zoe Room5 Room5 Zoe (a) (b) A constricted set demonstrating Bipartite graph with no perfect there is no perfect matching matching Figure 10.2: (a) A bipartite graph with no perfect matching. (b) A constricted set demon- strating there is no perfect matching. and Xin, taken together, has collectively provided only two options for rooms that would be acceptable to any of them. With three people and only two acceptable rooms, there is clearly no way to construct a perfect matching — one of these three people would have to get an option they didn’t want in any assignment of rooms. We call the set of three students in this example a constricted set , since their edges to the other side of the bipartite graph “constrict” the formation of a perfect matching. This example points to a general phenomenon, which we can make precise by defining in general what it means for a set to be constricted, as follows. First, for any set of nodes S on the right-hand side of a bipartite graph, we say that a node on the left-hand side is a neighbor ( of S . We define the neighbor set of S , denoted N if it has an edge to some node in S ), to S be the collection of all neighbors of S . Finally, we say that a set S on the right-hand side is constricted if S is strictly larger than N ( S ) — that is, S contains strictly more nodes than N S ) does. ( Any time there’s a constricted set S in a bipartite graph, it immediately shows that there can be no perfect matching: each node in S would have to be matched to a different node ), so this is not possible. N ( S ), but there are more nodes in S than there are in N ( S in

295 10.1. BIPARTITE GRAPHS AND PERFECT MATCHINGS 281 Valuations Valuations Room1 Xin Xin Room1 12, 2, 4 12, 2, 4 Yoram Room2 Yoram Room2 8, 7, 6 8, 7, 6 Zoe Room3 Room3 Zoe 7, 5, 2 7, 5, 2 (a) (b) An optimal assignment A set of valuations Figure 10.3: (a) A set of valuations. Each person’s valuations for the objects appears as a list next to them. (b) An optimal assignment with respect to these valuations. So it’s fairly easy to see that constricted sets form one kind of obstacle to the presence of perfect matchings. What’s also true, though far from obvious, is that constricted sets are in only Matching fact the kind of obstacle. This is the crux of the following fact, known as the . Theorem Matching Theorem: If a bipartite graph (with equal numbers of nodes on the left and right) has no perfect matching, then it must contain a constricted set. The Matching Theorem was independently discovered by Denes K ̈onig in 1931 and Phillip Hall in 1935 [280]. Without the theorem, one might have imagined that a bipartite graph could fail to have a perfect matching for all sorts of reasons, some of them perhaps even too complicated to explain; but what the theorem says is that the simple notion of a constricted set is in fact the obstacle to having a perfect matching. For our purposes in this chapter, only we will only need to use the fact that the Matching Theorem is true, without having to go into the details of its proof. However, its proof is elegant as well, and we describe a proof of the theorem in Section 10.6 at the end of this chapter. One way to think about the Matching Theorem, using our example of students and rooms, is as follows. After the students submit their lists of acceptable rooms, it’s easy for the dormitory administrators to explain to the students what happened, regardless of the outcome. Either they can announce the perfect matching giving the assignment of students to rooms, or they can explain that no assignment is possible by indicating a set of students who collectively gave too small a set of acceptable options. This latter case is a constricted set.

296 282 CHAPTER 10. MATCHING MARKETS Xin Room1 1, 1, 0 Xin Room1 Y Room2 oram 1, 0, 0 Room2 Y oram Zoe Room3 Room3 Zoe 0, 1, 1 A bipartite graph (a) A set of valuations encoding the search for a perfect (b) matching Figure 10.4: (a) A bipartite graph in which we want to search for a perfect matching. (b) A corresponding set of valuations for the same nodes so that finding the optimal assignment lets us determine whether there is a perfect matching in the original graph. 10.2 Valuations and Optimal Assignments The problem of bipartite matching from the previous section illustrates some aspects of a market in a very simple form: individuals express preferences in the form of acceptable op- tions; a perfect matching then solves the problem of allocating objects to individuals accord- ing to these preferences; and if there is no perfect matching, it is because of a “constriction” in the system that blocks it. We now want to extend this model to introduce some additional features. First, rather than expressing preferences simply as binary “acceptable-or-not” choices, we allow each individual to express how much they’d like each object, in numerical form. In our example of students and dorm rooms from Section 10.1, suppose that rather than specifying a list of acceptable rooms, each student provides a numerical score for each room, indicating how happy they’d be with it. We will refer to these numbers as the students’ for the valuations respective rooms. Figure 10.3(a) shows an example of this with three students and three rooms; for instance, Xin’s valuations for Rooms 1, 2, and 3 are 12, 2, and 4 respectively (while Yoram’s valuations for Rooms 1, 2, and 3 are 8, 7, and 6 respectively). Notice that students may disagree on which rooms are better, and by how much. We can define valuations whenever we have a collection of individuals evaluating a col- lection of objects. And using these valuations, we can evaluate the quality of an assignment of objects to individuals, as follows: it is the sum of each individual’s valuation for what

297 10.2. VALUATIONS AND OPTIMAL ASSIGNMENTS 283 1 they get. Thus, for example, the quality of the assignment illustrated in Figure 10.3(b) is 12 + 6 + 5 = 23. If the dorm administrators had accurate data on each student’s valuations for each room, then a reasonable way to assign rooms to students would be to choose the assignment of maximum possible quality. We will refer to this as the optimal assignment , since it maximizes the total happiness of everyone for what they get. You can check that the assignment in Figure 10.3(b) is in fact the optimal assignment for this set of valuations. Of course, while the optimal assignment maximizes total happiness, it does not necessarily give everyone their favorite item; for example, in Figure 10.3(b), all the students think Room 1 is the best, but it can only go to one of them. In a very concrete sense, the problem of finding an optimal assignment also forms a natural generalization of the bipartite matching problem from Section 10.1. Specifically, it contains the bipartite matching problem as a special case. Here is why. Suppose, as in Section 10.1, that there are an equal number of students and rooms, and each student simply submits a list of acceptable rooms without providing a numerical valuation; this gives us a bipartite graph as in Figure 10.4(a). We would like to know if this bipartite graph contains a perfect matching, and we can express precisely this question in the language of valuations and optimal assignments as follows. We give each student a valuation of 1 for each room they included on their acceptable list, and a valuation of 0 for each room they omitted from their list. Applying this translation to the graph in Figure 10.4(a), for example, we get the valuations shown in Figure 10.4(b). Now, there is a perfect matching precisely when we can find an assignment that gives each student a room that he or she values at 1 rather than 0 — that is, precisely when the optimal assignment has a total valuation equal to the number of students. This simple translation shows how the problem of bipartite matching is implicit in the broader problem of finding an optimal assignment. While the definition of an optimal assignment is quite natural and general, it is far from obvious whether there is a comparably natural way to find or characterize the optimal assignment for a given set of valuations. This is in fact a bit subtle; we will describe a way to determine an optimal assignment, in the context of a broader market interpretation of this problem, in the two next sections. 1 Of course, this notion of the quality of an assignment is appropriate only if adding individual’s valuations makes sense. We can interpret individual valuations here as the maximum amount the individuals are willing to pay for items, so the sum of their valuations for the items they are assigned is just the maximum amount the group would be willing to pay in total for the assignment. The issue of adding individuals’ payoffs was also discussed in Chapter 6, where we defined social optimality using the sum of payoffs in a game.

298 284 CHAPTER 10. MATCHING MARKETS 10.3 Prices and the Market-Clearing Property Thus far, we have been using the metaphor of a central “administrator” who determines a perfect matching, or an optimal assignment, by collecting data from everyone and then performing a centralized computation. And while there are clearly instances of market-like activity that function this way (such as our example of students and dorm rooms), a more standard picture of a market involves much less central coordination, with individuals making decisions based on prices and their own valuations. Capturing this latter idea brings us to the crucial step in our formulation of matching markets: understanding the way in which prices can serve to decentralize the market. We will see that if we replace the role of the central administrator by a particular scheme for pricing items, then allowing individuals to follow their own self-interest based on valuations and prices can still produce optimal assignments. To describe this, let’s change the housing metaphor slightly, from students and dorm rooms to one where the role of prices is more natural. Suppose that we have a collection , each with a house for sale, and an equal-sized collection of buyers , each of whom of sellers wants a house. By analogy with the previous section, each buyer has a valuation for each house, and as before, two different buyers may have very different valuations for the same j has for the house held by seller houses. The valuation that a buyer will be denoted v , i ij i j indicating that the valuation depends on both the identity of with the subscripts and i and the buyer j . We also assume that each valuation is a non-negative whole the seller , 1 , 2 ,... ). We assume that sellers have a valuation of 0 for each house; they care number (0 2 only about receiving payment from buyers, which we define next. Prices and Payoffs. Suppose that each seller i puts his house up for sale, offering to sell it for a price p ≥ 0. If a buyer j buys the house from seller i at this price, we will say that i the buyer’s is her valuation for this house, minus the amount of money she had to pay: payoff − v . So given a set of prices, if buyer j wants to maximize her payoff, she will buy from p ij i the seller i for which this quantity v is maximized — with the following caveats. First, − p i ij if this quantity is maximized in a tie between several sellers, then the buyer can maximize her payoff by choosing any one of them. Second, if her payoff v − p is negative for every ij i choice of seller i , then the buyer would prefer not to buy any house: we assume she can obtain a payoff of 0 by simply not transacting. j the preferred sellers We will call the seller or sellers that maximize the payoff for buyer 2 Our assumption that sellers all have valuations of 0 for their houses is done for the sake of simplicity; if we wanted, we could directly adapt the arguments here to the case in which “zero” is really some minimum base level, and all other valuations and prices represent amounts above this base level. It is also not hard to adapt our analysis to the case in which sellers each might have different valuations for their houses. Since none of these more general models add much to the underlying set of ideas, we will stick with the simple assumption that sellers have valuations of 0 for houses.

299 10.3. PRICES AND THE MARKET-CLEARING PROPERTY 285 Prices Buyers Sellers Buyers Valuations Sellers Valuations 5 12, 4, 2 a x x a 12, 4, 2 8, 7, 6 y b 2 b y 8, 7, 6 7, 5, 2 c z 7, 5, 2 c z 0 (b) Market-Clearing Prices (a) Buyer Valuations Prices Sellers Buyers Prices Sellers Valuations Valuations Buyers 2 3 12, 4, 2 12, 4, 2 x x a a 8, 7, 6 8, 7, 6 b y y b 1 1 z 7, 5, 2 7, 5, 2 z c c 0 0 (c) Market-Clearing Prices (Tie-Breaking Required) Prices that Don’t Clear the Market (d) (a) Three sellers ( a , b , and c Figure 10.5: x , y , and z ). For each buyer node, the ) and three buyers ( valuations for the houses of the respective sellers appear in a list next to the node. (b) Each buyer creates a link to her preferred seller. The resulting set of edges is the preferred-seller graph for this set of prices. (c) The preferred-seller graph for prices 2, 1, 0. (d) The preferred-seller graph for prices 3, 1, 0. of buyer j , provided the payoff from these sellers is not negative. We say that buyer j has . no preferred seller if the payoffs − p i are negative for all choices of v i ij In Figures 10.5(b)-10.5(d), we show the results of three different sets of prices for the same set of buyer valuations. Note how the sets of preferred sellers for each buyer change depending on what the prices are. So for example, in Figure 10.5(b), buyer x would receive a payoff of 12 − 5 = 7 if she buys from a , a payoff of 4 − 2 = 2 if she buys from b , and 2 − 0 = 2 if she buys from . This is why a is her unique preferred seller. We can similarly determine c the payoffs for buyers y (3, 5, and 6) and z (2, 3, and 2) for transacting with sellers a , b , and c respectively.

300 286 CHAPTER 10. MATCHING MARKETS Figure 10.5(b) has the particularly nice property that if each Market-Clearing Prices. buyer simply claims the house that she likes best, each buyer ends up with a different house: somehow the prices have perfectly resolved the contention for houses. And this happens a the highest; it is the despite the fact that each of the three buyers value the house of seller and z high price of 5 that dissuades buyers y from pursuing this house. market-clearing , since they cause each house to get bought We will call such a set of prices by a different buyer. In contrast, Figure 10.5(c) shows an example of prices that are not market-clearing, since buyers and z both want the house offered by seller a — so in this x case, when each buyer pursues the house that maximizes their payoff, the contention for a , b houses is not resolved. (Notice that although each of c is a preferred seller for y , , and since they all give y equal payoffs, this does not help with the contention between x and z .) Figure 10.5(d) illustrates one further subtlety in the notion of market-clearing prices. Here, if the buyers coordinate so that each chooses the appropriate preferred seller, then ’s house.) y c ’s house and z take each buyer gets a different house. (This requires that take b Since it is possible to eliminate contention using preferred sellers, we will say that this set of prices is market-clearing as well, even though a bit of coordination is required due to ties in the maximum payoffs. In some cases, ties like this may be inevitable: for example, if all buyers have the same valuations for everything, then no choice of prices will break this symmetry. Given the possibility of ties, we will think about market-clearing prices more generally as follows. For a set of prices, we define the preferred-seller graph on buyers and sellers by simply constructing an edge between each buyer and her preferred seller or sellers. (There will be no edge out of a buyer if she has no preferred seller.) So in fact, Figures 10.5(b)- 10.5(d) are just drawings of preferred-seller graphs for each of the three indicated sets of prices. Now we simply say: a set of prices is market-clearing if the resulting preferred-seller graph has a perfect matching. Properties of Market-Clearing Prices. In a way, market-clearing prices feel a bit too good to be true: if sellers set prices the right way, then self-interest runs its course and (potentially with a bit of coordination over tie-breaking) all the buyers get out of each other’s way and claim different houses. We’ve seen that such prices can be achieved in one very small example; but in fact, something much more general is true: Existence of Market-Clearing Prices: For any set of buyer valuations, there exists a set of market-clearing prices. So market-clearing prices are not just a fortuitous outcome in certain cases; they are always present. This is far from obvious, and we will turn shortly to a method for constructing market-clearing prices that, in the process, proves they always exist.

301 10.3. PRICES AND THE MARKET-CLEARING PROPERTY 287 Before doing this, we consider another natural question: the relationship between market- clearing prices and social welfare. Just because market-clearing prices resolve the contention among buyers, causing them to get different houses, does this mean that the total valuation of the resulting assignment will be good? In fact, there is something very strong that can be said here as well: market-clearing prices (for this buyer-seller matching problem) always provide socially optimal outcomes: Optimality of Market-Clearing Prices: For any set of market-clearing prices, a perfect matching in the resulting preferred-seller graph has the maximum total valuation of any assignment of sellers to buyers. Compared with the previous claim on the existence of market-clearing prices, this fact about optimality can be justified by a much shorter, if somewhat subtle, argument. M be The argument is as follows. Consider a set of market-clearing prices, and let total payoff a perfect matching in the preferred-seller graph. Now, consider the of this matching, defined simply as the sum of each buyer’s payoff for what she gets. Since each M buyer is grabbing a house that maximizes her payoff individually, has the maximum total payoff of any assignment of houses to buyers. Now how does total payoff relate to total M valuation, which is what we’re hoping that j chooses house i , then maximizes? If buyer her valuation is v . Thus, the total payoff to all buyers is simply and her payoff is v p − ij i ij the total valuation, minus the sum of all prices: . = Total Valuation of M − Total Payoff of M Sum of all prices But the sum of all prices is something that doesn’t depend on which matching we choose (it’s just the sum of everything the sellers are asking for, regardless of how they get paired up with buyers). So a matching M that maximizes the total payoff is also one that maximizes the total valuation. This completes the argument. There is another important way of thinking about the optimality of market-clearing prices, which turns out to be essentially equivalent to the formulation we’ve just described. Suppose that instead of thinking about the total valuation of the matching, we think about the total of the payoffs received by all participants in the market — both the sellers and the buyers. For a buyer, her payoff is defined as above: it is her valuation for the house she gets minus the price she pays. A seller’s payoff is simply the amount of money he receives in payment for his house. Therefore, in any matching, the total of the payoffs to all the sellers is simply equal to the sum of the prices (since they all get paid, and it doesn’t matter which buyer pays which seller). Above, we just argued that the total of the payoffs to all the buyers is equal to the total valuation of the matching M , minus the sum of all prices. Therefore, the total of the payoffs to all participants — both the sellers and the buyers — is exactly equal to the total valuation of the matching M ; the point is that the prices detract from

302 288 CHAPTER 10. MATCHING MARKETS the total buyer payoff by exactly the amount that they contribute to the total seller payoff, and hence the sum of the prices cancels out completely from this calculation. Therefore, to maximize the total payoffs to all participants, we want prices and a matching that lead to the maximum total valuation, and this is achieved by using market-clearing prices and a perfect matching in the resulting preferred-seller graph. We can summarize this as follows. Optimality of Market-Clearing Prices (equivalent version): A set of market- clearing prices, and a perfect matching in the resulting preferred-seller graph, produces the maximum possible sum of payoffs to all sellers and buyers. 10.4 Constructing a Set of Market-Clearing Prices Now let’s turn to the harder challenge: understanding why market-clearing prices must always exist. We’re going to do this by taking an arbitrary set of buyer valuations, and describing a procedure that arrives at market-clearing prices. The procedure will in fact be a kind of auction — not a single-item auction of the type we discussed in Chapter 9, but a more general kind taking into account the fact that there are multiple things being auctioned, and multiple buyers with different valuations. This particular auction procedure was described by the economists Demange, Gale, and Sotomayor in 1986 [129], but it’s actually equivalent to a construction of market-clearing prices discovered by the Hungarian mathematician Egerv ́ary seventy years earlier, in 1916 [280]. Here’s how the auction works. Initially all sellers set their prices to 0. Buyers react by choosing their preferred seller(s), and we look at the resulting preferred-seller graph. If this graph has a perfect matching we’re done. Otherwise — and this is the key point — there is S a constricted set of buyers N ( S ), which is a set of sellers. . Consider the set of neighbors The buyers in S only want what the sellers in N ( S ) have to sell, but there are fewer sellers N in S ) than there are buyers in S . So the sellers in ( ( S ) are in “high demand” — too N many buyers are interested in them. They respond by each raising their prices by one unit, and the auction then continues. There’s one more ingredient, which is a reduction operation on the prices. It will be useful to have our prices scaled so that the smallest one is 0. Thus, if we ever reach a point p > 0 — where all prices are strictly greater than 0 — suppose the smallest price has value then we reduce the prices by subtracting p from each one. This drops the lowest price to 0, and shifts all other prices by the same relative amount. A general round of the auction looks like what we’ve just described. (i) At the start of each round, there is a current set of prices, with the smallest one equal to 0. (ii) We construct the preferred-seller graph and check whether there is a perfect matching.

303 10.4. CONSTRUCTING A SET OF MARKET-CLEARING PRICES 289 (iii) If there is, we’re done: the current prices are market-clearing. (iv) If not, we find a constricted set of buyers and their neighbors N ( S ). S N ( S ) (simultaneously) raises his price by one unit. (v) Each seller in (vi) If necessary, we reduce the prices — the same amount is subtracted from each price so that the smallest price becomes zero. (vii) We now begin the next round of the auction, using these new prices. The full-page Figure 10.6 shows what happens when we apply the auction procedure to the example from Figure 10.5. The example in Figure 10.6 illustrates two aspects of this auction that should be empha- sized. First, in any round where the set of “over-demanded” sellers ( S ) consists of more N than one individual, all the sellers in this set raise their prices simultaneously. For example, in the third round in Figure 10.6, the set N ( S ) consists of both a and b , and so they both raise their prices so as to produce the prices used for the start of the fourth round. Second, while the auction procedure shown in Figure 10.6 produces the market-clearing prices shown in Figure 10.5(d), we know from Figure 10.5(b) that there can be other market-clearing prices for the same set of buyer valuations. Here is a key property of the Showing that the Auction Must Come to an End. auction procedure we’ve defined: the only way it can come to end is if it reaches a set of market-clearing prices; otherwise, the rounds continue. So if we can show that the auction must come to an end for any set of buyer valuations — i.e. that the rounds cannot go on forever — then we’ve shown that market-clearing prices always exist. It’s not immediately clear, however, why the auction must always come to an end. Con- sider, for example, the sequence of steps the auction follows in Figure 10.6: prices change, different constricted sets form at different points in time, and eventually the auction stops with a set of market-clearing prices. But why should this happen in general? Why couldn’t there be a set of valuations that cause the prices to constantly shift around so that some set of buyers is always constricted, and the auction never stops? In fact, the prices can’t shift forever without stopping; the auction must always come to an end. The way we’re going to show this is by identifying a precise sense in which a certain kind of “potential energy” is draining out of the auction as it runs; since the auction starts with only a bounded supply of this potential energy at the beginning, it must eventually run out. Here is how we define this notion of potential energy precisely. For any current set of prices, define the potential of a buyer to be the maximum payoff she can currently get from any seller. This is the buyer’s potential payoff; the buyer will actually get this payoff if the current prices are market-clearing prices. We also define the potential of a seller to be the

304 290 CHAPTER 10. MATCHING MARKETS Buyers Prices Buyers Sellers Valuations Valuations Sellers Prices 0 12, 4, 2 12, 4, 2 1 a x a x 8, 7, 6 8, 7, 6 b y y b 0 0 z 7, 5, 2 7, 5, 2 c z c 0 0 (a) Start of first round (b) Start of second round Sellers Prices Valuations Buyers Buyers Sellers Prices Valuations 3 12, 4, 2 2 12, 4, 2 a x x a 8, 7, 6 8, 7, 6 y b y b 0 1 7, 5, 2 c c 7, 5, 2 z z 0 0 Start of third round (c) Start of fourth round (d) Figure 10.6: The auction procedure applied to the example from Figure 10.5. Each separate picture shows steps (i) and (ii) of successive rounds, in which the preferred-seller graph for that round is constructed. (a) In the first round, all prices start at 0. The set of all buyers forms a constricted set , with S ( S N a . So a raises his price by one unit and the auction continues to the ) equal to the seller second round. (b) In the second round, the set of buyers consisting of x and z forms a constricted set S , with N ( ) again equal to the seller a . Seller a again raises his price by one unit and the auction S continues to the third round. (Notice that in this round, we could alternately have identified ) would have been the set of all buyers as a different constricted set N ( S S , in which case the set of sellers a and b . There is no problem with this — it just means that there can be multiple options for how to run the auction procedure in certain rounds, with any of these options leading to market-clearing prices when the auction comes to an end.) S , with N ( S ) equal to the (c) In the third round, the set of all buyers forms a constricted set set of two sellers a and b . So a and b simultaneously raise their prices by one unit each, and the auction continues to the fourth round. (d) In the fourth round, when we build the preferred-seller graph, we find it contains a perfect matching. Hence, the current prices are market-clearing and the auction comes to an end.

305 10.5. HOW DOES THIS RELATE TO SINGLE-ITEM AUCTIONS? 291 current price he is charging. This is the seller’s potential payoff; the seller will actually get this payoff if the current prices are market-clearing prices. Finally, we define the potential energy of the auction to be the sum of the potential of all participants, both buyers and sellers. How does the potential energy of the auction behave as we run it? It begins with all sellers having potential 0, and each buyer having a potential equal to her maximum valuation for any house — so the potential energy of the auction at the start is some whole number ≥ P 0. Also, notice that at the start of each round of the auction, everyone has potential 0 at least 0. The sellers always have potential at least 0 since the prices are always at least 0. Because of the price-reduction step in every round, the lowest price is always 0, and therefore each buyer is always doing at least as well as the option of buying a 0-cost item, which gives a payoff of at least 0. (This also means that each buyer has at least one preferred seller at the start of each round.) Finally, since the potentials of the sellers and buyers are all at least 0 at the start of each round, so is the potential energy of the auction. Now, the potential only changes when the prices change, and this only happens in steps (v) and (vi). Notice that the reduction of prices, as defined above, does not change the potential energy of the auction: if we subtract from each price, then the potential of each p p , but the potential of each buyer goes up by p seller drops by — it all cancels out. Finally, what happens to the potential energy of the auction in step (v), when the sellers in N ( S ) all raise their prices by one unit? Each of these sellers’ potentials goes up by one unit. But the potential of each buyer in S goes down by one unit, since all their preferred houses just S ) does, this means that the N ( S got more expensive. Since has strictly more nodes than potential energy of the auction goes down by at least one unit more than it goes up, so it strictly decreases by at least one unit. So what we’ve shown is that in each step that the auction runs, the potential energy of the auction decreases by at least one unit. It starts at some fixed value P , and it can’t drop 0 below 0, so the auction must come to an end within P steps — and when it comes to an 0 end, we have our market-clearing prices. 10.5 How Does this Relate to Single-Item Auctions? We talked in Chapter 9 about single-item auctions, and we’ve now seen a more complex type of auction based on bipartite graphs. It makes sense to ask how these different kinds of auctions relate to each other. In fact, there is a very natural way to view the single-item auction — both the outcome and the procedure itself — as a special case of the bipartite graph auction we’ve just defined. We can do this as follows. Suppose we have a set of n buyers and a single seller auctioning an item; let buyer j have valuation v for the item. To map this to our model based on perfect matchings, we j

306 292 CHAPTER 10. MATCHING MARKETS Sellers Valuations Buyers Prices Buyers Valuations Prices Sellers 2 0 3, 0, 0 3, 0, 0 x a x a 2, 0, 0 2, 0, 0 b b y y 0 0 z c 1, 0, 0 1, 0, 0 z c 0 0 Start of the Auction (b) End of the Auction (a) Figure 10.7: A single-item auction can be represented by the bipartite graph model: the item is represented by one seller node, and then there are additional seller nodes for which all buyers have 0 valuation. (a) The start of the bipartite graph auction. (b) The end of the bipartite graph auction, when buyer x gets the item at the valuation of buyer y . need an equal number of buyers and sellers, but this is easily dealt with: we create n − 1 “fake” additional sellers (who conceptually represent n − 1 different ways to fail to acquire the item), and we give buyer j a valuation of 0 for the item offered by each of these fake v = v , the valuation of buyer j sellers. With the real seller labeled 1, this means we have j j 1 v for the real item; and = 0 for larger values of i . ij Now we have a genuine instance of our bipartite graph model: from a perfect matching of buyers to sellers, we can see which buyer ends up paired with the real seller (this is the buyer who gets the item), and from a set of market-clearing prices, we will see what the real item sells for. Moreover, the price-raising procedure to produce market-clearing prices — based on finding constricted sets — has a natural meaning here as well. The execution of the procedure on a simple example is shown in Figure 10.7. Initially, all buyers will identify the real seller as their preferred seller (assuming that they all have positive valuations for the item). The first constricted set S we find is the set of all buyers, and N ( S ) is just the single real seller. Thus, the seller raises his price by one unit. This continues as long as at least two buyers have the real seller as their unique preferred seller: they form a constricted set S with N ( S ) equal to the real seller, and this seller raises his price by a unit. The prices of the fake items remain fixed at 0 throughout the auction. Finally, when all but one buyer has identified other sellers as preferred sellers, the graph has a perfect matching. This happens at precisely the moment that the buyer with the second-highest valuation drops out — in other words, the buyer with the highest valuation gets the item, and pays the second-highest valuation. So the bipartite graph procedure precisely implements an ascending bid (English) auction.

307 10.6. ADVANCED MATERIAL: A PROOF OF THE MATCHING THEOREM 293 10.6 Advanced Material: A Proof of the Matching Theorem The discussion in this chapter provides a complete proof that market-clearing prices exist, omitting the details in only one place: we deferred the proof of the Matching Theorem in Section 10.1, since it was enough for our purposes to use it in a “black-box” fashion. However, the standard proof of the Matching Theorem in fact provides important insights into the structure of bipartite graphs, and so we present it here. Recall that the statement is the following. Claim: If a bipartite graph (with equal numbers of nodes on the left and right) has no perfect matching, then it must contain a constricted set. The tricky part of proving this is to come up with some means of identifying a constricted set in a bipartite graph, knowing only that it contains no perfect matching. Our general plan for doing this will be as follows. We will take a bipartite graph, with equal numbers of nodes on the left and right but no perfect matching, and consider a matching that includes maximum matching . We then try to enlarge as many nodes as possible — we will call this a it, by searching for a way to include one more node from each side in the matching. This will fail (since the matching is already as large as possible for the given graph), and we show that when this search for a larger matching fails, it produces a constricted set. Naturally, there is a lot that needs to be fleshed out in this strategy, and the first step is to consider how one goes about “enlarging” a matching in a bipartite graph. This will turn out to be the key issue in the whole proof. Alternating and Augmenting Paths. With this in mind, let’s forget about constricted sets for a little while and simply think about matchings and how they can be enlarged. As a first example, consider the bipartite graph in Figure 10.8(a), with a matching indicated using matching edges are the edges used in a given matching, and bold edges. (We’ll say that the the non-matching edges are the other, unused ones.) The matching shown in Figure 10.8(a) is not a maximum matching — we could clearly pair up W with A and X with B to get a larger one, as in Figure 10.8(c). For examples this small, it’s easy just to look at the picture of the graph and see how to find a larger matching. But for bipartite graphs that are more complicated, it’s useful to have a more principled way of growing a smaller matching into a larger one. Here’s how we can do this in Figure 10.8(a). We start at node W , looking for a matching that would include it while still also including everyone who’s currently matched. It’s not immediately obvious whether we can pair up W with A , since A is already matched to X . So we try A “undoing” the pairing of and X , which would let W and A be matched. This in turn frees up X , which can then be matched with B , and we’ve enlarged the matching.

308 294 CHAPTER 10. MATCHING MARKETS A A W W A W B B X X B X (a) A larger (perfect) matching (b) An augmenting path (c) A matching that is not of max- imum size Figure 10.8: (a) A matching that does not have maximum size. (b) What a matching does not have maximum size, we can try to find an augmenting path that connects unmatched nodes on opposite sides while alternating between non-matching and matching edges. (c) If we then swap the edges on this path — taking out the matching edges on the path and replacing them with the non-matching edges — then we obtain a larger matching. This process is depicted in Figure 10.8(b). We followed a zigzag path through the bipartite graph, alternately adding unused edges to the matching while removing currently used edges from the matching: in particular, we added the edges A - W and B - X to the matching, while removing the edge A X . It was also important that the path was simple — it did not repeat - any nodes. We will call a simple path that alternates between non-matching and matching edges in this way an . alternating path This example illustrates a principle that’s true in general. In any bipartite graph with a matching, if we can find an alternating path that begins and ends at an unmatched node, then we can swap the roles of all edges on this path: each non-matching edge gets put into the matching, and each edge on the path that is currently in the matching gets removed from it. In this way, all the nodes in the path become paired up: we have managed to include the two formerly unmatched endpoints in the matching, thereby enlarging it. We can summarize this as follows: Claim: In a bipartite graph with a matching, if there is an alternating path whose endpoints are unmatched nodes, then the matching can be enlarged. In view of this, we call an alternating path with unmatched endpoints an augmenting path , since it gives us a way to augment the matching. Augmenting paths can get much longer than what we see in Figure 10.8. For example, in Figure 10.9, we show an augmenting path that includes a total of eight nodes, and succeeds in including its two endpoints W and D in the matching. It’s also the case that augmenting paths can be harder to find than these two simple examples might suggest. In each of these examples, looking for the augmenting path never involves any real choices — we just keep

309 10.6. ADVANCED MATERIAL: A PROOF OF THE MATCHING THEOREM 295 A A W A W W X B B X X B C Y C Y Y C Z Z Z D D D An augmenting path A matching that is not of max- (a) (b) (c) A larger (perfect) matching imum size Figure 10.9: The principle used in Figure 10.8 can be applied to larger bipartite graphs as well, sometimes producing long augmenting paths. following the only edge available at each step. But in more complicated bipartite graphs, there can be lots of dead-ends in the search for an augmenting path. Consider, for example, the graph with a matching in Figure 10.10(a). There is in fact an augmenting path that succeeds in including and D in the matching, but even on this relatively small example W one needs to check a bit carefully to find it. Moreover, there are other alternating paths starting from W — such as W - A - X and W - B - Y - C - Z — that don’t make it to the other Y unmatched node , as well as paths from W to D — such as W - B - Z - C - D - D — that are not alternating. Fortunately, however, there is a natural procedure Searching for an Augmenting Path. we can use to search for an augmenting path in a bipartite graph with a matching. It works by simply adapting the breadth-first search (BFS) procedure to include the requirement of alternation — as a result, we will refer to this new procedure as alternating BFS . Here is how it works. We start at any unmatched node on the right. Then, as in traditional BFS, we explore the rest of the graph layer by layer, adding new nodes to the next layer when they are connected by an edge to a node in the current layer. Because the graph is bipartite, these layers will alternate between consisting of nodes on the left

310 296 CHAPTER 10. MATCHING MARKETS A A W W A W B B X X B X Y C Y Y C C Z D D D Z Z An augmenting path (b) A matching that is not of max- (a) (c) A larger (perfect) matching imum size Figure 10.10: In more complex graphs, finding an augmenting path can require a more careful search, in which choices lead to “dead ends” while others connect two unmatched nodes. and nodes on the right. Now, here is the difference from traditional BFS: because we are searching specifically for an augmenting path, we want the paths that move downward layer- by-layer to all be alternating. Thus, when we build a new layer of nodes from the left-hand side, we should only use non-matching edges to discover new nodes; and when we build a new layer of nodes from the right-hand side, we should only use matching edges to discover new nodes. W Figure 10.11 shows how this works on the example from Figure 10.10(a). Starting at (which we’ll think of as layer 0), we build the first layer by following non-matching edges to A and B . We then build the second layer by only following matching edges, which leads us X to nodes Y . Following non-matching edges from this layer to new nodes not already and discovered, we get a third layer consisting of C and D ; and finally, taking the matching edge from C brings us to Z in the fourth layer. Notice that in this process, we never used the edge - Z : we couldn’t use it out of B in the first layer, because we were only allowed to B follow matching edges at that point; and we couldn’t use it out of Z in the fourth layer, because by then B had already been discovered. Now, the crucial thing to observe is that if this alternating BFS procedure ever produces

311 10.6. ADVANCED MATERIAL: A PROOF OF THE MATCHING THEOREM 297 W A B Y X B-Z edge not D C part of search Z Figure 10.11: In an alternating breadth-first search, one constructs layers that alternately use non-matching and matching edges; if an unmatched node is ever reached, this results in an augmenting path. a layer containing an unmatched node from the left-hand side of the graph, we have found an augmenting path (and can thus enlarge the matching). We simply move downward in a path from the unmatched node in layer 0 to the unmatched node from the left-hand side, proceeding one layer at a time. The edges on this path will alternate between being non-matching and matching, and so this will be an augmenting path. Augmenting Paths and Constricted Sets. This gives us a systematic procedure to search for an augmenting path. However, it leaves a basic question unresolved: if this search procedure fails to find an augmenting path, can we necessarily conclude that there is no perfect matching? This is certainly not a priori clear: why couldn’t it be that there is a perfect matching hidden somewhere in the graph, and we just need a more powerful way to find it? But in fact, alternating BFS is all that we need: what we’ll show now is that when

312 298 CHAPTER 10. MATCHING MARKETS Layer 0 W Layer 1 equal numbers of nodes } Layer 2 Layer 3 equal numbers of nodes } Layer 4 Figure 10.12: A schematic view of alternating breadth-first search, which produces pairs of layers of equal size. alternating BFS fails to find an augmenting path, we can in fact extract from this failed search a constricted set that proves there is no perfect matching. Here is how. Consider any bipartite graph, and suppose we are currently looking at a matching in it that is not perfect. Suppose further that we perform an alternating BFS from an unmatched node W on the right-hand side, and we fail to reach any unmatched node on the left-hand side. The resulting set of layers at the end of the search will look schematically like what’s depicted in Figure 10.12. More concretely, Figure 10.13(a) shows a specific example of a graph with no perfect matching, and Figure 10.13(b) shows a set of layers from a failed alternating BFS on this example. Let’s make some observations about the structure after a failed search. 1. First, the even-numbered layers consist of nodes from the right-hand side, while the odd-numbered layers consist of nodes from the left-hand side.

313 10.6. ADVANCED MATERIAL: A PROOF OF THE MATCHING THEOREM 299 A W A W W B X X B A B Y C C Y X Y Z D Z D The resulting constricted set A maximum matching that (a) A failed search for an aug- (b) (c) is not perfect menting path Figure 10.13: (a) A matching that has maximum size, but is not perfect. (b) For such a matching, the search for an augment path using alternating breadth-first search will fail. (c) The failure of this search exposes a constricted set: the set of nodes belonging to the even layers. 2. Moreover, each odd layer contains exactly the same number of nodes as the subsequent even layer. This is because we never reach an unmatched node in an odd layer: so in every odd layer, the nodes are all connected by their matching edges to distinct nodes in the next layer, as illustrated in Figure 10.12. W in layer 0, there are exactly the same number of nodes in even 3. So not counting node layers (numbered 2 and higher) as there are in odd layers. Counting the one extra node in layer 0, there are strictly more nodes in even layers overall than there are in odd layers. 4. Finally, every node in an even layer has all of its neighbors in the graph present in some layer. This is because each even-layer node other than W has its matched partner just above it in the previous layer; and if any of its other neighbors were not already present in a higher layer, they would be added to the next layer down, when we’re allowed to explore using non-matching edges. (Notice that it’s not necessarily true that every node in an odd layer has all of its

314 300 CHAPTER 10. MATCHING MARKETS neighbors in the graph present in some layer. For example, in Figure 10.13(b), node B is not present in any layer. This is because we were not allowed to add ’s neighbor Z B in the search, since we could only follow the matching edge out when we got to Z B .) of Putting these observations together, we discover the following fact: the set of nodes in all even layers, at the end of a failed alternating BFS, forms a constricted set. This is simply because it’s a set of nodes on the right-hand side whose set of neighbors — because they’re S S is. Figures 10.13(b) contained among the nodes in the odd layers — is strictly smaller than and 10.13(c) show how this works in one specific example. This completes our plan — to extract a constricted set from the failure of alternating BFS. Here is one way to summarize the conclusion. Claim: Consider any bipartite graph with a matching, and let W be any un- matched node on the right-hand side. Then either there is an augmenting path beginning at W , or there is a constricted set containing W . The Matching Theorem. The fact we’ve just discovered is the crucial step in proving the Matching Theorem; from here it’s easy, as follows. Consider a bipartite graph with an equal number of nodes on the left and right, and suppose it has no perfect matching. Let’s take a maximum matching in it — one that includes as many edges as possible. Since this matching is not perfect, and since there are W an equal number of nodes on the two sides of the bipartite graph, there must be a node on the right-hand side that is unmatched. We know there cannot be an augmenting path containing W , since then we’d be able to enlarge the matching — and that isn’t possible since we chose a matching of maximum size. Now, by our previous claim, since there is no augmenting path beginning at W , there must be a constricted set containing W . Since we’ve deduced the existence of a constricted set from the fact that the graph has no perfect matching, this completes the proof of the Matching Theorem. Computing a Perfect Matching. One final dividend from this analysis is that we actu- ally have a reasonably efficient method to determine whether a graph has a perfect matching — enormously more efficient than the brute-force approach of trying all ways to pair up the nodes on the left and right. The method works as follows. Given a bipartite graph with an equal number of nodes on the left and right, we will progress through a sequence of matchings, and each matching in the sequence will be one edge larger than the previous one. We can start from the empty matching — the trivial one in which no nodes at all are paired. Now in general, we look at our current matching and find an unmatched node W . We use alternating BFS to search for

315 10.6. ADVANCED MATERIAL: A PROOF OF THE MATCHING THEOREM 301 A W B X Y C Z D any node on the right-hand Figure 10.14: If the alternating breadth-first search fails from side, this is enough to expose a constricted set and hence prove there is no perfect matching. However, it is still possible that an alternating breadth-first search could still succeed from W some other node. (In this case, the search from Y would would fail, but the search from succeed.) an augmenting path beginning at . If we find one, we use this augmenting path to enlarge W the matching, and we continue with this new matching. If we don’t find one, we can stop with a constricted set that proves the graph has no perfect matching. Since the matchings get larger in every step while the process is running, the number of matchings we pass through can be at most the number of nodes on each side of the graph. By then, we will either have reached a perfect matching, or stopped earlier with a constricted set. An interesting question is the following: when the procedure stops with a constricted set, are we guaranteed to have a maximum matching? As we’ve described the procedure so far, the answer is no. Consider for example Figure 10.14. If we try to find an augmenting W , then we will fail (producing the constricted set consisting of path starting at and X ). W This is indeed enough to prove there is no perfect matching. However, it does not mean that the current matching has maximum size: if we instead had searched for an augmenting path - starting from , we would have succeeded, producing the path Y - B Y Z - D . In other words, if we’re looking for a maximum matching and not just a perfect matching, it can matter

316 302 CHAPTER 10. MATCHING MARKETS where we start our search for an augmenting path; certain parts of the graph can become “wedged,” while other still contain the potential for enlarging the matching. However, there is a variation on our procedure that is guaranteed to produce a maximum matching. We won’t go through all the details of this (see e.g. [260] for more), but the idea is as follows. By revisiting the analysis we’ve used thus far, and adapting it a little bit, one can show that if there is no augmenting path beginning at node on the right- any hand side, then in fact the current matching has maximum size. This shows that if, as we progress through larger and larger matchings, we always search for an augmenting path from every node on the right-hand side, then either one of these searches will succeed, or else we can conclude that the current matching has maximum size. And while this sounds like an expensive thing to do — having to search separately from each node on the right — in fact it can be done efficiently by making all the unmatched nodes on the right constitute layer 0 in the alternating BFS, and otherwise running it as before. Then if an unmatched node on the left is ever reached in some layer, we can follow the path from the appropriate node in layer 0 down to it, producing an augmenting path. A lot of work has gone into the design of efficient methods for finding maximum matchings in bipartite graphs, and there are a number of further improvements possible, including versions of alternating BFS that try to find many augmenting paths simultaneously, thereby cutting down the number of intermediate matchings one must pass through on the way to the maximum. Determining how efficiently maximum matchings can be found remains an open area of research. 10.7 Exercises a 1. Suppose we have a set of 2 sellers labeled b , and a set of 2 buyers labeled x and and y . Each seller is offering a distinct house for sale, and the valuations of the buyers for the houses are as follows. Buyer Value for for Value ’s house b a ’s house x 2 4 y 3 6 Suppose that a charges a price of 0 for his house, and b charges a price of 1 for his house. Is this set of prices market-clearing? Give a brief (1-3 sentence) explanation; as part of your answer, say what the preferred-seller graph is with this given set of prices, and use this in your explanation. 2. Suppose we have a set of 3 sellers labeled a , b , and c , and a set of 3 buyers labeled x , y , and z . Each seller is offering a distinct house for sale, and the valuations of the buyers for the houses are as follows.

317 10.7. EXERCISES 303 Value for Buyer Value for for Value ’s house c ’s house a ’s house b x 5 7 1 3 1 y 2 4 5 z 4 and b a c charges 1. Is this set of prices Suppose that sellers each charge 2, and seller market-clearing? Give a brief explanation. a 3. Suppose we have a set of 3 sellers labeled b , and c , and a set of 3 buyers labeled , x y , and z . Each seller is offering a distinct house for sale, and the valuations of the , buyers for the houses are as follows. Buyer for Value for Value for Value ’s house b a ’s house ’s house c x 2 4 6 y 3 5 1 z 4 7 5 Suppose that sellers a c each charge 1, and seller b charges 3. Is this set of prices and market-clearing? Give a brief explanation. 4. Suppose we have a set of 3 sellers labeled , b , and c , and a set of 3 buyers labeled a , y , and z . Each seller is offering a distinct house for sale, and the valuations of the x buyers for the houses are as follows. Buyer for for Value Value for Value ’s house ’s house a c ’s house b x 12 9 8 y 10 6 3 8 5 z 6 a charges a price of 3 for his house, Suppose that charges a price of 1 for his house, b and c charges a price of 0. Is this set of prices market-clearing? If so, explain which buyer you would expect to get which house; if not, say which seller or sellers should raise their price(s) in the next round of the bipartite-graph auction procedure from Chapter 10. 5. Suppose we have a set of 3 sellers labeled a , b , and c , and a set of 3 buyers labeled x , y , and z . Each seller is offering a distinct house for sale, and the valuations of the buyers for the houses are as follows.

318 304 CHAPTER 10. MATCHING MARKETS Value for Buyer Value for for Value ’s house c ’s house a b ’s house x 7 7 4 6 y 7 3 5 4 3 z Suppose that a charges a price of 4 for his house, b charges a price of 3 for his house, c charges a price of 1. Is this set of prices market-clearing? Give an explanation and for your answer, using the relevant definitions from Chapter 10. 6. Suppose we have a set of 3 sellers labeled , b , and c , and a set of 3 buyers labeled a , , and z . Each seller is offering a distinct house for sale, and the valuations of the x y buyers for the houses are as follows. Value for for Value Value Buyer for b ’s house a c ’s house ’s house x 6 3 2 y 10 5 4 z 7 8 6 charges a price of 4 for his house, charges a price of 1 for his house, a b Suppose that and c charges a price of 0. Is this set of prices market-clearing? If so, explain which buyer you would expect to get which house; if not, say which seller or sellers should raise their price(s) in the next round of the bipartite-graph auction procedure from Chapter 10. a , b , and c , and a set of 3 buyers labeled 7. Suppose we have a set of 3 sellers labeled , , and z . Each seller is offering a distinct house for sale, and the valuations of the y x buyers for the houses are as follows. for for Buyer Value for Value Value b ’s house a ’s house c ’s house x 8 7 6 y 5 6 6 z 3 6 5 Suppose that charges a price of 2 for his house, b charges a price of 5 for his house, a and c charges a price of 4. Is this set of prices market-clearing? If so, explain which buyer you would expect to get which house; if not, say which seller or sellers should raise their price(s) in the next round of the bipartite-graph auction procedure from Chapter 10.

319 10.7. EXERCISES 305 a and , and a set of 2 buyers labeled x and 8. Suppose we have a set of 2 sellers labeled b y . Each seller is offering a distinct house for sale, and the valuations of the buyers for the houses are as follows. for for Buyer Value Value b ’s house a ’s house 5 x 7 1 y 4 Describe what happens if we run the bipartite graph auction procedure to determine market-clearing prices, by saying what the prices are at the end of each round of the auction, including what the final market-clearing prices are when the auction comes to an end. a 9. Suppose we have a set of 3 sellers labeled b , and c , and a set of 3 buyers labeled , , , and z . Each seller is offering a distinct house for sale, and the valuations of the x y buyers for the houses are as follows. Value for for Buyer Value for Value ’s house b a c ’s house ’s house x 3 6 4 y 2 1 8 1 3 z 2 Describe what happens if we run the bipartite graph auction procedure from Chap- ter 10, by saying what the prices are at the end of each round of the auction, including what the final market-clearing prices are when the auction comes to an end. (Note: In some rounds, you may notice that there are multiple choices for the con- A . Under the rules of the auction, you can choose any such stricted set of buyers constricted set. It’s interesting to consider — though not necessary for this question — how the eventual set of market-clearing prices depends on how one chooses among the possible constricted sets.) 10. Suppose we have a set of 3 sellers labeled a , b , and c , and a set of 3 buyers labeled x , y , and z . Each seller is offering a distinct house for sale, and the valuations of the buyers for the houses are as follows. Value for Value for Value for Buyer b ’s house ’s house ’s house c a x 7 4 9 y 5 9 7 z 11 10 8

320 306 CHAPTER 10. MATCHING MARKETS Describe what happens if we run the bipartite graph auction procedure from Chap- ter 10, by saying what the prices are at the end of each round of the auction, including what the final market-clearing prices are when the auction comes to an end. (Note: In some rounds, you may notice that there are multiple choices for the con- stricted set of buyers A . Under the rules of the auction, you can choose any such constricted set. It’s interesting to consider — though not necessary for this question — how the eventual set of market-clearing prices depends on how one chooses among the possible constricted sets.) c z b a y x Figure 10.15: The map for a parking-space market. (Image from Google Maps, http://maps.google.com/) 11. Figure 10.15 shows a map of part of the Back Bay section of Boston. Suppose that the dark circles labeled x , y , and z represent people living in apartments in Back Bay who want to rent parking spaces by the month for parking their cars. (Due to the density of buildings, these parking spaces may be a short walk from where they live, rather than right at their apartment.) The dark circles labeled a , b , and c represent parking

321 10.7. EXERCISES 307 spaces available for rent. Let’s define the distance between a person and a parking space to be the number of blocks they’d have to walk from their apartment to the parking space. Thus, for and c , while y example, c is at a distance of 2 from space x z is at a distance of 5 from c . (We’ll ignore the fact that the block between Gloucester is at a distance of 6 from and Hereford is a bit shorter than the others; all blocks will be treated as the same in counting distance.) Suppose that a person has a valuation for a potential parking space equal to − . 8 (their distance to the parking space) (Notice that this formula gives higher valuations to closer parking spaces.) In terms of these valuations, we’d like to think about prices that could be charged for the parking spaces. Describe how you would set up this question as a matching market in the style of (a) Chapter 10. Say who the sellers and buyers would be in your set-up, as well as the valuation each buyer has for the item offered by each seller. (b) Describe what happens if we run the bipartite graph auction procedure from Chapter 10 on the matching market you set up in (a), by saying what the prices are at the end of each round of the auction, including what the final market-clearing prices are when the auction comes to an end. (Note: In some rounds, you may notice that there are multiple choices for the con- stricted set of buyers. Under the rules of the auction, you can choose any such con- stricted set. It’s interesting to consider — though not necessary for this question — how the eventual set of market-clearing prices depends on how one chooses among the possible constricted sets.) At a more informal level, how do the prices you determined for the parking spaces (c) x in (b) relate to these spaces’ intuitive “attractiveness” to the people in apartments , y , and z ? Explain. 12. Suppose we have a set of 2 sellers labeled a and b , and a set of 2 buyers labeled x and y . Each seller is offering a distinct house for sale, and the valuations of the buyers for the houses are as follows. Buyer Value for for Value ’s house a b ’s house x 4 1 y 3 2

322 308 CHAPTER 10. MATCHING MARKETS In general, there will be multiple sets of market-clearing prices for a given set of sellers, buyers, and valuations: any set of prices that produces a preferred-seller graph with a perfect matching is market-clearing. As a way of exploring this issue in the context of the example above, give three different sets of market-clearing prices for this matching market. The prices should be whole , 1 , 2 , 3 , 4 , 5 , numbers (i.e. they should be numbers from 0 ,... ). (Note that for two sets 6 of market-clearing prices to be different, it is enough that they not consist of exactly the same set of numbers.) Explain your answer. 13. Suppose you want to design an auction for the following type of situation: you have two identical copies of a valuable object, and there are four potential buyers for the i wants at most one copy, and has a value v object. Each potential buyer for either i copy. You decide to design the auction by analogy with the way in which we derived the single-item ascending-bid (English) auction from the general procedure for matching markets. In the present case, as there, you want to create a bipartite graph that encodes the situation, and then see what prices the bipartite graph auction procedure comes up with. (a) Describe how this construction would work using an example with four potential buyers. In creating your example, first choose specific valuations for the potential buyers, and then show how the auction proceeds and what the market-clearing prices are. (b) In the case of the single-item auction, the bipartite graph procedure yielded the simple rule from the ascending-bid (English) auction: sell to the highest bidder at the second-highest price. Describe in comparably simple terms what the rule is for the current case of two identical items (i.e. your description should not involve the terms “bipartite”, “graph,” or “matching”). 14. In Chapter 10, we discussed the notion of social-welfare maximization for matching markets: finding a matching M that maximizes the sum of buyers’ valuations for what they get, over all possible perfect matchings. We can call such a matching social- welfare-maximizing . However, the sum of buyers’ valuations is not the only quantity one might want to maximize; another natural goal might be to make sure that no individual buyer gets a valuation that is too small. With this in mind, let’s define the baseline of a perfect matching M to be the minimum valuation that any buyer has for the item they get in M . We could then seek a perfect matching M whose baseline is as large as possible, over all possible perfect matchings. We will call such a matching baseline-maximizing .

323 10.7. EXERCISES 309 For example, in the following set of valuations, Buyer Value for Value for for Value ’s house ’s house c ’s house a b 7 9 x 4 y 5 9 7 11 10 z 8 the matching consisting of the pairs a - x , b - y , and c M z has a baseline of 8 (this is - the valuation of z for what she gets, which is lower than the valuations of x and y for ′ M - consisting of the pairs b - x , c what they get), while the matching y , and a - z has a baseline of 7. In fact the first of these example matchings, M , is baseline-maximizing for this sample set of valuations. Now, finding a perfect matching that is baseline-maximizing is grounded in a kind of “egalitarian” motivation — no one should be left too badly off. This may sometimes be at odds with the goal of social-welfare maximization. We now explore this tension further. (a) Give an example of equal-sized sets of sellers and buyers, with valuations on the buyers, so that there is no perfect matching that is both social-welfare-maximizing and baseline-maximizing. (In other words, in your example, social-welfare maximization and baseline maximization should only occur with different matchings.) It is also natural to ask whether a baseline-maximizing matching can always be (b) supported by market-clearing prices. Here is a precise way to ask the question. For any equal-sized sets of sellers and buyers, with valuations on the buyers, is there always a set of market-clearing prices so that the resulting preferred- seller graph contains a baseline-maximizing perfect matching M ? Give a yes/no answer to this question, together with a justification of your answer. (If you answer “yes,” you should explain why there must always exist such a set of market- clearing prices; if you answer “no,” you should explain why there can be examples in which a baseline-maximizing matching cannot be found in the preferred-seller graph resulting from market-clearing prices.) 15. Consider again the set-up for the bipartite graph auction, with an equal number of buyers and sellers, and with each buyer having a valuation for the object being sold by each seller. Suppose that we have an instance of this problem in which there is a particular seller i who is the favorite : every buyer j has a higher valuation for seller i ’s object than for the object being sold by any other seller k . (In notation, we have and v > v i for all choices of j = k 6 .) ij kj

324 310 CHAPTER 10. MATCHING MARKETS Consider a set of market-clearing prices in this situation. Must it be the case that the price charged by seller i is at least as high as the price charged by any other seller? Give an explanation for your answer.

325 Chapter 11 Network Models of Markets with Intermediaries 11.1 Price-Setting in Markets In Chapter 10 we developed an analysis of trade and prices on a bipartite graph consisting of buyers, sellers, and the edges connecting them. Most importantly, we showed that market- clearing prices exist, and that trade at these prices results in maximal total valuation among the buyers and sellers; and we found a procedure that allowed us to construct market- clearing prices. This analysis shows in a striking way how prices have the power to direct the allocation of goods in a desirable way. What it doesn’t do is provide a clear picture of where prices in real markets tend to come from. That is, who sets the prices in real markets, and why do they choose the particular prices they do? Auctions, which we discussed in Chapter 9, provide a concrete example of price determi- nation in a controlled setting. In our discussion of auctions, we found that if a seller with a single object runs a second-price sealed-bid auction — or equivalently an ascending-bid auction — then buyers bid their true values for the seller’s object. In that discussion, the buyers were choosing prices (via their bids) in a procedure selected by the seller. We could procurement auction in which the roles of buyers and sellers are reversed, also consider a with a single buyer interested in purchasing an object from one of several sellers. Here, our auction results imply that if the buyer runs a second-price sealed-bid auction (buying from the lowest bidder at the second-lowest price), or equivalently a descending-offer auction, then the sellers will offer to sell at their true costs. In this case, the sellers are choosing prices (their offers) in a procedure selected by the buyer. D. Easley and J. Kleinberg. Networks, Crowds, and Markets: Reasoning about a Highly Connected World . Cambridge University Press, 2010. Draft version: June 10, 2010. 311

326 312 CHAPTER 11. NETWORK MODELS OF MARKETS WITH INTERMEDIARIES But who sets prices, and who trades with whom, if there are many buyers and many sellers? To get a feel for what happens, let’s look first at how trade takes place in an actual market. Trade with Intermediaries. In a wide range of markets, individual buyers and sellers do not interact directly with each other, but instead trade through intermediaries — brokers, market-markers, or middlemen who set the prices. This is true in settings that range from the trade of agricultural goods in developing countries to the trade of assets in financial markets. To get a sense for how markets with intermediaries typically work, let’s focus on the latter example, and consider how buyers and sellers interact in the stock market. In the U.S., buyers and sellers trade over a billion shares of stock daily. But there is no one market for trade in stocks in the U.S. Instead, trade occurs on multiple exchanges such as the New York Stock Exchange (NYSE) or the NASDAQ-OMX, as well as on alternative trading systems such as those run by Direct Edge, Goldman Sachs, or Investment Technologies Group (ITG), which arrange trades in stocks for their clients. These markets operate in various ways: some (such as NYSE or NASDAQ-OMX) determine prices that look very much like our market-clearing prices from Chapter 10, while others (like Direct Edge, Goldman, or ITG) simply match up orders to buy and sell stocks at prices determined in other markets. Some have people (called specialists in the NYSE) directly involved in setting prices, while others are purely electronic markets with prices set by algorithms; some trade continuously throughout the day, while others trade less frequently as they wait for batches of buy and sell orders to arrive; some allow anyone at least indirect access to the market, while others restrict the group of buyers and sellers that they will deal with (often to large institutional traders). order book for each stock that they Many of these markets create something called an trade. An order book is simply a list of the orders that buyers and sellers have submitted for that stock. A trader might for instance submit an order to sell 100 shares if the price is at $5 or more per share; another trader might submit an order to sell 100 shares if the price is at $5.50 or more per share. Two other traders might submit orders to buy 100 shares if the price is no more than $4 per share, and to buy 100 shares if the price is no more than $3.50 per share. Orders of this type are called limit orders , since they are commitments to buy or sell only once the price reaches some limit set by the trader. If these were the only orders that existed, then the order book for this stock would look like Figure 11.1(a). The highest outstanding offer to buy the stock is referred to as the current bid for the stock, while the lowest outstanding offer to sell it is referred to as the ask . If the market uses a specialist then this person knows the book of orders, and may choose to submit his or her own better offer to buy or sell out of inventory of the stock, which becomes the bid or ask respectively. For example, if Figure 11.1(a) describes the order book, then the specialist

327 11.1. PRICE-SETTING IN MARKETS 313 $5.50 ASK $5.50 ASK $5.00 $4.00 BID $4.00 BID $3.50 $3.50 (a) (b) Figure 11.1: (a) A book of limit orders for a stock with a bid of $4 and an ask of $5. (b) A book of limit orders for a stock with a bid of $4 and an ask of $5.50. may choose to display a bid of $4.25, based on his or her own better offer, and an ask of $5.00. These are the prices displayed to the trading public. Retail traders (small traders who buy and sell stocks using their own assets) most often do not submit limit orders; instead they typically submit orders to buy or sell at the existing quotes — the current bid and ask. This type of order to trade immediately at market prices market order . For example, if a trader submits a market order to buy 100 shares is called a of a stock with the order book described by Figure 11.1(a), then the seller whose limit order was placed at $5.00 sells 100 shares and the buyer who submitted the market order buys 100 shares at $5.00. (Note that the seller can be either a member of the public or the specialist.) The new order book would then be as displayed in Figure 11.1(b), and new ask would be $5.50. This process continues throughout the trading day with new limit orders, specialist offers, and market orders arriving over time, and transactions being performed. Of course, orders to buy or sell are not always for 100 shares, and in fact order sizes vary greatly. For example, if the order book is as depicted in Figure 11.1(a), and a market order to buy 200 shares arrives then both sellers on the book sell at their ask prices. The buyer will buy 100 shares at $5.00 and 100 shares at $5.50. We can think of this order as “walking up the book,” since executing it exposes multiple orders at different prices. Large mutual funds such as Fidelity or Vanguard, and other institutional traders such as banks, pension funds, insurance companies and hedge funds, buy and sell a very large number of shares each day. They don’t really want to trade many small lots of shares with retail traders and, as in our 200-share example, walk up or down the book. They also don’t want to submit a single large limit order to the market, as then other market participants

328 314 CHAPTER 11. NETWORK MODELS OF MARKETS WITH INTERMEDIARIES 1 will know their trading desires, and can take advantage of them. Instead of submitting a single large market or limit order these traders use a variety of orders and trading venues. They typically split their order into many pieces and trade these pieces over the trading day, or over several days, in order to minimize the impact of their trading desire on the price. One way in which large traders hide their trading desires is to submit pieces of it to many different trading systems to which they have access. One particularly interesting group of dark pools trading systems are called . Examples of these alternative trading systems are Goldman Sachs’s Sigma-X and the systems run by ITG. Access to these systems is limited and orders submitted to these systems are not displayed to the public. Instead these systems simply match orders submitted by their clients at prices established in the public market, and charge their clients a fee for the service. This is a relatively new, but growing segment of the market; in April 2009, for example, approximately 9% of the trade in U.S. equities was done on dark pools. As you might imagine, the actual structure of the stock market is very complex and rapidly evolving. There are many trading systems, many types of orders that buyers and sellers can use, and a wide variety of market participants. The questions of how prices evolve over time and how they relate to the underlying fundamental value of the assets being traded are also important, and we have ignored these issues so far. We will discuss some aspects of the evolution of prices and their relation to the underlying values in Chapter 22; more detailed analyses of the stock market are carried out in a number of books [206, 209, 332]. The collection of different trading venues for stocks ultimately results in a variety of markets with restricted participation. So when we take into account the full set of trading options for all market participants — both large and small — we see a network structure emerge, connecting buyers and sellers to different possible intermediaries. A fundamental question is how to reason about trade when there are multiple markets connected by a network in this way. In the next section, we develop a network model for trade which abstracts away the specific details of the stock market, focusing on the general issue of how the underlying structure constrains who can trade with whom, and how prices are set by market participants. 1 A large order to buy, for example, may provide information to other market participants suggesting that the stock is currently undervalued and its price is likely to increase. These other market participants may then jump into the market, perhaps getting ahead of the execution of some part of the large order, and drive the price up quickly. This would harm the trader who submitted the large order as he may then have to pay more than expected for his order. This is related to broader questions about the role of information in markets, a topic we discuss in Chapter 22.

329 11.2. A MODEL OF TRADE ON NETWORKS 315 11.2 A Model of Trade on Networks Our network model will be based on three fundamental principles that we saw in discussing the stock market: Individual buyers and sellers often trade through intermediaries, not all buyers and sellers have access to the same intermediaries, and not all buyers and sellers trade at the same price. Rather, the prices that each buyer and seller commands are determined in part by the range of alternatives that their respective network positions provide. Before specifying the model, let’s first look at another example of trade, in a very different setting, that exhibits these properties. This is the market for agricultural goods between local producers and consumers in a developing country. In many cases there are middlemen, or traders, who buy from farmers and then resell to consumers. Given the often poor transportation networks, the perishability of the products and limited access to capital by farmers, individual farmers can sell only to a limited number of intermediaries [46, 153]. Similarly, consumers can buy from only a limited number of intermediaries. A developing country may have many such partially overlapping local markets existing alongside modern, more global markets. We can use a graph to describe the trading opportunities available to sellers, buyers, and middlemen (traders). Figure 11.2 depicts a simple example of such a trading network, superimposed on its geographic setting. Here we’ve labeled seller nodes with S, buyer nodes with B, and trader nodes with T; and we’ve placed an edge between any two agents who can trade with each other. Notice in this example that the seller and buyer on the right-hand margin of the picture only have access to the trader on their side of the river. The buyer at the top of the figure has access to both traders — perhaps he has a boat. You might imagine that the extra trading opportunities available to this buyer, and the similar extra trading opportunities available to the seller on the west bank of the river, would result in better prices for them. We will see that this is exactly what happens in the trading outcomes determined by our model in networks of this type. Network Structure. We now describe a simple model of trade on a network which is general enough to incorporate important features of the trading and price-setting process for commodities as varied as financial assets traded in developed countries and agricultural goods in developing countries [63]. For the simplest form of the model, we don’t try to address the issue of multiple goods for sale, or multiple possible quantities; instead, we assume there is a single type of good that comes in indivisible units. Each seller i initially holds one unit of the good which he v ; he is willing to sell it at any price that is at least v values one . Each buyer j values at i i copy of the good at v , and will try to obtain a copy of the good if she can do it by paying j no more than v . No individual wants more than one copy of the good, so additional copies j are valued at 0. All buyers, sellers, and traders are assumed to know these valuations. As a

330 316 CHAPTER 11. NETWORK MODELS OF MARKETS WITH INTERMEDIARIES B T S S B T S B Figure 11.2: Trading networks for agricultural markets can be based on geographic con- straints, giving certain buyers (nodes labeled B) and sellers (nodes labeled S) greater access to traders (nodes labeled T). result, this model is best thought of as describing interaction between individuals who have a history of trade with each other, and hence know each other’s willingness to pay for goods. Trade takes place on a network that represents who can trade with whom. As in the example depicted in Figure 11.2 the nodes consist of buyers, sellers, and traders, with each edge representing an opportunity for trade. Since we are assuming that the traders act as intermediaries for the possible seller-buyer transactions, we require that each edge connects a buyer or seller to a trader. In Figure 11.3, we depict the same graph from Figure 11.2, redrawn to emphasize these features of the network model. (In all of our figures depicting trading networks we will use the following conventions. Sellers are represented by circles on the left, buyers are represented by circles on the right, and traders are represented by squares in the middle. The value that each seller and buyer places on a copy of the good is written next to the respective node that represents them.) Beyond the fact that we now have intermediaries, there are a few other differences between this model and our model of matching markets from Chapter 10. First, we are assuming

331 11.2. A MODEL OF TRADE ON NETWORKS 317 i v buyers j v sellers traders 1 B1 0 S1 T1 1 0 B2 S2 T2 1 0 S3 B3 Figure 11.3: A standardized view of the trading network from Figure 11.2: Sellers are represented by circles on the left, buyers are represented by circles on the right, and traders are represented by squares in the middle. The value that each seller and buyer places on a copy of the good is written next to the respective node that represents them. that buyers have the same valuation for all copies of a good, whereas in matching markets we allowed buyers to have different valuations for the goods offered by different sellers. The model in this chapter can be extended to allow for valuations that vary across different copies of the good; things become more complicated, but the basic structure of the model and its conclusions remain largely the same. A second difference is that the network here is fixed and externally imposed by constraints such as geography (in agricultural markets) or eligibility to participate (in different financial markets). In matching markets, we began the chapter with fixed graphs such as this, but then focused the core of the analysis on preferred-seller graphs that were determined not by external forces but by the preferences of buyers with respect to an evolving set of prices. Prices and the Flow of Goods. The flow of goods from sellers to buyers is determined by a game in which traders first set prices, and then sellers and buyers react to these prices. Specifically, each trader t offers a bid price to each seller i that he is connected to; we will denote this bid price by b . (The notation indicates that this is a price for a transaction ti to buy and i ). This bid price is an offer by between t i ’s copy of the good at a value of b . t ti Similarly, each trader t offers an ask price to each buyer j that he is connected to. This ask j price, denoted . In , is an offer by t to sell a copy of the good to buyer a at a value of a tj tj Figure 11.4(a), we show an example of bid and ask prices on the graph from Figure 11.3.

332 318 CHAPTER 11. NETWORK MODELS OF MARKETS WITH INTERMEDIARIES sellers traders buyers buyers traders sellers B1 0 B1 S1 0 1 S1 1 0.8 0.8 0.2 0.2 T1 T1 0.2 0.2 0.8 0.8 S2 B2 S2 1 0 0 B2 1 0.3 0.7 0.3 0.7 T2 T2 0 1 0 1 0 1 S3 B3 0 1 S3 B3 Prices (a) Flow of goods (b) Figure 11.4: (a) Each trader posts bid prices to the sellers he is connected to, and ask prices to the buyers he is connected to. (b) This in turn determines a flow of goods, as sellers and buyers each choose the offer that is most favorable to them. Once traders announce prices, each seller and buyer chooses at most one trader to deal with — each seller sells his copy of the good to the trader he selects (or keeps his copy of the good if he chooses not to sell it), and each buyer purchases a copy of the good from the trader she selects (or receives no copy of the good if she does not select a trader). This determines a flow of goods from sellers, through traders, to buyers; Figure 11.4(b) depicts such a flow of goods, with the sellers’ and buyers’ choices of traders indicated by the edges with arrows on them. Because each seller has only one copy of the good, and each buyer only wants one copy, at most one copy of the good moves along any edge in the network. On the other hand, there is no limit on the number of copies of the good that can pass through a single trader node. Note that a trader can only sell as many goods to buyers as he receives from sellers; we will include in the model a large penalty imposed on a trader who defaults on an offer to sell to a buyer as a result of not having enough goods on hand. Due to this, there are strong incentives for a trader not to produce bid and ask prices that cause more buyers than sellers to accept his offers. There are also incentives for a trader not to be caught in the reverse difficulty, with more sellers than buyers accepting his offers — in this case, he ends up with excess inventory that he cannot sell. We will see that neither of these will happen in the solutions we consider; traders will choose bid and ask prices such that the number of goods they receive from sellers is equal to the number of goods they pass on to buyers. Finally, notice something else about the flow of goods in this example: seller S 3 accepts the bid even though it is equal to his value, and likewise buyer B 3 accepts the ask even

333 11.2. A MODEL OF TRADE ON NETWORKS 319 S 3 and 3 is indifferent between accepting and though it is equal to his value. In fact, each of B rejecting the offer. Our assumption in this model is that when a seller or buyer is indifferent between accepting or rejecting, then we (as the modelers) can choose either alternative as the outcome that actually happens. Finding a way to handle indifference is an important aspect in most market models, since transactions will typically take place right at the boundary of an individual’s willingness to trade. This is similar to the tie-breaking issue inherent in the formulation of market-clearing prices in Chapter 10 as well. An alternate way to handle indifference in the present case is to assume a miniscule positive amount of payoff (e.g. a penny) that is required for an agent to be willing to trade, in which case we would see bid . 01 and 0 . 99. While this makes the tie-breaking decision more explicit, and ask values like 0 the model becomes much messier and ultimately harder to reason about. As a result, we will stick to the approach where we allow trades at zero payoff, with ties broken as needed; in doing so, we will remember that this is essentially a formal way to represent the idea of a price or a profit margin being driven to (almost) zero. And if it makes things simpler to think about, whenever you see an indifferent seller or buyer choosing to transact or not, you . 01 to account for this can imagine the price being shifted either upward or downward by 0 decision by the seller or buyer. Payoffs. Recall that specifying a game requires a description of the strategies and the payoffs. We have already discussed the strategies: a trader’s strategy is a choice of bid and ask prices to propose to each neighboring seller and buyer; a seller or buyer’s strategy is a choice of a neighboring trader to deal with, or the decision not to take part in a transaction. The payoffs follow naturally from the discussion thus far. • A trader’s payoff is the profit he makes from all his transactions: it is the sum of the ask prices of his accepted offers to buyers, minus the sum of the bid prices of his accepted offers to sellers. (As discussed above, we also subtract a large penalty if the trader has more accepted asks than bids, but the effect of this is primarily to ensure that traders will never expose themselves to this situation in the solutions we consider.) • For a seller i , the payoff from selecting trader t is b , while the payoff from selecting ti no trader is . In the former case, the seller receives b v units of money, while in the i ti latter he keeps his copy of the good, which he values at . (We will consider only cases v i in which all the seller v ’s are 0.) i For each buyer j , the payoff from selecting trader t is v − a • , while the payoff from tj j selecting no trader is 0. In the former case, the buyer receives the good but gives up units of money. a tj So for example, with prices and the flow of goods as in Figure 11.4(b), the payoff to the first trader is (0 . 8 − 0 . 2) = 0 4. 6 while the payoff to the second trader is (0 . 7+1 − 0 . 3 − 0) = 1 . .

334 320 CHAPTER 11. NETWORK MODELS OF MARKETS WITH INTERMEDIARIES buyers sellers traders 1 S1 B1 0 1 0 T1 0.4 0.6 1 0 S2 B2 0.3 0.7 T2 1 0 1 0 S3 B3 Figure 11.5: Relative to the choice of strategies in Figure 11.4(b), trader T 1 has a way to T 2 and performing the transaction that moves S improve his payoff by undercutting 2’s copy of the good to B 2. The payoffs to the three sellers are 0 . 2, 0 . 3, and 0 respectively, while the payoffs to the three buyers are 1 − 0 . 8 = 0 . 2, 1 − 0 . 7 = 0 . 3, and 1 − 1 = 0 respectively. The game we’ve defined here has a further important feature, which forms a contrast with other games we have discussed earlier. In earlier games, all players moved (i.e. executed their chosen strategies) simultaneously, while in this game the moves happen in two stages. In the first stage, all the traders simultaneously choose bid and ask prices. In the second stage, all the sellers and buyers then simultaneously choose traders to deal with. For us, this two-stage structure will not make things too complicated, particularly since the second stage is extremely simple: the best response for each seller and buyer is always simply to choose the trader with the best offer, and so we can essentially view the sellers and buyers as “drones” who are hard-wired to follow this rule. Still, we will have to take the two-stage structure into account when we consider the equilibria for this game, which we do next. Let’s think about the strategies that the two traders Best Responses and Equilibrium. T 1 is making several bad decisions. First, have chosen in Figure 11.4(b). The upper trader because of the offers he is making to seller S 2 and buyer B 2, he is losing out on this deal to the lower trader T 2. If for example he were to raise his bid to seller S 2 to 0 . 4, and lower his 2 and buyer ask to buyer 2 to 0 . 6, then he’d take the trade away from trader T 2: seller S B B 2 would both choose him, and he’d make a profit of 0 . 2. Second, and even more simply, there is no reason for trader T 1 not to lower his bid to seller

335 11.2. A MODEL OF TRADE ON NETWORKS 321 1, and raise his ask to buyer B S 1 and B 1 will still want to deal S 1. Even with worse offers, 1, since they have no other options aside from choosing not to transact. Given this, T with 1 will make more money with a lower bid to S 1 and a higher ask to B 1. Figure 11.5 shows T the results of a deviation by the upper trader that takes both of these points into account; his payoff has now increased to (1 + 0 6 − 0 − 0 . 4) = 1 . 2. Note that seller S 1 and buyer B 1 . are now indifferent between performing the transaction or not, and as discussed earlier, we give ourselves (as the modelers) the power to break ties in determining the equilibrium for such situations. This discussion motivates the equilibrium concept we will use for this game, which is a generalization of Nash equilibrium. As in the standard notion of Nash equilibrium from Chapter 6, it will be based on a set of strategies such that each player is choosing a best response to what all the other players are doing. However, the definition also needs to take the two-stage structure of the game into account. To do this, we first think about the problem faced by the buyers and sellers in the second stage, after traders have already posted prices. Here, we have a standard game among the buyers and sellers, and each of them chooses a strategy that is a best response to what all other players are doing. Next, we think about the problem faced by the traders, in deciding what prices to post in the first stage. Here, each trader chooses a strategy that is a best response both to the strategies the sellers and buyers will use (what bids and asks they will accept) and the strategies the other traders use (what bids and asks they post). So everyone is employing a best response just as in any Nash equilibrium. The one difference here is that since the sellers and buyers move second they are required to choose optimally given whatever prices the traders have posted, and the traders know this. This equilibrium is called a subgame perfect Nash equilibrium ; in this chapter, we will simply refer to it as an 2 . equilibrium The two-stage nature of our game here is particularly easy to think about, since the behavior of sellers and buyers is very simple. Thus, for purposes of reasoning about equilibria, we can mainly think about the strategies of the traders in the first stage, just as in a simultaneous-move game, knowing that sellers and buyers will simply choose the best offers (possibly with tie-breaking) once the traders post prices. In the next section, we’ll work out the set of possible equilibria for the trading network in Figures 11.3–11.5, by first dissecting the network into simpler “building blocks.” In particular, these building blocks will correspond to two of the basic structures contained within the network in Figures 11.3–11.5: buyers and sellers who are monopolized by having 2 The word “subgame” refers to the fact that once traders post prices, the buyers and sellers are faced with a new free-standing game in the second stage. The word “perfect” refers to the requirement that in the subgame, the players who have choices remaining are required to behave optimally given the choices that have already been made. This concept is considered at a general level, although without this particular terminology, in the discussion of games with sequential moves in Section 6.10.

336 322 CHAPTER 11. NETWORK MODELS OF MARKETS WITH INTERMEDIARIES sellers traders buyers 1 0 0 1 T1 S1 B1 Figure 11.6: A simple example of a trading network in which the trader has a monopoly and extracts all of the surplus from trade. only a single trader they can deal with, and buyers and sellers who benefit from perfect competition between multiple traders. In the process, we’ll see that network structure and access to alternatives can significantly affect the power of participants in the market. 11.3 Equilibria in Trading Networks We now discuss the process of analyzing equilibria in trading networks. We begin with simple network structures and build up to the example from the previous section. Following our plan, we’ll begin by considering simple networks corresponding to monopoly and perfect competition. Monopoly. Buyers and sellers are subject to monopoly in our model when they have access to only a single trader. Perhaps the simplest example of this is depicted in Figure 11.6. Here we have one seller who values the good at 0, one trader, and one buyer who values the good at 1. In this trading network the trader is in a monopoly position relative to both the seller and the buyer (there is only one trader available to each of them). The only equilibrium is for the trader to set a bid of 0 to the seller and an ask of 1 to the buyer; the seller and buyer will accept these prices and so the good will flow from the seller to the trader and then on to the buyer. Note that we are using the indifference of the seller and buyer as in the example from the previous section: since the seller and buyer are indifferent between engaging in a transaction or not, we as the modelers are choosing the outcome and having them perform the transaction. To see why this is the only equilibrium, we simply notice that with any other bid and ask between 0 and 1, the trader could slightly lower the bid or raise the ask, thereby performing the transaction at a higher profit.

337 11.3. EQUILIBRIA IN TRADING NETWORKS 323 sellers traders buyers T1 x x 0 1 B1 S1 x x T2 Figure 11.7: A trading network in which there is perfect competition between the two traders, T1 and T2. The equilibrium has a common bid and ask of x , where x can be any real number between 0 and 1. Perfect Competition. Now let’s look at a basic example showing perfect competition between two traders, as depicted in Figure 11.7. T T 2 to buy the copy of the In Figure 11.7 there is competition between traders 1 and S 1 and sell it to B good from 1. To help in thinking about what forms an equilibrium, let’s first think about things that are out of equilibrium, in a manner similar to what we saw in Figure 11.5. In particular, suppose trader T 1 is performing the trade and making a positive profit: suppose his bid to the seller is some number b , and his ask to the buyer is a number a > b . Since T 2 is not performing the trade, he currently has a payoff of zero. But then it must be that T T 1 is doing: T 2 could 2’s current strategy is not a best response to what b instead offer a bid slightly above a , thereby taking the trade and an ask slightly below away from T 1 and receiving a positive payoff instead of zero. So it follows that whichever trader is performing the trade at equilibrium must have a payoff of 0: he must be offering the same value x as his bid and ask. Suppose that trader T 1 is performing the trade. Notice that this equilibrium involves indifference on his part: he is indifferent between performing the trade at zero profit and not performing the trade. As in the earlier case of indifference by sellers and buyers, we assume that we (the modelers) can choose an outcome in this case, and we will assume that the transaction is performed. Here too, we could handle indifference by assuming a minimum increment of money (e.g. 0 . 01), and having the transaction take place with a bid and ask of x − 0 . 01 and x respectively, but again handling indifference via zero payoffs (and keeping in mind that they are designed to model profits that come arbitrarily close to 0) makes the analysis simpler without affecting

338 324 CHAPTER 11. NETWORK MODELS OF MARKETS WITH INTERMEDIARIES the outcome. Next we want to argue that the trader not performing the trade at equilibrium ( T 2 in . First, notice that in equilibrium, we cannot x this case) must also have bid and ask values of have a trader buy the good from the seller without also selling it to the buyer; therefore, b ≤ x , (or else the seller would sell to T T a ≥ x . (or 2 must be offering a bid 2) and an ask T 2). But if this bid and ask were not the same — that is, if else the buyer would buy from — then T a > b a 1 could lower his bid or raise his ask so that they still lie strictly between and . In that case T 1 could perform the trade while making a positive profit, and hence his b x T 2 is doing. current strategy of bidding and asking would not be a best response to what x . What can we say about the So the equilibrium occurs at a common bid and ask of value of ? It clearly has to be between 0 and 1: otherwise either the seller wants to sell but x the buyer wouldn’t want to buy, or conversely the buyer wants to buy but the seller wouldn’t want to sell. In fact this is all that we can say about x . Any equilibrium consists of a common bid and ask by each trader, and a flow of goods from the seller to the buyer through one of the traders. A key feature of the equilibrium is that the seller sells to the same trader that the buyer buys from: this is another kind of coordination in the face of indifference that is reminiscent of the tie-breaking issues in market-clearing prices from Chapter 10. It is also interesting that while traders make no profit in any equilibrium, the choice of equilibrium x — determines which of the seller or buyer receives a higher — captured in the value of payoff. It ranges from the extreme cases of x = 0 (where the buyer consumes all the available 1 (where = 1 (where the seller consumes it) to the intermediate value of x payoff) and x = 2 the seller and buyer receive equal payoffs). In the end, the choice of equilibrium reflects something about the relative power of the seller and buyer that can only be inferred by looking outside the formulation of the trading game — the game itself can determine only the range of possible equilibria. The Network from Section 11.2. Using the networks in Figures 11.6 and 11.7 as build- ing blocks, it is not hard to work out the equilibria in the example from Section 11.2. This is illustrated in Figure 11.8. Sellers S 1 and S 3, and buyers B 1 and B 3, are monopolized by their respective traders, and so in any equilibrium these traders will drive the bids and asks all the way to 0 and 1 respectively. Seller S 2 and buyer B 2, on the other hand, benefit from perfect competition between the two traders. Here the argument follows what we used in analyzing the simpler network in Figure 11.7: the trader performing the transaction must have bid and ask values equal to the same number x (for some real number x between 0 and 1), or else the other trader could take the trade away from him; and given this, the other trader must also have bid and ask values equal to x . These types of reasoning are useful in analyzing other more complex networks as well.

339 11.3. EQUILIBRIA IN TRADING NETWORKS 325 traders sellers buyers 1 B1 S1 0 0 1 T1 x x S2 B2 1 0 x x T2 0 1 1 S3 B3 0 Figure 11.8: The equilibria for the trading network from Section 11.2. This network can be analyzed using the ideas from the simpler networks representing monopoly and perfect competition. When you see a seller or buyer connected to only a single trader, they will receive zero payoff in any equilibrium, since the trader will drive the bid or ask to as extreme a value as possible. On the other hand, when two traders both connect the same seller and buyer, then neither can make a positive profit in conveying a good from this seller to this buyer: if one trader performed the trade at a positive profit, the other could undercut them. We now consider an example illustrating how the network structure can also produce more complex effects that are not explained by these two principles. Implicit Perfect Competition. In our examples so far, when a trader makes no profit from a transaction, it is always because there is another trader who can precisely replicate the transaction — i.e., a trader who is connected to the same seller and buyer. However, it turns out that traders can make zero profit for reasons based more on the global structure of the network, rather than on direct competition with any one trader. The network in Figure 11.9 illustrates how this can arise. In this trading network there is no direct competition for any one “trade route” from a seller to a buyer. However, in any equilibrium, all bid and ask prices take on some common value x between 0 and 1, and the goods flow from the sellers to the buyers. So all traders again make zero profit. It is easy to see that this is an equilibrium: we can simply check that each trader is using a best response to all the other traders’ strategies. It takes a bit more work to verify that in every equilibrium, all bid and ask prices are the same value x : this is most easily done by

340 326 CHAPTER 11. NETWORK MODELS OF MARKETS WITH INTERMEDIARIES 0 S1 x x T1 x 1 B1 x T2 x T3 x x 1 B2 x T4 0 S2 Figure 11.9: A form of implicit perfect competition : all bid/ask spreads will be zero in equilibrium, even though no trader directly “competes” with any other trader for the same buyer-seller pair. checking alternatives in which some trader posts a bid that is less than the corresponding ask, and identifying a deviation that arises. 11.4 Further Equilibrium Phenomena: Auctions and Ripple Effects The network model we’ve been considering is expressive enough that it can represent a diverse set of other phenomena. Here we consider two distinct examples: the first showing how the second-price auction for a single item arises from a trading network equilibrium, and the second exploring how small changes to a network can produce effects that ripple to other nodes. Second-price auctions. Figure 11.10 shows how we can represent the structure of a single-item auction using a trading network. Suppose there is a single individual 1 with S an item to sell, and four potential buyers who value the item at values w,x,y,z , listed in descending order w > x > y > z . We use four buyers in this example but our analysis would work for an arbitrary number of buyers. In keeping with our model in which trading happens through intermediaries, we assume

341 11.4. FURTHER EQUILIBRIUM PHENOMENA: AUCTIONS AND RIPPLE EFFECTS 327 w B1 B1 w T1 T1 w x x x T2 T2 x B2 x B2 0 S1 0 S1 y y B3 T3 B3 y T3 y z z B4 z B4 T4 z T4 (a) Equilibrium for the auction A single-item auction (b) Figure 11.10: (a) A single-item auction can be represented using a trading network. (b) Equilibrium prices and flow of goods. The resulting equilibrium implements the second- price rule from Chapter 9. that each buyer is represented by a distinct trader — essentially, someone who serves as the buyer’s “proxy” for the transaction. This gives us a trading network as depicted in Figure 11.10(a). T 1 has the ability to Now let’s consider a possible equilibrium for this network. Trader outbid all the other traders, since he has ability to sell to his buyer for an ask up to . In w equilibrium, he will outbid them by the minimum he needs to in order to make the trade, which he can do by offering x to outbid T 2. Here we use indifference to assume that the sale at x will go to T 1 rather than T 2, that buyer B 1 will buy from T 1 at a price of w , and that buyers B B 4 will choose not to buy the good from their respective traders. 2 through We therefore get the equilibrium depicted in Figure 11.10(b). Notice how this equilibrium has exactly the form of a second-price auction, in which the item goes to the highest bidder, 3 What’s interesting is with the seller receiving the second-highest valuation in payment. 3 With a little more work, we can describe the full set of equilibria for this network, and show that the second-price rule is unique over equilibria that avoid a certain “pathological” structure, as follows. In B 1, and each trader offers to sell the good to his any equilibrium, the good flows from the seller to buyer monopolized buyer for the buyer’s value. In the equilibrium we consider, T T 2 both bid x . However, 1 and T 2, T 3, or there are other bids that can be part of an equilibrium: Essentially, as long as one of traders T 4 bids between x and w , and T 1 matches this bid, we have an equilibrium. If the highest bid among T 2, T T 4 is strictly greater than x , then we have a situation in which this high-bidding trader among T 2 3, and through T 4 has a crossing pair of bid and ask values: his bid is higher than his corresponding ask. This is an equilibrium, since T 1 still makes the trade, the trader with the crossing bid/ask pair doesn’t lose money, and no one has an incentive to deviate. However, it is a pathological kind of equilibrium since there is a trader who is offering to buy for more than he is offering to sell [63]. Thus, if we consider only equilibria

342 328 CHAPTER 11. NETWORK MODELS OF MARKETS WITH INTERMEDIARIES 0 1 S1 B1 0 1 S1 B1 1 1 0 0 T1 T1 z y 0 x 2 B2 B2 2 0 S2 y S2 0 x z T2 T2 3 3 0 0 3 B3 B3 3 4 4 0 S3 S3 0 4 B4 4 B4 (a) (b) Equilibrium after new link. Equilibrium before new link. Figure 11.11: (a) Equilibrium before the new S 2- T 2 link is added. (b) When the S 2- T 2 edge is added, a number of changes take place in the equilibrium. Among these changes is the fact that buyer 1 no longer gets a copy of the good, and B 3 gets one instead. B that the second-price rule wasn’t in any sense “built in” to the formulation of the auction; it emerged naturally as an equilibrium in our network representation. Our network model also allows us to Ripple Effects from Changes to a Network. explore how small changes to the network structure can affect the payoffs of nodes that are not directly involved in the change. This suggests a way of reasoning about how “shocks” to highly interconnected trading networks can ripple to more distant parts of the network. This is a very general issue; we consider it here in a specific example where we can see concretely how such effects can arise. Consider the pair of networks in Figure 11.11: the second network is obtained from the first simply by adding a link from 2 to T 2. We first S work out the equilibria for each of these networks, and then consider how they differ. In Figure 11.11(a), all sellers and buyers are monopolized except for B 2, so their payoffs will all be zero; we use indifference to assume that B 3 will not buy the good but B 1 and B 4 will. (As in all our examples, we can view this as modeling the fact that T 2 can charge a B 3, dissuading price very slightly above 3 to 3 from purchasing.) The one part that needs B additional analysis is the pair of asks charged to B 2. In equilibrium, these must be the same (otherwise, the trader making the sale could slightly raise his ask), and this common value x can be anything between 0 and 2. We exploit indifference here to assume that B 2 buys from without such crossing pairs of bid/ask values, then T 2 bids the “second-price value” x , and this is what the good sells for. So the second-price rule is unique over equiibria without crossing pairs.

343 11.4. FURTHER EQUILIBRIUM PHENOMENA: AUCTIONS AND RIPPLE EFFECTS 329 T 1. Note that there cannot be an equilibrium in which buyer 2 buys from trader trader B B T 2 can sell the unit of the good he is able to buy 2 can pay only 2 while trader 2, since T at a price of 4. S T 2 has been added, we need to work out the In Figure 11.11(b), once the edge from 2 to 2 and B 2, and the flow of goods. Reasoning about equilibrium bids and asks charged to S these bids and asks requires a bit more work than we’ve seen in previous examples, so we build up to it in a sequence of steps. S 2 must be the same as each other, since otherwise the trader getting • The two bids to 2 must B the good could slightly lower his bid; for a similar reason, the two asks to z and the common ask y . also be the same as each other. Let’s call the common bid We can next determine how the seller-trader transactions work out in equilibrium. S 2 • T will sell to T 1 in equilibrium: if S 2 were selling to T 1, and T 1 were 2 rather than S receiving a positive payoff from this transaction, then 2 would be selling for at most T 2 could slightly outbid T 1 and sell S 2’s copy of the good to B 3. So 2. In this case, T 2 buys two copies of the good, while T 1 buys only one. in equilibrium, Now let’s figure out the possible values for the ask y • y must be at least 1: . The ask otherwise, one of the traders is selling to B 2 for a low price, and the trader performing this sale has an alternate trader whom he monopolizes, and from whom he would get a higher payoff. This can’t happen in equilibrium, so y is at least 1. Also, the ask y B 2 would not buy, and cannot be above 2 in equilibrium: in this case, T 1 could perform a payoff-improving change in strategy by lowering his ask to 2, so B 2 to buy from T B thereby getting 1 for a price between 1 and 2. • Next, we determine how the trader-buyer transactions work out in equilibrium. We’ve already concluded that T 2 is buying two copies of the good, and so he maximizes his B payoff by selling them to B 4. Therefore T 2 is not selling to B 2 in equilibrium. 3 and Since, the ask y is at least 1, trader T 1 will buy from S 1 and sell to B 2. • Finally, what do we know about the value of z ? It has to be at least 1, or else T 1 could outbid T S 2’s copy of the good, and receive a positive payoff by selling it to B 1. 2 for It also has to be at most 3, or else 2 would prefer not to buy it from S 2. T This sums up the analysis: in equilibrium, the bid z can be anything between 1 and 3, the ask y can be anything between 1 and 2, and the goods flow from S 1 through T 1 to B 2, and B from 2 and S 3 through T 2 to S 3 and B 4. Notice that this flow of goods maximizes the total valuation of the buyers who obtain the good, given the constraints on trade imposed by the network. We will see later in this chapter that any equilibrium has this efficiency property.

344 330 CHAPTER 11. NETWORK MODELS OF MARKETS WITH INTERMEDIARIES Let’s consider at a high level what’s going on in this pair of examples. In Figure 11.11(a), trader T 2 has access to a set of buyers who want copies of the good very badly (they value 1 on the other hand is able to use all T it highly), but his access to sellers is very limited. his available trading opportunities. The market in this respect has a “bottleneck” that is restricting the flow of goods. S 2 and T 2 form a link, creating the network in Figure 11.11(b), a number of things Once B 3 now gets a copy of the good while B 1 doesn’t. change. First, and most noticeably, buyer Essentially, the bottleneck in the market has been broken open, so the high-value buyers can now obtain the good at the expense of the low-value buyers. From 1’s perspective, this B is a “non-local” effect: a link formed between two nodes, neither of which are neighbors of hers, has caused her to no longer be able to obtain the good. S 2 is now in a much more powerful position, and There are other changes as well. Seller y will command a significantly higher price (since is at least 1 in any equilibrium). Moreover, the range of possible equilibrium asks to B 2 has been reduced from the interval [0 , 2] to the interval [1 , 2]. So in particular, if we were previously in an equilibrium where the ask to B 2 was a value x < 1, then this equilibrium gets disrupted and replaced by one in which the ask y 1. This indicates a subtle way in which B 2 was implicitly benefitting is a higher number ≥ from the weak position of the sellers, which has been now strengthened by the creation of S the edge between T 2. 2 and This is a simple example, but it already illustrates some of the complexities that can arise when the structure of a trading network changes to alleviate (or create) bottlenecks for the flow of goods. With more work, one can create examples where the effects of changes in the network ripple much farther through the structure. This style of reasoning also points to a line of questions in which we view the network as something malleable, and partially under the control of the market participants. For example, how much should 2 and T 2 be willing to spend to create a link between each other, S shifting the network structure from the one in Figure 11.11(a) to the one in Figure 11.11(b)? More generally, how should different nodes evaluate the trade-offs between investing resources to create and maintain links, and the benefits they get in terms of increased payoffs? This is a question that has been considered in other models of trading networks [150, 261], and it is part of a much broader research activity that investigates the formation of networks as a game-theoretic activity under a variety of different kinds of payoffs [19, 39, 121, 152, 227, 385]. 11.5 Social Welfare in Trading Networks When we’ve looked at games in earlier settings, we’ve considered not just equilibrium solu- tions, but also the question of whether these solutions are socially optimal . That is, do they maximize social welfare , the sum of the payoffs of all players?

345 11.5. SOCIAL WELFARE IN TRADING NETWORKS 331 i to a buyer contributes In the context of our game, each good that moves from a seller j , and the money − v to the social welfare. This is how much more j v i values the good than j i that is spent in moving the good from i to j is simply transferred from one player to another, creating a net effect of zero to the total payoff. In more detail, if the good moves through to t b i to i and an ask of a , then the sum of the payoffs of , who offers a bid of j trader tj ti and j , plus the portion of t ’s payoffs arising from this transaction, is equal to ( b . − v v ) + ( a − − b v ) + ( v ) = − a tj j j tj i i ti ti v over all goods that move from a seller − v Thus the social welfare is simply the sum of i j to a buyer j . This makes sense, since it reflects how much happier, in total, the new owners i of the goods are compared to the original owners of the goods. The maximum value of this quantity over all possible flows of goods — i.e., the socially optimal value — depends not just on the valuations of the sellers and buyers, but also on the network structure. Networks that are more richly connected can potentially allow a flow of goods achieving a higher social welfare than networks that are more sparsely connected, with bottlenecks that prevent a desirable flow of goods. For example, let’s go back to the pair of networks in Figure 11.11. In each case, the equilibrium yields a flow of goods that achieves the social optimum. In Figure 11.11(a), the best possible value of the social welfare is 1 + 2 + 4 = 7, since there is no way to use the network to get copies of the goods to both B B 4. However, when the single edge from 3 and 2 to S 2 is added, it suddenly becomes possible for both of these buyers to receive copies T of the good, and so the value of the social welfare increases to 2 + 3 + 4 = 9. This provides a simple illustration of how a more richly connected network structure can enable greater social welfare from trade. In our discussion of social optimality we count the gains to traders as part of the social welfare (since they are part of the society of players, along with sellers and buyers). In the next section, we will consider how the total payoffs are divided between sellers, buyers, and traders, and how it depends on the network structure. Equilibria and Social Welfare. In both the networks in Figure 11.11, the flow of goods achieving maximum social welfare can be achieved by an equilibrium. In fact, this holds for all the examples we’ve seen thus far, and it’s a fact that’s true in general: it can be shown that in every trading network, there is always at least one equilib- rium, and every equilibrium produces a flow of goods that achieves the social optimum [63]. While we won’t go into the details of the proof here, it is similar in structure to the Existence and Optimality of Market-Clearing Prices that we discussed in the previous chapter. There too, without intermediaries present, we were able to show that prices achieving a certain type of equilibrium always exist (in that case the market-clearing property), and that all such prices produce an allocation that maximize social welfare.

346 332 CHAPTER 11. NETWORK MODELS OF MARKETS WITH INTERMEDIARIES x 0 B1 S1 1 0 T1 x T2 x 1 x S2 1 B2 0 T3 x T4 x x 0 0 S3 B3 T5 1 Figure 11.12: Whether a trader can make a profit may depend on the choice of equilibrium. x = 1, traders T In this trading network, when T 5 make a profit, while when x = 0, 1 and only trader T 3 makes a profit. 11.6 Trader Profits We now consider the question of how the social welfare in an equilibrium is divided up as payoffs among the sellers, buyers, and traders. In particular, the examples we’ve studied so far suggest the informal principle that as the network becomes more richly connected, individual traders have less and less power, and their payoffs go down. Understanding this more precisely points to a basic question that can be expressed in terms of our network model: what is the structural basis of perfect competition? Our examples suggest that in order to make a profit (i.e. a positive payoff), a trader must in some way be “essential” to the functioning of the trading network. Certainly, if there is another trader who can replicate his function completely, then he cannot make a profit; nor can he make a profit in more complex settings like the implicit perfect competition in Figure 11.9. In fact, it will turn out that a version of this “essentiality” principle is true; but it is a bit more subtle than it might initially appear. To motivate it, we start with two illuminating examples. First, whether a trader makes a profit can depend on the equilibrium: in some networks, it can be possible for a trader to make a profit in some equilibria but not others. Figure 11.12 shows how this can occur. Any choice of x between 0 and 1 will result in an equilibrium, with the traders T 2 and T 4 who are left out of trades serving to “lock” the value of x in 5 make a profit, while when place. However, when = 1, traders T 1 and T x x = 0, only trader T 3 makes a profit. Furthermore, while every equilibrium produces a flow of goods to all three sellers, for a social welfare of 3, the amount of this social welfare that goes to the

347 11.6. TRADER PROFITS 333 S1 S1 S1 0 0 x x T1 T1 T1 B1 B1 B1 0 0 x x S2 S2 S2 y y 0 y 0 y T2 T2 T2 B2 B2 B2 S3 S3 S3 T1 A network in which trader T1 (a) An equilibrium where T1 (b) (c) An equilibrium where is essential. trades two goods. trades one good. Figure 11.13: Despite a form of monopoly power in this network, neither trader can make a profit in any equilibrium: we must have x y = 0. = x buyers and sellers — rather than the traders — varies between 1 and 2 as ranges from 1 to 0. The second example, in Figure 11.13, is even more counter-intuitive. Here, traders T 1 T 2 both have monopoly power over their respective sellers, and yet their profits are and every equilibrium. We can verify this fact as follows. First, we notice that any zero in equilibrium must look like one of the solutions in Figure 11.13(b) or Figure 11.13(c). The sellers are monopolized and will get bids of 0. For each buyer, the two asks must be the same, since otherwise the trader making the sale could slightly raise his ask. Now, finally, notice that if the common ask to either buyer were positive, then the trader left out of the trade on the higher one has a profitable deviation by slightly undercutting this ask. Therefore, in this example, all bids and asks equal 0 in any equilibrium, and so neither trader profits. This happens despite the monopoly power of the traders — and moreover T 1 fails to make a profit despite the fact that 2 can only perform one trade on his own. T We can interpret this as a situation in which a small trader competes with a larger trader across his full set of potential buyers, despite having access to an insufficient set of sellers to actually perform all the available trades — a situation in which “the threat is stronger than its execution.” While this fits naturally within the scope of the model, examples such as this one also suggest natural extensions to the model, in which each trader has an intrinsic limit on the number of trades he can perform, and this affects the behavior of competing traders. While the possibility of such intrinsic limits haven’t played a role in our earlier examples, Figure 11.13 suggests that allowing for such limits could change the outcome in certain settings. T With these examples in mind, let’s return to the question of when, for a given trader in a network, there exists an equilibrium in which T receives a positive payoff. It turns out that there exists such an equilibrium precisely when T has an edge e to a seller or buyer

348 334 CHAPTER 11. NETWORK MODELS OF MARKETS WITH INTERMEDIARIES e would change the value of the social optimum. In such a situation, we such that deleting is an essential edge from say that to the other node. The proof of this statement is e T somewhat involved, and we refer the reader to [63] for the details. Figures 11.6 and 11.8 are examples of trading networks in which each trader has an essential edge and thus makes a profit in equilibrium, while Figures 11.7 and 11.9 are examples in which no trader has an essential edge and no trader makes a profit. This essential-edge condition is a stronger form of monopoly power than we saw in Fig- ure 11.13(a). There, although deleting the node 1 would change the value of the social T optimum, there is no single edge whose deletion would reduce the value of the social opti- mum below 2; rather, after the removal of any one edge, there would still be a flow of goods to both buyers. This is the crux of why 1 is not able to make a profit in Figure 11.13(a), T despite his powerful position. The example in Figure 11.12 also shows that this condition only implies a profit in some equilibrium, as opposed to every equilibrium. In Figure 11.12, the available profit essentially “slides” smoothly from one trader to another as we vary the value of x in the equilibrium. 11.7 Reflections on Trade with Intermediaries In closing, it is useful to reflect on how our analysis of trade on networks relates to the motivating examples from the beginning of this chapter: trade in the stock market and the trade of agricultural goods in developing countries. The network model we analyzed in this chapter is an abstraction that captures some essential features of these real markets, and misses other features. Our trade model reflects the constraint that trade takes place through intermediaries and that there is differential access to these intermediaries. Equilibria in our trading networks reflect the fact that buyers and sellers in intermediated markets, such as the stock market, face a bid-ask spread. In our model, as in actual intermediated markets, the size of this spread, and how much profit intermediaries make, depends on the amount of competition between intermediaries for the trade flow. However, there are other interesting aspects of trade in intermediated markets that are not captured by our simple network model. In particular, we do not ask where buyers’ and sellers’ values come from; nor do we ask about how they might use information revealed by bids, asks, or trades to update these values. We discuss the role of beliefs and information in the stock market in Chapter 22. 11.8 Exercises 1. Consider a trading network with intermediaries in which there is one seller S , two buyers B 2. The seller is allowed to trade ,B 2 and two traders (intermediaries) T 1 ,T 1

349 11.8. EXERCISES 335 with either trader. The buyers can each trade with only one of the traders: buyer B 1; and buyer B 2 can only trade with trader T 2. The 1 can only trade with trader T seller has one unit of the object and values it at 0; the buyers are not endowed with the object. Buyer 1 values a unit at 1 and buyer B 2 values a unit at 2. B (a) Draw the trading network, with the traders as squares, the buyers and the seller as circles, and with edges connecting nodes who are able to trade with each other. Label S , B 1, B 2, T 1 or T each node as 2. (b) Suppose the traders offer prices as follows. 1 and an ask price of 1 to to S B 1. 1 offers a bid price of Trader T • 3 2 T 2 offers a bid price of • Trader to S and an ask price of 2 to B 2. 3 Does this set of prices form a Nash equilibrium? If you think the answer is yes, give a brief (1-3 sentence) explanation why. If you think the answer is no, describe a way in which one of the traders could changes its prices so as to increase its profit. 2. Consider a trading network in which there are two buyers (B1 and B2), two sellers (S1 and S2) and one trader (T1). All of the buyers and the sellers are allowed to trade with the trader. The sellers each have one unit of the object and value it at 0; the buyers are not endowed with the object, but they each want one unit; buyer B1 attaches a value of 1 to one unit, while buyer B2 attaches a value of 2 to one unit. (a) Draw the trading network, with the trader as a square, the buyers and the sellers as circles, and edges representing pairs of people who are able to trade with each other. Label the nodes as T1, B1, B2, S1, and S2. Find Nash equilibrium bid and ask prices. (You do not need to provide an explanation for your answer.) Suppose now that we add a second trader (T2) who can trade with each seller and (b) each buyer. In the new network is there a Nash equilibrium in which each trader’s bid price to each seller is 1; each trader’s ask price to buyer B1 is 1; each trader’s ask price to buyer B2 is 2; one unit of the good flows from S1 to B1 through trader T1; and, one unit of the good flows from S2 to B2 through trader T2? Draw the new trading network and give a brief (1-3 sentence) explanation for your answer. 3. Consider a trading network with intermediaries in which there are two sellers S 1 ,S 2, three buyers B 1 ,B 2 ,B 3, and two traders (intermediaries) T 1 ,T 2. Each seller can trade with either trader. Buyer 1 can only trade with trader T 1. Buyer B 2 can trade B with either trader. Buyer B 3 can only trade with trader T 2. The sellers each have one unit of the object and value it at 0; the buyers are not endowed with the object. Buyer B 1 values a unit at 1, buyer B 2 values a unit at 2 and buyer B 3 values a unit at 3.

350 336 CHAPTER 11. NETWORK MODELS OF MARKETS WITH INTERMEDIARIES Draw the trading network, with the traders as squares, the buyers and the seller as (a) circles, and with edges connecting nodes who are able to trade with each other. Label T 1, 2, B 1, B 2, B 3, T each node as S 2. S 1 or Suppose the prices and the flow of goods are as follows. (b) • Trader T 1 offers a bid price of 1 to each seller, an ask price of 1 to B 1, and an B ask price of 2 to 2. • T 2 offers a bid price of 1 to each seller, an ask price of 2 to B 2, and an Trader B ask price of 3 to 3. One unit of the good flows from seller S 1 to buyer B 2 through trader T • 1 and one unit of the good flows from seller S 2 to buyer B 3 through trader T 2. (If it is useful, it is okay to write these prices and this flow of goods on the picture you drew for part (a), provided the picture itself is still clear.) Do these prices and this flow of goods form a Nash equilibrium? If you think the answer is yes, give a brief (1-3 sentence) explanation why. If you think the answer is no, describe a way in which one of the traders could changes its prices so as to increase its profit. 4. Consider a trading network with intermediaries in which there is one buyer, one seller and two traders (intermediaries). The buyer and the seller each are allowed to trade with either trader. The seller has one unit of the object and values it at 0; the buyer is not endowed with the object but attaches a value of 1 to one unit of it. Draw the trading network, with traders as squares, the buyer and the seller as circles, and edges representing pairs of people who are able to transact directly. Then describe what the possible Nash equilibrium outcomes are, together with an explanation for your answer. S , two 5. Consider a trading network with intermediaries in which there is one seller B 1 ,B 2 and two traders (intermediaries) T 1 ,T 2. The seller is allowed to trade buyers with either trader. The buyers can each trade with only one of the traders: buyer B 1 can only trade with trader T 1; and buyer B 2 can only trade with trader T 2. The seller has one unit of the object and values it at 0; the buyers are not endowed with B 1 values a unit at 3 and buyer B 2 values a unit at 1. the object. Buyer (a) Draw the trading network, with the traders as squares, the buyers and the seller as circles, and with edges connecting nodes who are able to trade with each other. Label each node as S , B 1, B 2, T 1 or T 2. (b) Find Nash equilibrium bid and ask prices for this trading network. How much profit do the traders make?

351 11.8. EXERCISES 337 Suppose now that we add edges representing the idea that each buyer can trade (c) with each trader. Find a Nash equilibrium in this new trading game. What happens to trader profits? Why? S 1 2 ,S 3, 6. Consider a trading network with intermediaries in which there are three sellers ,S B 1 ,B 2, and two traders (intermediaries) T 1 ,T 2. Sellers S 1 and S 2 can two buyers T trade only with trader S 3 can trade only with trader T 2. The buyers 1; and, seller B T 1; can each trade with only one of the traders: buyer 1 can only trade with trader B 2 can only trade with trader and buyer 2. The sellers each have one unit of the T object and value it at 0. The buyers are not endowed with the object and they each value a unit at 1. (a) Draw the trading network, with the traders as squares, the buyers and sellers as circles, and with edges connecting nodes who are able to trade with each other. Label each node as S 1, S 2, S 3, B 1, B 2, T 1 or T 2. (b) Describe what the possible Nash equilibria are, including both prices and the flow of goods. Give an explanation for your answer. (c) B 2 and trader T 1. We want to Suppose now that we add an edge between buyer examine whether this new edge changes the outcome in the game. To do this, take the equilibrium from your answer to (b), keep the prices and the flow of goods on the edges from (b) the same as before, and then suppose that the ask price on the new B 2- T 1 edge is 1, and that no good flows on this new edge. Do these prices and this overall flow of goods still form an equilibrium? If you think that the answer is yes, give a brief (1-3 sentence) explanation why. If you think the answer is no, describe a way in which one of the participants in the game would deviate. 7. Consider a trading network in which there are two buyers (B1 and B2), two sellers (S1 and S2), and two traders (T1 and T2). The sellers each have one unit of the object and value it at 0; the buyers are not endowed with the object, but they each want one unit and attach a value of 1 to one unit. Seller S1 and Buyer B1 can trade only with trader T1; seller S2 and Buyer B2 can each trade with either trader. (a) Draw the trading network, with the traders as squares, the buyers and the sellers as circles, and edges representing pairs of people who are able to trade with each other. Label the nodes as T1, T2, B1, B2, S1, and S2. (b) Consider the following prices and flow of goods: • T1’s bid price to Seller S1 is 0, his bid price to Seller S2 is 1 / 2, his ask price to Buyer B1 is 1, and his ask price to Buyer B2 is 1 / 2.

352 338 CHAPTER 11. NETWORK MODELS OF MARKETS WITH INTERMEDIARIES T2’s bid price to Seller S2 is 1 2 and his ask price to Buyer B2 is 1 / 2. • / One unit of the good flows from Seller S1 to Buyer B1 through Trader T1; and, • one unit of the good flows from Seller S2 to Buyer B2 through trader T2. Do these prices and this flow of goods describe an equilibrium of the trading game? If you think that the answer is No, then briefly describe how someone should deviate. If you think that the answer is Yes, then briefly explain (1-3 sentences) why the answer is Yes. (c) Suppose now that we add a third trader (T3) who can trade with Seller S1 and Buyer B1. This trader cannot trade with the other seller or buyer, and the rest of the trading network remains unchanged. Consider the following prices and flow of goods: The prices on the old edges are unchanged from those in part (b). • • The prices on the new edges are: a bid of 1 / 2 to Seller S1 by Trader T3 and an ask of 1 / 2 to Buyer B1 by Trader T3. • The flow of goods is the same as in (b). Do these prices and this flow of goods describe an equilibrium of the trading game? If you think that the answer is No, then briefly describe how someone should deviate. If you think that the answer is Yes, then briefly explain (1-3 sentences) why the answer is Yes.

353 Chapter 12 Bargaining and Power in Networks In our analysis of economic transactions on networks, particularly the model in Chapter 11, we considered how a node’s position in a network affects its power in the market. In some cases, we were able to come up with precise predictions about prices and power, but in others the analysis left open a range of possibilities. For example, in the case of perfect competition between traders, we could conclude that the traders would make no profit, but it was not possible to say whether the resulting situation would favor particular buyers or sellers — different divisions of the available surplus were possible. This is an instance of a broader phenomenon that we discussed earlier, in Chapter 6: when there are multiple equilibria, some of which favor one player and some of which favor another, we may need to look for additional sources of information to predict how things will turn out. In this chapter, we formulate a perspective on power in networks that can help us further refine our predictions for the outcomes of different participants. This perspective arises dominantly from research in sociology, and it addresses not just economic transactions, but also a range of social interactions more generally that are mediated by networks. We will develop a set of formal principles that aim to capture some subtle distinctions in how a node’s network position affects its power. The goal will be to create a succinct mathematical framework enabling predictions of which nodes have power, and how much power they have, for arbitrary networks. 12.1 Power in Social Networks The notion of power is a central issue in sociology, and it has been studied in many forms. Like many related notions, a fundamental question is the extent to which power is a property of individuals (i.e. someone is particularly powerful because of their own exceptional attributes) D. Easley and J. Kleinberg. Networks, Crowds, and Markets: Reasoning about a Highly Connected World . Cambridge University Press, 2010. Draft version: June 10, 2010. 339

354 340 CHAPTER 12. BARGAINING AND POWER IN NETWORKS and the extent to which it is a property of network structure (i.e. someone is particularly powerful because they hold a pivotal position in the underlying social structure). The goal here is to understand power not just as a property of agents in economic settings, or in legal or political settings, but in social interaction more generally — in the roles people play in groups of friends, in communities, or in organizations. A particular focus is on the way in which power is manifested between pairs of people linked by edges in a larger social network. Indeed, as Richard Emerson has observed in his fundamental work on this subject, power is not so much a property of an individual as it is a property of a relation between two individuals — it makes more sense to study the conditions under which one person has power over another, rather than simply asserting that a particular person is “powerful” [148]. A common theme in this line of work is to view a social relation between two individuals as producing value for both of them. We will be deliberately vague in specifying what this value is, since it clearly depends on the type of social relation we are discussing, but the idea adapts naturally to many contexts. In an economic setting, it could be the revenue that two people can produce by working together; in a political setting, it could be the ability of each person in the relationship to do useful favors for the other; in the context of friendship, it could be the social or psychological value that the two people derive from being friends with one another. In any of these examples, the value may be divided equally or unequally between the two parties. For example, one of the two parties in the relationship may get more benefit from it than the other — they may get more than half the profits in a joint business relationship, or in the context of a friendship they may be the center of attention, or get their way more often in the case of disagreements. The way in which the value in the relationship is divided between the two parties can be viewed as a kind of social exchange , power then corresponds to the imbalance in this division — with the powerful party in and the relationship getting the majority of the value. Now, in some cases this imbalance in a relationship may be almost entirely the result of the personalities of the two people involved. But in other cases, it may also be a function of the larger social network in which the two people are embedded — one person may be more powerful in a relationship because they occupy a more dominant position in the social network, with greater access to social opportunities outside this single relationship. In this latter case, the imbalance in the relationship may be rooted in considerations of network structure, and transcend the individual characteristics of the two people involved. The ways in which social imbalances and power can be partly rooted in the structure of the social network has motivated the growth of a research area in sociology known as network exchange theory [417]. An Example of a Powerful Network Position. It is useful to discuss this in the context of a simple example. Consider the group of five friends depicted in Figure 12.1,

355 12.1. POWER IN SOCIAL NETWORKS 341 B C A D E Figure 12.1: A social network on five people, with node B occupying an intuitively powerful position. with strong friendships indicated by the social network links. Intuitively, node B appears to hold a powerful position in the network, and in particular to be powerful relative to two A and C of her three neighbors, . What general principle or principles should lead us to this conclusion? Here are several proposals, which we state informally here but make more precise in what follows. are completely Recalling that social relations confer value, nodes A and (i) Dependence. C dependent on B as a source of such value; B on the other hand, has multiple sources. (ii) Exclusion. Related to (i), B has the ability to exclude A and C . In particular, suppose each person were to choose a “best friend” in the group; then B has the unilateral A and C power to choose one of B does not have the , excluding the other. (However, analogous power over D .) (iii) Satiation. A somewhat different basis for B ’s power might be implicit in the psy- chological principle of : having diminishing rewards for increased amounts of satiation something. Again, viewing social relations as conferring value, B will acquire value at a greater rate than the other members of the group; having thus become satiated, B may be interested in maintaining these social relations only if she can receive an unequal share of their value. (iv) Betweenness. If we believe that the value generated in social relations flows not just across single edges but more generally along paths, then we are led to consider notions such as betweenness . Betweenness was considered extensively in Section 3.6; for our

356 342 CHAPTER 12. BARGAINING AND POWER IN NETWORKS purposes here, it is enough to think informally of a node as having high betweenness if it lies on paths (and particularly short paths) between many pairs of other nodes. In B has high betweenness because she is the unique access point between our example, multiple different pairs of nodes in the network, and this potentially confers power. More generally, betweenness is one example of a centrality measure that tries to find the “central” points in a network. We saw in our discussion of structural holes in Section 3.5 that evaluating a node’s power in terms of its role as an access point between different parts of the network makes sense in contexts where we are concerned about issues like the flow of information. Here, however, where we are concerned about power arising from the asymmetries in pairwise relations, we will see some concrete cases where a simple application of ideas about centrality can in fact be misleading. 12.2 Experimental Studies of Power and Exchange While all of these principles are presumably at work in many situations, it is difficult to make precise or quantify their effects in most real-world settings. As a result, researchers have turned to laboratory experiments in which they ask test subjects to take part in stylized forms of social exchange under controlled conditions. This style of research grew into an active experimental program carried out by a number of research groups in network exchange theory [417]. The basic idea underlying the experiments is to take the notion of “social value” and represent it under laboratory conditions using a concrete economic framework, of the type that we have seen in Chapters 10 and 11. In these experiments, the value that relationships produce is represented by an amount of money that the participants in a relationship get to share. This does not mean, however, that an individual necessarily cares only about the amount of money that he receives. As we shall see, it’s clear from the results that even subjects in the experiments may also care about other aspects of the relationship, such as the fairness of the sharing. While the details vary across experiments, here is the set-up for a typical one. Roughly, people are placed at the nodes of a small graph representing a social network; a fixed sum of money is placed on each edge of a graph; and nodes joined by an edge negotiate over how the money placed between them should be divided up. The final, crucial part of the set-up is that each node can take part in a division with only one neighbor, and so is faced with the choice not just of how large a share to seek, but also with whom. The experiment is run over multiple periods to allow for repeated interaction by the participants, and we study the divisions of money after many rounds. Here are the mechanics in more detail. 1. A small graph (such as the one in Figure 12.1) is chosen, and a distinct volunteer test subject is chosen to represent each node. Each person, representing a node, sits

357 12.2. EXPERIMENTAL STUDIES OF POWER AND EXCHANGE 343 at a computer and can exchange instant messages with the people representing the neighboring nodes. on each 2. The value in each social relation is made concrete by placing a resource pool edge — let’s imagine this as a fixed sum of money, say $1, which can be divided between the two endpoints of the edge. We will refer to a division of this money between the endpoints as an . Whether this division ends up equal or unequal will be exchange taken as a sign of the asymmetric amounts of power in the relationship that the edge represents. 3. Each node is given a limit on the number of neighbors with whom she can perform an exchange. The most common variant is to impose the extreme restriction that each node can be involved in a successful exchange with only of her neighbors; one 1-exchange rule this is called the B can . Thus, for example, in Figure 12.1, node ultimately make money from an exchange with only one of her three neighbors. Given this restriction, the set of exchanges that take place in a given round of the experiment can be viewed as a matching in the graph: a set of edges that have no endpoints in common. However, it will not necessarily be a perfect matching , since some nodes may not take part in any exchange. For example, in the graph in Figure 12.1, the exchanges will definitely not form a perfect matching, since there are an odd number of nodes. 4. Here is how the money on each edge is divided. A given node takes part in simultaneous sessions of instant messaging separately with each of her neighbors in the network. In each, she engages in relatively free-form negotiation, proposing splits of the money on the edge, and potentially reaching an agreement on a proposed split. These negotiations must be concluded by a fixed time limit; and to enforce the 1-exchange rule defined above, as soon as a node reaches an agreement with one neighbor, her negotiations with all other neighbors are immediately terminated. 5. Finally, the experiment is run for multiple rounds. The graph and the assignment of subjects to nodes as described in point (1) are kept fixed across rounds. In each round, new money is placed on each edge as in point (2), each node can take part in an exchange as in point (3), and the money is divided as in point (4). The experiment is run for multiple rounds to allow for repeated interactions among the nodes, and we study the exchange values that occur after many rounds. Thus the general notion of “social value” on edges is implemented using a specific eco- nomic metaphor: the value is represented using money, and people are negotiating explicitly over how to divide it up. We will mainly focus on the 1-exchange rule, unless noted other- wise. We can view the 1-exchange rule as encoding the notion of choosing “best friends”, which we discussed earlier when we talked about exclusion. That is, the 1-exchange rule

358 344 CHAPTER 12. BARGAINING AND POWER IN NETWORKS B A A B C 2-Node Path (b) 3-Node Path (a) B C D A E B C D A 4-Node Path (d) 5-Node Path (c) Figure 12.2: Paths of lengths 2, 3, 4, and 5 form instructive examples of different phenomena in exchange networks. models a setting in which the nodes are trying to form partnerships: each node wants to be in a partnership; and subject to this, the node wants to get a reasonable share of the value implicit in this partnership. Later in this chapter we will see that varying the number of successful exchanges in which a node can participate has effects on which nodes hold power, often in interesting ways. There are many variations in the precise way these experiments are implemented. One particularly interesting dimension is the amount of information provided to the participants about the exchanges made by other participants. This has ranged in experiments from a high-information version — in which each person sees not just what is happening on their edges, but also what is happening on every edge in the network, in real-time — to a low- information version — in which each person is told only what is happening on the edges she is directly involved in; for example, she may have no idea how many other potential partners each of her neighbors has. An interesting finding from this body of work is that the experimental results do not change much with the amount of information available [389]; this suggests a certain robustness to the results, and also allows us to draw some conclusions about the kinds of reasoning that participants are engaging in as they take part in these experiments. 12.3 Results of Network Exchange Experiments Let’s start by discussing what happens when one runs this type of experiment on some simple graphs using human test subjects. Since the results are intuitively reasonable and fairly robust, we’ll then consider — in increasing levels of detail — what sorts of principles can be inferred about power in these types of exchange situations. Figure 12.2 depicts four basic networks that have been used in experiments. Notice that these are just paths of lengths 2, 3, 4, and 5. Despite their simplicity, however, each

359 12.3. RESULTS OF NETWORK EXCHANGE EXPERIMENTS 345 introduces novel issues, and we will discuss them in order. The 2-Node Path. The 2-node path is as simple as it gets: two people are given a fixed amount of time in which to agree on a way to split $1. Yet even this simple setting introduces a lot of conceptual complexity — a large amount of work in game theory has been devoted precisely to the problem of reasoning about outcomes when two parties with oppositely aligned interests sit down to negotiate. As we will discuss more fully later in this 1 1 - chapter, most of the standard theoretical treatments predict a split. This seems to be 2 2 a reasonable prediction, and it is indeed approximately what happens in network exchange experiments on a 2-node graph. A , The 3-Node Path. , and C in order, node B On a 3-node path with nodes labeled B A and C . For example, as B negotiates with A , she has the intuitively has power over both ability to fall back on her alternative with , while A has no other alternatives. The same C reasoning applies to ’s negotiations with C . B A C must be excluded from an exchange in each round. In Moreover, at least one of or experiments, one finds that subjects who are excluded tend to ask for less in the next round A and C tends to drive in the hope of becoming included. Thus, the repeated exclusion of B indeed receives the overwhelming down what they ask for, and one finds in practice that / 6 in one recent set of experiments [281]). majority of the money in her exchanges (roughly 5 An interesting variation on this experiment is to modify the 1-exchange rule to allow B to take part in two exchanges in each round. One now finds that B negotiates on roughly equal footing with both A and C . This is consistent with the notions of dependence and exclusion discussed earlier: in order for B to achieve half the value from each exchange in A C as much as they need her. each round, she needs and This result for the version in which B is allowed two exchanges is less consistent with B were becoming satiated by money twice as quickly as A and C , one satiation, however: if A could expect to start seeing an effect in which C need to offer unequal splits to B in and order to keep interested. But this is not what actually happens. B The 4-Node Path. The 4-node path is already significantly more subtle than the previous two examples. There is an outcome in which all nodes take part in an exchange — A exchanges with B and C exchanges with D — but there is also an outcome in which B and C exchange with each other while excluding and D . A B should have some amount of power over A , but it is a weaker kind of power Thus, than in the 3-node path. In the 3-node path, B could exclude A and seek an exchange with C , who has no other options. In the 4-node path, on the other hand, if B excludes A , then B herself pays a price by having to seek an exchange with C , who already has an attractive

360 346 CHAPTER 12. BARGAINING AND POWER IN NETWORKS D B A C . B Figure 12.3: An exchange network with a weak power advantage for node D . In other words, B ’s threat to exclude A is a costly one to actually execute. alternative in : in A Experiments bear out this notion of B exchanges, B gets roughly between weak power - / 12 and 2 / 3 of the money, but not more [281, 373]. 7 Paths of length 5 introduce a further subtlety: node The 5-Node Path. , which intu- C itively occupies the “central” position in the network, is in fact weak when the 1-exchange C B and D , and rule is used. This is because ’s only opportunities for exchange are with A and E respectively. Thus, C each of these nodes have very attractive alternatives in can be excluded from exchange almost as easily as and E can. Put succinctly, C A ’s partners for negotiation all have access to very weak nodes as alternatives, and this makes C weak as well. In experiments, one finds that C does slightly better than A and E do, but only slightly. Thus, the 5-node path shows that simple centrality notions like betweenness can be mislead- ing measures of power in some kinds of exchange networks. C Note that the weakness of really does depend on the fact that the 1-exchange rule is used. Suppose, for example, that we instead allowed A , C , and E to take part in one B and exchange each, but allowed to take part in two exchanges each. Then suddenly D each of B and D need C to make full use of their exchange opportunities, and C is now the node with the ability to exclude some of his exchange partners. Other Networks. Many other networks have been studied experimentally. In a number of cases, the outcomes can be understood by combining ideas from the four basic networks in Figure 12.2. For example, the graph in Figure 12.1 has been extensively studied by network exchange theorists. Since B A and C , she tends to achieve highly favor- has the ability to exclude both able exchanges with them. Given these two alternatives, B and D almost never exchange; as a result, D doesn’t have a realistic second option besides E , and hence D and E tend to

361 12.3. RESULTS OF NETWORK EXCHANGE EXPERIMENTS 347 A B C Figure 12.4: An exchange network in which negotiations never stabilize. exchange on roughly equal footing. All these observations are borne out by the experimental results. Another interesting example that has been extensively studied is the “stem graph” shown and D typically exchange with each other, while in Figure 12.3. Here, exchanges with C B , obtaining favorable terms. The position of node B in this network is conceptually similar A B to the position of B has a power advantage in her in the 4-node-path: in the stem graph, dealings with , but it is a weak power advantage, since to exclude A she has to exchange A C or with , who each have exchange options in each other. Experiments have shown that D node B in the stem graph makes slightly more money than node B in the 4-node path, and there is an intuitive, if somewhat subtle, reason for this: B ’s threat over A in the 4-node path is to negotiate with the comparably powerful node C B ’s threat in the stem , while graph is to negotiate with people who are slightly weaker. An Unstable Network. A common theme in all the networks we have discussed thus far is that the negotiations among participants tend to wrap up reliably by the time limit, with fairly consistent outcomes. But there exist pathological networks in which negotiations tend to drag out until the very end, with unpredictable individual outcomes for the participants. To see why this might happen, we consider the simplest of these pathological examples, depicted in Figure 12.4: three nodes each connected to each other. It is not hard to see what happens when an exchange experiment is run on the triangle. Only one exchange can be A and completed among the three nodes; so as time is running out, two of the nodes — say, B — will be wrapping up negotiations, while the third node ( C in this case) is completely left out and stands to get nothing. This means that C will be willing to break into the A - B negotiations up to the very end, offering an exchange to either of these nodes in which they C can get a small amount. If this happens — say that C get almost everything as long as breaks up the A - B negotiations by offering highly favorable terms to A — then there will be a different node left out ( B in this case), who will in turn be willing to offer highly favorable terms to get back in on an exchange. This process, by itself, would cycle indefinitely — with some node always left out and

362 348 CHAPTER 12. BARGAINING AND POWER IN NETWORKS f of 1-x A receives good of value 1: payof sells to A for x: payof f of x B C D Figure 12.5: An exchange network built from the 4-node path can also be viewed as a buyer-seller network with 2 sellers and 2 buyers. trying anything to get back in — and it is only brought to a halt by the arbitrary arrival of the time limit. Under these conditions, you have nodes “playing for the last shot,” with the outcome for any one node correspondingly hard to predict. Again, this is not an issue that comes up in any of the earlier examples we discussed; what’s different in the triangle network is that no matter what tentative exchanges are being planned, excluded nodes have a natural way to “break in” to the negotiations. This prevents the outcome from ever stabilizing across the network. It is also worth noting that the mere presence of a triangle in a larger network does not necessarily cause problems: for example, the stem graph in Figure 12.3 contains a triangle, but the exchange possibilities provided by D A allows for robust outcomes in which A exchanges with B and C and the additional node exchange with each other. The problem with a “free-standing” triangle as in Figure 12.4 is fundamentally different: here, there is always a node who is left out, and yet has the ability to do something about it. 12.4 A Connection to Buyer-Seller Networks When we discussed matching markets in Chapter 10, we considered bipartite graphs con- sisting of buyers and sellers. Here, on the other hand, we have been talking about graphs in which the participants all play the same role (there is no division into buyers and sellers); and rather than conducting trade, they negotiate over the division of money on the edges. Despite these surface-level differences, there is a close connection between the two set- tings. To see this connection, let’s consider the 4-node path as an example. Suppose we

363 12.5. MODELING TWO-PERSON INTERACTION: THE NASH BARGAINING SOLUTION 349 A and to be buyers, and nodes B and D to be sellers. We give one unit of a declare nodes C C and , and one unit of money to each of A and B ; we assume that A and good to each of D each have a valuation of 1 for one copy of the good, and that C and D have no valuation B for the good. We now consider the prices at which sales of the good will take place. It takes a bit of thought, but this is completely equivalent to the exchange network experiment on the length-4 path, as indicated in Figure 12.5. For example, if sells to A B x , then B for a price of x (from the x units of money), and A gets a payoff gets a payoff of of 1 − x (from the 1 unit of value for the good, minus the x units of money he has to pay). Thus, the negotiation between A B over a price x in the buyer-seller network is just like and x A B the negotiation between x and 1 − and in an exchange over the division of $1 into network. Furthermore, the 1-exchange rule corresponds to the requirement that each seller can only sell a single unit of the good, and each buyer only wants one unit of the good. One can perform a comparable translation for all the graphs in Figures 12.1 and 12.2. However, it is important to note two caveats about this general observation on the relation- ship between exchange networks and buyer-seller networks. First, the translation is only possible for graphs that are bipartite (as all the graphs in Figures 12.1 and 12.2 are), even if they are not drawn with the nodes in two parallel columns. The triangle graph in Figure 12.4 is not bipartite, and although we can still talk about the exchange network experiment, it is not possible to label the nodes as buyers and sellers in such a way that all edges join a seller to a buyer. We can make one node a seller, and another node a buyer, but then we have no options for what to label the third node. Similarly, the stem graph in Figure 12.3 is not bipartite, and so the analogy to buyer-seller networks cannot be applied there either. A second caveat is that, for bipartite graphs, the two formulations are equivalent only at a mathematical level. It is not at all clear that human subjects placed in a buyer-seller experiment would behave in the same way as human subjects in a network exchange exper- iment, even on the very same graph. Indeed, there is recent empirical evidence suggesting that one may in fact see different outcomes from these two ways of describing the same process to test subjects [397]. 12.5 Modeling Two-Person Interaction: The Nash Bar- gaining Solution Thus far, we have seen a range of networks on which exchange experiments have been carried out, and we have developed some of the informal reasons why the outcomes turn out the way they do. We’d now like to develop a more mathematical framework allowing us to express predictions about what will happen when network exchange takes place in an arbitrary network. Among the phenomena we’d like to be able to explain are the distinctions between equal and asymmetric division of value across an edge; between strong power (when

364 350 CHAPTER 12. BARGAINING AND POWER IN NETWORKS B A outside outside option option x y Figure 12.6: Two nodes bargaining with outside options. imbalances go to extremes) and weak power (as in the four-node path, when the imbalance remains moderate); and between networks where outcomes stabilize and networks (like the triangle in Figure 12.4) where they don’t. In fact, we will be able to achieve this goal to a surprising extent, capturing each of these phenomena in a model based on simple principles. We begin the formulation of this model here and in the next section by developing two important ingredients, each based on a different type of two-person interaction. The first ingredient — the Nash bargaining solution — has a more mathematical flavor, while the second — the ultimatum game — is based primarily on human-subject experiments. The Nash Bargaining Solution. Let’s start with a simple formulation of two-person bargaining. Suppose, as in network exchange on a 2-node path, that two people A and B are negotiating over how to split $1 between them. Now, however, we extend the story to A also has an outside option of x assume that B has an outside option of y . By this we , and mean that if doesn’t like his share of the $1 arising from the negotiations with B , he can A x A is going to get less leave and take instead. This will presumably happen, for instance, if x from the negotiations. Similarly, B than has the option of abandoning the negotiation at any time and taking her outside option of y . Notice that if x + y > 1, then no agreement between A and B is possible, since they can’t divide a dollar so that one gets at least x and the other gets at least y x + y ≤ 1 when we consider this type . Consequently, we will assume of situation. Given these conditions, requires at least x from the negotiations over splitting the A B requires at least y . Consequently, the negotiation is really over how to split dollar, and the surplus s = 1 − x − y (which is at least 0, given our assumption that x + y ≤ 1 in the previous paragraph). A natural prediction is that if the two people B have equal and A A gets bargaining power, then they will agree on the division that splits this surplus evenly: 1 1 + x s , and B gets y + . This is the prediction of a number of general theories including s 2 2

365 12.5. MODELING TWO-PERSON INTERACTION: THE NASH BARGAINING SOLUTION 351 Nash bargaining solution the [312], and we will use this as our term for the outcome: Nash Bargaining Solution: When A and B negotiate over splitting a dollar, with x for A and an outside option of y for B (and x + y ≤ 1 ), an outside option of the Nash bargaining outcome is y − 1 + 1 x A = , and + x s to • 2 2 1 y + 1 − x s = • y + to B . 2 2 In the literature on network exchange theory, this division is sometimes referred as the equidependent outcome [120], since each person is depending equally on the other for conces- sions to make the negotiation work. At a high level, the formulation of the Nash bargaining solution emphasizes an important point about the process of negotiation in general: trying to ensure that you have as strong an outside option as possible, before the negotiations even begin, can be very important for achieving a favorable outcome. For most of this chapter, it is enough to take the Nash bargaining solution as a self-contained principle, supported by the results of experiments. In the final section of the chapter, however, we ask whether it can be derived from more fundamental models of behavior. We show there that in fact it can — it arises naturally as an equilibrium when we formulate the process of bargaining as a game. Experiments on Status Effects. When we think about bargaining in the context of experiments with human subjects, we of course need to consider the assumption that the two people have equal bargaining power. While in our models we will make use of this assumption, it is interesting to think about how external information could affect relative bargaining power in settings such as these. The effects of perceived social status on bargaining power have been explored experimen- tally by sociologists. In these experiments, two people are asked to divide money in situations where they are led to believe that one is “higher-status” and the other is “lower-status.” For example, in a recent set of these experiments, pairs of people A and B , each female college sophomores, negotiated in the presence of outside options using instant messaging. How- ever, each was given false information about the other: was told that B was a high-school A B was told that A was a graduate student with very high student with low grades, while grades [390]. Thus, A believed B to be low-status, while B believed A to be high-status. The results of these experiments illustrate interesting ways in which beliefs about differ- ential status can lead to deviations from theoretical predictions in bargaining. First, each subject had to communicate information about their own outside options to their partners as part of the negotiation (this information was not provided by the experimenters). It was found that people tended to inflate the size of their outside option when they believed

366 352 CHAPTER 12. BARGAINING AND POWER IN NETWORKS their negotiating partner was lower-status; and they tended to reduce the size of their out- side option when they believed their negotiating partner was higher-status. Compounding this effect, people tended to partially discount a negotiating partner’s claims about outside options when they believed this partner to be lower-status. (In other words, lower-status people tended to underreport the value of their outside options, and even these underre- ported values were discounted by their partners.) Overall, for these and other reasons, the subject who was believed to be higher-status by her partner tended to achieve significantly better bargaining outcomes than the theoretical predictions. Naturally, these status effects are interesting additional factors to incorporate into models of exchange. For developing the most basic family of models, however, we will focus on the case of interaction in the absence of additional status effects, using the Nash bargaining outcome as a building block. 12.6 Modeling Two-Person Interaction: The Ultima- tum Game The Nash bargaining outcome provides us with a way of reasoning about two people whose power differences arise through differences in their outside options. In principle, this applies even to situations with extreme power imbalances. For example, in network exchange on a 3-node path, we saw that the center node holds all the power, since it can exclude either of the two other nodes. But in exchange experiments on this network, the center is not generally able to drive its partners’ shares all the way down to 0; rather one sees splits like 1 5 - . 6 6 What causes the negotiations to “pull back” from a completely unbalanced outcome? This is in fact a recurring effect in exchange experiments: human subjects placed in bargain- ing situations with strong power imbalances will systematically deviate from the extreme predictions of simple theoretical models. One of the most basic experimental frameworks for exploring this effect is called the Ultimatum Game [203, 386], and it works as follows. Like the bargaining framework discussed in the previous section, the Ultimatum Game also involves two people dividing a dollar, but following a very different procedure than what we saw before: (i) Person A is given a dollar and told to propose a division of it to person B . That is, A should propose how much he keeps for himself, and how much he gives to . B (ii) Person B is then given the option of approving or rejecting the proposed division. (iii) If B approves, each person keeps the proposed amount. If B rejects, then each person gets nothing.

367 12.6. MODELING TWO-PERSON INTERACTION: THE ULTIMATUM GAME 353 A and are communicating by instant messaging from different Moreover, let’s assume that B rooms; they are told at the outset that they have never met each other before, and quite possibly will never meet again. For all intents and purposes, this is a one-shot interaction. Suppose first that both people are strictly interested in maximizing the amount of money they walk away with; how should they behave? This is not hard to work out. First, let’s should behave. If A proposes a division that gives any positive amount to consider how , B B B ’s choice is between getting this positive amount of money (by accepting) and getting then B nothing (by rejecting). Hence, should accept any positive offer. Given that this is how is going to behave, how should A behave? Since B will accept B A B any positive offer, something and otherwise maximizes should pick the division that gives ’s own earnings. Thus, A should propose $.99 for himself and $.01 for B , knowing that B A will accept this. A could alternately propose $1.00 for himself and $.00 for B , gambling that B — who would then be indifferent between accepting and rejecting — would still accept. B But for this discussion we’ll stick with the division that gives a penny. This, then, is a prediction of how purely money-maximizing individuals would behave in a situation of extreme power imbalance: the one holding all the power ( A ) will offer as little as possible, and the one with essentially no power will accept anything offered. Intuition — and, as we will see next, experimental results — suggests that this is not how human beings will typically behave. The Results of Experiments on the Ultimatum Game. In 1982, G ̈uth, Schmit- tberger, and Schwarze [203] performed a series of influential experiments in which they studied how people would actually play the Ultimatum Game. They found that people play- ing the role of A tended to offer fairly balanced divisions of the money — on average, about a third of the total, with a significant number of people playing A in fact offering an even split. Moreover, they found that very unbalanced offers were often rejected by the person B . playing the role of A large amount of follow-up work has shown these finding to be highly robust [386], even when relatively large amounts of money are at stake. The experiment has also been carried out in a number of different countries, and there are interesting cultural variations, although again the tendency toward relatively balanced divisions is consistent [93]. Relatively balanced offers in the Ultimatum Game, and rejections of positive amounts of money — can these observations be reconciled with the game-theoretic framework we’ve used in previous chapters? There are in fact a number of ways to do so. Perhaps the most natural is to keep in mind one of the basic principles we discussed when defining payoffs in game-theoretic situations: a player’s payoff should reflect his or her complete evaluation of a given outcome. So when a player B evaluates an outcome in which she walks away with only 10% of the total, one interpretation is that there is a significant negative emotional payoff to

368 354 CHAPTER 12. BARGAINING AND POWER IN NETWORKS 1/2 1/2 0 0 0 1 A B C C A B Not a stable outcome (b) A stable outcome (a) 1/2 1/2 1/2 1/2 1/2 1/4 3/4 1/2 B B C D D C A A Not a stable outcome (d) A stable outcome (c) 1/3 2/3 2/3 1/3 A B C D (e) A stable outcome Figure 12.7: Some examples of stable and unstable outcomes of network exchange on the 3-node path and the 4-node path. The darkened edges constitute matchings showing who exchanges with whom, and the numbers above the nodes represent the values. being treated unfairly, and hence when we consider B ’s complete evaluation of the options, finds a a greater overall benefit to rejecting the low offer and feeling good about it than B A accepting the low offer and feeling cheated. Moreover, since people playing the role of understand that this is the likely evaluation that their partner B will bring to the situation, they tend to offer relatively balanced divisions to avoid rejection, because rejection results in A getting nothing as well. It remains true that if you find yourself playing the role of in an instance of the A Ultimatum Game where player B is a money-maximizing robot, you should offer as little as possible. What the line of experiments on this topic have shown is simply that real people’s payoffs are not well modeled by strict money-maximization. Even a robot will reject low offers if you instruct it to care about feeling cheated. All these observations are useful when we think about network exchange experiments where there are strong power imbalances between adjacent nodes — in these situations, we should expect to see wide asymmetries in the division of resources, but not necessarily as wide as the basic models might predict.

369 12.7. MODELING NETWORK EXCHANGE: STABLE OUTCOMES 355 12.7 Modeling Network Exchange: Stable Outcomes Having now built up some principles — both theoretical and empirical — that govern two- person interactions, we apply these to build a model that can approximately predict the outcomes of network exchange on arbitrary graphs. Let’s begin by making precise what we mean by an We say that an outcome. Outcomes. outcome of network exchange on a given graph consists of two things: (i) A matching on the set of nodes, specifying who exchanges with whom. Recall that a matching, as discussed in Chapter 10, is a set of edges so that each node is the endpoint of at most one of them — this corresponds to the 1-exchange rule, in which each node can complete at most one exchange, and some nodes may be left out. , indicating how much this node value (ii) A number associated with each node, called its gets from its exchange. If two nodes are matched in the outcome, then the sum of their values should equal 1, indicating that they split the one unit of money in some fashion between them. If a node is not part of any matching in the outcome, then its value should equal 0, indicating that it does not take part in an exchange. Figure 12.7 depicts examples of outcomes on the 3-node and 4-node paths. Stable Outcomes. For any network, there is almost always a wide range of possible outcomes. Our goal is to identify the outcome or outcomes that we should expect in a network when an exchange experiment is actually performed. A basic property we’d expect an outcome to have is : no node X can propose an stability Y that makes both X offer to some other node Y better off — thus “stealing” node Y and away from an existing agreement. For example, consider Figure 12.7(a). In addition to C feeling left out by the outcome, there is something that C can do to improve the situation: for example, C can offer 2 / 3 to B (keeping 1 / 3 for himself), if B will break her agreement with A C instead. This offer from C to B would make B better off (as and exchange with / 3 instead of her current 1 / 2) and it would also make C better off (as he she would get 2 would get 1 / 3 instead of 0). There is nothing to prevent this from happening, so the current situation is unstable. (Although we’ve described this trade as having been initiated by C , it could equally well be initiated by , in an attempt to improve on her current value of 1 / 2.) B Compare this to the situation in Figure 12.7(b). Here too C is doing badly, but now there is nothing he can do to remedy the situation. B is already getting 1 — the most she possibly can — and so there is nothing that C can offer to B to break the current A - B exchange. The situation, even though it is bad for some parties, is stable.

370 356 CHAPTER 12. BARGAINING AND POWER IN NETWORKS instability We can make this idea precise for any network, defining an in an outcome to be a situation where two nodes have both the opportunity and the incentive to disrupt the existing pattern of exchanges. Specifically, we have the following definition. Instability: Given an outcome consisting of a matching and values for the nodes, an instability in this outcome is an edge not in the matching, joining two nodes and Y , such that the sum of X ’s value and Y ’s value is less than X . 1 Notice how this captures the kind of situation we’re been discussing: in an instability, the two X to disrupt the status quo (because they’re connected Y have the opportunity nodes and incentive — since the by an edge, and hence allowed to exchange), and they also have the sum of their values is less than 1, they can find a way to divide the dollar between them and each end up better than they currently are doing. In the example we discussed from Figure 12.7(a), the instability is the edge connecting B and C — the sum of the values is 1 / 2, and so both B and C can end up better off by exchanging with each other. On the other hand, Figure 12.7(b) has no instabilities; there are no inherent stresses that could disrupt the status quo in Figure 12.7(b). Thus, we introduce stability a further definition, which we call . Stability: An outcome of network exchange is stable if and only if it contains no instabilities. Given the inherent fragility of outcomes with instabilities, we expect to see stable outcomes in practice, and for networks that have stable outcomes, we in fact typically do see results that are close to stable outcomes. Figures 12.7(c)–12.7(e) provide some further opportunities to test these definitions on examples. There is an instability in Figure 12.7(c), since nodes B and C are connected by an edge and collectively making less than the one unit of money they could split by exchanging with each other. On the other hand, the outcomes in Figures 12.7(d) and 12.7(e) are both stable, since on the one edge not in the matching, the two nodes are collectively making at least one unit of money from the current situation. Applications of Stable Outcomes. In addition to being intuitively natural, the notion of a stable outcome helps to explain some of the general principles observed in network exchange experiments. First, stable outcomes are good at approximately capturing what’s going on in situations with extreme power imbalances. If we think a bit about Figures 12.7(a) and 12.7(b), we can convince ourselves that the only stable outcomes on the 3-node path are those in which B exchanges with one of A or C and gets the full one unit of value for herself. Indeed, if B got anything less than one unit, the unmatched edge would form an instability. Hence,

371 12.7. MODELING NETWORK EXCHANGE: STABLE OUTCOMES 357 B occupies the dominant position in this network. In fact, with a bit of stability shows why analysis, we can see that on the 5-node path from Figure 12.2(d), the only stable outcomes B D . So stable outcomes are also able to pick and give values of 1 to the “off-center” nodes up on the subtlety that the central node C on the 5-node path is in fact very weak. Now we know that in fact human subjects on the 3-node path or 5-node paths will not push things all the way to 0-1 outcomes; rather, the powerful nodes tend to get amounts 5 more like . But our discussion surrounding the Ultimatum Game shows that this is, in a 6 sense, the most extreme kind of outcome that we’ll see from real people. Since the notion of stability isn’t designed to avoid extremes, we’ll view this mismatch between theory and experiment as something that is relatively easy to explain and account for: when we see 1 5 - , we can think of this as being as close to 0-1 as strong-power outcomes in practice like 6 6 1 human players will get. Our current framework is also good at identifying situations where there is no stable outcome. In particular, recall the pathological behavior of network exchange on the triangle network in Figure 12.4, which never settles down to a predictable result. We can now explain what’s going on by observing that there is no stable outcome for the triangle network . To see why, notice first that in any outcome, some node will be unmatched and get a value of 0. Let’s suppose this is node – due to the symmetry of the situation, it doesn’t matter C C which node we choose for this argument. This unmatched node has edges to both other nodes, and no matter how these other two nodes divide the money, at least one of them (say, ) will get less than 1. But now the edge connecting B B C is an instability, since they and collectively are getting less than 1 and yet have the ability to perform an exchange. The fact that there is no stable outcome provides us with a way to think about the dynamics of negotiation on the triangle — no matter what tentative agreement is reached, the system necessarily contains internal stress that will disrupt it. The explanatory power of stability also has significant Limitations of Stable Outcomes. limitations, however. One source of limitations lies in the fact that it allows outcomes to go to extremes that people will not actually follow in real life. But as we’ve already observed, this difficulty is something where the theory is approximately accurate, and the discrepancies can be recognized and dealt with relatively easily. A more fundamental difficulty with the notion of a stable outcome is that it is too ambiguous in situations where there is a weak power imbalance between individuals. For example, let’s go back to Figures 12.7(d) and 12.7(e). Both of these represent stable outcomes on the 4-node path, but the first of these gives equal values to all the nodes, despite the power advantages of the middle nodes. In fact, there is a large range of possible stable 1 In fact, one can extend the theory of stability fairly easily to be able to handle this effect explicitly. For the discussion here, however, we’ll stick with the simpler version that allows things to go to extremes.

372 358 CHAPTER 12. BARGAINING AND POWER IN NETWORKS 1/2 1/2 1/2 1/2 C B D A outside outside outside outside option option option option 1/2 0 0 1/2 Not a balanced outcome (a) 1/3 2/3 2/3 1/3 B C D A outside outside outside outside option option option option 1/3 1/3 0 0 (b) A balanced outcome 1/4 3/4 3/4 1/4 B C D A outside outside outside outside option option option option 1/4 1/4 0 0 (c) Not a balanced outcome Figure 12.8: The difference between balanced and unbalanced outcomes. outcomes on the 4-node path: when the matching consists of the two outer edges, then any way of dividing the value on these edges so that B and C cumulatively get at least 1 will be stable. To summarize, while stability is an important concept for reasoning about outcomes of exchange, it is too weak in networks that exhibit subtle power differences. On these networks, it is not restrictive enough, since it permits too many outcomes that don’t actually occur. Is there a way to strengthen the notion of stability so as to focus on the outcomes that are most typical in real life? There is, and this will be the focus of the next section.

373 12.8. MODELING NETWORK EXCHANGE: BALANCED OUTCOMES 359 12.8 Modeling Network Exchange: Balanced Outcomes In cases where there are many possible stable outcomes for a given network, we will show in . this section how to select a particularly natural set of outcomes that we call balanced The idea behind balanced outcomes is perhaps best illustrated by considering the four- node path. In particular, Figure 12.7(d) is a stable outcome, but it doesn’t correspond to what one sees in real experiments. Moreover, there is something clearly “not right” about C are being severely out-negotiated. Despite the fact that each of them has it: nodes B and D respectively, even and A an alternate option, they are splitting the money evenly with and D have nowhere else to go. though A We can think about this issue by noticing that network exchange can be viewed as a type of bargaining in which the “outside options” — in the sense of the Nash bargaining solution from Section 12.5 — are provided by the other nodes in the network. Figure 12.8(a) depicts 1 outcome we’ve been considering. Given the values for each node, we observe this for the all- 2 1 1 that (or an amount very , since she can offer in effect has an outside option of C to B 2 2 1 ) and steal C away from his current agreement with D . For the same slightly higher than 2 1 reason, C to , by considering what he would need to offer B also has an outside option of 2 steal her away from her current agreement with A . On the other hand, the network with its current node values provides A and D with outside options of 0 — they have no alternatives to their current agreements. Defining Balanced Outcomes. The discussion above suggests a useful way to view the 1 outcome: the exchanges that are happening do not represent the Nash problem with the all- 2 bargaining outcomes with respect to the nodes’ outside options. And it is in this context that the outcome in Figure 12.8(b) starts to look particularly natural. With these values, B 1 , since to steal C away from his current partnership B would need has an outside option of 3 1 2 1 2 a value of , keeping B ’s C for herself. Thus, - to offer split with A represents the Nash 3 3 3 3 bargaining solution for B and A with outside options provided by the values in the rest of the network. The same reasoning holds for the - D exchange. Hence, this set of values on C the 4-node path has an elegant self-supporting property: each exchange represents the Nash bargaining outcome, given the exchanges and values elsewhere in the network. We can define this notion of balance in general for any network, as follows [120, 349]. First, for any outcome in a network, we can identify each node’s best outside option just as we did in the 4-node path: it is the most money the node can make by stealing a neighbor away from his or her current partnership. Now we define a balanced outcome as follows. Balanced Outcome: An outcome (consisting of a matching and node values) is balanced if, for each edge in the matching, the split of the money represents the Nash bargaining outcome for the two nodes involved, given the best outside

374 360 CHAPTER 12. BARGAINING AND POWER IN NETWORKS outside option 1/2 1/4 D 1/4 3/4 B A outside outside C option option outside 0 1/2 option 1/2 1/4 Figure 12.9: A balanced outcome on the stem graph. options for each node provided by the values in the rest of the network. Notice how this type of outcome really is “balanced” between different extremes. On B and C the one hand, it prevents from getting too little, as in Figure 12.8(a). But it also prevents B and C from getting too much — for example, the outcome in Figure 12.8(c) is not balanced either, because B and C are each getting more than their share under the Nash bargaining outcome. Notice also that all of the outcomes in Figure 12.8 are stable. So in this example it’s reasonable to think of balance as a refinement of stability. In fact, for any network, every balanced outcome is stable. In a balanced outcome each node in the matching gets at least its best outside option, which is the most the node could get on any unused edge. So no two nodes have an incentive to disrupt a balanced outcome by using a currently unused edge, and therefore the outcome is stable. But balance is more restrictive than stability, in that there can be many stable outcomes that are not balanced. Applications and Interpretations of Balanced Outcomes. In addition to its elegant definition, the balanced outcome corresponds approximately to the results of experiments with human subjects. We have seen this already for the 4-node path. The results for the stem graph provide another basic example. Figure 12.9 shows the unique balanced outcome for the stem graph: C and D exchange on 1 B with an outside option of even terms, providing , and hence leading to a Nash bargaining 2 1 3 . The balanced outcome thus captures not just weak power - outcome of B between A and 4 4 advantages, but also subtle differences in these advantages across networks — in this case, the idea that B ’s advantage in the stem graph is slightly greater than in the 4-node path.

375 12.9. ADVANCED MATERIAL: A GAME-THEORETIC APPROACH TO BARGAINING 361 Given the delicate self-reference in the definition of a balanced outcome — its values are defined by determining outside options in terms of the values themselves — it is natural to ask whether balanced outcomes even exist for all networks. Of course, since any balanced outcome is stable, a balanced outcome can only exist when a stable outcome exists, and we know from the previous section that for certain graphs (such as the triangle) there is no stable outcome. But it can be shown that in any network with a stable outcome, there is also a balanced outcome, and there are also methods to compute the set of all balanced outcomes for a given network [31, 242, 254, 349, 378]. In fact, the concepts of stability and balance from this section and the previous one can cooperative game theory be framed in terms of ideas from an area known as , which studies how a collection of players will divide up the value arising from a collective activity (such as network exchange in the present case). In this framework, stability can be formulated using a central notion in cooperative game theory known as the core solution , and balance can be formulated as a combination of the core solution and a second notion known as the kernel solution [234, 289, 349]. Finally, we note that balance is one of several definitions proposed for refining stable outcomes to produce reasonable alignment with experiments. There are competing theories, including one called , that achieve similar results [373]. It remains an open equiresistance research question to understand how closely the predictions of all these theories match up with human-subject experiments when we move to significantly larger and more complex networks. 12.9 Advanced Material: A Game-Theoretic Approach to Bargaining In Section 12.5 we considered a basic setting in which two people, each with outside options, bargain over a shared resource. We argued that the Nash bargaining solution provides a natural prediction for how the surplus available in the bargaining will be divided. When John Nash originally formulated this notion, he motivated it by first writing down a set of axioms he believed the outcome of any bargaining solution should satisfy, and then showing that these axioms characterize his bargaining solution [312]. But one can also ask whether the same solution can be motivated through a model that takes into account the strategic behavior of the people performing the bargaining — that is, whether we can formulate a game that captures the essentials of bargaining as an activity, and in which the Nash bargaining outcome emerges as an equilibrium. This was done in the 1980s by Binmore, Rubinstein, and Wolinsky [60], using a game-theoretic formulation of bargaining due to Rubinstein [356]. Here we describe how this strategic approach to the Nash bargaining solution works; it is based on the notion of a dynamic game as formulated in Section 6.10. In our formulation

376 362 CHAPTER 12. BARGAINING AND POWER IN NETWORKS of bargaining, we will use the basic set-up from Section 12.5. There are two individuals A A and B who negotiate over how to split $1 between them. Person has an outside option of . We assume that y < + y 1 as otherwise there is x x and person B has an outside option of no way to split the $1 that would be beneficial to both people. The first step is to formulate bar- Formulating Bargaining as a Dynamic Game. and gaining as a game. To do this, we imagine a stylized picture for how two people A might negotiate over the division of a dollar, as suggested by the following hypothetical B presumably has the stronger outside option): conversation (in which A A: I’ll give you 30% of the dollar. B: No, I want 40% . 34% ? A: How about 36% . B: I’ll take A: Agreed. To capture the intuition suggested by this conversation, we define a dynamic bargaining game that proceeds over a sequence of periods that can continue indefinitely. In the first period, A proposes a split of the dollar in which he gets a • and B gets b . 1 1 (The subscript “1” indicates that this is the split proposed in the first period.) We will denote this split by ( a ,b ). 1 1 • B A ’s proposal or reject it. If B accepts, the game ends and can then either accept each player gets their respective portion. Otherwise, the game continues to period 2. and B proposes a split ( a . ,b a • b gets In the second period, A ) in which she gets 2 2 2 2 Now A can either accept or reject; again, the game ends if A accepts, and it continues if A rejects. • The periods continue indefinitely in this fashion, with proposing a split in each odd- A B proposing a split in each even-number period. Any accepted numbered period, and offer ends the game immediately. A and B above fits the structure of this game, if we rewrite it The conversation between using our notation as follows. (Period 1) A: ( . 70 , 30) ? B: Reject. (Period 2) B: ( . 60 , 40) ? A: Reject.

377 12.9. ADVANCED MATERIAL: A GAME-THEORETIC APPROACH TO BARGAINING 363 ( . , 34) ? B: Reject. (Period 3) A: 66 ( 64 , 36) ? A: Accept. (Period 4) B: . There is one more important part to the game, which models the idea that the two parties experience some pressure to actually reach a deal. At the end of each round, and before the next round begins, there is a fixed probability 0 that negotiations abruptly break down. p > In the event of such a breakdown, there will be no further periods, and the players will be forced to take their respective outside options. This describes the full game: it proceeds through a sequence of alternating offers, and it continues until someone accepts an offer or negotiations break down. At its conclusion, each player receives a payoff — either the accepted split of the dollar, or the outside options in the event of a breakdown. The possibility of a breakdown in negotiations means that if B decides to reject the proposed split in the first period, for example, she is risking the possibility that there won’t be a second round, and she will have to fall back to her outside option. Each player has to take this risk into account each time they reject an offer. This breakdown probability is necessary for the results we derive on bargaining, and we can view it as reflecting the idea that each player believes there is some chance the game will end before they reach an agreement. Perhaps the other player will give up on the negotiation or will abruptly be drawn away by some unexpected better opportunity that comes along, or perhaps there is simply some outside reason that the game itself suddenly ends. The game we have just defined is a dynamic Analyzing the Game: An Overview. game in the sense of Section 6.10, but with two differences worth noting. The first difference is that each time a player makes a proposal, the set of available strategies is infinite rather than finite: he or she can propose to keep a portion of the dollar equal to any real number between 0 and 1. For our purposes, this difference ends up being relatively minor, and it doesn’t cause any trouble in the analysis. The second difference is more significant. In Section 6.10, we considered finite-horizon games that ran for at most a finite number of periods, whereas here we have an in which the sequence of periods infinite-horizon game can in principle go on forever. This poses a problem for the style of analysis we used in Section 6.10, where we reasoned from the final period of the game (with just a single move left to make) backward to the beginning. Here there is no final period, so we will need a different way to analyze the game. Despite this, the type of reasoning that we employed in Section 6.10 will help us to solve this game. The equilibrium we will look for is a subgame perfect equilibrium — a notion that we also saw in Chapter 11 associated with the trading game in which traders post prices, and buyers and sellers subsequently react. A subgame perfect equilibrium is simply a Nash

378 364 CHAPTER 12. BARGAINING AND POWER IN NETWORKS equilibrium with the property that the strategies, beginning from any intermediate point in the game, still form a Nash equilibrium for the play proceeding from that point onward. Our main result is twofold. First, the bargaining game has a subgame perfect equilibrium A ’s initial offer is accepted. Second, for this equilibrium, with a simple structure in which we can work out the values in the initial split ( a ,b ) that is proposed and accepted. These 1 1 a , and as and b quantities depend on the underlying value of the breakdown probability p 1 1 goes to 0, the split ( a p ,b ) converges to the Nash bargaining outcome. So the point is that 1 1 when two strategic bargainers interact through negotiations that are unlikely to break down quickly, the Nash bargaining solution is a good approximate prediction for the outcome. It is also worth considering how our formulation of bargaining here relates to the ex- perimental work in network exchange theory from earlier in this chapter. There are a few differences. First, of course, the experiments discussed earlier involve multiple interlinked negotiations that take place concurrently — one negotiation for each edge in a network. It is an interesting but largely open question to adapt the kind of bargaining game formulated here to a setting where negotiations take place simultaneously across all the edges of a net- work. But beyond this consideration, there are still differences between our game-theoretic model here and the exchange-theory experiments even when we look just at a single edge of the network. First, the experiments generally allowed for free-form discussion between the two endpoints of an edge, whereas we have a specified a fixed format in which the two bargainers take turns proposing splits, beginning with A . The fact that A gets to move first in our game gives him some advantage, but in the case we are mainly interested in for our results — as the breakdown probability becomes small — this advantage becomes p negligible. Second, the experiments generally imposed a fixed time limit to ensure that ne- gotiations would eventually end, while we are using a breakdown probability that applies to each round. It is not clear exactly how these two sources of time pressure in a negotiation relate to each other, since even with a fixed time limit, the fact that nodes may have multiple network neighbors in the exchange-theory experiments makes it hard to reason about how long the negotiation on any particular edge is likely to last. Because of the A First Step: Analyzing a Two-Period Version of Bargaining. complexity introduced by the infinite nature of the game, it is useful to get some initial insight by first analyzing a finite version of it. In particular, let’s take our earlier version of the game and assume that it ends for sure at the end of the second period. (As before, it may also end with probability p at the end of the first period.) Since this is now a game with a finite number of periods, we can solve it backward through time as follows. a • will accept B ’s proposal ( A ,b First, ) in period two provided that a is at least 2 2 2 as large as A ’s outside option x . (Since negotiations are guaranteed to end after this

379 12.9. ADVANCED MATERIAL: A GAME-THEORETIC APPROACH TO BARGAINING 365 A is simply choosing at this point between round, and x .) a 2 Given this, there is no reason for A more than x to offer B ’s period-two proposal • B , so 1 − x ). Since we have assumed x + y < 1, we have 1 − will be ( , and so B x, x > y 2 y . B prefers this split to the outcome in which negotiations end and gets only B Now, when A ’s offer in the first round, she • considers whether to accept or reject should compare it to the expected payoff she’d get by rejecting it and allowing the game to continue. If she rejects the offer, then with probability p , negotiations break down immediately and she gets . Otherwise, the game continues to its second and final y round, where we’ve already concluded that will get 1 − x . Therefore, B ’s expected B payoff if she rejects the offer is . p )(1 − x ) − py + (1 z ; our conclusion is that in the first round, B will accept any Let’s call this quantity z . offer of at least • Finally, we need to determine what will propose in the first round. There is no point A B A B anything more generous than (1 − in ), since ’s offering to will accept this, z,z so the question is simply whether A prefers this split to his outside option x . In fact, he does: since y < 1 − x , and z is a weighted average of y and 1 − x , it follows that z < 1 x , and so 1 − z > x . − A will propose (1 z,z ) in the first round, and it will be immediately Therefore, − accepted. This describes the complete solution to the two-period bargaining game, and it’s inter- esting to consider how the outcome for each player depends on the value of the breakdown p . When p is close to one, so that negotiations are very likely to break down in probability B ’s payoff z = the first round, + (1 − p )(1 − x ) is very close to her back-up option y ; py correspondingly, A gets almost all the surplus. On the other hand, when p is close to zero, so that negotiations are very likely to continue to the second round, B ’s payoff is very close to 1 − x , and so A is driven down to almost his back-up option. This makes sense intuitively. When p A has most of the leverage in is close to one, p the negotiations, since his offer is probably the only one that will get made. When is B has most of the leverage in the negotiations, since she will probably get close to zero, to make the final offer, and can therefore safely ignore an undesirable initial offer from . A 1 Notice also that when p is exactly equal to , the payoffs correspond to the Nash bargaining 2 2 We will exploit indifference, as in many of our previous models, to assume that A accepts the proposed split ( x, 1 − x ) rather than letting negotiations end. Alternately, as usual, we could imagine that B proposes an amount very slightly above x to A , to make sure A accepts.

380 366 CHAPTER 12. BARGAINING AND POWER IN NETWORKS outcome: each player gets an amount halfway between their backup option and their backup option plus the full surplus. So this in fact provides us with a first way to obtain the Nash bargaining solution from a two-player game: when the players take part in a two-round 1 after the first round. As a reasonable model of negotiation that ends with probability 2 bargaining, however, this structure is a bit artificial: why only two rounds, and moreover, 1 why a breakdown probability of exactly ? It feels more reasonable to consider negotiations 2 that are allowed to go on for a long time, with the small underlying breakdown probability imposing a mild form of pressure to reach an agreement. This is the infinite-horizon version that we formulated initially, and which we will analyze next. One way to build up to the analysis Back to the Infinite-Horizon Bargaining Game. of the infinite-horizon game would be to consider finite-horizon bargaining games that are allowed to last for a larger and larger number of rounds, and try to argue that these eventually B the last approximate the infinite-horizon version. Finite-horizon games of even length give offer, while those of odd length give the last offer; but as the length increases, the chance A that the last round is ever reached will go down. It is possible to carry out this analysis, but in fact it’s easier to use what we learned in the two-round version of the game to directly conjecture the structure of an equilibrium for the infinite-horizon game. In particular, we saw in the analysis of the two-round bargaining game that offers are not rejected in equilibrium. There are two reasons for this. First, both players stand to gain from splitting the surplus 1 x − y in some fashion, and delaying by rejecting offers − makes it possible that negotiations will break down and this surplus will be lost. Second, each player can reason about the minimum amount that the other is willing to accept, and so he or she can offer exactly this amount when given the opportunity to make an offer. At a general level, these considerations still apply to the infinite-horizon game, and so it is natural to conjecture there is an equilibrium in which A ’s initial offer is accepted. We will search for such an equilibrium — and in fact, more strongly for an equilibrium where from any intermediate point in the game, the next offer to be made would be accepted. There is another issue to consider: there is at least one sense in which the finite-horizon bargaining games actually have a more complicated structure than the infinite-horizon game. For a finite-horizon bargaining game, the reasoning in each period is slightly different — you have to evaluate the expected payoff a bit differently with 10 rounds left to go than you do with 9 rounds or 8 rounds left to go. This means that the splits being proposed will also change slightly in value as the time until the end of the game changes. The infinite-horizon game, on the other hand, is fundamentally different: after a back-and-forth pair of offers by A and B , there is another copy of exactly the same infinite-horizon game left to be played. The structure and payoffs in the game don’t change over time. Of course, the players do observe offers being made and rejected if the game actually continues past the first period,

381 12.9. ADVANCED MATERIAL: A GAME-THEORETIC APPROACH TO BARGAINING 367 and they could condition their behavior on this history of offers. But given the stationary nature of the game’s structure over time, it’s natural to look for an equilibrium among the : those in which each of A and B plans to propose the same split set of stationary strategies and also has a fixed A in every period in which they are scheduled to propose, and each of B amount that they require in order to accept a proposal. An equilibrium that uses stationary . strategies will be called a stationary equilibrium Analyzing the Game: A Stationary Equilibrium. A nice feature of stationary strate- gies is that they’re very easy to describe and work with. Although the game is complex, any pair of stationary strategies for A and B can be represented by just a few numbers, namely: The split ( a • ,b will offer whenever he is scheduled to propose a split; ) that A 1 1 the split ( ,b ) that B will offer whenever she is scheduled to propose a split; and • a 2 2 a respectively and b , constituting the minimum offers that A and B • reservation amounts will accept from the other. Moreover, since the offers constitute proposed splits of one dollar, the two parts of each split sum to 1; so we have b . = 1 − a b and a − = 1 2 1 1 2 Our plan is to write down a set of equations on the values describing the stationary strategies, such that any pair of stationary strategies satisfying these equations constitute an equilibrium. We will then solve these equations, obtaining a stationary equilibrium, and converges to 0, the payoffs to A show that as the breakdown probability B converge p and to the Nash bargaining outcome. A will offer The equations are as follows. First, as in the two-period version of the game, B the least he can in order to get to accept his offer, so we set B = b b. (12.1) 1 Similarly, B will offer the least she can in order to get A to accept her offer, so a (12.2) = a. 2 Again following the reasoning from the two-period version, will set her reservation amount B right at the level where she is indifferent between accepting ’s offer and rejecting it. If b A b she accepts, she gets ; if she rejects, she gets the expected payoff that comes from allowing 1 the game to continue. We can determine this expected value as follows. With probability p , the game ends right after her rejection, in which case she receives y . Otherwise, the game continues with an offer by to A , and this offer will be accepted since, by Equation (12.2), B we’ve set a = a . In this case, B receives b ; so her overall expected payoff from allowing the 2 2

382 368 CHAPTER 12. BARGAINING AND POWER IN NETWORKS py + (1 p ) b game to continue would be . For B to be indifferent between accepting and − 2 rejecting, we need b py + (1 − p ) b . = (12.3) 1 2 Similar reasoning applies to B and allows ’s reservation amount: if he rejects an offer from A , and so for him to be indifferent + (1 px ) a the game to continue, his expected payoff is − p 1 between accepting and rejecting B ’s offer we have (12.4) = px + (1 − p a a . ) 1 2 Following the reasoning above, we can check that these four equations (12.1)–(12.4) are enough to ensure that the pair of stationary strategies forms an equilibrium. Since = 1 − a , this gives us two linear equation in two unknowns: and b b = 1 − a 2 2 1 1 − a py = 1 + (1 − p ) b 2 1 − − a = px 1 b p ) + (1 2 1 Solving these, we get y − + 1 (1 − p ) x = a 1 − p 2 − + 1 − p ) y x (1 b . = 2 2 − p A ’s initial offer is accepted, so A gets a payoff of In this equilibrium, gets a payoff , and B a 1 of + (1 ) x − )(1 p y − = a − = 1 b 1 1 2 − p We can check how these values for a and b is close behave as a function of p . When p 1 1 − y and y to 1, they are approximately 1 A gets almost all the surplus and B respectively. gets very close to her outside option, because the negotiations are likely to break down after the opening offer by A , and A is taking advantage of this fact. More interestingly, as p converges to 0, so that the players can expect the negotiations to continue for a long time, the opening offer is still accepted in this stationary equilibrium, but the payoffs are converging to ) ( + 1 x y − y + 1 − x , , 2 2 which are the values for the Nash bargaining solution. This completes the analysis, and shows how the Nash bargaining outcome arises very naturally from a game-theoretic model in which the two bargainers behave strategically, following a simple model of negotiations.

383 12.10. EXERCISES 369 12.10 Exercises 1. Suppose a network exchange theory experiment is run on the graph depicted in Fig- ure 12.10 using the one-exchange rule. Say which node or nodes you would expect to make the most money (i.e. receive the most favorable exchanges), and give a brief (1-3 sentence) explanation for your answer. b a g f c d e Figure 12.10: A graph used for a network exchange theory experiment. 2. Suppose a network exchange theory experiment is run on the graph depicted in Fig- ure 12.11 (i.e. a graph that is a 3-node path), using the one-exchange rule. Now you, playing the role of a fourth node d , are told to attach by a single edge to one of the nodes in the network. How should you attach to the network to put yourself in as powerful a position as possible, where power will be determined by the result of a network exchange theory experiment run on the resulting 4-node network? Give a brief explanation for your answer. a b c Figure 12.11:

384 370 CHAPTER 12. BARGAINING AND POWER IN NETWORKS 3. Suppose a network exchange theory experiment is run on the graph depicted in Fig- ure 12.12 using the one-exchange rule with $10 placed on each edge. b c d e a Figure 12.12: A graph used for a network exchange theory experiment. (a) Say which node or nodes you would expect to make the most money (i.e. receive the most favorable exchanges), and give a brief (1-3 sentence) explanation for your answer. You do not need to give actual numbers for the amounts of money the nodes would receive. (b) Now the experimenters vary the network: they introduce a sixth node f , which is attached by a single edge to just the node c . A new person is brought in to play the role of f , and a new round of experiments is performed with this new six-node network. Explain what you think will happen to the relative power of the participants, compared to the situation in (a), and give a brief (1-3 sentence) explanation for your answer. Again, you do not need to give actual numbers for the amounts of money the nodes would receive. 4. Suppose a network exchange theory experiment is run on the graph depicted in Fig- ure 12.13 using the one-exchange rule with $10 placed on each edge. a b c d Figure 12.13: (a) Say which node or nodes you would expect to make the most money (i.e. receive the most favorable exchanges), and give a brief (1-3 sentence) explanation for your answer. You do not need to give actual numbers for the amounts of money the nodes would receive. (b) Now the experimenters vary the conditions slightly: instead of placing $10 on the b - c edge, they place only $2. Otherwise, the experiment is run exactly as before.

385 12.10. EXERCISES 371 Explain what you think will happen to the relative power of each of the participants, compared to the situation in (a), and give a brief (1-3 sentence) explanation for your answer. Again, you do not need to give actual numbers for the amounts of money the nodes would receive. 5. Suppose a network exchange theory experiment is run on the graph depicted in Fig- ure 12.14 using the one-exchange rule with $10 placed on each edge. a b d c Figure 12.14: A 4-node path in a network exchange theory experiment. (a) After running the experiment for a while, the experimenters vary the network: e they introduce two further nodes f , and bring in additional people to play the and roles of these nodes. Node is attached by a single edge to node b , while node e is f attached by a single edge to node c . A new round of experiments is performed with this new six-node network. Explain what you think will happen to the relative power of the participants, compared to the situation in the original four-node network. Give a brief (1-3 sentence) explanation for your answer. You do not need to give actual numbers for the amounts of money the nodes would receive. The experimenters now decide to vary the network again. They keep the same (b) e directly to set of nodes, but now they add an edge linking . (The existing edges f continue to remain in place as well.) A new round of experiments is performed with this modified six-node network. Explain what you think will happen to the relative power of the participants, compared to the situation on the previous six-node network in part (a). Give a brief (1-3 sentence) explanation for your answer. You do not need to give actual numbers for the amounts of money the nodes would receive. 6. (a) Suppose that two different network exchange theory experiments are run, using the one-exchange rule — one on the 3-node path depicted in Figure 12.15, and the other on the 4-node path depicted in Figure 12.15. In which set of experiments do you expect node b to receive more money (i.e. receive more favorable exchanges)? Give a brief (1-3 sentence) explanation for your answer. (You do not need to give actual numbers for the amounts of money the nodes would receive.) (b) Suppose a network exchange theory experiment is run on the graph depicted in

386 372 CHAPTER 12. BARGAINING AND POWER IN NETWORKS b c a c a b d 3-Node Path (a) (b) 4-Node Path Figure 12.15: A 3-node path (right) and a 4-node path (left). Figure 12.16 using the one-exchange rule. Say which node or nodes you would expect to make the most money (i.e. receive the most favorable exchanges). Also, do you think the advantage experienced by the most powerful nodes in Fig- ure 12.16 will be more similar to the advantage experienced by node on the 3-node b path from part (a), or more similar to the advantage experienced by node b on the 4-node path from part (a)? Give a brief (1-3 sentence) explanation for your answers. (Again, you do not need to give actual numbers for the amounts of money the nodes would receive.) A B C D E F Figure 12.16: The graph for the network exchange theory experiment in part (b).

387 Part IV Information Networks and the World Wide Web 373

388

389 Chapter 13 The Structure of the Web Up to this point in the book, we’ve considered networks in which the basic units being connected were people or other social entities, like firms or organizations. The links connect- ing them have generally corresponded to opportunities for some kind of social or economic interaction. In the next several chapters, we consider a different type of network, in which the basic units being connected are pieces of information, and links join pieces of information that are related to each other in some fashion. We will call such a network an . information network As we will see, the World Wide Web is arguably the most prominent current example of such a network, and while the use of information networks has a long history, it was really the growth of the Web that brought such networks to wide public awareness. While there are basic differences between information networks and the kinds of social and economic networks that we’ve discussed earlier, many of the central ideas developed earlier in the book will turn out to be fundamental here as well: we’ll be using the same basic ideas from graph theory, including short paths and giant components; formulating notions of power in terms of the underlying graph structure; and even drawing connections to matching markets when we consider some of the ways in which search companies on the Web have designed their businesses. Because the Web plays such a central role in the modern version of this topic, we begin with some context about the Web, and then look further back into the history of information networks that led up to the Web. D. Easley and J. Kleinberg. Networks, Crowds, and Markets: Reasoning about a Highly Connected World . Cambridge University Press, 2010. Draft version: June 10, 2010. 375

390 376 CHAPTER 13. THE STRUCTURE OF THE WEB I teach a class on Networks. Networks Course: W e have a class blog Networks Class Blog: This blog post is about Microsoft Microsoft Home Page Figure 13.1: A set of four Web pages. 13.1 The World Wide Web If you’re reading this book, it’s likely that you use the Web on a daily basis. But since the Web is so enmeshed in the broader information infrastructure of the world (including the Internet, wireless communication systems, and the global media industry), it’s actually useful to think a bit about what the Web is and how it came about, starting from first principles. At a basic level, the Web is an application developed to let people share information over the Internet; it was created by Tim Berners-Lee during the period 1989-1991 [54, 55]. Although it is a simplification, we can view the original conception and design of the Web as involving two central features. First, it provided a way for you to make documents easily available to anyone on the Internet, in the form of Web pages that you could create and store on a publically accessible part of your computer. Second, it provided a way for others to easily access such Web pages, using a browser that could connect to the public spaces on computers across the Internet and retrieve the Web pages stored there.

391 13.1. THE WORLD WIDE WEB 377 I teach a class on Networks Networks Course: W e have a class blog Networks Class Blog: This blog post is about Microsoft Microsoft Home Page Figure 13.2: Information on the Web is organized using a network metaphor: The links among Web pages turn the Web into a directed graph. To a first approximation, this is still how we experience the Web today: as a sequence of Web pages rendered inside a browser. For example, Figure 13.1 shows a set of four separate Web pages: the home page of a college instructor who teaches a class on networks; the home page of the networks class he teaches; the blog for the class, with a post about Microsoft listed at the top; and the corporate home page for Microsoft. Because of the underlying design, we can think of these pages both as part of a single coherent system (the Web), but also as files that likely reside on four separate computers, controlled by several different and completely independent organizations, and made publically accessible by a now-universal consensus to participate in the protocols of the Web. Hypertext. Beyond these basic features, there is a crucial design principle embedded in the Web — the decision to organize the information using a network metaphor. This is what turns the set of Web pages from Figure 13.1 into the “web” of Web pages in Figure 13.2: in writing a Web page, you can annotate any portion of the document with a virtual link to

392 378 CHAPTER 13. THE STRUCTURE OF THE WEB another Web page, allowing a reader to move directly from your page to this other one. The set of pages on the Web thereby becomes a graph, and in fact a directed graph: the nodes are the pages themselves, and the directed edges are the links that lead from one page to another. Much as we’re familiar with the idea of links among Web pages, we should appreciate that the idea to organize Web pages as a network was both inspired and non-obvious. There are many ways to arrange information — according to a classification system, like books in a library; as a series of folders, like the files on your computer; even purely alphabetically, like the terms in an index or the names in a phone directory. Each of these organizational systems can make sense in different contexts, and any of them could in principle have been used for the Web. But the use of a network structure truly brings forth the globalizing power of the Web by allowing anyone authoring a Web page to highlight a relationship with any other existing page, anywhere in the world. The decision to use this network metaphor also didn’t arise out of thin air; it’s an applica- tion of a computer-assisted style of authoring known as hypertext that had been explored and refined since the middle of the twentieth century [316, 324]. The motivating idea behind hy- pertext is to replace the traditional linear structure of text with a network structure, in which any portion of the text can link directly to any other part — in this way, logical relationships within the text that are traditionally implicit become first-class objects, foregrounded by the use of explicit links. In its early years, hypertext was a cause passionately advocated by a relatively small group of technologists; the Web subsequently brought hypertext to a global audience, at a scale that no one could have anticipated. 13.2 Information Networks, Hypertext, and Associa- tive Memory The hypertextual structure of the Web provides us with a familiar and important example of an information network — nodes (Web pages in this case) containing information, with explicit links encoding relationships between the nodes. But the notion of an information network significantly predates the development of computer technology, and the creators of hypertext were in their own right motivated by earlier networks that wove together large amounts of information. Intellectual Precursors of Hypertext. A first important intellectual precursor of hy- pertext is the concept of citation among scholarly books and articles. When the author or authors of a scholarly work wish to credit the source of an idea they are invoking, they include a citation to the earlier paper that provides the source of this idea. For example, Figure 13.3 shows the citations among a set of sociology papers that provided some of the

393 13.2. INFORMATION NETWORKS, HYPERTEXT, AND ASSOCIATIVE MEMORY 379 Kossinets- W atts 2006 Burt 2004 Burt 2000 Coleman 1988 Granovetter 1985 Feld 1981 Granovetter 1973 T ravers- Davis 1963 Milgram 1969 Milgram Rapoport Lazarsfeld- Cartwright- 1953 Merton 1954 Harary 1956 1967 Figure 13.3: The network of citations among a set of research papers forms a directed graph that, like the Web, is a kind of information network. In contrast to the Web, however, the passage of time is much more evident in citation networks, since their links tend to point strictly backward in time.

394 380 CHAPTER 13. THE STRUCTURE OF THE WEB key ideas in the first part of this book. (At the bottom of this figure are seminal papers on — from left to right — triadic closure, the small-world phenomenon, structural balance, and homophily.) We can see how work in this field — as in any academic discipline — builds on earlier work, with the dependence represented by a citation structure. We can also see how this citation structure naturally forms a directed graph, with nodes representing books and articles, and directed edges representing citations from one work to another. The same structure arises among patents, which provide citations to prior work and earlier inventions; and among legal decisions, which provide citations to earlier decisions that are being used as precedents, or are being distinguished from the present case. Of course, the example in Figure 13.3 is a tiny piece of a much larger directed graph; for instance, Mark Granovetter’s 1973 paper on the strength of weak ties has been cited several thousand times in the aca- demic literature, so in the full citation structure we should imagine thousands of arrows all pointing to this single node. One distinction between citation networks and the Web is that citations are governed much more strongly by an underlying “arrow of time.” A book, article, patent, or legal decision is written at a specific point in time, and the citations it contains — the edges pointing outward to other nodes — are effectively “frozen” at the point when it is written. In other words, citations lead back into the past: if paper X cites paper Y , then we generally won’t find a citation from Y back to X for the simple reason that Y was written at a time before X existed. Of course, there are exceptions to this principle — two papers that were written concurrently, with each citing the other; or a work that is revised to include more recent citations — but this flow backward in time is a dominant pattern in citation networks. On the Web, in contrast, while some pages are written once and then frozen forever, a significant portion of them are evolving works in progress where the links are updated over long periods of time. This means that while links are directed, there is no strong sense of “flow” from the present into the past. Citation networks are not the only earlier form of information network. The cross- references within a printed encyclopedia or similar reference work form another important example; one article will often include pointers to other related articles. An on-line reference work like Wikipedia (even when viewed simply as a collection of linked articles, independent of the fact that it exists on the Web) is structured in the same way. This organizing principle is a clear precursor of hypertext, in that the cross-referencing links make relationships among the articles explicit. It is possible to browse a printed or on-line encyclopedia through its cross-references, pursuing serendipitious leads from one topic to another. For example, Figure 13.4 shows the cross-references among Wikipedia articles on certain 1 topics in game theory, together with connections to related topics. We can see, for example, 1 Since Wikipedia changes constantly, Figure 13.4 necessarily represents the state of the links among these articles only at the time of this writing. The need to stress this point reinforces the contrast with the “frozen” nature of the citations in a collection of papers such as those in Figure 13.3.

395 13.2. INFORMATION NETWORKS, HYPERTEXT, AND ASSOCIATIVE MEMORY 381 Game Nash Equilibrium Theory John Forbes Nash RAND Beautiful A Mind (fi lm) Conspiracy Apollo 13 Ron Howard (fi Theories lm) NASA Figure 13.4: The cross-references among a set of articles in an encyclopedia form another kind of infor- mation network that can be represented as a directed graph. The figure shows the cross-references among a set of Wikipedia articles on topics in game theory, and their connections to related topics including popular culture and government agencies. how it’s possible to get from the article on Nash Equilibrium to the article on NASA (the U.S. National Aeronautics and Space Administration) by passing through articles on John Nash (the creator of Nash equilibrium), A Beautiful Mind (a film about John Nash’s life), Ron Howard (the director of A Beautiful Mind ), Apollo 13 (another film directed by Ron Howard), and finally on to the article about NASA (the U.S. government agency that managed the real Apollo 13 space mission). In short: Nash equilibrium was created by someone whose life was the subject of a movie by a director who also made a movie about NASA. Nor is this the only short chain of articles from Nash equilibrium to NASA. Figure 13.4 also contains a sequence of cross-references based on the fact that John Nash worked for a period of time at RAND, and RAND is the subject of several conspiracy theories, as is NASA. These short paths between seemingly distant concepts reflect an analogue, for information

396 382 CHAPTER 13. THE STRUCTURE OF THE WEB networks, of the “six degrees of separation” phenomenon in social networks from Chapter 2, where similarly short paths link apparently distant pairs of people. Indeed, browsing through chains of cross-references is closely related to the stream-of- consciousness way in which one mentally free-associates between different ideas. For example, suppose you’ve just been reading about Nash equilibrium in a book, and while thinking about it during a walk home your mind wanders, and you suddenly notice that you’ve shifted to thinking about NASA. It may take a bit of reflection to figure out how this happened, and to reconstruct a chain of free-association like the one pictured in Figure 13.4, carried out entirely among the existing associations in your mind. This idea has been formalized in semantic network , in which nodes literally represent another kind of information network: a concepts, and edges represent some kind of logical or perceived relationship between the concepts. Researchers have used techniques like (e.g. “Tell me word association studies what you think of when I say the word ‘cold’ ”) as a way to probe the otherwise implicit structure of semantic networks as they exist in people’s minds [381]. Thus, information networks date back into much Vannevar Bush and the Memex. earlier periods in our history; for centuries, they were associated with libraries and scholarly literature, rather than with computer technology and the Internet. The idea that they could assume a strongly technological incarnation, in the form of something like the Web, is generally credited to Vannevar Bush and his seminal 1945 article in the Atlantic Monthly , entitled “As We May Think” [89]. Written at the end of World War II, it imagined with eerie prescience the ways in which nascent computing and communication technology might revolutionize the ways we store, exchange, and access information. In particular, Bush observed that traditional methods for storing information in a book, a library, or a computer memory are highly linear — they consist of a collection of items sorted in some sequential order. Our conscious experience of thinking, on the other hand, exhibits what might be called an associative memory , the kind that a semantic network represents — you think of one thing; it reminds you of another; you see a novel connection; some new insight is formed. Bush therefore called for the creation of information systems that mimicked this style of memory; he imagined a hypothetical prototype called the that Memex functioned very much like the Web, consisting of digitized versions of all human knowledge connected by associative links, and he imagined a range of commercial applications and knowledge-sharing activities that could take place around such a device. In this way, Bush’s article foreshadowed not only the Web itself, but also many of the dominant metaphors that are now used to think about the Web: the Web as universal encyclopedia; the Web as giant socio-economic system; the Web as global brain. The fact that Vannever Bush’s vision was so accurate is not in any sense coincidental; Bush occupied a prominent position in the U.S. government’s scientific funding establish-

397 13.2. INFORMATION NETWORKS, HYPERTEXT, AND ASSOCIATIVE MEMORY 383 ment, and his ideas about future directions had considerable reach. Indeed, the creators of early hypertext systems explicitly invoked Bush’s ideas, as did Tim Berners-Lee when he set out to develop the Web. The Web and its Evolution. This brings us back to the 1990s, the first decade of the Web, in which it grew rapidly from a modest research project to a vast new medium with global reach. In the early phase of this period, the simple picture in Figure 13.2 captured the Web’s essential nature: most pages were relatively static documents, and most links served navigational functions — to transport you from one page to another, according to primarily the relational premise of hypertext. This is still a reasonable working approximation for large portions of the Web, but the Web has also increasingly outgrown the simple model of documents connected by navigational links, and it is important to understand how this has happened in order to be able to interpret any analysis of the Web’s structure. In the earliest days of the Web, the computers hosting the content played a relatively passive role: they mainly just served up pages in response to requests for them. Now, on the other hand, the powerful computation available at the other end of a link is often brought more directly into play: links now often trigger complex programs on the computer hosting the page. Links with labels like “Add to Shopping Cart,” “Submit my Query,” “Update my Calendar,” or “Upload my Image,” are not intended by their authors primarily to transport you to a new page (though they may do that incidentally as part of their function) — such links exist to activate computational transactions on the machine that runs the site. Here’s an example to make this concrete. If we continued following links from the Microsoft Home Page in the example from Figure 13.2, we could imagine taking a next step to the on-line shopping site that Microsoft hosts for its products. From this page, clicking on a link labeled “Buy Now” next to one of the featured products would result in a charge to your credit card and the delivery of the product to your home in the physical, off-line world. There would also be a new page providing a receipt, but the purpose of this last “Buy Now” link was not primarily to transport you, hypertextually, to a “receipt page”; rather, it was to perform the indicated transaction. In view of these considerations, it is useful to think of a coarse division of links on the Web into navigational and transactional , with the former serving the traditional hypertex- tual functions of the Web and the latter primarily existing to perform transactions on the computers hosting the content. This is not a perfect or clear-cut distinction, since many links on the Web have both navigational and transactional functions, but it is a useful dichotomy to keep in mind when evaluating the function of the Web’s pages and links. While a lot of content on the Web now has a primarily transactional nature, this content still remains largely linked together by a navigational “backbone” — it is reachable via relatively stable Web pages connected to each other by more traditional navigational links.

398 384 CHAPTER 13. THE STRUCTURE OF THE WEB This is the portion of the Web we will focus on in our analysis of its global structure. Sorting out what should belong to this navigational backbone and what shouldn’t is ultimately a type of judgment call, but fortunately there is a lot of experience in making and even codifying such judgments. This is because distinguishing between navigational and transactional links has long been essential to Web search engines, when they build their indexes of the available content on the Web. It’s clearly not in a search engine’s interest to index, for the general public, every receipt from an on-line purchase that every user of the Web has ever made, or every query result for available airline flight times or product specifications that every Web user ever has made. As a result, search engines have developed and refined automated rules that try to assess whether the content they are collecting is relatively stable and intended for public consumption, and they tend to collect content that is reachable via navigational links. We will implicitly be following such working definitions when we talk about the structure of the Web; and when we discuss empirical data on large sets of Web pages in Section 13.4, it will be based on collections assembled by search engines according to such rules. 13.3 The Web as a Directed Graph Viewing social and economic networks in terms of their graph structures provides significant insights, and the same is true for information networks such as the Web. When we view the Web as a graph, it allows us to better understand the logical relationships expressed by its links; to break its structure into smaller, cohesive units; and — as we will see in Chapter 14 — to identify important pages as a step in organizing the results of Web searches. To begin with, it is important to note two things. First, in discussing the graph structure of the Web, we will be following the plan outlined at the end of Section 13.2 and focusing on its navigational links. As we observed in that discussion, the navigational links still form the bulk of the Web’s structural backbone, despite the increasing richness of Web content as a whole. Second, we need to appreciate that the fundamentally directed nature of the Web makes it different from many of the networks we’ve considered thus far. Recall that in a directed graph, the edges don’t simply connect pairs of nodes in a symmetric way — they point from one node another. This is clearly true on the Web: just because you write a blog post and to include a link to the Web page of a company or organization, there is no reason to believe that they will necessarily reciprocate and include a link back to the blog post. This distinction between directedness and undirectedness is an important aspect of the difference between social and information networks; an analogy here is to the difference between the global friendship network that we discussed in Chapter 2, showing who is friends global name-recognition network , in which there is a link from person with whom, and the A to person . This latter network is directed and in fact quite if A has heard of B B

399 13.3. THE WEB AS A DIRECTED GRAPH 385 I'm a student I'm applying to Univ . of X at Univ . of X college My song Classes lyrics USNews College Rankings I teach at Networks . of X Univ USNews Featured Colleges Networks class blog Blog post about college rankings Blog post about Company Z Company Z's home page Our Press Releases Founders Contact Us Figure 13.5: A directed graph formed by the links among a small set of Web pages. asymmetric — famous celebrities are recognizable to millions of people, and in fact millions closely track the goings-on in their lives, but one doesn’t expect that such celebrities are in any sense aware of the names or identities of all these fans. In other words, the global name-recognition network is structurally more similar to an information network like the Web than it is to a traditional social network defined by friendship. Paths and Strong Connectivity. The connectivity of undirected graphs was defined in terms of paths: two nodes are linked by a if we can follow a sequence of edges from path one to the other; a graph is connected if every pair of nodes is linked by a path; and we can break up a disconnected graph into its connected components . Now that we’re dealing with a directed graph, we’re going to try following the same general strategy for talking about connectivity; but to do this, we first need to rework the definition of a path to take directions

400 386 CHAPTER 13. THE STRUCTURE OF THE WEB into account, and this will necessarily make the subsequent definitions more subtle. First, a A to a node B in a directed graph is a sequence of nodes, path from a node and ending with A , with the property that each consecutive pair of nodes beginning with B in the sequence is connected by an edge pointing in the forward direction. This “pointing in the forward direction” condition makes the definition of a path in a directed graph different from the corresponding definition for undirected graphs, where edges have no direction. On the Web, this notion of following links only in the forward direction corresponds naturally to the notion of viewing Web pages with a browser: we can follow a link when it’s emanating to from the page we’re on, but we aren’t in general aware of all the links that point the page we’re currently visiting. We can try out this definition on the example in Figure 13.5, which shows the directed graph formed by the links among a small set of Web pages; it depicts some of the people and classes associated with the hypothetical University of X, which we imagine to have once been a Featured College in a national magazine. By following a sequence of links in this example (all in the forward direction), we can discover that there’s a path from the node Univ. of X to the node labeled US News College Rankings labeled : we can follow a link from Univ. of X to its Classes page, then to the home page of its class entitled Networks , then to the Networks class blog , then to a class blog post about college rankings, and finally via a link from this blog post to the page US News College Rankings . On the other hand, Company Z’s home page US News there’s no path from the node labeled to the node labeled College Rankings — there would be if we were allowed to follow directed edges in the reverse Company Z’s home page , we can only reach direction, but following edges forward from Our Founders , Press Releases , and Contact Us. With the definition of a path in hand, we can adapt the notion of connectivity to the setting of directed graphs. We say that a directed graph is strongly connected if there is a path from every node to every other node. So for example, the directed graph of Web pages in Figure 13.5 is not strongly connected, since as we’ve just observed, there are certain pairs of nodes for which there’s no path from the first to the second. Strongly Connected Components. When a directed graph is not strongly connected, it’s important to be able to describe its reachability properties: identifying which nodes are “reachable” from which others using paths. To define this notion precisely, it’s again useful to draw an analogy to the simpler case of undirected graphs, and try to start from there. For an undirected graph, its connected components serve as a very effective summary of reachability: if two nodes belong to the same component, then they can reach each other by paths; and if two nodes belong to different components then they can’t. But reachability in a directed graph is a harder thing to summarize. In a directed graph, we can have pairs of nodes for which each can reach the other (like Univ. of X and US

401 13.3. THE WEB AS A DIRECTED GRAPH 387 I'm a student I'm a applying to at Univ . of X college . of X Univ My song lyrics Classes USNews: College Rankings I teach at Networks Univ . of X USNews: Featured Colleges Networks class blog Blog post about college rankings Blog post Company Z's about home page Company Z Our Press Founders Releases Contact Us Figure 13.6: A directed graph with its strongly connected components identified.

402 388 CHAPTER 13. THE STRUCTURE OF THE WEB ), pairs for which one can reach the other but not vice versa (like News College Rankings Company Z’s home page ), and pairs for which neither can US News College Rankings and and I’m applying to college I’m a student at Univ. of X reach the other (like ). Moreover, the conceptual complexity of reachability in a directed graph corresponds to a kind of “visual” complexity as well: whereas the components of an undirected graph naturally correspond to separate chunks of the graph with no edges between them, a directed graph that is not strongly connected does not break equally obviously into non-interacting pieces. How then should we describe its reachability properties? The key is to find the right notion of a “component” for directed graphs, and in fact one can do this with a definition that strictly mirrors the formal definition of a component in an undirected graph. strongly connected component We say that a (SCC) in a directed graph is a subset of the nodes such that: (i) every node in the subset has a path to every other; and (ii) the subset is not part of some larger set with the property that every node can reach every other. As in the undirected case, part (i) of this definition says that all nodes within a strongly connected component can reach each other, and part (ii) of this definition says that the strongly connected components correspond as much as possible to separate “pieces,” not smaller portions of larger pieces. It helps to consider an example: in Figure 13.6 we show the strongly connected compo- nents for the directed graph from Figure 13.5. Notice the role that part (ii) of the definition plays in producing the separate pieces of the graph in this picture: the set of four nodes consisting of Univ. of X , Classes , Networks , and I teach at Univ. of X collectively satisfy part (i) of the definition, but they do not form a strongly connected component because they belong to a larger set that also satisfies (i). Looking at this picture, one can see how the SCCs serve as a compact summary of the reachability properties of the directed graph. Given two nodes and B , we can tell if there A A is a path from B as follows. First, we find the SCCs containing A and B respectively. If to A and B belong to the same SCC, then they can each reach each other by paths. Otherwise, viewing the SCCs themselves as larger “super-nodes”, we see if there is a way to walk from A to the SCC of B , following edges between SCCs in the forward direction. If the SCC of there is a way to do this, then this walk can be opened up into a path from A to B in the graph; if there is no way to do this, then there is no path from A to B . 13.4 The Bow-Tie Structure of the Web In 1999, after the Web had been growing for the better part of a decade, Andrei Broder and his colleagues set out to build a global map of the Web, using strongly connected components

403 13.4. THE BOW-TIE STRUCTURE OF THE WEB 389 Figure 13.7: A schematic picture of the bow-structure of the Web (image from [80]). Al- though the numbers are now outdated, the structure has persisted. as the basic building blocks [80]. For their raw data, they used the index of pages and links from one of the largest commercial search engines at the time, AltaVista. Their influential study has since been replicated on other, even larger snapshots of the Web, including an early index of Google’s search engine [56] and large research collections of Web pages [133]. Similar analyses have been carried out for particular well-defined pieces of the Web, including the links among articles on Wikipedia [83], and even for complex directed graph structures arising in other domains, such as the network of interbank loans depicted in Figure 1.3 from Chapter 1 [50]. In this way, although the actual snapshot of the Web used by Broder et al. in their original study comes from an earlier time in the Web’s history, the mapping paradigm they proposed continues to be a useful way of thinking about giant directed graphs in the context of the Web and more generally.

404 390 CHAPTER 13. THE STRUCTURE OF THE WEB A “map” of the Web clearly can’t resemble A Giant Strongly Connected Component. a map of the physical world in any serious sense, given the scale and complexity of the network being analyzed. Rather, what Broder et al. [80] wanted was something more conceptual — an abstract map dividing the Web into a few large pieces, and showing in a stylized way how these pieces fit together. Their first finding was that the Web contains a giant strongly connected component. Recall from our discussions in Chapter 2 that many naturally occuring undirected graphs have a giant connected component — a single component containing a significant fraction of all the nodes. The fact that the directed analogue of this phenomenon holds for the Web is not hard to believe based on analogous thought experiments. Roughly, the point is that a number of major search engines and other “starting page” sites have links leading to directory-type pages from which you can, in turn, reach the home pages of major educational institutions, large companies, and governmental agencies. From here one can reach most of the pages within each of these large sites. Further, many of the pages within these sites US News link back to the search engines and starting pages themselves. (The path from College Rankings to a class blog and back in Figures 13.5 and 13.6 suggests a concrete example for how this happens.) Thus, all these pages can mutually reach one another, and hence all belong to the same strongly connected component. Given that this SCC contains (at least) the home pages of many of the major commercial, governmental, and non-profit organizations in the world, it is easy to believe that it is a giant SCC. From here, we can invoke an argument — familiar from the undirected case as well — that there is almost surely at most one giant SCC. For if there were two giant SCCs — call them X and Y — all it would take is a single link from any node in X to any node Y , and Y another link from any node in X , and X and to any node in would merge to become Y part of a single SCC. The Bow-Tie Structure. The second step in the analysis by Broder et al. [80] was to position all the remaining SCCs in relation to the giant one. This involves classfying nodes by their ability to reach and be reached from the giant SCC. The first two sets in this classification are the following. IN : nodes that can reach the giant SCC but cannot be reached from it — i.e., nodes (1) that are “upstream” of it. (2) OUT : nodes that can be reached from the giant SCC but cannot reach it — i.e., nodes are “downstream” of it. Figure 13.6 forms a useful example for trying out these definitions. Although the network in Figure 13.6 is much too small for any of its SCCs to be considered “giant,” we can imagine its largest SCC as the giant one and consider how the other nodes are positioned in relation to

405 13.4. THE BOW-TIE STRUCTURE OF THE WEB 391 I’m a student at Univ. of X and constitute it. In this case, the pages I’m applying to college Blog post about Company Z , and the pages and the whole SCC involving Company Z IN . And this is roughly what one intuitively expects to find in these sets: IN constitute OUT OUT contains pages that have not been “discovered” by members of the giant SCC, while contains pages that may receive links from the giant SCC, but which choose not to link back. Figure 13.7 shows the original schematic image from Broder et al. [80], depicting the IN relation of OUT , and the giant SCC. Because of the visual effect of IN and OUT as , large lobes hanging off the central SCC, Broder et al. termed this the “bow-tie picture” of the Web, with the giant SCC as the “knot” in the middle. The actual sizes of the different pieces shown in the Figure come from the 1999 AltaVista data, and are long since obsolete — the main point is that all three of these pieces are very large. As Figure 13.7 also makes clear, there are pages that belong to none of IN OUT , or the , giant SCC — that is, they can neither reach the giant SCC nor be reached from it. These can be further classified as (3) Tendrils: The “tendrils” of the bow-tie consist of (a) the nodes reachable from IN that cannot reach the giant SCC, and (b) the nodes that can reach OUT but cannot be My song lyrics reached from the giant SCC. For example, the page in Figure 13.6 is IN but has no path to the giant an example of a tendril page, since it’s reachable from SCC. It’s possible for a tendril node to satisfy both (a) and (b), in which case it’s IN to OUT without touching the giant SCC. (For part of a “tube” that travels from example, if the page My song lyrics happened to link to Blog post about Company Z in Figure 13.6, it would be part of a tube.) Disconnected: Finally, there are nodes that would not have a path to the giant SCC (4) even if we completely ignored the directions of the edges. These belong to none of the preceding categories. Taken as a whole, then, the bow-tie picture of the Web provides a high-level view of the Web’s structure, based on its reachability properties and how its strongly connected components fit together. From it, we see that the Web contains a central “core” containing many of its most prominent pages, with many other nodes that lie upstream, downstream, or “off to the side” relative to this core. It is also a highly dynamic picture: as people create pages and links, the constituent pieces of the bow-tie are constantly shifting their boundaries, with nodes entering (and also leaving) the giant SCC over time. But subsequent studies suggest that the aggregate picture remains relatively stable over time, even as the detailed structure changes continuously. While the bow-tie picture gives us a global view of the Web, it doesn’t give us insight into the more fine-grained patterns of connections within the constituent parts — connections which could serve to highlight important Web pages or communities of thematically related

406 392 CHAPTER 13. THE STRUCTURE OF THE WEB pages. Addressing these latter issues will require more detailed network analysis, which we undertake in Chapter 14; as we will see, this requires us to think about what it means for a Web page to occupy a “powerful” position, and it leads to methods that bear directly on the design of Web search engines. More generally, network analysis of the Web forms one ingredient in a broader emerging research agenda that aims to understand the structure, behavior, and evolution of the Web as a phenomenon in itself [220]. 13.5 The Emergence of Web 2.0 The increasing richness of Web content, which we’ve encountered through the distinction between navigational and transactional links, fueled a series of further significant changes in the Web during its second decade of existence, between 2000 and 2009. Three major forces behind these changes were (i) the growth of Web authoring styles that enabled many people to collectively create and maintain shared content; (ii) the movement of people’s personal on-line data (including e-mail, calendars, photos, and videos) from their own computers to services offered and hosted by large companies; and (iii) the growth of linking styles that emphasize on-line connections between people, not just between documents. Taken together, this set of changes altered user experience on the Web sufficiently that technologists led by Tim O’Reilly and others began speaking in 2004 and 2005 about the emergence of Web 2.0 [335]. While the term evokes images of a new software release, there is agreement that Web 2.0 is principally “an attitude, not a technology” [125]. There has never been perfect consensus on the meaning of the term, but it has generally connoted a major next step in the evolution of the Web, driven by versions of principles (i), (ii), and (iii) above (as well as others), and arising from a confluence of factors rather than any one organization’s centralized decisions. Indeed, there was an explosion of prominent new sites during the period 2004–2006 that exemplified these three principles (i), (ii), and (iii), sometimes in combination. To name just a few examples: Wikipedia grew rapidly during this period, as people embraced the idea of collectively editing articles to create an open encyclopedia on the Web (principle (i)); Gmail and other on-line e-mail services encouraged individuals to let companies like Google host their archives of e-mail (principle (ii)); MySpace and Facebook achieved widespread adoption with a set of features that primarily emphasized the creation of on-line social networks (principle (iii)).

407 13.5. THE EMERGENCE OF WEB 2.0 393 Many sites during this period combined versions of all three principles. For example, the photo-sharing site Flickr and subsequently the video-sharing site YouTube provided users with a centralized place to store their own photos and videos (principle (ii)), simultane- ously enriched this content by allowing a large user community to tag and comment on it (principle (i)), and allowed users to form social connections to others whose content they followed (principle (iii)). The micro-blogging service Twitter extended principle (ii) further, by creating an on-line forum for personal data (in the form of short real-time descriptions of one’s experiences, thoughts, and questions) that would otherwise never have been recorded at all. Because many people will all comment at roughly the same time on a current event in the news, Twitter also creates collective summaries of worldwide reactions to such events (principle (i)), and allows users to construct links by which they follow the writings of other users (principle (iii)). Even if some (or many) of these specific sites are replaced by others in the coming years, the principles they embody have clearly brought about a lasting change in perspective on Web content. These principles have also led to a point that we discussed early in Chapter 1: designers of Web sites today need to think not just about organizing information, but also about the social feedback effects inherent in maintaining an audience of millions of users — users who are able to interact directly not just with the site itself but with one another. This helps to explain why many of the central concepts in this book relate to phenomena that surround this current phase of the Web’s evolution. For example, many of the key rallying cries that accompanied the emergence of Web 2.0 are in a sense shorthand for social phenomena that we discuss in other chapters: “Software that gets better the more people use it.” A core principle of Web 2.0 is that • on-line Web sites and services can become more appealing to users — and in fact, often genuinely more valuable to them — as their audiences grow larger. When and how this process takes place forms a central focus in chapters from the next two parts of the book, particularly Chapters 16, 17, and 19. • “The wisdom of crowds.” The collaborative authoring of an encyclopedia by millions on Wikipedia, the elevation of news content by group evaluation on Digg, the fact that photos of breaking news now often appear on Flickr before they do in the mainstream news, and many similar developments highlighted the ways in which the audience of a Web 2.0 site — each contributing specific expertise and sometimes misinformation — can produce a collective artifact of significant value. But the “wisdom of the crowds,” as this process is now often called, is a subtle phenomenon that can fail as easily as it can succeed. In Chapter 22 we discuss some of the basic work in the theory of markets that helps explain how collective information residing in a large group can be synthesized successfully; and in Chapter 16 we describe ways in which this process can also lead to unexpected and sometimes undesirable outcomes.

408 394 CHAPTER 13. THE STRUCTURE OF THE WEB “The Long Tail.” With many people contributing content to a Web 2.0 site, the system • will generally reach a balance between a small amount of hugely popular content and a “long tail” of content with various levels of niche appeal. Such distributions of popularity have important consequences, and will be the topic of Chapter 18. In addition to the ideas suggested by such mantras, the premises underlying Web 2.0 appear in many other contexts in the book as well. The social-networking aspects of Web 2.0 sites provide rich data for large-studies of social network structure, as discussed in Chapter 2. They offer a basis for empirical studies of the ideas of triadic closure and group affiliation from Chapters 3 and 4, and have been used to evaluate the theories underlying the small-world phenomenon in Chapter 20. Moreover, many of the features that are common to Web 2.0 sites are designed to ex- plicitly steer some of the underlying social feedback mechanisms in desirable directions. For reputation systems and trust systems example, enable users to provide signals about the behavior — and misbehavior — of other users. We discussed such systems in the context of structural balance in Chapter 5, and will see their role in providing information essential to the functioning of on-line markets in Chapter 22. Web 2.0 sites also make use of recommen- dation systems , to guide users toward items that they may not know about. In addition to serving as helpful features for a site’s users, such recommendation systems interact in com- plex but important ways with distributions of popularity and the long tail of niche content, as we will see in Chapter 18. The development of the current generation of Web search engines, led by Google, is sometimes seen as a crucial step in the pivot from the early days of the Web to the era of Web 2.0. In the next two chapters we will discuss how thinking of the Web as a network helped form the foundation for these search engines, and how models based on matching markets helped turn search into a profitable business. 13.6 Exercises 1. Consider the set of 18 Web pages drawn in Figure 13.8, with links forming a directed graph. Which nodes constitute the largest strongly connected component (SCC) in this graph? Taking this as the giant SCC , which nodes then belong to the sets IN and OUT defined in Section 13.4? Which nodes belong to the tendrils of the graph? 2. As new links are created and old ones are removed among an existing set of Web pages, the pages move between different parts of the bow-tie structure. (a) Name an edge you could add or delete from the graph in Figure 13.8 so as to increase the size of the largest strongly connected component.

409 13.6. EXERCISES 395 1 5 4 2 3 10 6 8 9 7 16 15 1 1 13 12 14 17 18 Figure 13.8: A directed graph of Web pages. (b) Name an edge you could add or delete from the graph in Figure 13.8 so as to IN . increase the size of the set (c) Name an edge you could add or delete from the graph in Figure 13.8 so as to increase the size of the set OUT . 3. In Exercise 2, we considered how the consistuent parts of the bow-tie structure change as edges are added to or removed from the graph. It’s also interesting to ask about the magnitude of these changes. (a) Describe an example of a graph where removing a single edge can reduce the size of the largest strongly connected component by at least 1000 nodes. (Clearly you shouldn’t attempt to draw the full graph; rather, you can describe it in words, and also draw a schematic picture if it’s useful.) (b) Describe an example of a graph where adding a single edge can reduce the size of the set OUT by at least 1000 nodes. (Again, you should describe the graph rather than actually drawing it.)

410 396 CHAPTER 13. THE STRUCTURE OF THE WEB

411 Chapter 14 Link Analysis and Web Search 14.1 Searching the Web: The Problem of Ranking When you go to Google and type “Cornell,” the first result it shows you is www.cornell.edu, the home page of Cornell University. It’s certainly hard to argue with this as a first choice, but how did Google “know” that this was the best answer? Search engines determine how to rank pages using automated methods that look at the Web itself, not some external source of knowledge, so the conclusion is that there must be enough information intrinsic to the Web and its structure to figure this out. Before discussing some of the ideas behind the ranking of pages, let’s begin by considering a few of the basic reasons why it’s a hard problem. First, search is a hard problem for com- puters to solve in any setting, not just on the Web. Indeed, the field of information retrieval [36, 360] has dealt with this problem for decades before the creation of the Web: automated information retrieval systems starting in the 1960s were designed to search repositories of newspaper articles, scientific papers, patents, legal abstracts, and other document collections in reponse to keyword queries. Information retrieval systems have always had to deal with the problem that keywords are a very limited way to express a complex information need; in addition to the fact that a list of keywords is short and inexpressive, it suffers from the problems of synonymy (multiple ways to say the same thing, so that your search for recipes involving scallions fails because the recipe you wanted called them “green onions”) and pol- ysemy (multiple meanings for the same term, so that your search for information about the animal called a jaguar instead produces results primarily about automobiles, football players, and an operating system for the Apple Macintosh.) For a long time, up through the 1980s, information retrieval was the province of reference D. Easley and J. Kleinberg. Networks, Crowds, and Markets: Reasoning about a Highly Connected World . Cambridge University Press, 2010. Draft version: June 10, 2010. 397

412 398 CHAPTER 14. LINK ANALYSIS AND WEB SEARCH librarians, patent attorneys, and other people whose jobs consisted of searching collections of documents; such people were trained in how to formulate effective queries, and the documents they were searching tended to be written by professionals, using a controlled style and vocabulary. With the arrival of the Web, where everyone is an author and everyone is a searcher, the problems surrounding information retrieval exploded in scale and complexity. To begin with, the diversity in authoring styles makes it much harder to rank documents according to a common criterion: on a single topic, one can easily find pages written by experts, novices, children, conspiracy theorists — and not necessarily be able to tell which is which. Once upon a time, the fact that someone had the money and resources to produce a professional-looking, typeset, bound document meant that they were very likely (even if not always) someone who could be taken seriously. Today, anyone can create a Web page with high production values. There is a correspondingly rich diversity in the set of people issuing queries, and the problem of multiple meanings becomes particularly severe. For example, when someone issues the single-word query “Cornell,” a search engine doesn’t have very much to go on. Did the searcher want information about the university? The university’s hockey team? The Lab of Ornithology run by the university? Cornell College in Iowa? The Nobel-Prize-winning physicist Eric Cornell? The same ranking of search results can’t be right for everyone. These represent problems that were also present in traditional information retrieval sys- tems, just taken to new extremes. But the Web also introduces new kinds of problems. One is the dynamic and constantly-changing nature of Web content. On September 11, 2001, many people ran to Google and typed “World Trade Center.” But there was a mismatch between what people thought they could get from Google and what they really got: since Google at the time was built on a model in which it periodically collected Web pages and indexed them, the results were all based on pages that were gathered days or weeks earlier, and so the top results were all descriptive pages about the building itself, not about what had occurred that morning. In response to such events, Google and the other main search engines built specialized “News Search” features, which collect articles more or less contin- uously from a relatively fixed number of news sources, so as to be able to answer queries about news stories minutes after they appear. Even today, such news search features are only partly integrated into the core parts of the search engine interface, and emerging Web sites such as Twitter continue to fill in the spaces that exist between static content and real-time awareness. More fundamental still, and at the heart of many of these issues, is the fact that the Web has shifted much of the information retrieval question from a problem of scarcity to a problem of abundance . The prototypical applications of information retrieval in the pre-Web era had a “needle-in-a-haystack” flavor — for example, an intellectual-property attorney might express the information need, “find me any patents that have dealt with the design

413 14.2. LINK ANALYSIS USING HUBS AND AUTHORITIES 399 of elevator speed regulators based on fuzzy-logic controllers.” Such issues still arise today, but the hard part for most Web searches carried out by the general public is in a sense the opposite: to filter, from among an enormous number of relevant documents, the few that are most important. In other words, a search engine has no problem finding and indexing literally millions of documents that are genuinely relevant to the one-word query “Cornell”; the problem is that the human being performing the search is going to want to look at only a few of these. Which few should the search engine recommend? An understanding of the network structure of Web pages will be crucial for addressing these questions, as we now discuss. 14.2 Link Analysis using Hubs and Authorities So we’re back to our question from the beginning of the chapter: in response to the one-word query “Cornell,” what are the clues that suggest Cornell’s home page, www.cornell.edu, is a good answer? Voting by In-Links. In fact, there is a natural way to address this, provided we start from the right perspective. This perspective is to note that there is not really any way to use features purely internal to the page www.cornell.edu to solve this problem: it does not use the word “Cornell” more frequently or more prominently than thousands of other pages. and so there is nothing on the page itself that makes it stand out. Rather, it stands out because of features on other Web pages: when a page is relevant to the query “Cornell,” very often www.cornell.edu is among the pages it links to. This is the first part of the argument that links are essential to ranking: that we can use them to assess the authority of a page on a topic, through the implicit endorsements that other pages on the topic confer through their links to it. Of course, each individual link may have many possible meanings: it may be off-topic; it may convey criticism rather than endorsement; it may be a paid advertisement. It is hard for search engines to automatically assess the intent of each link. But we hope that in aggregate, if a page receives many links from other relevant pages, then it is receiving a kind of collective endorsement. In the case of the query “Cornell,” we could operationalize this by first collecting a large sample of pages that are relevant to the query — as determined by a classical, text-only, information retrieval approach. We could then let pages in this sample “vote” through their links: which page on the Web receives the greatest number of in-links from pages that are relevant to Cornell? Even this simple measure of link-counting works quite well for queries such as “Cornell,” where, ultimately, there is a single page that most people agree should be ranked first.

414 400 CHAPTER 14. LINK ANALYSIS AND WEB SEARCH SJ Merc 2 votes News all St. W 2 votes Journal Y ork New 4 votes imes T oday T USA 3 votes Facebook 1 vote ahoo! Y 3 votes Amazon 3 votes Figure 14.1: Counting in-links to pages for the query “newspapers.” A List-Finding Technique. It’s possible to make deeper use of the network structure than just counting in-links, and this brings us to the second part of the argument that links are essential. Consider, as a typical example, the one-word query “newspapers.” Unlike the query “Cornell,” there is not necessarily a single, intuitively “best” answer here; there are a number of prominent newspapers on the Web, and an ideal answer would consist of a list of the most prominent among them. With the query “Cornell,” we discussed collecting a sample of pages relevant to the query and then let them vote using their links. What happens if we try this for the query “newspapers”? What you will typically observe, if you try this experiment, is that you get high scores for a mix of prominent newspapers (i.e. the results you’d want) along with pages that are going to receive a lot of in-links no matter what the query is — pages like Yahoo!, Facebook, Amazon, and others. In other words, to make up a very simple hyperlink structure for purposes of

415 14.2. LINK ANALYSIS USING HUBS AND AUTHORITIES 401 SJ Merc 2 votes News 8 W all St. 2 votes Journal 1 1 7 Y New ork 4 votes imes T 3 5 6 6 USA T oday 3 votes 3 Facebook 1 vote 3 ahoo! Y 3 votes Amazon 3 votes Figure 14.2: Finding good lists for the query “newspapers”: each page’s value as a list is written as a number inside it. this example, we’d see something like Figure 14.1: the unlabeled circles represent our sample of pages relevant to the query “newspapers,” and among the four pages receiving the most votes from them, two are newspapers (New York Times and USA Today) and two are not (Yahoo! and Amazon). This example is designed to be small enough to try by hand; in a real setting, of course there would be many plausible newspaper pages and many more off-topic pages. But votes are only a very simple kind of measure that we can get from the link structure — there is much more to be discovered if we look more closely. To try getting more, we ask a different question. In addition to the newspapers themselves, there is another kind of useful answer to our query: pages that compile lists of resources relevant to the topic. Such pages exist for most broad enough queries: for “newspapers,” they would correspond to lists

416 402 CHAPTER 14. LINK ANALYSIS AND WEB SEARCH SJ Merc new score: 19 News 8 W all St. new score: 19 Journal 1 1 7 Y ork New new score: 31 T imes 3 6 5 6 T USA oday new score: 24 3 Facebook new score: 5 3 Y ahoo! new score: 15 Amazon new score: 12 Figure 14.3: Re-weighting votes for the query “newspapers”: each of the labeled page’s new score is equal to the sum of the values of all lists that point to it. of links to on-line newspapers; for “Cornell,” one can find many alumni who maintain pages with links to the University, its hockey team, its Medical School, its Art Museum, and so forth. If we could find good list pages for newspapers, we would have another approach to the problem of finding the newspapers themselves. In fact, the example in Figure 14.1 suggests a useful technique for finding good lists. We notice that among the pages casting votes, a few of them in fact voted for many of the pages that received a lot of votes. It would be natural, therefore, to suspect that these pages have some sense where the good answers are, and to score them highly as lists. Concretely, we could say that a page’s value as a list is equal to the sum of the votes received by all pages that it voted for. Figure 14.2 shows the result of applying this rule to the pages casting votes in our example.

417 14.2. LINK ANALYSIS USING HUBS AND AUTHORITIES 403 If we believe that pages scoring well as lists The Principle of Repeated Improvement. actually have a better sense for where the good results are, then we should weight their votes more heavily. So, in particular, we could tabulate the votes again, but this time giving each page’s vote a weight equal to its value as a list. Figure 14.3 shows what happens when we do this on our example: now the other newspapers have surpassed the initially high-scoring Yahoo! and Amazon, because these other newspapers were endorsed by pages that were estimated to be good lists. In fact, you can recognize the intuition behind this re-weighting of votes in the way we evaluate endorsements in our everyday lives. Suppose you move to a new town and hear restaurant recommendations from a lot of people. After discovering that certain restaurants people in fact had mentioned most get mentioned by a lot of people, you realize that certain of these highly-recommended restaurants when you asked them. These people play the role of the high-value lists on the Web, and it’s only natural to go back and take more seriously the more obscure restaurants that they recommended, since you now particularly trust their judgment. This last step is exactly what we are doing in re-weighting the votes for Web pages. The final part of the argument for link analysis is then the following: Why stop here? If we have better votes on the right-hand-side of the figure, we can use these to get still more refined values for the quality of the lists on the left-hand-side of the figure. And with more refined estimates for the high-value lists, we can re-weight the votes that we apply to the right-hand-side once again. The process can go back and forth forever: it can be viewed as a , in which each refinement to one side of the figure Principle of Repeated Improvement enables a further refinement to the other. Hubs and Authorities. This suggests a ranking procedure that we can try to make precise, as follows [247]. First, we’ll call the kinds of pages we were originally seeking — the prominent, highly endorsed answers to the queries — the for the query. We’ll call authorities the high-value lists the hubs for the query. Now, for each page p , we’re trying to estimate its value as a potential authority and as a potential hub, and so we assign it two numerical scores: auth ( p ) and hub ( p ). Each of these starts out with a value equal to 1, indicating that we’re initially agnostic as to which is the best in either of these categories. Now, voting – in which we use the quality of the hubs to refine our estimates for the quality of the authorities – is simply the following: Authority Update Rule: For each page p , update auth ( p ) to be the sum of the hub scores of all pages that point to it. On the other hand, the list-finding technique – in which we use the quality of the authorities to refine our estimates for the quality of the hubs, is the following:

418 404 CHAPTER 14. LINK ANALYSIS AND WEB SEARCH SJ Merc normalized .152 News 8 all St. W normalized .152 Journal 1 1 7 Y ork New normalized .248 T imes 3 6 5 6 USA T oday normalized .192 3 Facebook normalized .040 3 ahoo! Y normalized .120 Amazon normalized .096 Figure 14.4: Re-weighting votes after normalizing for the query “newspapers.” Hub Update Rule: For each page p , update hub ( p ) to be the sum of the authority scores of all pages that it points to. Notice how a single application of the Authority Update Rule (starting from a setting in which all scores are initially 1) is simply the original casting of votes by in-links. A single application of the Authority Update Rule followed by a single application the Hub Update Rule produces the results of the original list-finding technique. In general, the Principle of Repeated Improvement says that to obtain better estimates, we should simply apply these rules in alternating fashion, as follows. • We start with all hub scores and all authority scores equal to 1. • We choose a number of steps k .

419 14.2. LINK ANALYSIS USING HUBS AND AUTHORITIES 405 SJ Merc limit .199... News .249 all St. W limit .199... Journal .321 .181 ork New Y limit .304... imes T .015 .018 .123 .088 USA oday T limit .205... .003 Facebook limit .043... .003 Y ahoo! limit .042... Amazon limit .008... Figure 14.5: Limiting hub and authority values for the query “newspapers.” • We then perform a sequence of k hub-authority updates. Each update works as follows: – First apply the Authority Update Rule to the current set of scores. – Then apply the Hub Update Rule to the resulting set of scores. • At the end, the hub and authority scores may involve numbers that are very large. But we only care about their relative sizes, so we can to make them smaller: we normalize divide down each authority score by the sum of all authority scores, and divide down each hub score by the sum of all hub scores. (For example, Figure 14.4 shows the result of normalizing the authority scores that we determined in Figure 14.3.) What happens if we do this for larger and larger values of k ? It turns out that the normalized values actually converge to limits as k goes to infinity: in other words, the

420 406 CHAPTER 14. LINK ANALYSIS AND WEB SEARCH results stabilize so that continued improvement leads to smaller and smaller changes in the values we observe. We won’t prove this right now, but we provide a proof in Section 14.6 at the end of this chapter. Moreover, the analysis in that section shows that something even deeper is going on: except in a few rare cases (characterized by a certain kind of degenerate property of the link structure), we reach the same limiting values no matter what we choose as the initial hub and authority values, provided only that all of them are positive. In other words, the limiting hub and authority values are a property purely of the link structure, not of the initial estimates we use to start the process of computing them. (For the record, the limiting values for our “newspapers” example are shown, to three decimal places, in Figure 14.5.) Ultimately, what these limiting values correspond to is a kind of equilibrium: their relative sizes remain unchanged if we apply the Authority Update Rule or the Hub Update Rule. As such, they reflect the balance between hubs and authorities that provided the initial intuition for them: your authority score is proportional to the hub scores of the pages that point to you, and your hub score is proportional to the authority scores of the pages you point to. 14.3 PageRank The intuition behind hubs and authorities is based on the idea that pages play multiple roles in the network, and in particular that pages can play a powerful endorsement role without themselves being heavily endorsed. For queries with a commercial aspect — such as our query for newspapers in the previous section, or searches for particular products to purchase, or more generally searches that are designed to yield corporate pages of any type — there is a natural basis for this intuition. Competing firms will not link to each other, except in unusual circumstances, and so they can’t be viewed as directly endorsing each other; rather, the only way to conceptually pull them together is through a set of hub pages that link to all of them at once. In other settings on the Web, however, endorsement is best viewed as passing directly from one prominent page to another — in other words, a page is important if it is cited by other important pages. This is often the dominant mode of endorsement, for example, among academic or governmental pages, among bloggers, or among personal pages more generally. It is also the dominant mode in the scientific literature. And it is this mode of endorsement that forms the basis for the PageRank measure of importance [79]. As with hubs and authorities, the intuition behind PageRank starts with simple voting based on in-links, and refines it using the Principle of Repeated Improvement. In particular, the Principle is applied here by having nodes repeatedly pass endorsements across their out-going links, with the weight of a node’s endorsement based on the current estimate of its PageRank: nodes that are currently viewed as more important get to make stronger

421 14.3. PAGERANK 407 A B C F D E G H A has the largest PageRank, followed by B Figure 14.6: A collection of eight pages: C and (which collect endorsements from A ). endorsements. The basic definition of PageRank. Intuitively, we can think of PageRank as a kind of “fluid” that circulates through the network, passing from node to node across edges, and pooling at the nodes that are the most important. Specifically, PageRank is computed as follows. • In a network with nodes, we assign all nodes the same initial PageRank, set to be n /n 1 . • We choose a number of steps k . • We then perform a sequence of k updates to the PageRank values, using the following rule for each update: Basic PageRank Update Rule: Each page divides its current PageRank equally across its out-going links, and passes these equal shares to the pages it points to. (If a page has no out-going links, it passes all its current PageRank to itself.) Each page updates its new PageRank to be the sum of the shares it receives.

422 408 CHAPTER 14. LINK ANALYSIS AND WEB SEARCH 4/13 A 2/13 2/13 B C 1/13 1/13 1/13 1/13 F E D G 1/13 H Figure 14.7: Equilibrium PageRank values for the network of eight Web pages from Fig- ure 14.6. Notice that the total PageRank in the network will remain constant as we apply these steps: since each page takes its PageRank, divides it up, and passes it along links, PageRank is never created nor destroyed, just moved around from one node to another. As a result, we don’t need to do any normalizing of the numbers to prevent them from growing, the way we had to with hub and authority scores. As an example, let’s consider how this computation works on the collection of 8 Web pages in Figure 14.6. All pages start out with a PageRank of 1 / 8, and their PageRank values after the first two updates are given by the following table: A Step C D E F G H B 1 1/2 1/16 1/16 1/16 1/16 1/16 1/16 1/8 2 3/16 1/4 1/32 1/32 1/32 1/32 1/16 1/4 A gets a PageRank of 1 / 2 after the first update because it gets all of F ’s, For example, G ’s, and H ’s PageRank, and half each of D ’s and E ’s. On the other hand, B and C each get half of ’s PageRank, so they only get 1 / 16 each in the first step. But once A acquires A a lot of PageRank, B and C benefit in the next step. This is in keeping with the principle of repeated improvement: after the first update causes us to estimate that A is an important page, we weigh its endorsement more highly in the next update.

423 14.3. PAGERANK 409 As with hub-authority computations, one can prove Equilibrium Values of PageRank. that except in certain degenerate special cases the PageRank values of all nodes converge to goes to infinity. limiting values as the number of update steps k Because PageRank is conserved throughout the computation — with the total PageRank in the network equal to one — the limit of the process has a simple interpretation. We can think of the limiting PageRank values, one value for each node, as exhibiting the following : if we take the limiting PageRank values and apply one step of the Basic kind of equilibrium PageRank Update Rule, then the values at every node remain the same. In other words, the limiting PageRank values regenerate themselves exactly when they are updated. This description gives a simple way to check whether an assignment of numbers to a set of Web pages forms such an equilibrium set of PageRank values: we check that they sum to 1, and we check that when we apply the Basic PageRank Update Rule, we get the same values back. For example, on the network of Web pages from Figure 14.6, we can check that the values shown in Figure 14.7 have the desired equilibrium property — assigning a PageRank , and 1 / A , 2 / 13 to each of B and C 13 to page / 13 to the five other pages achieves this of 4 equilibrium. Now, depending on the network structure, the set of limiting values may not be the only ones that exhibit this kind of equilibrium. However, one can show that if the network is strongly connected — that is, each node can reach each other node by a directed path, following the definition from Chapter 13 — then there is a unique set of equilibrium values, and so whenever the limiting PageRank values exist, they are the only values that satisfy this equilibrium. Scaling the definition of PageRank. There is a difficulty with the basic definition of PageRank, however: in many networks, the “wrong” nodes can end up with all the PageRank. Fortunately, there is a simple and natural way to fix this problem. yielding the actual definition of PageRank that is used in practice. Let’s first describe the problem and then its solution. To trigger the problem, suppose we take the network in Figure 14.6 and make a small F and change, so that now point to each other rather than pointing to A . The result is G shown in Figure 14.8. Clearly this ought to weaken A somewhat, but in fact a much more extreme thing happens: PageRank that flows from C to F and G can never circulate back into the rest of the network, and so the links out of function as a kind of “slow leak” that C eventually causes all the PageRank to end up at F and G . We can indeed check that by repeatedly running the Basic PageRank Update Rule, we converge to PageRank values of 1 / 2 for each of F and G , and 0 for all other nodes. This is clearly not what we wanted, but it’s an inevitable consequence of the definition.

424 410 CHAPTER 14. LINK ANALYSIS AND WEB SEARCH A C B E G F D H F and G have changed their links to Figure 14.8: The same collection of eight pages, but A point to each other instead of to . Without a smoothing effect, all the PageRank would go to and G . F And it becomes a problem in almost any real network to which PageRank is applied: as long as there are small sets of nodes that can be reached from the rest of the graph, but 1 have no paths back, then PageRank will build up there. Fortunately, there is a simple and natural way to modify the definition of PageRank to get around this problem, and it follows from the “fluid” intuition for PageRank. Specifically, if we think about the (admittedly simplistic) question of why all the water on earth doesn’t inexorably run downhill and reside exclusively at the lowest points, it’s because there’s a counter-balancing process at work: water also evaporates and gets rained back down at higher elevations. scaling factor s that should be strictly between 0 We can use this idea here. We pick a and 1. We then replace the Basic PageRank Update Rule with the following: First apply the Basic PageRank Update Rule. Scaled PageRank Update Rule: Then scale down all PageRank values by a factor of s . This means that the total PageRank in the network has shrunk from 1 to s . We divide the residual 1 − s units of PageRank equally over all nodes, giving (1 s ) /n to each. − 1 If we think back to the bow-tie structure of the Web from Chapter 13, there is a way to describe the problem in those terms as well: there are many “slow leaks” out of the giant SCC, and so in the limit, all nodes in the giant SCC will get PageRank values of 0; instead, all the PageRank will end up in the set OUT of downstream nodes.

425 14.3. PAGERANK 411 This rule also preserves the total PageRank in the network, since it is just based on redis- tribution according to a different “water cycle” that evaporates 1 units of PageRank in − s each step and rains it down uniformly across all nodes. One can show that repeated appli- The Limit of the Scaled PageRank Update Rule. cation of the Scaled PageRank Update Rule converges to a set of limiting PageRank values goes to infinity. Moreover, for any network, these limiting values as the number of updates k form the unique equilibrium for the Scaled PageRank Update Rule: they are the unique set of values that remain unchanged under the application of this update rule. Notice, of course, that these values depend on our choice of the scaling factor s : we should really think of there s being a different update rule for each possible value of . This is the version of PageRank that is used in practice, with a scaling factor that is s 2 . . 9. The use of the scaling factor also turns out to usually chosen to be between 0 8 and 0 make the PageRank measure less sensitive to the addition or deletion of small numbers of nodes or links [268, 422]. Random walks: An equivalent definition of PageRank. To conclude our discussion in this section, we now describe an equivalent formulation of PageRank that looks quite different on the surface, but in fact leads to exactly the same definition. It works as follows. Consider someone who is randomly browsing a network of Web pages, such as the one in Figure 14.6. They start by choosing a page at random, picking each page with equal probability. They then follow links for a sequence of k steps: in each step, they pick a random out-going link from their current page, and follow it to where it leads. (If their current page has no out-going links, they just stay where they are.) Such an exploration of random walk on the network. We nodes performed by randomly following links is called a should stress that this is not meant to be an accurate model of an actual person exploring the Web; rather, it is a thought experiment that leads to a particular definition. In Section 14.6, we analyze this random walk and show the following fact: Claim: The probability of being at a page X after k steps of this random walk is precisely the PageRank of X after k applications of the Basic PageRank Update Rule. 2 As an aside about our earlier motivating example, one can check that using a value of s in this range doesn’t completely fix the problem with Figure 14.8: nodes F and G still get most (though no longer all) of the PageRank under the scaled update rule with such values of s . The problem is that an eight-node example is simply too small for the redistribution of the PageRank to truly offset the problem of a slow leak into a dead-end region of the network: on only eight nodes, a “slow leak” isn’t actually so slow. However, on large networks such as are used in real applications, the redistribution of PageRank works well to give nodes outside the giant strongly connected component of the network very small limiting PageRank values.

426 412 CHAPTER 14. LINK ANALYSIS AND WEB SEARCH Given that the two formulations of PageRank — based on repeated improvement and random walks respectively — are equivalent, we do not strictly speaking gain anything at a formal level by having this new definition. But the analysis in terms of random walks provides some additional intuition for PageRank as a measure of importance: the PageRank of a page X is the limiting probability that a random walk across hyperlinks will end up at X , as we run the walk for larger and larger numbers of steps. This equivalent definition using random walks also provides a new and sometimes useful perspective for thinking about some of the issues that came up earlier in the section. For F and example, the “leakage” of PageRank to nodes in Figure 14.8 has a natural interpre- G tation in terms of the random walk on the network: in the limit, as the walk runs for more F or G is converging to 1; and once it and more steps, the probability of the walk reaching F or G , it is stuck at these two nodes forever. Thus, the limiting probabilities reaches either of being at F and G are converging to 1 / 2 each, and the limiting probabilities are converging to 0 for all other nodes. We will also show in Section 14.6 how to formulate the Scaled PageRank Update Rule in terms of random walks. Rather than simply following a random edge in each step, the walker performs a “scaled” version of the walk as follows: With probability , the walker s − s follows a random edge as before; and with probability 1 , the walker jumps to a random node anywhere in the network, choosing each node with equal probability. 14.4 Applying Link Analysis in Modern Web Search The link analysis ideas described in Sections 14.2 and 14.3 have played an integral role in the ranking functions of the current generation of Web search engines, including Google, Yahoo!, Microsoft’s search engine Bing, and Ask. In the late 1990s, it was possible to produce reasonable rankings using these link analysis methods almost directly on top of conventional search techniques; but with the growth and enormously expanding diversity of Web content since then, link analysis ideas have been extended and generalized considerably, so that they are now used in a wide range of different ways inside the ranking functions of modern search engines. It is hard to say anything completely concrete about the current ranking functions of the main search engines, given that they are constantly evolving in complexity, and given that the search engine companies themselves are extremely secretive about what goes into their ranking functions. (There are good reasons for this secrecy, as we will discuss later.) But we can make general observations, coupled with sentiments that represent the conventional wisdom of the search community. In particular, PageRank was one of the original and central ingredients of Google, and it has always been a core component of its methodology. The importance of PageRank as a feature in Google’s ranking function has long been claimed

427 14.4. APPLYING LINK ANALYSIS IN MODERN WEB SEARCH 413 to be declining over time, however. For example, in 2003 and 2004, a significant overhaul of Google’s ranking function was generally believed to involve non-PageRank styles of link analysis, including a method called Hilltop [58], developed by Krishna Bharat and George Mihaila as an extension of the two-sided form of endorsement behind hubs and authorities. Around a similar time period, the search engine Ask rebuilt its ranking function around hubs and authorities, though its recent extensions have increasingly blended this in with many other features as well. Combining links, text, and usage data. While our emphasis on link analysis in this chapter was meant to motivate the ideas in a clean setting, in practice one clearly needs to closely integrate information from both network structure and textual content in order to produce the highest-quality search results. One particularly effective way to combine text and links for ranking is through the analysis of anchor text , the highlighted bits of clickable text that activate a hyperlink leading to another page [102]. Anchor text can be a highly succinct and effective description of the page residing at the other end of a link; for example, ” on someone’s Web page, it’s a good guess if you read “I am a student at Cornell University that clicking on the highlighted link associated with the text “Cornell University” will take 3 you to a page that is in some way about Cornell. In fact, the link analysis methods we have been describing can be easily extended to incorporate textual features such as anchor text. In particular, the basic forms of both hubs and authorities and PageRank perform updates by simply adding up values across links. But if certain links have highly relevant anchor text while others don’t, we can weight the contributions of the relevant links more heavily than the others; for example, as we pass hub or authority scores, or PageRank values, across a link, we can multiply them by a factor that indicates the quality of the anchor text on the link [57, 102]. In addition to text and links, search engines use many other features as well. For example, the way in which users choose to click or not click on a search result conveys a lot of information: if among a search engine’s ranked results for the query “Cornell,” most users skip the first result and click on the second, it suggests that the first two results should potentially be reordered. There is ongoing research on methods for tuning search results based on this type of feedback [228]. A final important aspect of Web search serves to illustrate a basic A moving target. game-theoretic principle that we have encountered many times already — that you should always expect the world to react to what you do. As search grew into the dominant means of accessing information on the Web, it mattered to a lot of people whether they ranked highly 3 Of course, not all anchor text is useful; consider the ubiquitous bit of Web page text, “For more informa- tion, click here .” Such examples make you realize that creating useful anchor text is an aspect of hypertext authoring style worth paying attention to.

428 414 CHAPTER 14. LINK ANALYSIS AND WEB SEARCH in search engine results. For example, many small companies had business models that increasingly depended on showing up among the first screen of Google’s results for common queries ranging from “Caribbean vacations” to “vintage records.” An update to Google’s ranking function that pushed them off the first screen could spell financial ruin. Indeed, search-industry publications began naming some of Google’s more significant updates to its core ranking function in the alphabetic style usually reserved for hurricanes — and the analogy was an apt one, since each of these updates was an unpredictable act of nature (in this case, Google) that inflicted millions of dollars of economic damage. With this in mind, people who depended on the success of their Web sites increasingly began modifying their Web-page authoring styles to score highly in search engine rankings. For people who had conceived of Web search as a kind of classical information retrieval application, this was something novel. Back in the 1970s and 1980s, when people designed information retrieval tools for scientific papers or newspaper articles, authors were not overtly 4 writing their papers or abstracts with these search tools in mind. From the relatively early days of the Web, however, people have written Web pages with search engines quite explicitly in mind. At first, this was often done using over-the-top tricks that aroused the ire of the search industry; as the digital librarian Cliff Lynch noted at the time, “Web search is a new kind of information retrieval application in that the documents are actively behaving badly.” Over time though, the use of focused techniques to improve a page’s performance in search engine rankings became regularized and accepted, and guidelines for designing these techniques emerged; a fairly large industry known as (SEO) came search engine optimization into being, consisting of search experts who advise companies on how to create pages and sites that rank highly. And so to return to the game-theoretic view: the growth of SEO followed naturally once search became such a widespread application on the Web; it simply mattered too much to too many people that they be easily findable through search. These developments have had several consequences. First, they mean that for search engines, the “perfect” ranking function will always be a moving target: if a search engine maintains the same method of ranking for too long, Web-page authors and their consultants become too effective at reverse-engineering the important features, and the search engine is in effect no longer in control of what ranks highly. Second, it means that search engines are incredibly secretive about the internals of their ranking functions — not just to prevent competing search engines from finding out what they’re doing, but also to prevent designers of Web sites from finding out. And finally, with so much money at stake, the search industry turned these developments into a very successful business model based on advertising. Rather than simply showing results computed by a ranking function, the search engine offered additional slots on the 4 One can argue, of course, that at a less overt level, the development of standard authoring styles in these domains has been motivated by the goal of making these kinds of documents easier to classify and organize.

429 14.5. APPLICATIONS BEYOND THE WEB 415 main results page through a market in which sites could pay for placement. Thus, when you look at a search results page today, you see the results computed by the ranking function alongside the paid results. We have just seen some of the ideas behind ranking functions; the paid results, as we will see in the next chapter, are allocated using the kinds of matching markets discussed in Chapter 10. 14.5 Applications beyond the Web Link analysis techniques of the kind we’ve been discussing have been applied to a wide range of other settings, both before and after their use on the Web. In essentially any domain where information is connected by a network structure, it becomes natural to infer measures of authority from the patterns of links. As we discussed in Chapters 2 and 13, the study of citations among Citation Analysis. scientific papers and journals has a long history that significantly predates the Web [145]. A standard measure in this field is Garfield’s impact factor for a scientific journal [177], defined to be the average number of citations received by a paper in the given journal over the past two years. This type of voting by in-links can thus serve as a proxy for the collective attention that the scientific community pays to papers published in the journal. In the 1970s, Pinski and Narin extended the impact factor by taking into account the idea that not all citations should be counted equally — rather, citations from journals that are themselves high-impact should be viewed as more important [341]. This can be viewed as a use of the principle of repeated improvement, in the context of the scientific literature, just as we’ve seen it used for Web-page ranking. Pinski and Narin used this to formulate a notion of influence weights for journals [180, 341] that is defined very similarly to the notion of PageRank for Web pages. Link Analysis of U.S. Supreme Court Citations. Recently, researchers have adapted link analysis techniques from the Web to study the network of citations among legal decisions by U.S. courts [166, 377]. Citations are crucial in legal writing, to ground a decision in precedent and to explain the relation of a new decision to what has come before. Using link analysis in this context can help in identifying cases that play especially important roles in the overall citation structure. In one example of this style of research, Fowler and Jeon applied hub and authority measures to the set of all U.S. Supreme Court decisions, a collection of documents that spans more than two centuries [166]. They found that the set of Supreme Court decisions with high authority scores in the citation network align well with the more qualitative judgments of legal experts about the Court’s most important decisions. This includes some cases that

430 416 CHAPTER 14. LINK ANALYSIS AND WEB SEARCH th Figure 14.9: The rising and falling authority of key Fifth Amendment cases from the 20 century illustrates some of the relationships among them. (Image from [166].) acquired significant authority according to numerical measures shortly after they appeared, but which took much longer to gain recognition from the legal community. Supreme Court decisions also provide a rich setting for looking at how authority can change over long time periods. For example, Fowler and Jeon analyzed the rising and falling th century, as illustrated authority of some of the key Fifth Amendment cases from the 20 in Figure 14.9. In particular, Brown v. Mississippi — a 1936 case concerning confessions obtained under torture — began rising rapidly in authority in the early 1960s as the Warren Court forcefully took on a range of issues surrounding due process and self-incrimination. This development ultimately led to the landmark case Miranda v. Arizona in 1966 — and with this clear precedent established, the need for citations to quickly Brown v. Mississippi declined as the authority of Miranda shot upward. The analysis of Supreme Court citations also shows that significant decisions can vary widely in the rate at which they acquire authority. For example, Figure 14.10 (also from [166]) shows that Roe v. Wade — like Miranda — grew in authority very rapidly from the time it was first issued. On the other hand, the equally consequential Brown v. Board of Education only began acquiring significant authority in the citation network roughly a decade after it was issued. Fowler and Jeon argue that this trajectory aligns with legal scholars’

431 14.6. ADVANCED MATERIAL: SPECTRAL ANALYSIS, RANDOM WALKS, AND WEB SEARCH 417 Roe v. Wade and Brown v. Board of Education acquired authority at very Figure 14.10: different speeds. (Image from [166].) views of the case, writing, “Judicial specialists often point towards the ruling issued in Brown as an example of a precedent that was legally weak when first issued, and was strengthened through the Civil Rights Act of 1964 and its application in subsequent civil rights cases” [166]. This style of analysis thus shows how a strictly network-based analysis of a topic as intricate as legal precedent can reveal subtleties that align well with the views of the scholarly community. It also indicates some of the interesting effects that emerge when one tries to track the rising and falling pattern of authority in a complex domain — an activity that stands to provide important insights in many other settings as well. 14.6 Advanced Material: Spectral Analysis, Random Walks, and Web Search We now discuss how to analyze the methods for computing hub, authority, and PageRank values. This will require some basic familiarity with matrices and vectors. Building on this, we will show that the limiting values of these link-analysis measures can be interpreted as coordinates in eigenvectors of certain matrices derived from the underlying networks. The use of eigenvalues and eigenvectors to study the structure of networks is often referred to as

432 418 CHAPTER 14. LINK ANALYSIS AND WEB SEARCH node 1 0 1 1 0 1 0 0 1 node 2 0 1 0 0 0 1 0 0 node 4 node 3 adja- Figure 14.11: The directed hyperlinks among Web pages can be represented using an M cency matrix M : the entry is equal to 1 if there is a link from node i to node j , and ij M = 0 otherwise. ij spectral analysis the of graphs, and we will see that this theory forms a natural language for discussing the outcome of methods based on repeated improvement. A. Spectral Analysis of Hubs and Authorities Our first main goal will be to show why the hub-authority computation converges to limiting values for the hub and authority scores, as claimed in Section 14.2. As a first important step in this, we show how to write the Authority Update and Hub Update Rules from that section as matrix-vector multiplications. n pages as We will view a set of Adjacency Matrices and Hub/Authority Vectors. , 2 , 3 ,...,n , let’s encode a set of nodes in a directed graph. Given this set of nodes, labeled 1 th th × n matrix M as follows: the entry in the i row and the links among them in an j n , and column of , denoted M M , will be equal to 1 if there is a link from node i to node j ij it will be equal to 0 otherwise. We will call this the adjancency matrix of the network. Figure 14.11 shows an example of a directed graph and its adjacency matrix. Given a large set of pages, we expect that most of them will have very few outlinks relative to the total number of pages, and so this adjacency matrix will have most entries equal to 0. As a result, the adjacency matrix is not necessarily a very efficient way to represent the network, but as we will see, it is conceptually very useful. Now, since the hub and authority scores are lists of numbers — one associated with each

433 14.6. ADVANCED MATERIAL: SPECTRAL ANALYSIS, RANDOM WALKS, AND WEB SEARCH 419 node 1 1 1 2 9 0 0 6 0 7 0 1 1 = node 2 0 4 0 0 1 2 0 3 1 0 0 4 node 4 node 3 Figure 14.12: By representing the link structure using an adjacency matrix, the Hub and Authority Update Rules become matrix-vector multiplication. In this example, we show how multiplication by a vector of authority scores produces a new vector of hub scores. n nodes of the network — we can represent them simply as vectors in n dimensions, of the th i i . Specifically, we write h where the coordinate gives the hub or authority score of node h equal to the hub score of node i , and we similarly write for the vector of hub scores, with i a for the vector of authority scores. Let’s consider Hub and Authority Update Rules as Matrix-Vector Multiplication. the Hub Update Rule in terms of the notation we’ve just defined. For a node i , its hub score is updated to be the sum of a h over all nodes j to which i has an edge. Note that these j i nodes j M are precisely the ones for which = 1. Thus we can write the update rule as ij a ← M (14.1) , a a + h M + ··· + M 2 i 2 i in 1 n 1 i where we use the notation “ ← ” to mean that the quantity on the left-hand-side is updated to become the quantity on the right-hand-side. This is a correct way to write the update rule, since the values M as multipliers select out precisely the authority values that we wish ij to sum. But Equation (14.1) corresponds exactly to the definition of matrix-vector multiplication, so we can write it in the following equivalent way: h ← Ma. Figure 14.12 shows this for the example from Figure 14.11, with the authority scores (2 , 6 , 4 , 3) producing the hub scores (9 , 7 , 2 , 4) via the Hub Update Rule. Indeed, this is an example of a general principle: if you’re updating a collection of variables according to a rule that

434 420 CHAPTER 14. LINK ANALYSIS AND WEB SEARCH selects out certain ones to add up, you can often write this update rule as a matrix-vector multiplication for a suitably chosen matrix and vector. Specifying the Authority Update Rule in this style is strictly analogous, except that the is updated to be the sum of scores flow in the other direction across the edges. That is, a i over all nodes that have an edge to i , so h j j ← M a (14.2) h . + M h M h + + ··· 2 i 2 n 1 i ni 1 i This too corresponds to a matrix-vector multiplication, but using a matrix where the entries have all been “reflected” so that the roles of rows and columns are interchanged. This can T M , denoted M be specified using the , and defined by the property transpose of the matrix T T . Then Equation (14.2) M = M j,i : that is, M is the ( M ) entry of i,j that the ( ) entry of ji ij corresponds to the update rule T a h. ← M -step hub-authority computation. k So far we have discussed a single Unwinding the application of each of the update rules. What happens when we perform the k -step hub- ? authority computation for some large value of k 〉 0 〈 0 〉 〈 We start with initial vectors of authority and hub scores that we denote a and , h k 〉 〈 k 〉 〈 each of them equal to the vector all of whose coordinates are 1. Now, let and h denote a k the vectors of authority and hub scores after applications of the Authority and then Hub Update Rules in order, as in Section 14.2. If we simply follow the formulas above, we first find that 〈 〉 T 1 〈 0 〉 M = a h and 〉 〈 1 〉 0 〈 〈 1 〉 T MM = Ma = h . h That’s the result of the 1-step hub-authority computation. In the second step, we therefore get 2 〉 〉 〈 T 0 〈 1 〉 〈 T T MM h = h M = a M and 2 〉 〉 〈 2 〉 T T 〈 0 〉 T 2 〈 0 〈 Ma = ( MM MM ) h h h = MM . = One more step makes the pattern clear: 〉 〈 3 〉 0 〈 T T 〈 2 〉 2 T T 〉 T 0 T 〈 a M = ( M M M ) = MM h h M = h MM and 〉 〈 3 〉 0 〈 〈 3 〉 3 T T 〉 T 0 T 〈 ) h MM = ( MM MM MM = h Ma = h .

435 14.6. ADVANCED MATERIAL: SPECTRAL ANALYSIS, RANDOM WALKS, AND WEB SEARCH 421 〈 k 〉 〈 k 〉 and h are products of the Proceeding for larger numbers of steps, then, we find that a T 〈 k 〉 T and in alternating order, where the expression for M begins with M and M terms a 〈 k 〉 h begins with M . We can write this much more compactly as the expression for 〈 k 〉 〉 0 T 〈 T k − 1 M a M M = ( ) h and 0 〉 〈 〉 T k 〈 k h ) MM . = ( h -step hub-authority computation: So that’s a direct picture of what’s happening in the k the authority and hub vectors are the results of multiplying an initial vector by larger and T T larger powers of and MM M respectively. We now consider why this process converges M to stable values. Thinking about multiplication in terms of eigenvectors. Let’s keep in mind that, since the actual magnitude of the hub and authority values tend to grow with each update, they will only converge when we take normalization into account. To put it another way, it is the directions of the hub and authority vectors that are converging. Concretely, what we 〈 〉 〉 k k 〈 h a d and and c will show is that there are constants so that the sequences of vectors k k d c goes to infinity. converge to limits as k We’ll talk first about the sequence of hub vectors, and then we’ll consider the authority vectors largely by pursuing a direct analogy to the analysis of hub vectors. If 0 〈 k T 〉 k 〈 〉 ) ( h MM h = k k c c 〈∗〉 〈∗〉 , what properties do we expect h h should have? Since the is going to converge to a limit 〈∗〉 h shouldn’t change direction is converging, we expect that at the limit, the direction of T . That is, ), although its length might grow by a factor of c when it is multiplied by ( MM 〈∗〉 h will satisfy the equation we expect that T 〈∗〉 〈∗〉 ) h ( = MM ch . Any vector satisfying this property — that it doesn’t change direction when multiplied by a given matrix — is called an of the matrix, and the scaling constant c is called the eigenvector 〈∗〉 corresponding to the eigenvector. So we expect that should be an eigenvector eigenvalue h T of the matrix , with MM a corresponding eigenvalue. We now prove that the sequence of c 〈 k 〉 h T vectors MM indeed converges to an eigenvector of . k c To prove this, we use the following basic fact about matrices. We say that a square A is symmetric for each choice A = A matrix if it remains the same after transposing it: ji ij T of i and j , or in other words A = A . The fact we will use is the following [268]:

436 422 CHAPTER 14. LINK ANALYSIS AND WEB SEARCH A Any symmetric matrix n columns has a set of n eigenvectors n with rows and basis that are all unit vectors and all mutually orthogonal — that is, they form a n R for the space . T is symmetric, we can apply this fact to it. Let’s write the resulting mu- MM Since c , with corresponding eigenvalues z ,c tually orthogonal eigenvectors as ,...,c ,z ,...,z 2 1 n 1 n 2 c | ≥ | c respectively; and let’s order the eigenvalues so that | ≥ ··· ≥ | | . Furthermore, | c n 1 2 c c | > | to make things simpler in this explanation, let’s suppose that | | . (This essentially 2 1 always happens in link analysis applications; and below we explain the small changes that need to be made in the discussion if this assumption does not hold.) Now, given any vector T ) x is to first write x as a MM , a good way to think about the matrix-vector product ( x + ,...,z linear combination of the vectors . That is, with x = p for z z z p p z + + ··· 2 n 1 n 1 1 n 2 ,...,p , we have coefficients p n 1 T T ) x ( MM = ( )( p ) z z + p p z + + ··· MM 2 n 1 n 1 2 T T T p z + p = MM z z MM + ··· + p MM n 2 1 n 1 2 p , c z z + + p p c = z ··· + c 2 2 n 1 n 1 n 1 2 z where the third equality follows from the fact that each is an eigenvector. i ,z z ,...,z is a very useful set of coordinate axes for representing What this says is that 1 n 2 T MM x consists simply of replacing each term p in the representation : multiplication by z i i x c p of z by . We now see how this makes it easy to analyze multiplication by larger powers i i i T of MM , which will be the last step we need for showing convergence. Convergence of the hub-authority computation. We’ve seen that when we take any T vector p and write it in the form z produces + ··· + p x z MM , multiplication by n 1 n 1 T . When we multiply repeatedly by z c + ··· + c , each successive multipli- p p MM z n 1 n n 1 1 th c in front of the i term. Therefore we have cation introduces an additional factor of i k T k k k = c x c p . z z + ) MM p p ( z c + ··· + 2 n 1 n 1 2 2 1 n k 〉 T 〉 〈 0 〈 h Now let’s think of this in the context of the vectors of hub scores, where = ( MM ) . h 〈 0 〉 Recall that is just the fixed starting vector in which each coordinate is equal to 1; h z ,...,z as some linear combination it can be represented in terms of the basis vectors 1 n 〉 〈 0 ··· h z = + q . So z z q + q 1 n 2 n 2 1 〈 k 〉 k k T k k 〉 〈 0 z h c ) q , z z + c MM q q = h + ··· + c = ( (14.3) 2 2 n 1 n 1 n 2 1 k and if we divide both sides by , then we get c 1 ( ( ) ) k k 〉 〈 k c c h 2 n q = q (14.4) z . q + z z + ··· + 1 2 n n 2 1 k c c c 1 1 1

437 14.6. ADVANCED MATERIAL: SPECTRAL ANALYSIS, RANDOM WALKS, AND WEB SEARCH 423 c Recalling our assumption that | > | c | | (which we’ll relax shortly), we see that as k goes 2 1 to infinity, every term on the right-hand side but the first is going to 0. As a result, the k 〈 〉 h k is converging to the limit q as z goes to infinity. sequence of vectors 1 1 k c 1 Wrapping up. We’re essentially done at this point; but to round out the picture of con- vergence, we will show two important things. First, we need to make sure that the coefficient in the argument above is not zero, so as to be able to ensure so that the limit q q z is in 1 1 1 . Second, we will find that in fact a limit in the z fact a non-zero vector in the direction of 1 〉 〈 0 h z regardless is reached essentially : it is of our choice of starting hub scores direction of 1 in this sense that the limiting hub weights are really a function of the network structure, not the starting estimates. We will show these two facts in reverse order, considering the second point first. To begin with, then, let’s suppose we began the computation of the hub vector from a 0 〉 〈 different starting point: rather than having h be the vector with all coordinates equal to x x has a positive number 1, we picked some other starting vector . Let’s suppose only that in each coordinate — we’ll call such a vector a positive vector . As we noted before, any x can be written as x = p ··· z , and + vector p ,...,p z p , for some choice of multipliers 1 n n 1 n 1 k 〉 k 〈 k T k k p — in other words, z is converging to . Then /c h p z + z p ··· c so ( MM c = ) x 1 1 n n 1 1 1 n 1 still converging to a vector in the direction of even with this new choice for the starting z 1 〈 0 〉 = x . h vector q p and Now, let’s show why above are not zero (hence showing that the limits are 1 1 x p non-zero vectors). Given any vector , there is an easy way to think about the value of 1 x = p and in its representation as + ··· + p z : we just compute the inner product of z z n 1 n 1 1 x ,...,z are all mutually orthogonal, we have . Indeed, since the vectors z 1 n p · x = z , · ( p p z ) = + ··· p z z · ) = p z ( z ( · z p ) + z + ( z ··· · z ) + 1 2 2 1 n 1 1 1 n n n 1 1 1 1 1 p p z · z ) = ( . Since p since all terms in the last sum are 0 except for is just the inner 1 1 1 1 1 product of x and , we see that our sequence of hub vectors converges to a non-zero vector z 1 〉 〈 0 z provided only that our starting hub vector in the direction of = x is not orthogonal h 1 z to . 1 We now argue that no positive vector can be orthogonal to , which will conclude the z 1 picture of convergence that we’ve been seeking to establish. The argument works via the following steps. z 1. It is not possible for every positive vector to be orthogonal to , and so there is some 1 T k k MM ) positive vector x x/c for which ( converges to a non-zero vector z . p 1 1 1 k k T MM 2. Since the expressions for ( ) x/c only involve non-negative numbers, and their 1 values converge to p has only non-negative coordinates; and z z , it must be that p 1 1 1 1 p z must have at least one positive coordinate, since it is not equal to zero. 1 1

438 424 CHAPTER 14. LINK ANALYSIS AND WEB SEARCH 3. So if we consider the inner product of any positive vector with z p , the result must 1 1 z no . This be positive. Hence we conclude that positive vector can be orthogonal to 1 establishes that in fact the sequence of hub vectors converges to a vector in the direction z of when we start from any positive vector (including the all-ones vector), which is 1 what we wanted to show. This is pretty much the complete story, with the only loose end being our assumption c | | > that c | | . Let’s now relax this assumption. In general, there may be ` > 1 eigenvalues 1 2 c that are tied for the largest absolute value: that is, we can have | = ··· | | c , and then | = ` 1 c are all smaller in absolute value. While we won’t go through all the eigenvalues ,...,c n +1 ` T MM details here, it is not hard to show that all the eigenvalues of are non-negative, so in > c = ··· = c fact we have c 0. In this case, going back to Equations ≥ ≥ ··· ≥ c n +1 ` 1 ` (14.3) and (14.4), we have ( ) ) ( k k k k 〈 k 〉 z q + ··· + z q c c c c h n n 1 1 +1 ` n n 1 = + ··· + q = z q + z . z q q + z ··· + ` ` 1 1 +1 ` n n ` +1 k k c c c c 1 1 1 1 ` + 1 through n of this sum go to zero, and so the sequence converges to q + Terms ··· + z 1 1 c . Thus, when c = q z , we still have convergence, but the limit to which the sequence ` 1 ` 2 〈 0 〉 converges might now depend on the choice of the initial vector h (and particularly its inner product with each of z ). We should emphasize, though, that in practice, with real ,...,z ` 1 M with the and sufficiently large hyperlink structures, one essentially always gets a matrix T | has | c property that | > MM c . | 2 1 Finally, we observe that while this whole discussion has been in terms of the sequence of hub vectors, it can be adapted directly to analyze the sequence of authority vectors as T well. For the authority vectors, we are looking at powers of ( M M ), and so the basic result T M is that the vector of authority scores will converge to an eigenvector of the matrix M associated with its largest eigenvalue. B. Spectral Analysis of PageRank The analysis we’ve just seen emphasizes how eigenvectors arise naturally as the limits of repeated improvement. We now discuss how PageRank can be similarly analyzed using matrix-vector multiplication and eigenvectors. Recall that like hub and authority scores, the PageRank of a node is a numerical quantity that is repeatedly refined using an update rule. Let’s start by thinking about the Basic PageRank Update Rule from Section 14.3, and then move on to the scaled version. Under the basic rule, each node takes its current PageRank and divides it equally over all the nodes it points to. This suggests that the “flow” of PageRank specified by the update rule can be naturally represented using a matrix N as depicted in Figure 14.13: we define N to be the ij share of i ’s PageRank that j should get in one update step. This means that N = 0 if i ij

439 14.6. ADVANCED MATERIAL: SPECTRAL ANALYSIS, RANDOM WALKS, AND WEB SEARCH 425 node 1 0 1/2 1/2 0 1/2 0 1/2 0 node 2 0 1 0 0 0 1 0 0 node 4 node 3 Figure 14.13: The flow of PageRank under the Basic PageRank Update Rule can be repre- derived from the adjacency matrix sented using a matrix : the entry N N specifies the M ij portion of i ’s PageRank that should be passed to j in one update step. j , and otherwise N i is the reciprocal of the number of nodes that doesn’t link to points ij i links to , then N to. In other words, when = 1 /` j , where ` is the number of links out of i ij i . (If i has no outgoing links, then we define N = 1, in keeping with the rule that a node i ii with no outgoing links passes all its PageRank to itself.) In this way, is similar in spirit N M . i links to j to the adjacency matrix , but with a different definition when Now, let’s represent the PageRanks of all nodes using a vector , where the coordinate r r is the PageRank of node i . Using this notation, we can write the Basic PageRank Update i Rule as r r N (14.5) . r ← + N N + r ··· + 2 i 2 ni 1 n i 1 i This corresponds to multiplication by the transpose of the matrix, just as we saw for the Authority Update Rule; thus, Equation (14.5) can be written as T N ← r. (14.6) r The Scaled PageRank Update Rule can be represented in essentially the same way, but ̃ with a different matrix N to represent the different flow of PageRank, as indicated in Fig- ure 14.14. Recall that in the scaled version of the update rule, the updated PageRank is scaled down by a factor of s , and the residual 1 − s units are divided equally over all nodes. ̃ s Thus, we can simply define to be sN , and then the scaled update rule can + (1 − N ) /n ij ij be written as ̃ ̃ ̃ (14.7) N r r r N + . N ← + r ··· + 2 i 2 1 ni 1 n i i

440 426 CHAPTER 14. LINK ANALYSIS AND WEB SEARCH node 1 .05 .45 .45 .05 .45 .05 .05 .45 node 2 .85 .05 .05 .05 .05 .85 .05 .05 node 4 node 3 Figure 14.14: The flow of PageRank under the Scaled PageRank Update Rule can also be represented using a matrix derived from the adjacency matrix M (shown here with scaling ̃ ̃ . 8). We denote this matrix by s N ; the entry factor N = 0 specifies the portion of i ’s ij PageRank that should be passed to in one update step. j or equivalently T ̃ N ← r. r (14.8) Repeated Improvement Using the Scaled PageRank Update Rule. As we apply 〈 0 〉 the scaled update rule repeatedly, starting from an initial PageRank vector r , we produce 〉 2 〈 1 〉 〈 ,r r where each is obtained from the previous via multiplication a sequence of vectors ,... T ̃ N . Thus, unwinding this process, we see that by 〉 〉 〈 k k 〈 0 T ̃ r N ) = ( . r Moreover, since PageRank is conserved as it is updated — that is, the sum of the PageRanks at all nodes remains constant through the application of the scaled update rule — we don’t have to worry about normalizing these vectors as we proceed. So by analogy with the limiting values of the hub-authority computation (but with the added fact that normalization isn’t needed), one expects that if the Scaled PageRank Update 〈∗〉 T 〈∗〉 〈∗〉 ̃ N Rule converges to a limiting vector r , this limit should satisfy = r r — that is, we 〈∗〉 T 〈∗〉 ̃ r N to be an eigenvector of with corresponding eigenvalue 1. Such an r should expect has the property that it will not change under further refinements by the Scaled PageRank Update Rule. In fact, all this turns out to be true: repeated application of the Scaled PageRank Update 〈∗〉 Rule converges to precisely such an r . To prove this, however, we can’t use the same

441 14.6. ADVANCED MATERIAL: SPECTRAL ANALYSIS, RANDOM WALKS, AND WEB SEARCH 427 approach that we applied in the case of the hub-authority computation: there, the matrices T T MM M ) were symmetric, and so they had eigenvalues that were real involved ( M and numbers and orthogonal eigenvectors that formed a basis. In general, for matrices such as ̃ that are not symmetric, the eigenvalues can be complex numbers, and the eigenvectors N may have less clean relationships to one another. Convergence of the Scaled PageRank Update Rule. Fortunately, for matrices such ̃ ̃ ̃ N N > 0 for all entries in which all entries are positive (i.e. N as ), we can use a powerful ij ij result known as Perron’s Theorem [268]. For our purposes, Perron’s Theorem says that any P in which all entries are positive has the following properties. matrix ′ ′ | c > | c P c > for all other eigenvalues c 0 such that . has a real eigenvalue (i) y with positive real coordinates corresponding to the largest (ii) There is an eigenvector c y is unique up to multiplication by a constant. , and eigenvalue c is equal to 1, then for any starting vector x 6 = 0 with non- (iii) If the largest eigenvalue k P negative coordinates, the sequence of vectors x converges to a vector in the direction of y k goes to infinity. as Interpreted in terms of the (scaled) version of PageRank, Perron’s Theorem tells us that there is a unique vector that remains fixed under the application of the scaled update rule, y and that repeated application of the update rule from any starting point will converge to y . y thus corresponds to the limiting PageRank values we have been seeking. This vector C. Formulation of PageRank Using Random Walks To close this chapter, we consider how to formulate PageRank in terms of a random walk on the nodes of the network, following the discussion at the end of Section 14.3. First let’s make the description of the random walk precise. A walker chooses a starting node at random, picking each node with equal probability. (When a random choice is made with equal probability over the options, we will say it is made .) Then, in uniformly at random each step, the walker follows an outgoing link selected uniformly at random from its current node, and it moves to the node that this link points to. In this way, a random path through the graph is constructed one node at a time. Let’s ask the following question: if b denote the probabilities of the walk being ,b ,...,b 2 1 n at nodes 1 , 2 ,...,n respectively in a given step, what is the probability it will be at node i in the next step? We can answer this by reasoning as follows. 1. For each node j that links to i , if we are given that the walk is currently at node j , in the next step, where then there is a 1 is the chance that it moves from j to i /` ` j j number of links out of j .

442 428 CHAPTER 14. LINK ANALYSIS AND WEB SEARCH j for this to happen, so node contributes 2. The walk has to actually be at node j (1 ) = b /` to the probability of being at i in the next step. b /` j j j j j /` 3. Therefore, summing over all nodes b that link to i gives the probability the walk j j b is at in the next step. i in the next step is the sum of b /` So the overall probability that the walk is at i over j j all nodes that link to i . We can use the matrix N defined in the analysis of PageRank to write this update to the probability as follows: b i ··· . b + N b + N + N b b ← (14.9) 1 i 2 ni i i 2 1 n If we represent the probabilities of being at different nodes using