April 2019 ATO Safety Management System Manual

Transcript

1 Safety Management System Manual April 201 9 Air Traffic Organization

2 FOREWORD The fundamental mission of the Air Traffic Organization (ATO) is to ensure the safe provision of air traffic services in the National Airspace System (NAS). Thanks to its employees, the ATO operates the safest, most efficient air traffic m in the world. syste As the ATO helps build the Next Generation Air Transportation System, the resulting cross -organizational changes to the NAS require an intensive, proactive, and systematic focus on assuring safety. ATO uses the Safety Management System ( SMS) to achieve this. The SMS constitutes the operating principles that support the ATO in objectively examining the safety of its operations. This document is the result of an ATO -wide effort, and reflects current international best practices and intr a- agency lessons learned. It marks an important next step toward a mature and integrated SMS in the FAA. Therefore, it is important that all ATO personnel work diligently to uphold and follow the procedures and guidance in this SMS Manual to manage safet y risk and help promote a positive safety culture in the ATO and the FAA. Teri Bristol Chief Operating Officer Air Traffic Organization

3 Contents 1. Safety Management System Overview Overview 1.1 About the SMS Manual 1.1.1 Establishment and Continuous Support of the ATO SMS 1.1.2 SMS Continuous Improvement 1.1.3 -Wide ATO 1.1.3.1 Measuring NAS Safety Performance SMS Benefits 1.1.4 1.2 The Four Components of SMS 1.2.1 SMS Components Safety Culture and Promotion: Valuing Safety in the ATO 1.2.2 Overview of Safety Culture, Safety Assurance, and SRM 1.2.2.1 1.2.2.2 Safety Programs and Initiatives 1.3 SMS Policy 1.3.1 SMS Policy Derivations ICAO SMS Policy 1.3.1.1 FAA SMS Policy 1.3.1.2 1.3.1.3 AOV Order 1.3.1.4 ATO SMS Policy and Requirements Policy Compliance with SMS 1.3.2 1.3.3 FAA Documents Related to SMS Requirements Safety Reporting 1.3.3.1 Facilities and Equipment Management 1.3.3.2 1.3.3.3 Hardware and Software System Development Safety Management and R isk Assessment 1.3.3.4 Managing Safety Risk in a System of Systems 2. SRM and Safety Assurance 2.1 2.1.1 Introduction to Managing System Safety 2.1.2 Safety Assessment Using the Tenets of SRM and Safety Assurance 2.1.3 SRM: Proactive and Reactive Hazard and Risk Reduction 2.1.4 Safety Assurance: Identifying and Closing Safety Gaps 2.1.4.1 Audits and Assessments ATO Quality Assurance and Quality Control 2.1.4.2 2.2 Identifying and Addressing System Vulnerabilities 2.2.1 System Gaps and Hazard Defenses 2.2.1.1 Overview and Causes of System Gaps 2.2.1.2 Hazard Defenses 2.2.2 The Human Element’s Effect on Safety 2.2.3 Closing Gaps Using SRM and Safety Assurance Principles and Processes 2.2.4 Safety Order of Precedence i

4 3. The Safety Analysis and Risk Mitigation Process 3.1 Overview Overview of the SRM Process 3.1.1 SRM Safety Analysis Phases 3.1.2 3.2 Scope of the SRM Process 3.2.1 When to Perform a Safety Analysis 3.2.2 When a Safety Analysis May Not Be Required Overview 3.2.2.1 NAS Change Proposal 3.2.2.2 Examples of NAS Changes Unlikely to Require a Safety 3.2.2.3 Analysis IAAT Phase 1: Describe System 3.3 D Overview 3.3.1 3.3.2 Bounding and Scoping Safety Analyses 3.3.2.1 Bounding Safety Analyses in an Integrated NAS 3.3.2.2 Required Depth and Breadth of the Analysis Involving Other FAA LOBs 3.3.2.3 Setting the Scope of the Analysis 3.3.2.4 3.3.3 Defining the System / NAS Change Describe the System and the NAS Change 3.3.3.1 Overview 3.3.3.1.1 3.3.3.1.2 Considerations when Defining the System 3.3.3.2 5M Model Method AAT Phase 2: Identify Hazards 3.4 DI 3.4.1 Overview 3.4.2 Potential Sources of Hazards 3.4.2.1 Existing Hazards 3.4.2.1.1 Identified but Not in the Scope of an Ongoing NAS Change Hazards Identified by Audits 3.4.2.1.2 Hazards Identified by Top 5 3.4.2.1.3 3.4.2.1.4 Emergency Modifications 3.4.2.1.5 Existing High -Risk Hazards 3.4.3 Elements of Hazard Identification Techniques for Hazard Identification and Analysis 3.4.3.1 Developing a PHL 3.4.3.1.1 3.4.3.1.2 Developing a HAW 3.4.3.1.3 Other Accepted Tools and Techniques 3.4.4 Causes and System State Defined 3.4.5 Addressing Hazards that Cross FAA Lines of Business 3.4.5.1 Hazard Escalation and Reporting 3.5 DIAAT Phase 3: Analyze Risk 3.5.1 Overview ii

5 3.5.2 Controls Defining a Credible Hazard Effect 3.5.3 3.5.4 Defining Risk How to Define and Determine Risk 3.5.4.1 Determining Severity 3.5.4.2 3.5.4.2.1 Assessing Severity of NAS Equipment Hazard Effects 3.5.4.2.2 Using the NAS Equipment Worst Credible Severity Table 3.5.4.3 Determin ing Likelihood 3.5.4.3.1 Likelihood versus Frequency What to Consider When Defining Likelihood 3.5.4.3.2 Calculating Likelihood with Quantitative Data 3.5.4.3.3 Determining Likelihood When No Data Are 3.5.4.3.4 Available 3.6 DIAA T Phase 4: Assess Risk 3.6.1 Overview Risk Levels and Definitions 3.6.2 High Risk 3.6.2.1 3.6.2.2 Medium Risk Low Risk 3.6.2.3 Plotting Risk for Each Hazard 3.6.3 3.7 DIAAT Phase 5: Treat Risk 3.7.1 Overview Risk Management Strategies 3.7.2 3.7.2.1 Risk Control 3.7.2.2 Risk Avoidance Risk Transfer 3.7.2.3 3.7.2.4 Risk Assumption 3.7.3 Documenting Safety Requirements 3.7.4 Determining Predicted Residual Risk 4. Developing Safety Performance Targets and Monitoring Plans 4.1 Developing Safety Performance Targets Developing the Monitoring Plan 4.2 Monitoring Activities 4.2.1 4.2.2 Frequency and Duration Monitoring 4.3 Post -SRM Monitoring 4.3.1 Monitoring and Current Risk 4.3.2 Predicted Residual Risk Is Not Met 4.3.3 Pred icted Residual Risk Is Met 4.3.4 Residual Risk 4.3.5 Monitoring and Tracking of Changes Added to the Operating NAS iii

6 5. Preparing, Performing, and Documenting a Safety Analysis 5.1 Overview Safety Analysis Process Flow 5.1.1 5.2 Preparing a Safety Analysis Planning and Initial Decision -Making 5.2.1 5.2.1.1 Scope 5.2.1.2 Detecting Potential for Hazards Preparing for In- 5.2.2 Depth Safety Analyses SRM Panel Facilitator 5.2.2.1 SRM Panel Co -Facilitator 5.2.2.2 Facilitation by AJI Safety Case Leads 5.2.2.3 Pre 5.2.2.4 -SRM Panel Assessment of the Scope of the Analysis Involving AOV during a Safety Analysis 5.2.2.5 SRM Panel Membership 5.2.2.6 5.2.2.6.1 Overview 5.2.2.6.2 SRM Panel Guidance for Bargaining Unit Participation Participation on SRM Panels Outside of a Service 5.2.2.6.3 Unit or the ATO Primary SRM Panel Roles 5.2.2.6.4 Examples of Skills and Backgrounds for SRM 5.2.2.6.5 Panel Members 5.3 Performing a Safety Analysis 5.3.1 SRM Documents Administering the SRM Panel Meeting 5.3.2 5.3.3 Factors that Jeopardize Safety Assessment Results 5.3.4 SRM Panel Deliberations Safety Risk Management Documentation 5.4 Hazard Analysis Worksheet 5.4.1 Monitoring Plan 5.4.2 SRM Documents 5.4.3 5.4.3.1 Safety Finding With Hazards 5.4.3.2 Safety Finding Without Hazards 5.4.4 Writing the SRM Document Executive Summary 5.4.4.1 5.4.4.2 SRM Document Signatures 5.4.4.3 Current System 5.4.4.4 Description of Change / Existing Safety Issue 5.4.4.5 Rationale for a Safety Finding Without Hazards (If No Hazards Are Identified) 5.4.4.6 Hazard and Risk Analysis (If Hazards A re Identified) 5.4.4.7 Monitoring Plan (If Hazards Are Identified) 5.4.4.8 Dissention iv

7 5.4.4.9 SRM Panel Attendees Appendices 5.4.4.10 SMTS 5.4.5 5.4.5.1 Implementation Dates in SMTS Special SRM Efforts/Considerations 5.5 5.5.1 Deactivation, Removal, or Decommissioning of NAS Equipment Emergency Modifications 5.5.2 5.5.3 Existing High -Risk Hazards 5.5.4 Documentation, Review, and Approval Process for Waivers to Separation Minima Initiate the Request for a New Waiver or Waiver Renewal 5.5.4.1 ppropriate 5.5.4.2 Waiver Development Guidance: Identify A Hazards Relationship between the Waiver Request and the SRM 5.5.4.3 Document Waiver Renewals 5.5.4.3.1 Waiver Approval 5.5.4.3.2 6. Risk Acceptance and Safety Documentation Review 6.1 Risk Acceptance and Approval and Overview 6.2 Scope of NAS Changes 6.2.1 Local Implementation 6.3 Approving Safety Requirements Appropriate Signatories 6.3.1 6.3.2 Endorsing Implementation of Safety Requirements 6.3.2.1 Safety Requirements Not Planned for Implementation Safety Requirements Planned for Implementation 6.3.2.2 Safety Recommendations 6.3.2.3 6.4 Risk Acceptance 6.4.1 Authority to Accept Safety Risk 6.4.2 Risk Acceptance Outside of the Air Traffic Organization 6.5 SRM Document Concurrence 6.6 SRM Document Approval 6.6.1 Service Unit SRM Documentation Approval or Concurrence 6.6.2 AJI Review and Approval 6.6.2.1 AJI Participation in System Acquisition Safety Analyses 6.6.3 and Acceptance AOV Approval 6.6.3.1 Items Requiring AOV Approval 6.6.3.2 Items Requiring AOV Acceptance 6.6.4 Coordination of SRM Documentation 6.7 Revising an SRM Document v

8 7. ATO Audit and Assessment Programs Audit and Assessment Programs 7.1 7.1.1 Overview Air Traffic Compliance Verification Evaluation Program 7.1.2 7.1.3 Difference between ATC Facility Audits and Assessments 7.1.4 National Airspace System Technical Evaluation Program Independent Operational Assessments 7.1.5 7.1.6 Independent Assessments Safety Data Reporting, Tracking, and Analysis 7.2 7.2.1 and Evaluation Purpose of Safety Data Collection 7.2.2 AJI’s Role in Safety Data Collection and Evaluation 7.2.3 Safety Data Collection and Reporting Processes 7.3 Safety Incident and Accident Reporting and Analysis Reported Safety Data about Serviceability of Equipment, Systems, and 7.4 Facilities Voluntary Data Reporting 7.5 7.5.1 Unsatisfactory Condition Report 7.5.2 Aviation Safety Hotline 7.5.3 Administrator’s Hotline 7.5.4 Air Traffic Safety Action Program / Technical Operations Safety Action Program 8. Safety Data and Information Repositories 8.1 Overview 9. Definitions and Acronyms 9.1 Definitions Acronyms 9.2 vi

9 Section 1 Safety Management System Overview 1.1 Overview 1.1.1 About the SMS Manual The S afety Management System (SMS) is a formalized and proactive approach to system safety. It directly supports the mission of the Federal Aviation Administration (FAA), which is “to Air Traffic Organization provide the safest, most efficient aerospace system in the world.” The is an integrated collection of principles, policies, processes, procedures, and SMS (ATO) programs used to identify, analyze, assess, manage, and monitor safety risk in the provision of air traffic management and communication, navigation, and surveillance services. This SMS Manual informs ATO employees and contractors about the goal of the ATO SMS, describes the interrelationship among the four components of the SMS, and instructs readers on the process of identifying safety hazards and mitigating risk in the National Airspace System (NAS). Use this document and its complements, such as the Safety Risk Management Guidance for System Acquisitions, ATO Safety Guidance documents, and other FAA safety documents, to carry out the safety mission of the FAA and requirements of the SMS. 1.1.2 Establishment and Continuous Support of the ATO SMS Safety , the principal consideration of all ATO activities, is defined as the state in which the risk of harm to persons or property damage is acceptable. Managing and ensuring the safety of operations using the SMS has long been a focus of air navigation service providers worldwide, with the International Civil Aviation Organization having provided the guiding principles and the mandate for member organizations to have an SMS. The ATO’s SMS efforts support the FAA safety mission, which emphasizes continuous improvement of safety and the integration of . Business safety management activities across FAA organizations, programs, and Lines of Efforts to deve Transportation lop and implement complex, integrated Next Generation Air System systems to improve the safety and efficiency of air travel in the United States make clear the relevance of the SMS. 1.1.3 SMS Continuous Improvement The SMS is the framework that the ATO uses to measure and help ensure the safety of its operations. In an evolving NAS, it is necessary to continuously seek improvement in ATO processes and policies that support ATO safety efforts and, by extension, support the SMS. The ATO and external organizations conduct audits and assessments to measure and determine compliance with the policies and procedures used to manage safety in the NAS. By assessing SMS maturity, the ATO is able to identify gaps in SMS performance, opportunities for improv ement, and areas in which to focus new policy development. -W 1.1.3.1 Measuring NAS ide ATO Safety Performance As part of the effort to support the FAA Strategic Initiatives, and to help the FAA achieve the Next Level of Safety, Event Rate as a measure of its the ATO has developed the System Risk safety performance. month rolling rate that compares The System Risk Event Rate metric, a 12- the number of high- risk losses of standard separation to the number of total losses of separation, is based on Risk Analysis Events. Risk Analysis Events are losses of standard separation in which less than two- thirds of the required separation is maintained. Risk Analysis Events are identified and assessed as part of the Risk Analysis Process, which considers causal factors and pilot and controller performance when assessing the severity and Through the Risk Analysis Process, Risk Analysis repeatability of the event(s) that occurred. Events replace the long -standing measures of safety performance in the ATO, allowing to be drawn between events and potential causes. From performance of individual relationships facilities up to the NAS- wide system level, the Risk Analysis Process helps focus ATO safety 1.1_SMSM_20150 9 1 Originally published September 2015 Uncontrolled copy w hen downloaded

10 Section 1 Safety Management System Overview initiatives on significant causes, events, and hazards that necessitate remedial action, thus, advancing risk -based decision-m aking initiatives. 1.1.4 SMS Benefits ocesses and tools that support the SMS help: ATO pr • Provide a common framework to proactively and reactively identify and address safety hazards and risks associated with NAS equipment, operations, and procedures; • Encourage intra- agency stakeholders to participate in solving the safety challenges of an increasingly complex NAS; Reduce isolated analysis and decision- • making using integrated safety management principles; Improve accountability for safety through defined managerial roles and responsibilities • anagement and S afety R isk M processes; • Integrate Safety Assurance processes that enable the ATO to effectively measure safety performance; • Promote a continuous cycle of assessing, correcting/mitigating, and monitoring the safety of air navigation services; Foster a positive safety culture that can help improve system safety; and • • Measure the performance and support the improvement of the SMS. 1.1_SMSM_201509 2 Originally published September 2015 Uncontrolled copy when downloaded

11 Section 1 Safety Management System Overview The Four Components of SMS 1.2 1.2.1 SMS Components T he four components of the S afety Management System (SMS) combine to create a systemic approach to managing and ensuring safety. These components are: • Safety Policy: The documented organizational policy that defines management’s commitment, responsibility, and accountability for safety. Safety Policy identifies and assigns responsibilities to key safety personnel. • A process within the SMS composed of describing Safety Risk Management (SRM): the system; identifying the hazards; and analyzing, assessing, and controlling risk. SRM includes processes to define strategies for monitoring the safety risk of the National Airspace System (NAS) . SRM complements Safety Assur ance. • Safety Assurance: A set of processes within the SMS that verify that the organization meets or exceeds its safety performance objectives and that function systematically to determine the effectiveness of safety risk controls through the collection, analysis, and assessment of information. • Safety The communication and distribution of information to improve the Promotion: safety culture and the development and implementation of programs and/or processes that support the integration and continuous improvement of the SMS within the A ir Organization (ATO). Safety Promotion allows the ATO to share and provide Traffic evidence of successes and lessons learned. in an integrated model. The Figure 1.1 represents the relationship of the four SMS components integration and interaction of the four components is essential to managing the SMS effectively and fostering a positive safety culture. 3 1.2_SMSM_201509 Originally published September 2015 Uncontrolled copy when downloaded

12 Section 1 Safety Management System Overview egrated Components of the SMS Figure 1.1: The Int e and Promotion: Valuing S afety in the ATO Safety Cultur 1.2.2 Overview of Safety Culture, Safety Assurance, and SRM 1.2.2.1 Safety culture is defined as the way safety is perceived and valued in an organization. It represents the priority given to safety at all levels in the organization and reflects the real commitment to safety. The ATO uses its SMS to promote a positive safety culture through policies that align safety goals with organizational standards, training, voluntary reporting, and best practices. A strong safety culture helps ensure that personnel are trained and competent to perform their duties and that continual training and updates on safety progress are provided. Promoting strong safety values means that all ATO employees share lessons learned from investigations and experiences, both internally and from other organizations. SRM and Safety Assurance are the performance -oriented components and results of the SMS, but programs and work that contribute to the Safety Promotion component are vital to achieving positive safety outcomes throughout the ATO. The tenets of Safety Promotion are used to foster a positive safety culture in which ATO employees understand why safety is important and how they affect it, providing a sense of purpose to safety efforts. Each employee must consider the potential effect their decisions may have on safety and is responsible for understanding the significance of his or her job as it relates to safety. SMS training identifies the importance of the 4 1.2_SMSM_20150 9 Originally published September 2015 Uncontrolled copy when downloaded

13 Section 1 Safety Management System Overview SMS and how each employee and contractor fits into the mission of using the SMS to improve safety in the ATO. For more information on SMS training, refer to the SMS website. Open communication is critical to a positive safety culture. The ATO communicates safety objectives to all operational personnel to improve the way safety is perceived, valued, and prioritized. In an organization with a strong safety culture, individuals and groups take responsibility for safety by communicating safety concerns and striving to learn, adapt, and modify individual and organizational behavior based on lessons learned. 1.2.2.2 Safety Programs and Initiatives The ATO maintains a positive safety culture using programs and initiatives such as: • Recurrent Training: Collaboratively -developed instruction for controllers, designed to maintain and update previously learned skills while promoting a positive safety culture. • Top 5: -priority factors that contribute to the risk in the NAS. The Top 5 is High determined based on data obtained from the Risk Analysis Process, Voluntary Safety Reporting Programs, and other databases used to log and report unsafe occurrences. Fatigue Risk Management: A group that provides operational fatigue risk expertise, • guidance, and support to the ATO in developing fatigue reduction strategies and policy recommendations to mitigate and manage operational fatigue risks in the NAS. • A joint effort between the ATO and the National Air Traffic Partnership for Safety: Controllers Association that encourages employees to become actively engaged in identifying local hazards and developing safety solutions before incidents occur. • Voluntary Safety Reporting Programs o Air Traffic Safety Action Program (ATSAP): A confidential system for controllers and other employees to voluntarily identify and report safety and operational conce AT SAP website . rns. For more information, refer to the Confidential Information Share Program: A program for the sharing and o analysis of information collected through the ATSAP and airlines' Aviation Safety Action Programs to provide a more complete representation of the NAS. For more information , visit the Confidential Information Share Program website. o Technical Operations Safety Action Program (T -SAP): A system for reporting safety to operations, equipment, personnel, or -related events or issues pertaining anything believed to affect safety in the NAS for technicians and other Technical Operations employees. For more information, refer to the T- SAP website . • Lessons Learned: Lessons learned are used to improve ATO processes, address deficiencies proactively, and empower employees to play a direct role in the safety of the NAS by providing valuable safety information. 5 1 .2_SMSM_201509 Originally published September 2015 Uncontrolled copy when downloaded

14 Section 1 Safety Management System Overview SMS Policy 1.3 Policy Deriv SMS 1.3.1 ations TO ) S afety Management System (S MS The A ir Traffic Organization (A ) is supported by Figure 1.2 . Some relevant numerous levels of policy and requirements, as depicted in programs that pre -date the SMS are detailed in other Federal Aviation Administration (FAA) publications and processes. This SMS Manual only references those documents when Section 1.3.3 necessary. lists many of the related documents. Figure 1.2: SMS Policy and Requirements Hierarchy CAO 1.3.1.1 I SMS Policy A der ives i ts hi gh -level S MS pol icy f The FA nterna rom I tional C ivil A viation Organizati on ( ICAO ) d recommended pr andards an Safety M policy. ICAO A nnex 19, actices anagement , pr ovides st for sa tates and ai r t raffic se rvice providers. Additionally, ICAO anagement for member s fety m Safety M anagement Manual , p rovides g uidance for t he development and Document 9859, ICAO D he SMS for ai r traffic se rvice providers. ocument 9859 also provides implementation of t guidance for safety pr n acco rdance with the international st andards and recommended ograms i practices co ntained i n Annex 19. 1.3.1.2 FAA SMS Policy urrent version of FAA Order 8000.369, Safety Management System , describes the The c essential aspects of an SMS and provides implementation guidance to FAA organizations. This document is designed to create a minimum SMS standard that each FAA Line of B usiness can follow to implement an SMS. (LOB) 6 1.3_SMSM_201509 Originally published September 2015 Uncontrolled copy when downloaded

15 Section 1 Safety Management System Overview Order 8040.4, The current version of FAA Safety Risk Management Policy , provides risk management policy for FAA LOBs to follow when hazards, risks, and associated safety analyses affect multiple LOBs. The ATO must consider and, when necessary, use the provisions in this order when coordinating safety assessments with other FAA organizations. Safety and Technical Training (AJI) will function as the ATO liaison to interface with outside organizations. Within the ATO, AJI will adjudicate discrepancies among Service Units. 1.3.1.3 AOV Order afety O versight S ervice (AOV) p rovides i ndependent sa fety ov ersight o The A ir Traffic S f the AA Order 1100. Air T raffic S afe ty O versight 161, ovides hi gh- level S MS r equirements ATO. F , pr he liaison nvolvement i JI will function as OV. When AOV i t of the ATO and A s required, A Additional guidance from A between AOV and other ATO Service Units and organizations. OV irculars (SOCs) rovide information and guidance ia Safety Oversight C that p will be submitted v material t o develop and implement i nternal p rocedures. AOV hat may be used by t he ATO t shes al l S OCs on t he i ntranet publi . 1.3.1.4 ATO S olicy and Requirements MS P 1000.37, gh- affic Organization Safety Management S ystem , docu ments hi FAA Order JO Air Tr level SMS req uirements, r oles, and responsibilities. Additional r equirements a re contained , uidance Manual. F AA Order J O 1030.1, Air Tr affic Organization Safety G MS within this S roviding t TO w ith supplemental guidance material for p he A establishes a method and a process MS. The Safety R isk M anagement Guidance pertinent t ystem A cquisitions provides o t he S for S SMS requirements and guidance pertinent to programs proceeding through the FAA Acquisition Management System process. The ATO has also established Quality Assurance and Quality Control orders that govern safety data collection and the establishment of safety-related ons. Those orders a re as follows: corrective acti FAA • Air Traffic Organization Occurrence Reporting Order JO 7210.632, ir Traffic Organization Quality Assurance Program (QAP) FAA A • Order JO 7210.633, ir Traffic Organization (ATO) Quality Control Order JO 7210.634, A • FAA FAA Order JO 7200.20, • Voluntary Safety Reporting Program (VSRP) ons and individuals under the purview of FAA Order JO 1000.37 must adhere All ATO organizati to the provisions of the aforementioned documents and this SMS Manual. If discrepancies exist between this SMS Manual and FAA orders and guidance, including those that originate outside 1 the ATO, notify the ATO Safety Manager. 1.3.2 Policy C ompliance with SMS As the ATO’s SMS matures, the tenets of the SMS components are integrated into new and existing ATO policy. For a directive to be considered compliant with the SMS, it must incorporate safety measures and SMS requirements to help manage safety. 1.3.3 FAA Documents Related to S MS R equ irements The following documents (orders, directives, handbooks, and manuals) address National that support the ATO SMS. Airspace System safety management and are core documents This list is not all-i nclusiv e and only represents a small portion of ATO documents that pertai n to safety m ana gement. Some docu ments listed may hav e been updated since the publication of this S MS M anual. 1. The role of the ATO Safety Manager is defined in the current version of FAA Order JO 1000.37. 7 1.3_SMSM_201509 Originally published September 2015 Uncontrolled copy when downloaded

16 Section 1 Safety Management System Overview Safety Reporting 1.3.3.1 a. Runway Safety Program FAA Order 7050.1, , Voluntary Safety Reporting Programs (VSRP) FAA Order JO 7200.20 b. , Air Traffic Organization Occurrence Reporting c. FAA Order JO 7210.632 , Air Traffic Organization Quality Assurance Program (QAP) FAA Order JO 7210.633 d. e. FAA Order JO 7210.634 , Air Traffic Organization (ATO) Quality Control Air Traffic Organization Aircraft Accident and Incident Notification, , FAA Order JO 8020.16 f. Investigation, and Reporting Facilities and Equi 1.3.3.2 pment Management Instructions for Writing Notices, Maintenance Technical Handbooks, FAA Order JO 1320.58, a. and System Support Directives b. FAA Order 1800.66, Configuration Management Policy c. FAA Order JO 1900.47 , Air Traffic Control Operational Contingency Plans FAA Order 6000.15, G d. eneral Maintenance Handbook for National Airspace System (NAS) Facilities National Airspace System Maintenance Policy e. FAA Order 6000.30, f. FAA Order JO 6000.50 , National Airspace System (NAS) Integrated Risk Management Hardware and So 1.3.3.3 ftware System Development: a. FAA Acquisition Management System neering Manual (SEM) b. FAA Systems Engi d Risk Assessment: Safety Management an 1.3.3.4 AOV SOC 07 AOV Concurrence/Approval at Various Phases of Safety Risk a. -02, Risk Hazards Management Documentation and Mitigations for Initial High- afety Risk Modeling and Simulation of Hazards and AOV SOC 07 -05A, Guidance on S b. Mitigations c. AOV SOC 13-1 A , Corrective Action Plan Development and Acceptance in Response 3 to Safety Compliance Issues d. FAA Order JO 1000.37 , Air Traffic Organization Safety Management System e. FAA Order 1100.161, Air Traffic Safety Oversight FAA Order 8000.369, f. Safety Management System Safety Risk Management Policy FAA Order 8040.4, g. 8 1.3_SMSM_201509 Originally published September 2015 Uncontrolled copy when downloaded

17 Section 2 Managing Safety Risk in a System of Systems SRM and Safety Assurance 2.1 2.1.1 Introduction to Managing System Safety TO ) operational procedures and National Airspace System (N AS) As Air Traffic Organization (A equipment (i.e., hardware and software) evolve, their interaction and interdependency across Administration (FAA) must be organizations within the ATO and throughout the Federal Aviation addressed. In a system as large and diverse as the NAS, the discovery of a safety hazard and of its risk often falls within the purview of multiple organizations. reduction The effects of safety hazards and associated across multiple risk reduction methods implementation timelines must be properly understood to achieve organizations, domains, and the highest practical level of safety. Safety risk deemed acceptable for an individual element of the NAS may lead to unintentional safety risk in another if a safety assessment is not condu cted with a “system of systems” philosophy. As emerging NAS equipment, operations, and procedures are tested and implemented, safety risk assessments must account for their potential safety impact on existing/legacy tools and procedures and vice versa. Sharing safety data and conducting cooperative analyses using an integrated safety management approach helps identify and resolve issues requiring the consideration of multiple disciplines. The goal of an integrated approach to safety management is to eliminate gaps in safety analyses by assessing NAS equipment, operations, and procedures across three planes: vertical, horizontal, and temporal. The vertical plane is hierarchical, providing assessments from a specific project up to the NAS -level system of systems of which the project is a part. The horizontal plane spans organizations, programs, and systems. Finally, the temporal plane attempts to eliminate safety gaps across program and system implementation timelines. 2.1 depicts several factors in each of the three planes that should be considered to Figure ensure an integrated approach to safety management. Refer to the current version of the for more information. Safety Risk Management Guidance for System Acquisitions Integrated Safety Management Procedures Training Policy Capability Technology People ntegrated Safety Factors Figure 2.1: I Safety Assessment Using the Tenets of S RM and Safety Assurance 2.1.2 In acknowledging the complexity of the NAS and its various system interdependencies, the ATO uses the systematic processes and tenets of Safety Risk Management (S RM ) and Safety Assurance to identify and address safety hazards and risks across the NAS. The remainder of this chapter discusses the foundational concepts and practices used to identify and address safety issues and consider potential ramifications in an integrated way. It 2.1_SMSM_2016 07 9 Originally published July 2016 Uncontrolled copy when downloaded

18 Section 2 Managing Safety Risk in a System of Systems will describe at a high level the underlying causes of safety hazards and the means by which the ATO manages safety risk. The SRM process provides the framework to track a NAS change after it has been using implemented, s Safety Assurance functions like assessments to determine whether control and/or recommended safety requirements are performing as intended/designed. Refer to Figure 2.2 for a depiction of the relationship between SRM and Safety Assurance. Safety Assurance SRM System Describe System Description Operation System Identify Data Information Management Hazards Acquisition Analyze Analysis Analysis Risk Meeting No Yes Operational/Safet y Assessment Assess Risk Performance Targets? Implement Problem Resolution Safety Treat Risk Requirements Figure 2.2 : SRM / Safety Assurance Process Flow 2.1.3 SRM: Proactive and Reactive Hazard and Risk Reduction SRM is a formalized approach to integrated system safety. It both informs decision -makers hazards, about the potential risks, and ways to reduce risk associated with a particular safety proposal and identifies ways to mitigate existing hazards in the NAS. The methodology is applied to all NAS equipment, operations, and procedures to identify s afety ha zards and address risk. It is important to understand that though the ATO uses SRM as a formal safety and risk assessment process, its philosophy is easily understood outside of the technical realm of aviation. For example, a person performs SRM each time he or she crosses the street. The individual identifies hazards (cars passing), analyzes and assesses the risk (potential to be 2.1_SMSM_2016 07 10 Originally published July 2016 Uncontrolled copy when downloaded

19 Section 2 Managing Safety Risk in a System of Systems ways to reduce the perceived risk (looking both struck and severity if he or she is), and explores ways for traffic and/or heeding pedestrian signals) to an acceptable level before proceeding. It is necessary to make the approach to managing safety risk into a formalized, objective of a safety hazard’s risk s. reduction process. This helps ensure the effective management and SRM provides a means to: • Identify potential hazards and analyze and assess safety risk in ATO operations and NAS equipment ; Define safety requirements to reduce risk to an acceptable level; • Identify safety performance targets • measurable goals used to verify the predicted , the and residual risk of a hazard; Create a plan that an organization can use to determine if expected risk levels are met • and maintained. Section 3 for further guidance and the process for using SRM to perform a safety Refer to analysis. 2.1.4 Safety Assurance: Identifying and Closing Safety Gaps SRM alone does not assure the safety of the services the ATO provides; equally important are the efforts performed under the umbrella of Safety Assurance. Safety Assurance builds on SRM efforts by collecting and assessing data to monitor compliance, assess the performance of safety measures, and identify safety trends. The Safety Assurance component of the S afety Management System (S MS ) encompasses all of the ATO processes and programs that survey the NAS. These processes and programs can lead to the discovery of previously unidentified existing hazards and/or risk controls that are outdated or no longer effective. Safety Assurance —and provides the means to determine whether NAS equipment, operations, and procedures changes to them —meet or exceed acceptable safety levels. 2.1.4.1 Audits and Assessments To continuously improve the safety of its NAS equipment, operations, and procedures, the ATO conducts audits and assessments to determine whether the NAS is performing as expected. ATO employees also use audit and assessment techniques to test, validate, and verify safety data obtained and produced by the various entities and organizations in the NAS. Furthermore, ATO audits and assessments identify causes and correlations that can improve the unders tanding of saf ety performance. Audits and assessments verify suspected positive and negative safety trends identified through analysis. In the event that a safety hazard is identified through an audit and/or assessment, SRM is used to identify potential and/or known risk reduction methods . In this sense, Safety Assurance and SRM complement each other by providing a continuous loop of hazard identification and risk reduction methods . Audits and assessments may be scheduled or unscheduled formal reviews, examinations, or verifications of activities, controls, ATO operations, and ATO systems. The scope of safety audit and assessment activities can vary. An audit or assessment can either focus on a single procedure or piece of NAS equipment, or it can broadly examine multiple elements of a system. 2.1_SMSM_2016 07 11 Originally published July 2016 Uncontrolled copy when downloaded

20 Section 2 Managing Safety Risk in a System of Systems ATO assessments fall into two categories: • Operational: An assessment to address the effectiveness and efficiency of the organization. The objective of an operational assessment is to determine the organization’s ability to achieve its goals and accomplish its mission. • Compliance: An audit that evaluates conformance to established criteria, processes, and work practices. The objective of a compliance audit is to determine whether employees and processes have followed established policies and procedures. The ATO uses both operational assessments and compliance audits at the facility, district, Service Area, and national levels. Using the above described methodologies, the ATO assesses safety performance through: • Proactive evaluation of facilities, equipment, documentation, and procedures (e.g., internal assessments); • verifying the fulfillment Proactive evaluation of Service Delivery Point performance, thus of Service Delivery Point safety responsibilities (e.g., periodic competency checks in the form of Quality Control, operational skills assessments, and system safety reviews); and • of safety Periodic evaluations to verify a system’s performance in control and reduction risks (e.g., internal and external audits and/or assessments). 2.1.4.2 ATO Quality Assurance and Quality Control Requirements and guidance for Quality Assurance and Quality Control are contained in three ATO orders: FAA Order JO 7210.632, Air Traffic Organization Occurrence Reporting ; FAA O AP) ; and FAA Order Air Traffic Organization Quality Assurance Program ( Q rder JO 7210.633, , Air Traffic Organization (ATO) Quality Control . JO 7210.634 These orders provide specific direction for the reporting, investigation, and recording of air traffic incid ents. Responsibilities for assessing trends and non- compliance are also provided, along with guidance for identifying and correcting performance deficiencies. Continuous improvement of the safety of the NAS can occur only when an organization is in monitoring the performance of its operations and its corrective actions. Refer to vigilant Section 7 for more i nformation about the ATO programs that fit within the Safety Assurance component of the SMS. 2.1_SMSM_2016 07 12 Originally published July 2016 Uncontrolled copy when downloaded

21 Secti on 2 Managing Safety Risk in a System of Systems Identifying and Addressing System Vulnerabilities 2.2 Before assessing safety risk or auditing safety performance, it is important to acknowledge the AS). Daily operations in potential origins of safety hazards in the National Airspace System (N -changing air traffic environment can present varying hazards and levels of safety risk . an ever Given the complex interplay of human, material, and environmental factors in Air Traffic (ATO) Organization operations, the complete elimination of all hazards and safety risk is unachievable. Even in organizations with excellent training programs and a strong safety culture, mechanical and electronic equipment will fail, software will function in an unintended manner, and human operators will make errors. 2.2.1 System Gaps and Hazard Defenses verview and Causes of System Gaps 2.2.1.1 O Dev ng a safe procedure, hardware, or software system requires that the procedure/system elopi contain multiple defenses, ensuring that no single event or sequence of events results in an incident or accident. Failures in the defensive layers of an operational system can create gaps in defenses, some known and others unknown. Gaps “open” and “close” as the operational situation, environment, or equipment serviceability state changes. A gap may sometimes be the result of a momentary oversight on the part of a controller or operator, typically described as an latent failures active failure . Other gaps may represent long-standing in the system. Latent conditions exist in the system before negative effects can occur. The consequences of a latent illustrates how an incident or condition may lie dormant for extended periods of time. Figure 2.3 accident can penetrate all of a system’s defensive layers. epth : Defenses in D Figure 2.3 e gaps may occ Thes ur due to: Undiscovered and long-standing shortcomings in the defenses, • The temporary unavailability of some elements of the system due to maintenance action, • Equipment failure, • 3 2.2_SMSM_201904 1 Originally published April 201 9 Uncontrolled copy when downloaded

22 Secti Managing Safety Risk in a System of Systems on 2 Human interaction, and/or • Policy/decision-making. • Hazard Def 2.2.1.2 enses f NAS hardware and software must strive to design systems that will not impose Designers o hazardous conditions during abnormal performance. Using a key systems engineering concept, such systems are referred to as being fault tolerant. A fault-tolerant system includes mechanisms that will preemptively recognize a fault or error so that corrective action can be A subset of a fault-tolerant system taken before a sequence of events can lead to an accident. system is designed such that if it fails, it is a system that is designed to be fail safe. A fail-safe fails in a way that will cause no harm to other devices or present a danger to personnel. , another systems engineering concept, is a system attribute in which, to the Error tolerance maximum extent possible, systems are designed and implemented in such a way that errors do not result in an incident or accident. An error-tolerant design is the human equivalent of a fault-tolerant design. Design attributes of an error -tolerant system include: • Errors are made apparent, • Errors are trapped to prevent them from affecting the system, Errors are detected and warnings/alerts are provided, and • Systems are able to recover from errors. • For an accident or incident to occur in a well -designed system, gaps must develop in all of the defensive layers of the system at a critical time when defenses should have been capable of detecting the earlier error or failure. Functions, equipment, procedures, and airspace components of the NAS interact though numerous complex relationships. Given the temporal nature of these relationships, the ATO must continuously monitor safety risk to maintain an acceptable level of safety performance and prevent gaps. The Human Element 2.2.2 ’s Effect on Safety is estimated to be a causal factor in the majority of aviation accidents and is Human error directly linked with system safety error and risk. For this reason, hardware and software system designers must eliminate as many errors as possible, minimize the effects of errors that cannot be eliminated, and reduce the negative effect of any remaining potential human errors. Human performance variability is a limitation that necessitates careful and complete analysis of the potential effect of human error. Human capabilities and attributes differ in areas such as: Manner and ability of the senses (e.g., seeing, hearing, touching), • Cognitive functioning, • Reaction time, • • Physical size and shape, and • Physical strength. Fatigue, illness, and other factors, such as stressors in the environment, noise, and task also affect human performance. Optimally, the system is designed to resist, or to at on, interrupti least tolerate, human error. 2.2_SMSM_201904 1 9 Originally published April 201 Uncontrolled copy when downloaded

23 Secti Managing Safety Risk in a System of Systems on 2 When examining adverse events attributed to human error, it is often determined that elements of the human-to-system interface (such as display design, controls, training, workload, or manuals and documentation) are flawed. The analysis of human reliability and the application of human performance knowledge must influence system design for safety systems and be an integral part of risk management. Recognizing the critical role that humans and human error play in complex systems and applications has led to the development of the human-centered des ign approach. This approach is central to the concept of managing human error that affects safety risk. 2.2.3 Closing Gaps Using SRM and Safety Assurance Principles and Processes can be reduced proactively and reactively. Monitoring operational data, carefully Safety ri sk analyzing the system, and reporting safety issues make it possible to proactively detect and prevent sequences of events where system deficiencies (i.e., faults and errors, either separately or in combination) could lead to an incident or accident before it actually occurs. The same approach also can be used to reactively analyze the chain of events that led to an accident or incident. With adequate information, safety professionals can take corrective action to strengthen the system’s defenses when devising new air traffic procedures, operations, and NAS equipment, or when making changes to them. The following is an illustrative, but not comprehensive, list of typical defenses used in combination to close gaps in defenses: Equipment Defense Strategies: • Redundancy: Full redundancy, which provides the same level of functionality when operating o on the alternate system Partial redundancy, which results in some reduction in functionality (e.g., local o copy of essential data from a centralized network database) • Independent checking of design and assumptions • System design that ensures that critical functionality is maintained in a degraded mode if individual elements fail • Policy and procedures regarding maintenance to prevent a loss of some functionality in the active system or a loss of redundancy • Automated aids or diagnostic processes designed to detect system failures or those failures reporting of appropriate he processing errors and t Scheduled maintenance • Operating Procedures: Adherence to standard phraseology and procedures • Readback of critical items in clearances and instructions • Checklists and habitual actions (e.g., requiring a controller to follow through the • projected flight path of an aircraft, looking for conflicts, receiving immediate coordination from the handing-off sector) 2.2_SMSM_201904 15 9 Originally published April 201 Uncontrolled copy when downloaded

24 Secti Managing Safety Risk in a System of Systems on 2 Inclusion of a validity indicator in designators for Standard Instrument Departures and • Standard Terminal Arrival Routes • Training, analysis, and reporting methods Organizational Factors: Management commitment to • safety • A strong, positive safety culture Safety policy implementation with adequate funding provided for safety management • activities • Oversight to ensure that correct procedures are followed -tolerance policy toward willful violations or shortcuts • A zero Control over the activities of contractors • 2.2_SMSM_201904 16 201 Originally published 9 April Uncontrolled copy when downloaded

25 Secti Managing Safety Risk in a System of Systems on 2 Safety Order of Precedence 2.2.4 The methods for reducing safety risk generally fall under one of the four categories that make up the Safety Order of Precedence. The Safety Order of Precedence categorizes safety risk mitigations in the following order of preference: Safety Order of Precedence and Examples Table 2.1: Example Definition Priority - Design the system (e.g., Design for minimum risk Remove intersection point by 1. operation, procedure, human- to -system interface, or removing concrete, preventing NAS equipment) to eliminate risks. If the identified planes from ever crossing the risk cannot be eliminated, reduce it to an acceptable . same area level by selecting alternatives. If identified risks cannot - Incorporate safety devices Install physical stop bars and 2. be eliminated through alternative selection, reduce the lights to ensure pilots cannot risk by using fixed, automatic, or other safety features cross unauthorized runways or or devices, and make provisions for periodic function intersection points . checks. - When alternatives and safety Provide warning Install a lighting system to alert 3. use devices do not effectively eliminate or reduce risk, pilots/controllers of potential warning devices or procedures to detect the condition unauthorized crossings. and produce an adequate warning. The warning is Provide new runway or taxiway designed to minimize the likelihood of inappropriate markings. and must be provided human reaction and response, in time to avert the hazard’s effects. Procedures and - Develop procedures and training Develop new taxi and 4. hen it is impractical to eliminate training are used w departure/arrival procedures for features, risks through alternative selection, safety intersecting runway operations. . However, management must and warning devices in pilots and air traffic Tra concur when procedures and training alone are controllers on new procedures applied to reduce risks of catastrophic or hazardous for intersecting runways. severity. Note: Reliance solely on training is not normally a sufficient means to mitigate safety risk. 2.2_SMSM_201904 1 9 Originally published April 201 Uncontrolled copy when downloaded

26 Section 3 The Safety Analysis and Risk Mitigation Process 3.1 Overview Overview of the SRM Process 3.1.1 SRM ) process to follow, guidelines to This chapter provides a linear Safety Risk Management ( identify safety hazards and mitigate their risks, and requirements for the development of consistent and thorough safety analyses. Using the steps in this chapter to perform a safety analysis will not always result in an exhaustive study of air traffic procedures, operations, or NAS National Airspace System ( ) equipment (i.e., hardware and software). The appropriate level of detail in a safety analysis depends on the complexity, size, and potential effect of the or existing safety issue. NAS change This chapter focuses solely on describing the key concepts and five phases of the safety Section 5 analysis process. Section 6 Refer to and for more detailed information on the development of safety documentation and the adminis trative requirements regarding the System using of hazards and risk Management Tracking the . Refer to Section 5 tracking Safety Figure 3.1 provides a high -level depiction of the key for SRM documentation requirements. steps, decision points, and outputs of the SRM process. ribe the System and Identify Hazards Desc Treat Risk Analy ze and Assess Risk Existing Saf ety Issue De fi ne s af et y r is k Determine Introdu ces De t er m ine s ev er it y Aff ects safe mitigation controls and new hazards or and lik elihoo d ; provision ing of strategy ; de fin e NAS Change Yes Yes potential hazar d increases ide nt ify ri sk l ev el ? AT M s er vi ce s monitoring plan effects ? safety risk 3.5.4 , 3.2.2 3.2.1 3.7 , 4 3.5.2 , 3.5.3 3.4 Sa fe ty Ri sk No Management document No 5.4 No further analysis Process Output r eq uir ed SRM Process Figure 3.1: SRM Safety Analysis Phases 3.1.2 Performance of a safety analysis is broken down into a five- phase process called the DIAAT, presented in F igure 3.2 . Consistent with International Civil Aviation Organization guidelines and best practices, these five SRM phases apply to all SRM activity, whether the activity pertains to Air Traffic Organization , maintenance, procedures, or equipment development. operations Systematically completing the steps outlined in the five phases supports a thorough and consistent safety analysis. The DIAAT phases are described in detail in Section 3.3 through Section 3.7. 3.1_SMSM_2016 07 1 9 Originally published July 2016 Uncontrolled copy when downloaded

27 Section 3 The Safety Analysis and Risk Mitigation Process Define scope and objectives DESCRIBE Define stakeholders Identify criteria and plan for SRM efforts (including modeling and simulations) D THE SYSTEM Define system or change (use, environment, intended function, future configuration, etc.) Identify hazards IDENTIFY Use a structured approach Be comprehensive and do not dismiss hazards prematurely I HAZARDS Employ lessons learned and experience supplemented by checklists ANALYZE Identify controls Determine risk based upon the severity and likelihood of the outcome A RISK ASSESS Assign risk level for each hazard based on severity and likelihood A RISK Identify risk management strategies Develop safety performance targets TREAT RISK T Develop monitoring plan Figure 3.2: DIAAT Process 3.1_SMSM_2016 07 2 0 Originally published July 2016 Uncontrolled copy when downloaded

28 Section 3 The Safety Analysis and Risk Mitigation Process 3.2 Scope of the SRM Process ational The Safety Risk Management (SRM ) process is used to assess the safety risk of N System (NAS) Airspace associated with the provision of air changes or existing safety issues traffic management services. These services include the acquisition, operation, and maintenance of hardware and software; management of airspace and airport facilities; and development of operations and procedures. Security (e.g., physical, information, cyber), environmental, or occupational safety and health issues that potentially affect the provision of air traffic management services (i.e., causes of air traffic safety hazards) should be assessed analysis. These issues should not be assessed through SRM if they do not during the safety have an effect on the safe provision of air traffic management services (i.e., if they are not causes of air traffic safety hazards). Likewise, the SRM process is not designed to and should not be used to account for programmatic considerations that are related to the environment, finance, budget, or labor/human resources. Safety hazards associated with the environment, occupational safety, or security that can or do affect the provision of air traffic management services must be reported to the appropriate authority. When to Perform a Safety Analysis 3.2.1 AS change. NAS changes Safety analyses are most frequently performed in response to a N may be proposed and initiated as part of implementation plans for new/modified air traffic safety issues currently in existing procedures, operations, or NAS equipment, or in response to ), a the NAS. For the Air Traffic Organization (ATO is a modification to any NAS change element of the NAS that pertains to or could affect the provision of air traffic management and communication, navigation, and surveillance services. Air traffic controllers and technicians, their training, and their certification are elements of the NAS and directly relate to the provision of air traffic services. In some cases, a safety analysis is performed in response to a request to take action on an Safety existing safety issue. Requests for action may be proposed and initiated as part of a usually For the ATO, this is Assurance function. Assurance, audits, or uality a result of Q If a request to take action on an existing safety issue is received, a safety assessment findings. analysis must be performed. Though not all NAS changes will require a documented safety analysis, the decision and justification to forgo performing a safety analysis is a safety decision. If there is uncertainty as AJI) safety Safety and Technical Training ( to the appropriate path to take, contact the engineering team manager via the Safety Management System (SMS ) mailbox for assistance. 1 that will require a safety analysis. It is important to The following list presents NAS changes does not constitute a complete list or explanation of all NAS changes that note that this list require a safety analysis. Contact the AJI safety engineer team manager for assistance determining whether a safety analysis is required. Operational/procedural c that are not defined in an existing order hanges or waivers • flight trials, tests, demonstrations, and prototypes that are live in the NAS) (e.g., • Any waiver or change to an order, if the order implements a procedure that, when followed, could affect the provision of air traffic services Do not use the SRM process to assess editorial or administrative changes. 1. 3.2_SMSM_201904 2 1 201 Originally published April 9 ntrolled cop Unco y when downloaded

29 Section 3 The Safety Analysis and Risk Mitigation Process • Introduction of new types of navigation procedures into the NAS Changes to separation minima (refer to the current ATO Safety Guidance • on documents separation minima) Addition, modification, closure, or removal of an airport, runway, or taxiway; airport • building construction; and lighting changes (Note: Many of the changes that fall into this category are proposed and sponsored by ederal Aviation the Office of Airports; their SMS req uirements are documented in F FAA Airports (ARP) Safety Management System . Administration (FAA) Order 5200.11, The ATO must remain vigilant to ensure an appropriate safety assessment is conducted on construction projects to maintain continued compliance with air traffic procedures and operations.) • New NAS systems used in Air Traffic Control (ATC) or pilot navigation (or new uses for such existing systems), regardless of their applicability to the Acquisition Management System (A ) MS System Support Directives that introduce new requirements and/or change requirements • -assessed operational systems/equipment in the NAS, such as: for risk Communication, navigation, and surveillance systems o Weather products/services o Displays o o Alerting and advisory systems Service provider equipment (e.g., Automatic Dependent Surveillance– o Broadcast, FAA Telecommunications Infrastructure) o Local patches o Decision support tools RTCA System Support Directives that are built with different levels of rigor (e.g., • level development assurance levels) than what was required during initial acquisition- SRM analysis Changes to system certification and maintenance standards, requirements, and • practices (e.g., technical handbooks) Deactivation, removal, or decommissioning of ATO equipment, procedures, systems, or • services • Site adaptations, if the acceptable technical limits for such adaptations are not defined in the system -level SRM work approved prior to In- Service Decision, or if such limits are to be exceeded • ATC facility changes, including: Tower siting or relocation o o Facility relocation Cab replacement or redesign o of facilities Permanent consolidation or de-consolidation o Facility split o Temporary tower o 3.2_SMSM_201904 2 2 9 Originally published April 201 Uncontrolled copy when downloaded

30 Section 3 The Safety Analysis and Risk Mitigation Process • All charting specification changes prior to submission to the Inter -Agency Air Cartographic Committee for final signature (e.g., symbology, color changes in routes, route identifiers) changes, including Airspace • routes, airways, sectors, and the addition or deletion of a position or sector • Changes to policies, procedures, or NAS equipment for which training exists Removal of or modifications/waivers to existing national and/or local training • requirements except for the purposes of could affect the NAS or NAS operations, that individual performance management Establishment of or modifications to the Technical Training orders, architecture, and • curricula the conditions Changes to of the assumptions during the SRM process (potential • initial risk must be reassessed) implications to When a Safety Analysis May Not Be Required 3.2.2 3.2.2.1 Overview Not all S changes require a s NA afety analysis using the SRM process; there are exceptions. Section 3.2.1 to make this The change proponent must use the criteria in this section and determination. A safety analysis using the SRM process does not need to be performed for NAS changes that are compliant with policies/processes that have undergone SRM and have been documented and approved by the appropriate management official. If these policies or procedures are changed, or if any NAS change deviates from these policies or procedures, a safety analysis must be performed using SRM to manage the safety risk. Note that editorial and administrative changes (i.e., any changes that do not affect the substantive elements of a procedure or system) do not require SRM. documents (e.g., policies, directives, manuals, Standard Operating FAA and/or ATO Procedures, Letters of Agreement, Letters of Procedure) for developing and implementing many routine and repeatable NAS changes could be considered compliant with the SMS, meaning that SRM was performed, documented, and approved. For example, routine procedures such Order 8200.1, United States as flight inspections are conducted in accordance with FAA . If there are no changes to those procedures, then a safety Standard Flight Inspection Manual analysis is not required. However, if there is a change to the frequency of flight inspections, a safety analysis is requir ed. Modifications made to systems to meet initial operational specifications (e.g., Problem Trouble Reports) may not require additional assessments if the system specifications have undergone a ocesses must also be documented safety assessment. The modification and testing pr compliant with the SMS. 3.2_SMSM_201904 2 3 9 Originally published April 201 Uncontrolled copy when downloaded

31 Section 3 The Safety Analysis and Risk Mitigation Process NAS Change Proposals 3.2.2.2 om not process may NAS Change Proposal The configuration management requirements fr the specifically relate to safety effects. When a NAS change covered by a NAS Change Proposal requires SRM, the appropriate safety analysis and documentation must be included in the material provided to the Configuration Control Board. In terms of SRM, a NAS Change Proposal can be categorized as one of the following: • Not requiring any safety assessment • (refer to ocument Requiring a complete safety analysis by an SRM panel and an SRM d Section 5) For more information on NAS Change Proposals, refer to FAA Order 1800.66, Configuration Management Policy . Examples of NAS Changes Unlikely to Require a Safety Analysis 3.2.2.3 The following list presents NAS changes that will likely not require SRM. It is not a complete list or explanation of all NAS changes that do not require a safety analysis. • Facility layout/redline/end-state drawings (e.g., Air Route Surveillance Radar, Air Traffic Control Tower, Terminal Radar Approach Control Facility, Air Route Traffic Control Center), as identified in the Configuration Control Board Charter, Appendix A System Support Directives that do not change requirements and have followed AMS • development assurance processes Changes to directives for those directives with no safety functionality • Installation or moving of equipment if defined installation siting processes are not • violated maintenance technical handbooks • Maintenance actions, as specified in m AJI safety engineering team Contact the anager via the SMS mailbox for assistance determining if a safety analysis is required. 3.2_SMSM_201904 2 4 9 Originally published April 201 Uncontrolled copy when downloaded

32 Section 3 The Safety Analysis and Risk Mitigation Process D IAAT Phase 1: Describe System 3.3 Define scope and objectives DESCRIBE Define stakeholders Identify criteria and plan for SRM efforts (including modeling and simulations) D THE SYSTEM Define system or change (use, environment, intended function, future configuration, etc.) 3.3.1 Overview in Section 3.2.1 , N ational A irspace As discussed NAS) ch anges may be proposed and System ( implementation plans for new or modified air traffic procedures, operations, initiated as part o f or NAS equipment, or in response to existing safety issues currently in the NAS . As part of any aking and follow-on analysis, it is important to develop a detailed description of initial decision-m affected elements. When deciding on the correct scope and level of the NAS change and its he safety analysis, determine the information required about the NAS change and/or detail of t current system. ety analyses initiated for mitigations to existing hazards that were identified through Note: Saf safety audits or post -event safety risk analysis should use the event or situation that led to the realization of the hazard’s effect(s) as the basis for the documented system description. Use this section as guidance, but refer to and 3.4.2.1.3 for further information. Sections 3.4.2.1.2 3.3.2 Bounding and Scoping Safety Analyses 3.3.2.1 Bounding Safety Analyses in an Integrated NAS Bounding refers to limiting the analysis of a change or system to only the elements that affect or interact with each other to accomplish the central function of that change or system. In many cases, there may be a limited or incomplete understanding of the air traffic environment in which the NAS equipment, operation, or procedure will be employed, or the interconnected systems with which the changing system must be integrated for effective operation. Furthermore, the scope of assessment for other associated NAS equipment, operations, or procedures may be unknown. Thus, it becomes difficult to ensure that there are no gaps across the boundaries of these safety analyses. As a result, the scope may be inadvertently set at an inappropriate level. In l ight o f the se pot ential difficulties, the sco pe o f a safety anal ysis m ust be set su ch that gaps ve, and i are eliminated. As sy stems be come i ncreasingly m ore co mplex, i nteracti nterrelated, the assessment of pot fety r isk must be i ntegrat ed temporally, by dom ain, and ac ross ential sa locations. Figure 3. 3 provides a visual r epresentation of t his i ntegration. Where t ime i s ned, i t i s i mportant to consider w hether pot ential sa fety r isk mitigations i mplemented in concer the short term will be adequate years into the future when other systems are introduced in the NAS or whether other follow -on mitigations will negate the effect of those implemented in the past. 3.3_SMSM_201509  Originally published September 2015 Uncontrolled copy when downloaded

33 Section 3 The Safety Analysis and Risk Mitigation Process Figure 3.3: The Complex Integration Aspects of a Capability ) required RM scope and level of Safety Risk Management (S the potential Figure 3.4 depicts -tiered safety based on the potential impact and scope of the NAS change. The lowest assessments focus on identifying hazards associated with individual projects/programs and analyzing individual changes to the NAS that are often associated with new system acquisitions. The middle tier is the capability level. Examples of capabilities include Performance Based Navigation, Surface Operations , or Data Communications. Here, system safety risk assessments become more complex, considering multiple combinations of dependent functions. The top tier represents high -level SRM activities associated with service levels and/or domains to reflect a str ategic view of safety across the NAS. Safety management at this level is more static in nature (i.e., essentially non- recurring system safety engineering). It employs high -level functional hazard analyses to identify NAS -level hazards and safety requirem ents that fl ow down vertically to the other -tiered levels. Figure 3.4: Three Tiers of Integrated Safety Management 3.3_SMSM_201509 2 6 Originally published September 2015 Uncontrolled copy when downloaded

34 Section 3 The Safety Analysis and Risk Mitigation Process Required Depth and Breadth of the Analysis 3.3.2.2 The required depth and breadth of the safety analysis and the amount of collaboration across organizations can vary based the following factors: The complexity of the NAS change. The complexity and nature (i.e., operational or • system acquisition) of the NAS change will dictate the type, depth, and number of analyses requi red. The scope of an SRM activity will require additional The breadth of the NAS change. • details when the NAS change affects more than one organization or Line of Business (LOB ). -reaching NAS changes will require a In general, safety analyses for more complex and far greater scope and more detail. When evaluating a NAS change, consider any potential effect on organizations outside the A TO ) (e.g., the Office of Aviation Safety ir Traffic Organization (A and the Office of Airports). AA L OBs 3.3.2.3 Involving Other F TO sa fety an alysis i When an A ral A viati on Administrati on (F AA) LOBs and /or mpacts Fede organizations out side the A TO, the provisions and guidance i n the current version of FAA O rder 8040.4, Safety Risk Management Policy , apply. Refer to Section 3.4.5 for information on existing and Section 6.4.2 coordinating and addressing safety issues. Refer to Section 6.4.1 for discussion on cross-L OB r isk acceptance. 3.3.2.4 cope of the Analysis Setting the S Guidelines to help determine the scope of the SRM effort include: • Having a sufficient understanding of system boundaries, including interfaces with peer syst ems, larger systems of which the system is a component, and users and maintainers; • system components that may be Determining the system elements that interact or sub- affected; and • Limiting the system to those elements that affect or interact with each other to accomplish the mission or function. When setting the scope of a safety analysis: • Define the relationships/interactions of the NAS change. • Identify temporal aspects of these relationships/interactions. • Collect safety documentation that has assessed the building blocks of the NAS change. • Set the scope wide enough to determine the aggregated risk and assess any gaps. 3.3_SMSM_201509 2 Originally published September 2015 Uncontrolled copy when downloaded

35 Section 3 The Safety Analysis and Risk Mitigation Process Defining the System / NAS Change 3.3.3 3.3.3.1 Describe the System and the NAS Change Overview 3.3.3.1.1 System descriptions need to exhibit two essential characteristics: correctness and completeness. Correctness means that the description accurately reflects the system without ambiguity or error. Completeness means that nothing has been omitted and that everything stated is essential and appropriate to the level of detail. The system description provides information that serves as the basis for identifying all hazards icient and associated safety risks. The system/operation must be described and modeled in suff detail to allow the safety analysis to proceed to the hazard identification stage. For example, modeling might entail creating a functional flow diagram to help depict the system and its systems. interface with the users, other systems, or sub- As discussed, the system is always a component of some larger system. For example, even if the analysis encompasses all services provided within an entire Air Route Traffic Control which in turn is a Center, that Center can be considered a subset of a larger body of airspace, subset of the NAS. 3.3.3.1.2 Considerations when Defining the System Complex NAS changes may require a detailed system description that includes numerous charts, drawings, design descriptions, and/or narratives. Simple NAS changes may only require one or two paragraphs describing the system and NAS change. The description must be clear and complete before continuing the safety analysis. Questions to consider include: • What is the purpose of the NAS change? What issue is necessitating the NAS change? • How will the change be used/function in the NAS? • • What are the boundaries and external interfaces of the NAS change or system? In what environment will the system or NAS change operate? • How is the system or NAS change interconnected/interdependen t with other systems? • • How will the NAS change affect system users/maintainers? • If the NAS change is a waiver/renewal, how could other waivers in effect interact with it? The following are examples of information to consider when describing the system: • age annual approaches to each runway Aver • Fleet mix • Number and type of airport operations • Number of aircraft controlled (ground, pattern, and transitions) • Number of hours the airport operates and number of aircraft controlled under Visual Flight Rules versus Instrument Flight Rules • Availability and reliability of both hardware and software 3.3_SMSM_201509 28 Originally published September 2015 Uncontrolled copy when downloaded

36 Section 3 The Safety Analysis and Risk Mitigation Process identifies sources of data to use in the SRM analysis. Section 8 Once the sys tem elements are listed, a careful review of the NAS change description should be conducted. A bounded system limits the analysis to the components necessary to adequately assess the safety risk associated with the NAS change, system, and/or operation. W hen there is doubt about whether to include a specific element in the analysis, it is preferable to include that item, even though it might prove irrelevant during the hazard identification phase. 5M Model Method 3.3.3.2 The 5M Model can be used to capture the information needed to describe the system and aid in hazard identification. The 5M Model uses a Venn diagram to depict the interrelationships among its five elements, as seen in Figure 3.5 . To adequately bound and describe a sys tem, it is important to understand the relationships between the elements of the 5M Model. The 5M Model illustrates five integrated elements that are present in any system: • Mission: The clearly defined and detailed purpose of the NAS change proposal or system/operation being assessed • The human operators, maintainers, and affected stakeholders (hu)Man/Person: Machine: The equipment used in the system, including hardware, firmware, sof tware, • to-system interfaces, system -to-system interfaces, and avionics human- • Management: The procedures and policies that govern the system’s behavior • Media: The environment in which the system is operated and maintained Figure 3.5: 5M Model Model and similar techniques are used to deconstruct the proposed NAS change in The 5M order to distinguish elements that are part of or affected by the proposed NAS change. These elements later help to identify sources, causes, hazards, and current and proposed risk mitigation strategies. 3.3_SMSM_201509 2 Originally published September 2015 Uncontrolled copy when downloaded

37 Section 3 The Safety Analysis and Risk Mitigation Process For an example of assessing elements outside the scope of the NAS change in question, consider the following: A panel of stakeholders and Subject Matter Experts (see Section 5.1.1) is tasked with assessing the risk of changing the required longitudinal separation from 3 nautical miles to 2.5 nautical miles on the final approach course between 10 and 20 nautical miles at XYZ Airport. The panel does not limit the description of the environment to the final approach course at XYZ Airport; instead it also considers hazards involved with allowing 2.5 nautical miles’ separation on the base and downwind legs. By considering these additional legs, the panel has failed to properly bound its analysis . 3.3_SMSM_201509 30 Originally published September 2015 Uncontrolled copy when downloaded

38 Section 3 The Safety Analysis and Risk Mitigation Process 3.4 Phase 2: Identify Hazards AAT DI Identify hazards IDENTIFY Use a structured approach Be comprehensive and do not dismiss hazards prematurely I HAZARDS Employ lessons learned and experience supplemented by checklists Overview 3.4.1 ing the hazard identification phase, identify and document safety issues, their possible Dur causes, and corresponding effects. A hazard is defined as a ny real or potential condition that can cause injury, illness, or death to people; damage to or loss of a system, equipment, or property; or damage to the environment. A hazard is a prerequisite to an accident or incident. ) and its employees are responsible for identifying and The Air Traffic Organization (ATO mitigating hazards with unacceptable risk (i.e., high risk). Likewise, the ATO should determine if hazards with acceptable risk (i.e., medium and low risk) can be further mitigated. The hazard identification stage is integral to all preliminary safety analyses and follow -on, in-depth analyses in determining the appropriate means to address any safety risks associated with a N ational Airspace System (N AS) change. At this point, decide whether the Safety Risk document will contain a safety analysis finding with or without hazards Management (SRM) discussion of risk (refer to for further reduction strategies. Section 5.4). Refer to Section 3.7.2 for guidance on the signatures required for implementation of the NAS Section 6 Refer to change. The following res ources and methods can be used to identify hazards: • The safety analysis that accompanies the proposed implementation of a new or modified operation, process, or piece of NAS equipment; Air Traffic Safety Action Program Technical Operations Safety Action Program • and reports ; • Air Traffic Safety Oversight Service (AOV) compliance audits ; • ; Risk Analysis Processes National Transportation Safety Board safety recommendations ; • -level Quality Control efforts or Safety and Technical • Audits performed as part of facility Quality Assurance efforts; and Training (AJI) Reports of unsafe conditions in daily operations . • Refer to Section 7 for information about the various audit and reporting programs and tools. Potential Sources of Hazards 3.4.2 The hazard identification s tage considers al l po ssible causes of hazards . The use of previous nd can onsistency i M a hazard analyses w hen identifying hazards i s important, as i t provides c n SR r reduce the time needed to identify haz imila on s ards. For example, approved SRM documents rganizational safety luding applicable cross-o NAS changes or earlier integrated assessments, inc assessments and Independent Operational Assessments, may be useful. Refer to Section 7 for ssessment. information on the Independent Operational A 3 3.4_SMSM_ 201904 1 Originally published April 201 9 Uncontrolled copy when downloaded

39 Section 3 The Safety Analysis and Risk Mitigation Process Depending on the nature and size of the system under consideration, the causes may include: • function, NAS equipment failure/mal • Operating environment (including physical conditions, airspace, and air route design), • Human operator failure/error, machine interface problems, • Human- • Operational procedures limitations/design, Maintenance procedures limitations/design, and/or • • Ex ternal services. 3.4.2.1 Existing Hazards ard is any hazard that is currently in the NAS. Existing hazards often fall into the An exis ting haz following categories . ified but Not in the Scope of an Ongoing NAS Change 3.4.2.1.1 Ident These hazards must typically be addressed through a separate, follow -on safety analysis can assist in performed by the organization deemed responsible. An AJI safety case lead identified. determining the organization responsible for assessing existing hazards azards Identified by Audits H 3.4.2.1.2 When an audit i dentifies a potential safety issue, the issue must be addressed. Refer to Federal Air Traffic Organization Audits and , FAA) Order JO 2900.2 Aviation Administration ( . Assessments azards Identified by Top 5 3.4.2.1.3 H am identifies safety issues, the safety issues must be addressed using a When the Top 5 Progr Corrective Action Plan (CAP) that identifies means to reduce safety risk. If there are potential changes to the NAS, those changes must go through the SRM process. As with safety risks identified in SRM documents, risk treated through a CAP must be monitored. This requires determining the appropriate risk level using the matrix in this manual (see Figure 3.7), assigning a predicted residual risk, creating safety performance targets (or other means to measure safety performance) and monitoring activities, and obtaining approval for risk acceptance and Section 4.3 for more for more information on monitoring and Section 6 to implementation. Refer information on risk acceptance and approval. Emergency Modifications 3.4.2.1.4 There m ay be unusual, unforeseen, or extraordinary issues or conditions that require the implementation of hardware or software solutions in a timeframe that does not allow proceeding through the formal SRM process. Emergency modifications are temporary fixes installed to maintain continuity of air navigation, air t control , communications, or support services raffic during unusual or emergency conditions. Such NAS changes may result from unforeseen -time situations that natural occurrences, a lack of replacement parts, software patches, or real Order 6032.1, require immediate action. Refer to the current edition of FAA National Airspace , for more information on emergency modifications. Refer System (NAS) Modification Program for information on how to properly document emergency modifications. to Section 5.5.2 3.4.2.1.5 Existing High-Risk Hazards When the A TO she must Chief Safety Engineer validates an existing hazard as high risk, he or notify the ATO Chief Operating Officer (COO) and AOV of the high risk and the interim actions needed to mitigate the risk. The ATO COO must approve the interim action and accept the associated risk or require the operation to be stopped. The responsible Service Unit must 3.4_SMSM_ 201904 3 2 9 Originally published April 201 Uncontrolled copy when downloaded

40 Section 3 The Safety Analysis and Risk Mitigation Process coordinate with the ATO Chief Safety Engineer to address the risk and any potential corrective actions. Section 3.7.2 for risk management strategies. Refer to Section 5.5.3 for information on Refer to the administrative process of addressing existing high-risk hazards and obtaining approval for risk reduction. their 3.4.3 Elements of Hazard Identification When considering new N equipment and procedures or planned modifications to current AS NAS equipment and procedures, define the data sources and measures necessary to identify hazards. The elements of a thorough system description contain the potential sources of hazards associated with the proposed NAS change. There are numerous ways to do this, but all require at least three elements: • Operational expertise that relates specifically to the operation or equipment, Training or experience in various hazard analysis techniques, and • • fined hazard analysis tool. A de 3.4.3.1 Techniques for H azard Identification and Analysis ds, a Preliminary H azard List (PHL ) and the In many c ases, to identify and analyze safety haz ar equire required Hazard Analysis Worksheet ( HAW) w ill s uffice. Some cases, however , may r other tool hniques (refer to Section 3.4.3.1.3). s or tec 3.4.3.1.1 Developing a PH L The process of describing the system us ing a tool l ike the 5M M odel i s des igned to facilitate brainstorming for s ources of hazards. The next step in the hazard identification process i s to may be a combination of hazards tates. PHL , causes, effects, and system s develop a PHL . The sted in the PHL all hav i The items l nto the HAW. It is at the panel’s e the potential to be placed i discreti ong in the HAW during the Identify H azards and Analyze ide which items bel on to dec Risk phas es of the SRM process. For more guidance on which items should be placed into the HAW , refer to Section 3.5. 3.4.3.1.2 Developing a HAW , a worksheet us HAW ed to document a safety anal When hazards are identified, the ysis, is required as part of the ATO S RM pr ocess. It is also used both for O perations and S econd-L evel En gineering. When developing the HAW, it is c rucial to consider the hazards i nherent to all ofessionals use the HAW in nearly aspects of an operati on without regard to risk. ATO s afety pr isk m ritical s ituations. anagement applications, except in the most time-c all r Using the HAW helps p ercome the tendency to focus on safety risk i n on e aspect of an anels ov operation and overlook m ore serious i ssues el sewhere in the operati on. Its br oad scope guides the identification of issues that may require analysis w ith more detailed hazard identification tools. Refer to Section 5.4.1 and Section 5.4.4.6 for a description of the expected contents of the HAW. 3.4.3.1.3 Other A ccepted Tools and Techniques eans sis c alls for an additional m to identify hazards and c ompare solutions, anal y If the safety select the methodology that is m ost appropri ate for the type of system bei ng evaluated. The as e lead can provide additional gu idance on which tool(s) Service Center and/or an AJI safety c to use for v ). hanges (refer to Table 3.1 pes of NAS c arious ty 3.4_SMSM_ 201904 3 3 9 Originally published April 201 Uncontrolled copy when downloaded

41 Section 3 The Safety Analysis and Risk Mitigation Process When selecting hazard identification/analysis tools, it is important to consider: The necessary information and its availability; • • The timeliness of the necessary information; • The amount of time required to conduct the analysis; and The tool that will provide the appropriate systematic approach for: • o Identifying the greatest number of relevant hazards, Identifying the causes of the hazards, o Predicting the effects associated with the hazards, and o Assisting in identifying and recommending risk management o strategies. Table 3.1 : Evaluation and Hazard Identification Techniques Summary Description Analysis The Failure Mode and Effect Analysis determines the results or effects of Failure Mode and Effect sub-element failures on a system operation and classifies each potential failure Analysis according to its severity. The Failure Modes, Effects, and Criticality Analysis is an essential function in design from concept through development. The Failure Modes, Effects, and Criticality Analysis is iterative to correspond with the nature of the design process itself. It Failure Modes, Effects, system failure modes (including the effect of human identifies component and sub- and Criticality Analysis error), evaluates the results of the failure modes, determines rates and probability, and demonstrates compliance with safety requirements. is a deductive method of analysis that can be used The Fault Hazard Analysis exclusively as a qualitative analysis or, if desired, can expand to a quantitative one. requires a detailed investigation of sub-systems to The Fault Hazard Analysis Fault Hazard Analysis determine component hazard modes, causes of these hazards, and resultant effects on the sub-system and its operation. A Fault Tree Analysis is a graphical design technique that can provide an alternative to block diagrams. It is a top-down, deductive approach structured in terms of Fault Tree Analysis events. It is used to model faults in terms of failures, anomalies, malfunctions, and human errors. The foundation of the performance of a Human Error Analysis is a Job Task Analysis, which describes each human task and subtask within a system in terms of the perceptual (information intake), cognitive (information processing and decision-making), and manual (motor) behaviors required of an operator, maintainer, Job Task Analysis should also identify the skills and or support person. The Job Task Analysis information required to complete tasks; equipment requirements; the task setting, time, and accuracy requirements; and the probable human errors and consequences relating to these areas. There are several tools and techniques for performing task analyses , depending on the level of analysis needed. is a qualitative severity assessment of The Operational Hazard Assessment (OHA) Operational Hazard des tabular worksheets and inclu the hazards associated with the system. The OHA Assessment the PHL . The Scenario Analysis tool identifies and corrects potentially hazardous situations by Scenario postulating accident scenarios in cases where it is credible and physically logical to Analysis do so. -If Analysis methodology identifies hazards, hazardous situations, or The What What specific accident events that could produce an undesirable consequence. One can -If Analysis use the What -If Analysis as a brainstorming method. 3.4_SMSM_ 201904  9 Originally published April 201 Uncontrolled copy when downloaded

42 Section 3 The Safety Analysis and Risk Mitigation Process 3.4.4 Causes and System S tate Defined Identify and document potential s afety i ssues, their pos sibl e causes, and the conditions under which the safety i s sues a re revealed (i.e., the system s tate). lure. d or fai Causes are events oc c urring independently or i n combination that resul t in a hazar They i an error, latent failure, active failure, desi gn flaw, nclude, but are not limited to, hum component failure, and software error. A characterized by quantities or ate is t he expression of the various c onditions ( system st qualities) i tate that most n which a system c an exist. It is important to capture the system s exposes a hazard, while remaining within the confines of any operational c onditions and assumptions defined in existing documentation. Refer to Section 9 for the definition of assumption. The system s tate can be described using a combination of, but not limited to, the following terms: Oper • Visual Flight Rules versus Instrument Flight Rules, ational and Procedural: simultaneous procedures versus visual approach procedures, etc. Met versus Visual Meteorological Conditions Instrument Conditional: • eorological Conditions , peak traffic versus low traffic, etc. • Physical: Electromagnetic environment effects, precipitation, primary power source versus back -up power source, closed runways versus open runways, dry runways versus contaminated runways, environmental conditions, etc. Any given hazard may have a different risk level in each possible system state. Hazard assessment must consider all possibilities while allowing for all system states. In a hazard analysis, it is important to capture different system states when end results lead to the application of different risk reduction methods and tools . Addressing Hazards that Cross FAA Lines of Bus 3.4.5 iness Safety Risk Management Policy , provides risk rder 8040.4, The current versi O on of FAA management policy to follow when hazards, risks, and associated safety analyses affect necessary, use the provisions in multiple Lines of Business. The ATO must consider and, when this order when coordinating safety assessments with other FAA organizations. AJI will function as the ATO liaison to interface with organizations outside of the ATO when the provisions of FAA Order 8040.4 are invoked. 3.4.5.1 Hazard Escalation and Reporting There may be cases in which the ATO and another FAA organization disagree on key issues 1 surrounding a NAS change. The ATO Safety Manager and ATO Chief Safety Engineer must be made aware of such NAS changes and must work to determine the appropriate course of action. The ATO Chief Safety Engineer will determine whether such hazards and issues need to be elevated to an FAA-level mediation process facilitated by the FAA Safety Management ) Committee. System (S MS . For more information, refer to the FAA SMS Hazard Escalation Reporting Process The primary function 1. is to provide leadership and expertise to ensure that of the ATO Chief Safety Engineer He or she also operational safety risk in the air traffic services that the ATO provides is identified and managed. ensures that safety risk is considered and proactively mitigated in the early development, design, and integration of for a description of the solutions. Refer to the Safety Risk Management Guidance for System Acquisitions ATO Chief Safety Engineer’s roles and responsibilities. 3.4_SMSM_ 201904 3 5 9 Originally published April 201 Uncontrolled copy when downloaded

43 Section 3 The Safety Analysis and Risk Mitigation Process 3.5 DIAAT Phase 3: Analyze Risk ANALYZE Identify controls Determine the severity and likelihood of the hazard’s effect RISK A Overview 3.5.1 A n accident or incident rarely results from a single failure or event. Consequently, risk analysis is seldom a binary (e.g., on/off, open/closed, broken/operational) process. Risk and hazard analyses can identify failures from primary, secondary, or even tertiary events. phase: During the risk analysis • Evaluate each hazard (identified during the “Identify Hazards” phase) and the system state (from the “Describe the System” and “Identify Hazards” phases) to determine the controls, • Analyze how the operation would function should the hazard occur, and • Determine the hazard’s associated severity and likelihood and provide supporting rationale. Controls 3.5.2 anything that currently reduces a hazard’s causes or effects. Polic contr , , procedures ies A ol is s can part of the control s if they are hardware, software, or other tool only be considered operating National Airspace System (N demonstrated effectiveness . AS) and have Understanding controls affects the ability to determine credible effects. Certain controls, such as the Traffic Collision Avoidance System, may only be in place in certain operating environments or under certain system states. Do not document safety requirements as only are ; safety requirements . Refer to controls planned or proposed ways to reduce risk Section 3.7.3 for information about documenting safety requirements. supporting data and/or Provide a rationale that confirms the control’s use, applicability, and availability related to the hazard. For instance, if orders are identified as controls, cite the specific version, paragraph, and/or section number(s). Alternatively, if equipment is identified as or manages the risk. Only document the controls associated a control, discuss how it reduces with the NAS change under evaluation. When considering existing hazards identified through safety audits or post-event risk analys that either minimized es, consider any control the (s) hazard’s effect or failed. provides broad examples of controls. This is not a comprehensive list of controls; Table 3.2 each identified control should be directly applicable to the hazard being addressed. 36 3.5_SMSM_201904 Originally published April 201 9 Uncontrolled copy when downloaded

44 Section 3 Safety Analysis and Risk Mitigation Process The Table 3.2 : Examples of Controls Pilot Equipment Controller • Radar Surveillance • Traffic Collision Avoidance • Preventive Maintenance System - Ground and Airborne • Failure Warnings / Ground Proximity Warning • Maintenance • Controller Scanning System Alerts • - Radar Visual Scanning (Out Window) • • Redundant Systems - Visual (Out Window) Radar Surveillance • - Triple Redundant Radio • Conflict Alert, Minimum Safe • Airborne Altitude Warning, Airport - Software Redundancy Movement Area Safety System • Checklists Diverse Points of Delivery • Procedures • • Redundancies / Back -up Systems -back Systems Fall • - Specific Standard Operating • Pilot Intervention (Evasive Action) - Center Radar Processing Procedure Reference • Software/Hardware Designs - Order Reference Triple Redundant Radio • • Controller Intervention Management Oversight • • Completed Training 3.5.3 Determining a Credible Hazard Effect Effect refers to the real or credible harmful outcome that has occurred or can be expected if the hazard occurs in the defined system state. A single hazard can have multiple effects. Credible means that it is reasonable to expect that the assumed combination of conditions that define the system state will occur within the operational lifetime of a typical Air Traffic Control (A ) TC system. Credible effects should be determined with respect to controls. Document all identified credible effects. Often, there is confusion when distinguishing the effects of a hazard from the credible possible effects; possible is not necessarily the same as credible. The credibility of an effect is a nuanced and key consideration in the analysis. A thorough understanding of this concept can save time in determining the risk level of a specific hazard. When determining the credibility of the effect, it is important to: • Recall and Understand the Defenses in Depth Model. It is well established that incidents and accidents cannot typically be attributed to a single cause, or even to a single individual. Rather, aviation safety issues are the end result of a number of causes. Based on this model (see Section 2.2.1), it is critical to consider the defenses that already exist in the NAS when deciding the credibility of an effect. • Review History. Check the historical record. Have there been similar NAS changes? What happened? How does the experience gained from the activities affect the credibility of the outcomes that have been identified for the NAS change? Section 3.5.4.3.3 • discuss the use of and 3.5.4.3.4 Section Rely on Quantitative Data. quantitative and qualitative data, respectively. Do the quantitative data support the credibility of the outcomes identified? If so, the hazard severity determination can be based on statistical data, and the safety assessment will be more objective. Section 8 3.5_SMSM_201904 37 9 Originally published April 201 Uncontrolled copy when downloaded

45 Section 3 Safety Analysis and Risk Mitigation Process The provides additional information about the aviation safety databases available for gathering data. Put the hazard in its proper Visualize the Occurrence of the Accident or Incident. • context within the given system state and determine the sequence of events (causes) ir Traffic Organization that could lead to the worst credible outcome. Given that the A (ATO ) strives to build error-tolerant systems (in accordance with the Defenses in Depth Model ), consider how many controls (redundancies, procedures, warning devices, an identified hazard to breach equipment, etc.) would have to fail for every defense to result in a catastrophic event. Is it reasonable (i.e., credible) to expect that the necessary combination of extreme conditions will simultaneously occur within the operational lifetime of the system? 3.5.4 Defining Risk 3.5.4.1 How to Define and Determine Risk is the composite of predicted severity and likelihood of the potential effect of a hazard. Risk the highest , the likelihood of th is effect is present severity While the worst credible effect may often very low. A less severe effect may occur more frequently and therefore present a higher risk than the more severe effect. The ways to reduce the risk overall for the two effects may be and their associated risks in different, and both must be identified. Consider all credible effects order to identify the highest risk for the safety hazard. Attempt to obtain and document objective evidence (e.g., historical evidence of similar NAS changes, testing data, modeling or simulation results) to support the assessed level of risk. If —including the data sources quantitative data are not available, document the research methods reviewed—in addition to qualitative assessments. Because different system states can affect the hazard will exist in several both severity and likelihood in unique ways, determine whether system states and assess the risk accordingly. Severity etermining D 3.5.4.2 is the consequence or impact of a hazard’s effect or outcome in terms of degree of Severity is loss or harm. It is independent of likelihood and must be determined before likelihood when determining severity, and use the calculated . Assess all effects and consider controls measure yielding the most conservative estimate (i.e., the higher severity). Table 3.3 is the severity table used by the ATO to assess the severity of a hazard when performing Safety Risk Management. Provide a rationale for the chosen severity level in the H azard Analysis ederal Aviation Administration (F Worksheet (H ines AA) L AW). When a NAS change crosses F ), consult with the affected parties; the provisions of FA of Business (LOBs A Order 8040.4, , apply. Safety Risk Management Policy 3.5_SMSM_201904 3 8 9 Originally published April 201 Uncontrolled copy when downloaded

46 Section 3 Safety Analysis and Risk Mitigation Process The Severity Table Table 3.3: Hazard Severity Classification Note: Severities related to ground - based effects apply to movement areas only. 4 Hazardous Major Catastrophic Minimal Minor 4 5 1 3 2 OLLOWING: F THE F CONDITIONS RESULTING IN ANY ONE O 5 Ground collision High Risk Analysis Medium Risk Low Risk Analysis A minimal 3 Event severity, four Analysis Event Event severity, n reduction i two Mid-air collision indicators fail severity, three ATC services or fewer indicators indicators fail fail Controlled flight into CAT A runway CAT D runway terrain or obstacles 1 incursion CAT B runway CAT C runway incursion incursion incursion Proximity ATC Event , Services Operational Deviation, or measure of compliance greater than or equal to 66 2 percent Low Risk Analysis Medium Risk High Risk Analysis Discomfort to A collision with a Event severity, two Analysis Event Event severity, four those on the manned aircraft or fewer indicators severity, three indicators fail gr ound Fatality or fatal fail indicators fail Incapacitation to Loss of injury to persons Non-serious injury to unmanned aircraft Non-serious injury to separation other than the three or fewer people system crew more than three leading to a unmanned aircraft on the ground people on the ground measure of system crew Proximity of less than compliance 500 feet to a manned A reduced ability of greater than or aircraft the crew to cope with equal to 66 adverse oper ating Unmanned Serious injury to percent conditions to the Aircraft persons other than extent that there Systems unmanned aircraft the would be a significant System crew reduction in safety margins Manned aircraft making an evasive maneuver, but proximity from unmanned aircraft remains greater than 500 feet  3.5_SMSM_201904 201 9 Originally published April Uncontrolled copy when downloaded

47 Section 3 The Safety Analysis and Risk Mitigation Process Hazard Severity Classification Note: Severities related to ground - based effects apply to movement areas only. 4 Major Catastrophic Minor Hazardous Minimal 3 4 5 2 1 IN ANY ONE OF THE F CONDITIONS RESULTING OLLOWING: Fat al injuries to Serious injury to Physical distress to Minimal injury Phy sical discomfort 8 7 persons on board persons on board to passenger(s) (e.g., or discomfort passengers (e.g., to persons on extreme braking abrupt evasive action, action, clear air severe turbulence board causing unexpected turbulence causing aircraft movements) unexpected movement of aircraft Minor injury to greater Flying resulting in injuries to than 10 percent of Public one or two persons on board passengers out of their seats) Minor injury to less than or equal to 10 percent of persons 6 on board lision between Col e reduction in Larg Large increase in flight Increase in Flight crew aircraft and safety margin ATC workload crew workload inconvenience NAS obstacles or terrain Significant reduction Significant increase Slight increase Equipment in safety margin in ATC workload in ATC able (with T workload 3.4) Slight reduction in safety margin 3.5_SMSM_201904  Originally published April 201 9 Uncontrolled copy when downloaded

48 Section 3 The Safety Analysis and Risk Mitigation Process Hazard Severity Classification - Note: Severities related to ground based effects apply to movement areas only. 4 Catastrophic Hazardous Minimal Minor Major 4 5 3 2 1 CONDITIONS RESULTING OLLOWING: IN ANY ONE OF THE F Pilot deviation where Pilot deviation where Pilot is aware ound collision Pilot deviation where Gr loss of airborne of traffic loss of airborne loss of airborne Mid-air collision separation falls within separation falls within (identified by separation falls the same parameters the same parameters Traffic Collision within the same Controlled flight into of a h igh Risk Avoidance of a medium Risk parameters of a low terrain or obstacles Analysis Event System traffic Analysis Event Risk Analysis Event severity Hull loss to manned severity alert, issued by severity aircraft ATC, or Reduction in safety Reduction in safety Reduction of observed by margin and functional Failure conditions margin or functional functional capability flight crew) in capability of the that would prevent capability of the of aircraft, but overall close enough aircraft requiring crew continued safe flight aircraft, requiring safety not affected proximity to to follow emergency and landing crew to follow (e.g., normal require procedures as per abnormal procedures procedures as per focused Airplane Flight as per Airplane Flight Airplane Flight attention, but Manuals Manuals Manuals) no action is Near mid- air collision required Circumstances Circumstances encounters with requiring a flight crew requiring a flight crew 9 Pilot deviation less than separation to reject landing (i.e., to abort takeoff Flight Crew 10 where loss of 100 feet balked landing) at or (rejected takeoff); airborne near the runway however, the act of separation falls threshold aborting takeoff does within the not degrade the same Circumstances aircraft performance parameters of requiring a flight crew capability a Proximity to abort takeoff (i.e., Event or Near mid- air collision rejected takeoff); the measure of encounters with act of aborting takeoff compliance separation greater degrades the aircraft 10 greater than or than 500 feet performance equal to 66 capability percent Near m id-air collision Circumstances encounters with requiring a less than separation 10 flight crew to 500 feet initiate a go-around . Runway Safety Program Order 7050.1, Refer to the current version of FAA 1. 2. Proximity Events and Operational Deviations are no longer used to measure losses of separation, but they are applicable when validating old data. The minimal loss of standard separation is now represented as a measure of compliance of greater than or equal to 66 percent. Risk Analysis Event severity indicators are as follows: 3. Proximity. a. Failure transition point of 50 percent of required separation or less. Rate of Closure. Failure transition point greater than 205 knots or 2,000 feet per minute b. (consider both aspects and utilize the higher of the two if only one lies above the transition point). ATC able to implement separation actions in a timely manner. ATC Mitigation. c. 3.5_SMSM_201904 1 Originally published April 201 9 Uncontrolled copy when downloaded

49 Section 3 Safety Analysis and Risk Mitigation Process The Pilot executed ATC mitigation in a timely manner. Pilot Mitigation. d. An effect categorized as catastrophic is one that results in a fatality or fatal injury. 4. 5. Ground Collision. An airplane on the ground collides with an object or person. Minor Injury. Any injury that is neither fatal nor serious. 6. Serious Injury. Any injury that: 7. Requires hospitalization for more than 48 hours, commencing within seven days from a. the date the injury was received; Results in a fracture of any bone (except simple fractures of fingers, toes, or nose); b. Causes severe hemorrhages, nerve, muscle, or tendon damage; c. Involves any internal organ; or d. e. Involves second- or third -degree burns, or any burns affecting more than five percent of the body’s surface. Fatal Injury. Any injury that results in death within 30 days of the accident. 8. Refer to FAA Order JO 8020.16, Air Traffic Organization Aircraft Accident and Incident 9. , for more information about pilot deviations . Notification, Investigation, and Reporting Near Flight Standards 8900.1, FAA Order 10. mid -air collision definitions are derived from critical, which defines the following categories: Information Management System, Volume 7, Refer to Section potential. potential, and low 9 for the complete definitions of these categories. 3.5_SMSM_201904  9 Originally published April 201 Uncontrolled copy when downloaded

50 Section 3 The Safety Analysis and Risk Mitigation Process 3.5.4.2.1 Assessing Severity of NAS Equipment Hazard Effects Acquisition NAS equipment is subjected to thorough safety analysis through the FAA Management System (AMS). Refer to the Safety Risk Management Guidance for System Ac quisitions , or go to the FAA Acquisition System Toolset website for more information on the AMS. As such, the inherent functional severity of certain NAS equipment hazard effects has been assessed and documented. performing a safety analysis on NAS equipment that was previously assessed through When the AMS, it is recommended to use the data, methodology, and results of the previous work as the starting point for the new safety analysis. If there are differences in functionality between the original, previously assessed system and the system undergoing analysis, the differences should be accounted for and documented in the new safety analysis. In general, NAS equipment can fail such that one of two effects is expected: Loss of Function. The service is no longer provided. • • Malfunction. The service is being provided inaccurately or with diminished integrity. When identifying functional failures that lead to hazards, the loss of function and the malfunction of constituent parts must be considered. The severity of malfunctions and losses of function , is dependent from infrastructure systems , such as telecommunications and power systems upon the services they support. but are not limited to, the following: Examples of the systems that provide services include, Navigation (NAV) Instrument approach systems • , : Localizer, glide slope (e.g., visual glide slope indicators such as Precision Approach Path Indicator and Visual Approach Slope Indicator), Ground-Based Augmentation System, markers, approach lights, Distance Measuring Directional Aid, and Runway Visual Range Equipment, Localizer -Type -High -Range Radio, Frequency Omnidirectional : Very • En Route guidance systems Tactical Air Navigation, Distance Measuring Equipment, and Wide -Area Augmentation System Communication (COMM) • Air-to-ground COMM : Headsets/microphones, speakers, voice switches, radio control equipment, and radios Ground-to-ground COMM : Headsets/microphones, speakers, and voice switches • Surveillance • Automatic Dependent Surveillance – Broadcast (ADS-B), Airport Movement Area Safety System (AMASS), Automated Radar Terminal System (ARTS), Airport Surface Detection Equipment (ASDE), Air Route Surveillance Radar (ARSR), Air Traffic Control Mode Select Beacon System (MODES), Wide Area (ATCRB System S), Radar Beacon Multilateration (WAM), and Standard Terminal Automation Replacement System (STARS) 3.5_SMSM_201904  9 Originally published April 201 Uncontrolled copy when downloaded

51 Section 3 The Safety Analysis and Risk Mitigation Process Weather • Automated Surface Observing System (ASOS), Automated Weather Observing System -Level Wind Shear Alert System, Flight Service automation system, (AWOS), Low Operations and Supportability Implementation System, NextGen Weather Radar, Terminal Doppler Weather Radar, Weather and Radar Processor, and Weather Messaging Switching Center Replacement 3.5.4.2.2 Using the NAS Equipment Worst Credible Severity Table sessing the severity of hazards related to NAS equipment, use the “NAS Equipment” When as . Table 3.4 , the NAS Equipment Worst Credible row in Table 3.3 in conjunction with Table 3.4 Severity Table, is the starting point for severity assessments of NAS equipment. The severity of hazards that result from specific equipment changes may be lower or higher than the worst case due to the possible controls that limit exposure or the interactions and presented in Table 3.4 dependencies that exist with other systems. Because effects of losses in equipment functionality and equipment malfunctions may not necessarily be traceable to a loss in separation, equipment safety effects may require separate assessment from operational effects (i.e., assess the severity of equipment loss or malfunction irrespective of operational severity). The severity levels in Table 3.4 are derived from the operational safety analyses and other documentation produced during initial safety assessments completed as part of the AMS processes that define severity based on the inherent functionality of systems. References to high or low traffic are relative indications during a period of time at any given facility. 1 Table 3.4 : NAS Equipment Worst Credible Severity Table Worst Credible Environment / Failure Effect Service Functionality Severity/Rating System State Condition/Hazard Insufficient IMC, CAT III, critical reaction time Hazardous phase of flight (i.e., for pilot to Large reduction near or immediately execute in safety margin after touchdown) missed approach IMC, CAT I/II Loss of function Minor Missed All, CAT III, Increased flight Instrument approach non-critical phase of crew workload approach flight NAV guidance Minimal Pilot has to VMC take over Flight crew manual control inconvenience Hazardously Minor Misleading Malfunction Da y, VMC Information Increased flight (HMI), missed crew workload approach 1. Risk should be assessed and determined with regard to its operational impact on the provision of air traffic management, communication, navigation, or surveillance services. 3.5_SMSM_201904 44 9 Originally published April 201 Uncontrolled copy when downloaded

52 Section 3 The Safety Analysis and Risk Mitigation Process Worst Credible Failure Environment / Functionality Effect Service Severity/Rating Condition/Hazard System State Pilot Major penetrates Significant Obstacle Night, VMC reduction in Clearance safety margin Surface (OCS) Catastrophic HMI exceeds monitor limits Collision and penetrates between aircraft OCS and obstacles IMC HMI exceeds Hazardous monitor limits Large reduction but does not in safety margin penetrate OCS Visual Glide Night, VMC None Loss of function No safety effect Slope Indicators (Precision Major Approach Path Pilot Significant Indicator / Visual penetrates Malfunction Night, VMC reduction in Approach Slope OCS safety margin Indicator) Pilot Minor transitions to alternate IMC Loss of function reduction Slight NAV navigation in safety margin En route method guidance Hazardous HMI exceeds Malfunction IMC minimum en Large reduction route altitude in safety margin Minor Runway visual Loss of function / Missed IMC Increased flight range malfunction approach crew workload Major Large increase in ATC workload High traffic Significant or Pilots slight reduction unable to in safety margin Loss of single communicate frequency Minor with ATC on -to Air COMM -ground that frequency Significant increase in ATC Low traffic workload Slight reduction in safety margin Hazardous Simultaneous loss Pilots of multiple High traffic Large reduction unable to frequencies in safety margin communicate 45 3.5_SMSM_201904 9 201 Originally published April Uncontrolled copy when downloaded

53 Section 3 Safety Analysis and Risk Mitigation Process The Worst Credible Failure Environment / Functionality Service Effect Condition/Hazard Severity/Rating System State with ATC on Major multiple Significant Low traffic frequencies reduction in safety margin Minor ATC transitions to - Ground-to Significant Loss of function All alternate ground increase in ATC communication workload Major Significant High traffic reduction in ATC loss of safety margin situational Loss of function awareness Minor Aircraft/ vehicle Low traffic Slight reduction position in safety margin Major ATC makes Significant All Malfunction decisions reduction in based on HMI safety margin Minor ATC loss of ability to Significant Loss of function All differentiate increase in ATC aircraft among workload ATC makes Aircraft data decisions Major Surveillance based on Significant incorrect Malfunction All reduction in aircraft safety margin identification information ATC not Major alerted when aircraft exceed Significant Loss of function All established reduction in safety safety margin Alerts parameters Minimal False alarms All Malfunction Slight increase in ATC workload Minor ATC transitions to Significant Interfacility data Loss of function All manual increase in ATC methods workload 46 3.5_SMSM_201904 9 201 Originally published April Uncontrolled copy when downloaded

54 Section 3 The Safety Analysis and Risk Mitigation Process Worst Credible Failure Environment / Service Functionality Effect Severity/Rating Condition/Hazard System State Adverse Minimal weather information All Loss of function Flight crew Adverse reported as inconvenience weather unavailable information Major (Adverse Weather Adverse weat her Malfunction: Significant weather not All includes wind to detect failure reduction in reported shear, safety margin thunderstorms, icing, IMC, etc.) Minimal Adverse Malfunction: All weather falsely Flight crew false detection reported inconvenience 3.5.4.3 Determining Likelihood Likelihood versus Frequency 3.5.4.3.1 Likelihood i timated probability or frequency, in quantitative or qualitative s defined as the es More specifically, the concept of likelihood can be terms, of a hazard’s effect or outcome. Frequency is how often a separated into two components: likelihood/probability and frequency. given effect occurs ; it is a known value determined (for example) by monitoring a hazard and its effects to identify initial, current, or residual risk (see Section 4.3.1 and Section 4.3.4) . a hazard’s Conversely, likelihood is a n expression of the probability of (i.e., a effects occurring estimate to ), which is used rate of how often a given effect is expected to occur initial and predicted residual risk. Provide a rationale for likelihood estimations in the HAW. 3.5.4.3.2 What to Consider When Defining Likelihood Frequency and Modeling Frequency is sometimes used to help estimate likelihood, but historical data do not always represent future conditions. Historical frequency may be zero for a given procedure, but that kelihood is also zero. For example, a facility may conduct a not mean that the future li does procedure that has unreported i ncidents that could lead to an undesirable outcome, such as a loss of separation or a collision. Likewise, a facility may not have encountered the scenario or system state that exposes the more severe outcome. Consider all potential effects that are derived from indicators of the operation in all credible scenarios. This practice is required to challenge the philosophy of, “It has not happened in the past, so it will not happen in the future.” When possible, use modeling to examine the effects of hazards that are too rare to have 2 If modeling is required and data are available, significant historical statistical data available. the risk assessment should be based on statistical or observational data (e.g., radar tracks). Where there are insufficient data to construct statistical risk assessments, input from Subject Matter Experts (SMEs) can be used. This means that if the true rate of a particular type of operation is unknown, it can be estimated using expert judgment. It is important to note that complex proposed NAS changes, such as changes to separation standards, require quantitative data to support the associated risk analysis. 2. For guidance on how to design and conduct modeling in support of safety risk analyses, refer to Air Traffic Safety A, Guidance on Safety Risk Modeling and Simulation of Hazards ircular 07-05 Oversight C Oversight Service Safety and Mitigations . 3.5_SMSM_201904  9 Originally published April 201 Uncontrolled copy when downloaded

55 Section 3 Safety Analysis and Risk Mitigation Process The Credible Effects and Controls and the highest potential risk Determine Analyze the likelihood of all credible effects to: 1) 2) Identify all system states that expose the risk. Remember that less severe effects may occur more frequently, producing a higher risk, which is why it is important to determine the likelihood may of all credible effects. Consider controls when determining likelihood because they minimize the likelihood of an effect. Crossing FAA LOBs ; the provisions of When a NAS change crosses FAA LOBs, consult with the affected parties FAA Order 8040.4 apply. Calculating Likelihood with Quantitative Data 3.5.4.3.3 Once the credible effects and the estimated rates of occurrence have been determined, it is is the official source possible to calculate a likelihood rating. The Operations Network database data. of NAS air traffic operations To estimate the likelihood, first determine the expected number of times the credible effect will occur (i.e., the number of times that the hazard will occur in the system state that will expose the risk). Then, divide that value by the number of ATO operations, flight hours, or operational hours in which the effect is exposed (i.e., the number of ATO operations, flight hours, or operational hours affected by the proposed NAS change or the existing hazard). Finally, compare the result of this calculation (presented below) to the ranges presented in Table 3.5 to determine the likelihood rating. Identify which li kelihood unit to use to assess the effect’s maximum exposure rate (i.e., number the following of ATO operations, flight hours, or operational hours). For example, for environments, the number of ATO operations will often be the most appropriate likelihood unit to Terminal Radar Approach Control use when assessing the exposure of an effect: a (TRACON) an airport traffic ; or center; Air Route Traffic Control Center (ARTCC) with small, busy sectors RTCC control tower. However, when assessing an effect in the Oceanic an A with domain or for a larger sector, often the number of flight hours may be more appropriate. System acquisitions or modifications will use units of operational hours. Whether the NAS change applies to a single facility or to an entire NAS domain, it is important to use the relevant number of ATO operations in which the hazard may occur when calculating likelihood. 3.5_SMSM_201904 48 9 Originally published April 201 Uncontrolled copy when downloaded

56 Section 3 Safety Analysis and Risk Mitigation Process The ATO Operations and NAS Equipment : Likelihood of the Effect Standards – Table 3.5 Operations: Expected Occurrence Rate 3 (per operation / flight hour / operational hour ) Quantitative (ATC / Flight Procedures / Systems Engineering) Frequent (Probability) ≥ 1 per 1000 A Probable 1 per 1000 > (Probability) ≥ 1 per 100,000 B Remote 1 per 100,000 > (Probability) ≥ 1 per 10,000,000 C Extremely Remote 1 per 10,000,000 > (Probability) ≥ 1 per 1,000,000,000 D Extremely Improbable 14 1 per 1,000,000,000 > (Probability) ≥ 1 per 10 E om an analysis of historical ATC data mapped to the The values i n Table 3.5 are derived fr the System current version of Advisory Circular 25.1309-1, established engineering standard ( Design Analysis ) and can be applied to both ATC and Flight Procedures. The ratios binding each expected occurrence rate range were determined through calculations made using ten years of aviation data. In each calculation, the numerator was the number of occurrences of a given severity level occurring during a ten-year period, as obtained from various relevant databases. The denominator was the number of ATO operations (or flight hours) in that or the National ten-year period, as obtained through the Operations Network database Transportation Safety Board database. The value was adjusted to reflect a forecasted air traffic -14 was established to define the boundaries of credible events for increase. A cut-off point of 10 the purposes of calculating likelihood. Figure 3.6 depicts the likelihood continuum and the expected occurrence rate ranges. inuum Figure 3.6: Likelihood Cont 3.5.4.3.4 Determining Likelihood When No Data Are Available For some NAS changes, the necessary data are not available. There may not be a similar enough change/procedure/situation in the NAS to provide similar data from which to estimate a rate of occurrence. In situations where modeling is not feasible, pure subject matter expertise is the only input available, providing a qualitative approach to determining likelihood. This approach is only recommended when all avenues of data collection have been exhausted or when the change proponent is attempting to implement a new operation for which no data exist. For a majority of changes to the NAS, SMEs can collect and analyze data from a similar NAS change to determine the number of expected occurrences of an effect. It is important to note that the close correlation between flight hours and operations is entirely coincidental; 3. average flight time is roughly two hours, and each flight has about two T ower and two TRACON operations. The two numbers are not interchangeable. 3.5_SMSM_201904 49 9 Originally published April 201 Uncontrolled copy when downloaded

57 Section 3 Safety Analysis and Risk Mitigation Process The -wide effect occurrences. This table -based approximations of NAS presents calendar Table 3.6 only applies if the proposed NAS change or existing hazard affects all ATO operations in a particular air traffic domain. Operations/Domain-Wide Table 3.6 : Calendar -Based Likelihood of the Effect Definitions – Operations: Expected Occurrence Rate based) - (Calendar -wide, Terminal, or En Route) -wide: NAS (Domain Frequent Equal to or more than once per week A Probable Less than once per week and equal to or more than once per three B months Remote Less than once per three months and equal to or more than once per C three years Extremely Remote Less than once per three years and equal to or more than once per 30 D years Extremely Improbable Less than once per 30 years E 3.5_SMSM_201904 50 9 Originally published April 201 Uncontrolled copy when downloaded

58 Section 3 The Safety Analysis and Risk Mitigation Process DIA 3.6 AT Phase 4: Assess Risk ASSESS Assign risk level for each hazard based on severity and likelihood A RISK 3.6.1 Over view In this phase, identify each hazard’s associated initial risk and plot each hazard on a risk matrix. When assessing and mitigating safety risk, first determine the risk level prior to the implementation of any safety requirements (see Section 3.7.3) . describes the Initial risk composite of the severity and likelihood of a hazard, considering only control s and documented safet assumptions for a given system state. It describes the risk before any of the y requirements are implemented. When assessing National Airspace System (N AS) equipment or existing hazards, the initial risk current risk may be equated to the , which is defined as the assessed severity and frequency of a hazard’s effects in the present state. 3.6.2 Risk Levels and Definitions Record all hazards and their associated risk levels. Hazards are assigned one of three risk levels: 3.6.2.1 High Risk This is unacceptable risk, and the NAS change cannot be implemented unless the hazard’s associated risk is mitigated to medium or low. Existing high -risk hazards also must be reduced - or low hazards . The predicted residual risk must be monitored and tracked in to medium -risk relation to the safety performance targets . The predicted residual risk must be confirmed with . objective evidence suggesting an impact to the hazard’s causes or effects Hazards with catastrophic effects that are caused by single point events or failures, common cause events or failures, or undetectable latent events in combination with single point or common cause events are considered high risk, even if the possibility of occurrence is extremely improbable. When a system has a single point failure , there is a failure of one independent element of the system that causes or could cause the whole system to fail. The system does not have a back ve procedure to compensate for the failed component. An -up, redundancy, or alternati example of a single point failure is found in a system with redundant hardware, in which both pieces of hardware rely on the same battery for power. In this case, if the battery fails, the entire sy stem will fail. A common cause failure is a single fault resulting in the corresponding failure of multiple components. An example of a common cause failure is found in a system with redundant computers running on the same software, which is susceptible to the same software bugs. 3.6_SMSM_2016 07 5 1 Originally published July 2016 Uncontrolled copy when downloaded

59 Section 3 The Safety Analysis and Risk Mitigation Process Medium Risk 3.6.2.2 Although initial medium risk is acceptable, it is recommended and desirable that safety requirements be developed to reduce severity and/or likelihood. The risk must be monitored and tracked in relation to the safety performance targets . The predicted residual risk must be confirmed with objective evidence suggesting an impact to the hazard’s causes or effects . Refer to Section 4.2 for information on monitoring. A catastrophic severity and corresponding extremely improbable likelihood qualify as medium risk, provided that the effect is not the result of a single point or common cause failure. If the risk. cause is a single point or common cause failure, the hazard is categorized as high 3.6.2.3 Low Risk This is acceptable risk without restriction or limitation. It is not mandatory to develop safety ow -risk hazards; however, develop a monitoring plan with at least one safety requirements for l performance target. Plotting Risk for Each Hazard 3.6.3 The risk matrix shown in Figure 3.7 is used to determine risk levels. Plotting the risk for each hazard on the matrix helps to prioritize treatment. The rows in the matrix reflect the likelihood categories, and the columns reflect the severity categories. Adhere to the following guidelines when plotting risk for each hazard: • Plot a hazard’s risk according to its associated severity and likelihood. • To plot the risk for a hazard on the risk matrix, select the appropriate severity column (based on the severity definitions in 3 Table 3. ) and move down to the appropriate Table 3.5 or likelihood row (based on the likelihood definitions used from either 3.6 Table ). • Plot the hazard in the box where the severity and likelihood of the effect associated with the hazard intersect. • If the plotted box is red, the risk associated with the hazard is high; if the box is yellow, the risk associated with the hazard is medium; and if the box is green, the risk assoc iated with the hazard is low. As shown in the split cell in the bottom right corner of the matrix, hazards with a catastrophic severity and extremely improbable likelihood can be medium or high risk, depending on the cause, as explained in Section 3.6.2.1. The current edition of Federal Aviation Administration (FAA) Order 8040.4, Safety Risk , pres cribes the use of a ris k matrix that is different from the risk Management Policy matrix depicted in Figure 3. 7 so appl ies w . The order al d to acceptability ith regar of risk l evels at t he agency l evel w hen c rossing L ines o f B usiness (LOBs) . Use t he A ir Traffic O rganization (A TO ) fety risk matrix and risk a icy i n this S afety M anagement S ystem M anual f or al l sa ssessment pol risk analyses in which the ATO accepts the risk. When the safety analysis involves acceptance of safety risk by FAA LOBs other than the ATO, the current edition of FAA Order 8040.4 applies. 3.6_SMSM_2016 07 5 2 Originally published July 2016 Uncontrolled copy when downloaded

60 Section 3 The Safety Analysis and Risk Mitigation Process Severi Likelih Major Hazardous Minor Minimal Catastrophic t y 1 4 2 5 3 oo d Frequent High Medium High High Low A Probable High High High Medium Low B Remote Medium Low Medium High High C Extremely Medium Remote Medium Low Low High D High* Extremely Medium Improbable Low Low Low E Medium *Risk is high when there is a single point or common cause failure. sk Matrix Figure 3.7: Ri 3.6_SMSM_2016 07 5 3 Originally published July 2016 Uncontrolled copy when downloaded

61 Section 3 The Safety Analysis and Risk Mitigation Process DIAAT 3.7 Phase 5: Treat Risk Choose risk management strategies Develop safety performance targets TREAT RISK T Develop monitoring plan Overview 3.7.1 In this phase, identify appropriate means to mitigate or manage the safety risk. Treating risk involves: appropriate safety requirements, • Identifying • Defining safety performance targets or a sound alternate method to verify the predicted residual risk for each hazard, and • Developing a monitoring plan that prescribes tasks and review cycles for comparing the current risk to the predicted residual risk. rategies 3.7.2 Risk t Management S address safety risk, identify and evaluate means To either manage the risk or reduce it to that an acceptable level. s The four ri k management strategies are risk control, risk avoidance, risk transfer, and risk assumption. Assess how the proposed risk management strategy affects the on o overall risk. Consider using a combinati f actions to best manage the risk to an or reduce acceptable level. When determining the appropriate strategy, consider how the safety t performance arget (see Section 4.1 ) will be used to evaluate the safety performance of the chosen course of action. R 3.7.2.1 isk Control risk control strategy involv A es the development of safety requirements , defined as planned or proposed means to reduce a hazard’s caus es Examples include policies or or effects. procedures, redundant systems and/or components, and alternate sources of production. R efer 3 on to Secti for info r .7.3 mation on developing safety requirements. ly —ultimate An explanation of how a safety requirement reduced the hazard’s risk level or supported with objective evidence through testing, monitoring, hod — another met must be requiremen provided for each safety t . All safety requirements that are implemented and are successful l determined to have y addressed the hazard or safety issue become part of the operating National Airspace System (NAS). At that t i me, they will be considered “controls” that form the basis for future safety hazard and risk analysis efforts. Refer to Sec t ion 3.5.2 for tion on controls informa . 3.7.2.2 Risk Avoidance risk avoidance The strategy av erts the potential occurrence and/or consequence of a hazard by either selecting a different approach or not implementing a specific proposal. This technique may be pursued when multiple alternatives or options are available, such as determining where to construct an air traffic control tower. In some cases, a decision may be made to limit the NAS change to certain conditions or system states, thereby avoiding the risk associated with other conditions. An example of this is allowing simultaneous operations on one runway that is over -flown by three other runway flight paths. It may be discovered that the risk associated with unways but simultaneous operation can be mitigated to an acceptable level f r the or two of the 3.7_SMSM_2016 54 07 Originally published July 2016 Uncontrolled copy when downloaded

62 Section 3 The Safety Analysis and Risk Mitigation Process the third. It may be decided that aircraft will not be allowed to operate on the third not for runway while simultaneously landing on the crossing runway, thereby avoiding risk. may be used when multiple systems or procedures are A Comparative Safety Assessment mitigated to an acceptable level, then another system, available. If one alternative cannot be method, or procedure may be chosen. When no alternatives are available, the risk avoidance strategy is more likely to be used as the basis for a “go” or “no-g o” decision at the start of an operation or program. Risk must be avoided from the perspective of all affected stakeholders. Thus, an avoidance strategy is one that involves all of the stakeholders associated with the proposed NAS change. 3.7.2.3 Risk Transfer risk transfer strategy The shifts the ownership of risk to another party; the recipient may be better equipped to mitigate the risk at the operational or organizational level. Organizations transfer risk primarily to assign responsibility to the organization or operation most capable of managing it. The recipient must accept the risk, and the transfer must then be documented (e.g., through a Letter of Agreement, Statement of Agreement, or Memorandum of Agreement). Examples of risk transfer may include: • The transfer of aircraft separation responsibility in applying visual separation from the air traffic controller to the pilot, The development of new policies or procedures to change ownership of a NAS • component to a more appropriate organization, • The procurement of contracts for specialized tasks from more appropriate sources (e.g ., contract maintenance), and • The transfer of A ir Traffic Control systems from the acquisition organization to the organization that provides maintenance. Transfer of risk can risk hazard. Identify safety not be the only method used to treat a high- requirements to lower the safety risk to medium or low before it can be accepted in the NAS. All transferred risks must be monitored until the predicted residual risk is verified by the appropriate organization. 3.7.2.4 Risk Assumption risk assumption The at str egy simply means accepting the risk. The risk acceptor assumes responsibility for the risk as it is. When a risk acceptor agrees to implement a NAS change, he or she agrees to implem ent it based on the predicted residual risk being medium or low and assumes responsibility for the risk. When this management strategy is used, the predicted residual risk is derived from the controls. Under this strategy, controls serve as the basis on which safety performance targets or alternate methods to verify predicted residual risk are I t developed. is recommended and desirable that safety requirements be developed to further ood or severity. mitigate risk or reduce likelih It is not permissible to use a risk assumption strategy to treat an initial or current high risk ed with a hazard. The predicted residual risk for initial high- risk hazards must be associ at medium or low before it can be accepted into the NAS. 3.7_SMSM_2016 07 55 Originally published July 2016 Uncontrolled copy when downloaded

63 Section 3 The Safety Analysis and Risk Mitigation Process Documenting Safety Requirements 3.7.3 SRM and i nc All safety requirements identified by the Safety Risk Management ( ) panel luded in and the Hazard Analysis Worksheet (HAW) are considered to be recommendations for review at approval by the appropri e signatories . After appropriate means of managing risk have been by the SRM developed and documented , m panel anagement officials may of identify the effect on other org safety requirements ani zations and coordinate with the affected organizations. on the saf e It may be necessary to perform separate safety analyses to ty requirements determine their effects on the NAS. If so, the associated safety analyses must be developed, completed, and approved for implementation before proceeding with implementation of the original NAS change. Refer to S ection 6.3 for more information on safety requirements approval and implementation aking and signatures. decision -m D 3.7.4 esidual Risk i eterm ning Predict ed R Predicted residual risk s is the ri k that is estimated to exist after the safety requirements are implemented or after all avenues of risk reduction have been explored. The predicted residual risk is based on the assumption that controls are in place and/or all sa f ety requirements are implemented and are valid. If safety requirements are not documented in the HAW, predicted residual risk should be the same as the initial risk. If the risk cannot be reduced to an acceptable level after attempting all possible risk reduction strategies, either revise the original objectives or abandon the proposed NAS change. If an ac ceptable propos al is not identified, the NAS change cannot be implemented. Similarly, if a NAS change was implemented without safety requirements and the predicted residual risk was not met, the safety analysis must be revisited, which may require the development o f safety requirements. Refer to nformation. Section 4.3.2 for more i 3.7_SMSM_2016 07 56 Originally published July 2016 Uncontrolled copy when downloaded

64 Section 4 Developing Safety Performance Targets and Monitoring Plans 4.1 Safety Performance Targets Developing are measurable goals used to verify the predicted residual risk of Safety performance targets a hazard. A safety performance target is the preferred means to relate the performance of risk to the expected risk level. The safety performance target is included as part of reduction efforts the monitoring plan (see ). Section 4.2 s and Safety performance targets are used to assess safety performance with respect to control newly implemented safety requirements. Do not define the worst credible effect or effects producing the highest risk level as the safety performance target; instead, look at the less the number of unauthorized vehicle deviations on taxiways per severe effects or indicators (e.g., a specific number of airport operations over a period of time). Safety performance targets Airspace should be related to the hazard or N ational System (NAS) change. to determine the When developing safety performance targets, use subject matter expertise appropriate metrics to monitor. The sources of data used when preparing to assess the NAS be evaluated when developing safety performance targets. If there is no must change established data source to support a proposed safety performance target, a means to begin collecting this data should be identified and documented. The pre– Safety Risk Management panel data analysis also serves as the basis for comparison against the post-implementation metrics. performance target may not be possible in terms of Mapping a hazard to a specific safety establishing a causal relationship. In such cases, identify a sound alternate method to verify the predicted residual risk and determine whether control s and/or safety requirements are are functioning as intended. appropriate and 4.1_SMSM_201904  9 Originally published April 201 Uncontrol led copy when downloaded

65 Section 4 Developing Safety Performance Targets and Monitoring Plans Developing the Monitoring Plan 4.2 The monitoring plan should be comprehensive to verify the predicted residual risk. The monitoring plan includes the safety performance targets or another sound method for verifying the predicted residual risk. Create a plan for each hazard that defines: • Monitoring activities; • The frequency and duration of tracking monitoring results; and • How to determine, measure, and analyze any adverse effects on adjoining systems. 4.2.1 Monitoring Activities The risk acceptor, or the monitoring Point of Contact (POC) identified by the risk acceptor, must verify that the control s and/or safety requirements were indeed implemented and are functioning as designed. Specifically, this means that procedures must be stringently followed and hardware or software must function within the established design limits. identified by the risk Detail the methods by which the risk acceptor, or the monitoring POC will gather the performance data or monitoring results. The organization that acceptor, ensuring that the monitoring plan is being upheld (i.e., that accepted the risk is accountable for the the defined safety performance targets (or to are being compared the monitoring results t). Refer results alone are being used) to determine whether predicted residual risk is being me to Section 6.4 for information about risk acceptance. It is important to retain objective evidence that the safety requirements have been implemented. is simply documented proof. The evidence must not be circumstantial; it Objective evidence must be obtained through observation, measurement, testing, or other means. Frequency and Duration of Monitoring 4.2.2 When considering the frequency and duration of tracking monitoring results, account for: change, • The complexity of the National Airspace System The hazard’s initial risk level, • How often the hazard’s effect is expected to occur (i.e., likelihood), • • s, Control • The types of safety requirements that are being implemented (if any), and The amount of time needed to verify the predicted residual risk. • For example, when considering a hazard associated with the familiarity of a new procedure, a relatively short tracking period would be required until a person or population could reasonably be expected to adapt to the new procedure and the predicted residual risk could be verified. However, the monitoring plan for a hazard associated with new separation criteria may require several years of tracking to verify the predicted residual risk. able 5.3 Refer for the documentation requirements of a summarized monitoring plan. Refer to T for documentation requirements of a complete monitoring plan for an individual Table 5.5 to hazard. 4.2_SMSM_201 904  Originally published April 201 9 Uncontrolled copy when downloaded

66 Section 4 Developing Safety Performance Targets and Monitoring Plans 4.3 Post -SRM Monitoring It is critical to obtain feedback on safety performance indicators through continuous monitoring. Organizations responsible for performing Quality Control and/or Quality Assurance use audits and assessments to monitor the safety risk and performance of an implemented N ational Airspace System (N AS) change documented in the monitoring plan. The responsible organization determines whether an implemented NAS change is meeting the safety performance targets documented in the monitoring plan. Results of post-implementation monitoring help determine whether a change can be made part of the operating NAS or must be reassessed through the Safety Risk Management (SRM ) s. proces 4.3.1 Monitoring and Current Risk A hazard’s current risk is updated at each monitoring interval (in accordance with stated monitoring frequency). Current risk provides an indicator of whether safety requirements are meeting the predicted residual risk. The risk acceptor is accountable for ensuring that the monitoring plan is being upheld and that monitoring reports, as dictated by the monitoring frequency, are being analyzed to determine whether the safety performance targets are being met . 4.3.2 Predicted Residual Risk Is Not Met Through monitoring current risk and the safety performance of a recently implemented NAS change, it may become clear that the predicted residual risk is not being met. If this occurs, notify the risk acceptor. The risk acceptor may choose to accept the current risk as the new The SRM document must be revised with the new predicted residual predicted residual risk. for 6.4.1 Section to approval and risk acceptance signatures must be reobtained (refer and risk, ). There are several reasons why the predicted residual authority information on risk acceptance risk may not be met: ements or control s may not be properly mitigating the risk, • The safety requir • The initial risk may have been assessed inaccurately, Uni • ntended consequences may have occurred, or • identified. New hazards may be Sec Refer to Section 6.7 tion 5 for information about SRM panels and for information on updating safety documentation. 4.3.3 Is Predicted Residual Risk Met The successful completion of monitoring is a prerequisite to hazard and NAS change closeout. This includes the achievement of safety performance targets and/or the predicted residual risk. The monitoring procedures used to verify the predicted residual risk must also be documented, as they will be used to evaluate the safety performance of the change after it is added to the operating NAS. The established monitoring requirements must be followed, even after meeting the goals of the monitoring plan. Residual Risk 4.3.4 by completing a thorough monitoring sk that has been verified Residual risk i is the level of r plan with achieved measurable safety performance targets. It is the assessed severity of a hazard’s effects and the frequency of the effect’s occurrence. 59 4.3_SMSM_2019 04 Originally published April 201 9 Uncontrolled copy when downloaded

67 Section 4 Developing Safety Performance Targets and Monitoring Plans Monitoring and Tracking of Changes Added to the Operating NAS 4.3.5 A change is considered to be part of the operating NAS only after monitoring is completed, the safety performance target is achieved and maintained, and/or the predicted residual risk is verified. At that point, the NAS change is monitored through existing Safety Assurance safety is maintained. The NAS change processes to determine whether an acceptable level of and all of the associated safety requirements become part of the operating NAS, which will become the basis from which all future NAS changes will be measured. If a safety requirement change that was made part of the operating NAS, a new SRM is altered or removed from a NAS analysis must be performed. The documentation that was developed during the SRM process is critical to Safety Assurance as inputs to assessments and evaluations. The functions, which often use S RM d ocuments 5. Section process for preparing, performing, and documenting the safety analysis is described in 4.3_SMSM_2019 04  Originally published April 201 9 Uncontrolled copy when downloaded

68 Section 5 Preparing, Performing, and Documenting a Safety Analysis 5.1 Overview 5.1.1 Safety Analysis Process Flow he ov erall pr ocess for pe rforming the sa fety anal ysis f Figure 5.1 depicts t existing s afety or an issue or a N ational A irspace System ( NAS) change and proceeding through t he adm inistrative process for getting t he safety anal ysis and its ass ociated safety requirements through the from t approval pr e figure separates the safety anal ysis pr ocess . Th he documentation ocess approval and review process (see Section 6), in which the analysis is recorded in a Safety Risk Management (SRM) document. Refer to Section 5.4 for additional information on SRM documentation. 5.1_SMSM_2016 07 6 1 Originally published July 2016 Uncontrolled copy when downloaded

69 Section 5 Preparing, Performing, and Documenting a Safety Analysis Figure 5.1: Safety Analysis Development and Approval Process 5.1_SMSM_2016 2 6 07 Originally published July 2016 Uncontrolled copy when downloaded

70 Section 5 Preparing, Performing, and Documenting a Safety Analysis afety Analysis 5.2 Preparing a S aking 5.2.1 Planning and Initial Decision-M The scope of a Sa fety R isk M anagement (SRM) effort is bas ed on the type, complexity, and sue. It is critical that S) isting safety i s ational Airspace System (NA effect of the N change or ex of the associated the level of detail in a safety analysis matches the scope and complexity hange NAS c or existing safety issue (see Section 3.3.2.4). To support this activity, the change uality C ontrol G vice Center Q roup, er proponent for the SRM effort should c onsult his or her S er (who can be the Safety and Technical Training (AJI) Safety Engineering Team M anag stem (SMS) contacted through the Air Traffic O rganization (ATO) Sa fety M anagemen t Sy mailbox nt of contact when initiating the process. The following steps are afety poi ), or a local s nitial de cision-m aking, as w ell as pl anning and prepari ng for a essential to performing any i sessment of all NAS c hanges and existing safety i ssues: safety as early define the NAS change or existing safety issue. Cl • • Scope the operational system and/or environment affected. • Decide the extent to which SRM must be performed and/or documented. • Coordinate with other organizations that may be affected by the NAS change or the potential risk management strategies. • Identify an SRM panel facilitator, if necessary. • Identify a facility/organization/program/technical lead (i.e., a Subject Matter Expert (SME)). facilitator. • Identify appropriate SRM p anel members by consulting with the SRM p anel ssues identified i onfirmed existing safety i ded to address c s nee SRM i e.g., Top 5 n the NAS ( safety i ssues and other i dentified in national Corrective Action Requests). When ssues i mined. It is not ng existing safety i ssues, the hazard and its r addressi evel m ay be pre-d eter isk l necessary t o reassess t he validity or current risk l evel of any ex i sting safety i s sue identified and pos t- event safety r isk anal ysis. The purpose of performing SRM confirmed by a safety audi t or on existing safety i ssues is to identify s afety r equirements or other ac tions t o reduce the associated risk to an acceptable level. At minimum, apply S RM to the existing safety i ssue; o account for the risk i however, the risk as sessment should als mpact of any pr oposed safety requirements. 5.2.1.1 Scope must properly define the The change proponent, along with a small group of technical experts, purpose and scope of the NAS change (see Section 5.3). The group should follow the guidance and requirements in Section 3.1.1 to determine the impact of the NAS change on relevant NAS equipment, operations , and procedures. 5.2.1.2 Detecting Potential for Hazards ning the sc ope and purpose of the NAS c hange, the change proponent and a small After defi ociated al s afety haz ards as s group of technical ex perts should determine if there are any potenti with the NAS change or if the NAS change could increase risk associated with NAS equipment, operations, and/or procedures. Review Section 3.4 for assistance in determining the existence of safety haz ards as sociated with the NAS c hange. When the NAS c hange does no t have the avigation, and/or ng of ai r t raffic m anagemen t, c ommunicati on, n potential to affect safe provisioni surveillanc ysis i ther anal e services, no fur al for the s potenti equired. Conversely, if there i s r 63 5.2_SMSM_201904 Originally published April 201 9 Uncontrolled copy when downloaded

71 Section 5 Preparing, Performing, and Documenting a Safety Analysis NAS change to affect the safety of the NAS, the change proponent should proceed to perform an in-d ction 5.2. 2). epth safety analysis (refer to Se epth S 5.2.2 Preparing for In-D afety Analyses If the change proponent and initial g roup of SMEs deter mine that there are safety haz ards associat ed with a NAS change, a more in-d epth s afety anal ysis m ust be performed. Likewise, when usi ssue, a more in-d epth approach is w arranted. ng SRM to address an existing safety i alled an SRM ision will nec essitate a larger group of SMEs and stakeholders, ty pi cally c This dec s to objectively ex panel. The role of the SRM panel i ards and effects amine potential haz as ed with the NAS c hange. The SRM p anel onl y as sesses the safety of the NAS c hange, sociat must not use panel del iberations to define essity. SRM panel s not its s uitability, validity, or nec w hat the NAS c hange should be or at tempt to reassess the purpose or i ntent of the NAS change defined by the organization(s) s ponsoring the NAS c hange. 5.2.2.1 SRM Panel Facilitator e led by hange proponent selects or r equests an SRM panel fac ilitator. All S RM panel s ar The c expert in facilitati on and SRM. The role of the facilitator i s to work a facilitator , who is a trai ned with the change proponent to help scope the safety anal ysis and moderate the deliberations of ilitator s -versed in the subjec t matter hould become well the SRM pa nel. The SRM panel fac (e.g., by requesting briefings and collecting all av as nformation), ailable and relevant safety i necessary onvenes. The facilitator w ill ensure all relevant informati on , before the SRM panel c about the NAS c hange or ex isting safety i ssue is s ent to the S RM panel members be fore the panel m eeting. An effective SRM panel f acilitator ensures the SRM pr ocess i s fol lowed in an unbiased manner and works t o achieve consensus. He or s he captures the decisions of the panel m embers, mediates any di ssenting opinions, and remains neutr al sagreements, documents any di throughout the process without advocating for a s pecific out come. The facilitator (or his or her designee) m ay w rite the document describing the safety fi ndings of the SRM panel m eeting. Facilitator duties and responsibilities m ust be discussed w ith the change proponent and ticipants. communicated to the SRM panel par 5.2.2.2 SRM Panel Co-Facilitator -facilitator shares the same duties and responsibilities as the panel facilitator. anel co The SRM p -fac Like the facilitator, the co wr ilitator (o r his or her des ignee) may ite the document describing must be the safety fi ndings of the SRM pane l meeting. Co -facilitator duties and responsibilities anel participants di . scussed with the change proponent and communicated to the SRM p JI Safety C ase Leads 5.2.2.3 Facilitation by A ase lead may fac An AJ I safety c ilitate SRM effor ts for N AS c hanges that meet any of the following criteria: • The NA ) impact on (potential ly political, economic, or financial the S change has a high Federal Aviation Administration (FAA), the NAS, or the flying public. • The NAS change is the result of financial or operational decisions made by FAA abinet-level executives, or Congress. executive management, c risks identified as part of the Top any safety The NAS change includes means to reduce • 5 Program. • The NAS change modifies safety policy that must be incorporated in a directive. 5.2_SMSM_201904 64 9 Originally published April 201 Uncontrolled copy when downloaded

72 Section 5 Preparing, Performing, and Documenting a Safety Analysis • The NAS change can or does present operational or technical conflicts to multiple affected Service Units or FAA L ines of Business (LOBs). 5.2.2.4 Pre –SRM Panel Assessment of the Scope of the Safety Analysis After selecting a facilitator, the change proponent and the facilitator (and co-facilitator , if one is selected) will have an initial meeting to prepare for the SRM panel. During this time, the facilitator will provide a briefing to the change proponent on the SRM process. This meeting will be used to define: also NAS change or existing safety issue, The • The system state(s) in which the change will be operational, • • Assumptions (not controls) that may influence the analysis, and • The components of the 5M Model. When defining the components of the 5M Model, adhere to the following guidelines: • Mission: There should be agr eement on the language for the NAS change or existing safety issue that the SRM panel is tasked to assess. Ensure that the language is unambiguous, concise, and clearly reflective of the NAS change. or existing safety hange • Human: Identify stakeholders that are affected by the NAS c hange affected by the NAS c ly , identify organizations that are . First issue or existing safety issue. Secondly, proceed to identify SMEs from each of those organizations. Be mindful that further discussions may identify the need to add other organizations to the SRM panel. There may be times where it is not feasible to obtain participation from some of the identified stakeholders. In those cases, other avenues of collecting input or data may be used, such as telephone interviews, worksheets, surveys, etc. • Machine: Define the hardware and software involved in the NAS c hange or existing safety issue. • Management: Define the documents that are relevant to the NAS c hange or existing (e.g., directives, policies, Standard Operating Procedures, Letters of safety issue Agreement). hange Define the elements of the NAS that are affected by the NAS c • Media: or existing safety issue. on between the change proponent and facilitator/co-facilitator will Coordination and preparati result in the development of a briefing package to provide to SRM panel members. The briefing package should include the SRM panel meeting invitation, an agenda, briefing materials, and should be shared with SRM panel members in directions to the meeting venue. All documents advance of the panel meeting. Involving AOV during a Safety Analysis 5.2.2.5 hange t evaluate the NAS c or existing safety issue to determine whether it SRM panel mus An will require approval or acceptance from the ffic Safety Oversight Service (AOV). Air Tra Contact the ATO Chief Safety Engineer for guidance, if necessary. If AOV approval or acceptance is required, the SRM panel facilitator or change proponent will coordinate with AJI to ensure compliance with AOV require ments. 5.2_SMSM_201904 65 9 Originally published April 201 Uncontrolled copy when downloaded

73 Section 5 Preparing, Performing, and Documenting a Safety Analysis 5.2.2.6 SRM Panel Membership 5.2.2.6.1 Overview hange proponent works c losely w ith the SRM panel fac ilitator /co-f acilitator to identify the The c e safety of the NAS c hange. The size and SRM panel participants necessary to assess th composition of the SRM panel w ill v ary w ith the type and complexity of the proposed N AS change or c urrent risk. The SRM p anel m ust be limited to an appropriately s ized team of considered to be an entity that could be affected by stakeholders and SMEs. A s takeholder i s the proposed NA hange from a safety r isk per spective (i.e., an entity responsible f or any of S c the following tas ks : implementing the NAS change when approved, accepting the residual r isk , implementi ng safety r equirements, or affi rming controls). gaining Unit P ar ticipation 5.2.2.6.2 SRM Panel Guidance for B ar nel attendee , adhere to the Collective Bargaining Agreement between electing SRM pa s When s the FAA and affected bargaining uni hange or ex i t representatives. When a NAS c sting safety issue cr osses Se rvic e Area boundaries and LOBs , the change proponent will ens ure the M anagement Services (AJG) Tec hni cal Labor Group, AJG-L1, is notified. on, m M ultiple bargaining uni t members, w hen represented by the same labor uni ay be SRM panel members. Ensure that all fac are given ilities, including their respective bargaining units, notification of the upcoming SRM panel. Labor organizations, such as the National A ir Traffic (N ATCA), represent several different bargaining units (engineers, Controllers Association ontrollers, attorneys , etc.) c . In some cases, multiple bargaining units m ay need to be present on the panel to ensure that the appropriate expertise is available. In all cases, the labor organization representative will i dentify a lead representative that speaks f or the labor or ganization during the safety anal ysis. viation on r If you need assistance finding a labor uni epresentative (e.g., NATCA, Professional A ore information. for m 1 -L pecialists), please contact AJG Safety S 5.2.2.6.3 Participation on SRM P anels O vice Unit or t he ATO utside of a S er takeholders or S ted to participate as s pl M Es on SRM panels oyees are often reques ATO em ide o f their S ervice Unit or t he ATO. It is important to support ponsored by organizations outs s these requests, whether they originate within or outside of the ATO. Participation as an SME or not necessarily m ean that the organizati on represented by an SRM panel stakeholder does member i esponsible f or dev el oping or i m s r plementing safety requirements, accepting risk, or appr oving the safety anal ysis. Refer to Section 6 for information on safety requirement approval and implementation, risk ac ceptance, and documentation approval. When requesting the participation of an ATO S ervice Unit, the requestor should contact the oordination. e or S ervice Unit for c c appropri ate program offi 5.2.2.6.4 Primary S R anel R oles M P east one of the roles s ny S M p A eeting attendee should fulfill a t l R pec ified as f ol lows: anel m C hange Proponent: An individual , program offi c e, facility, or or gani zation within the FAA that hange or means to has i fied the need for S RM or h as proposed o r is sponsoring a NAS c denti addr ess an existing safety issue. Func tional D escription: Am ong other responsibilities , the change proponent works with the SRM ssue, sting safety i i hange or ex litator to defi e and scope of the NAS c ne the purpos panel fac i 5.2_SMSM_201904 66 9 Originally published April 201 Uncontrolled copy when downloaded

74 Section 5 Preparing, Performing, and Documenting a Safety Analysis ndi ngs fr om the SRM panel meeting in an SRM doc ument, and ensure that capture the safety fi cument is r ecorded in SMTS . the SRM do Note: The safety c ase approver s ember. hould not be a panel m A tr SRM Panel Facilitator/Co-Facilitator: ained expert on the SRM process who moderates the deliberations of the SRM panel members from a neutral pos ition. Functional Description: Refer to Section 5.2.2.1. SRM Panel Member: A s elec ted FAA em ployee or FA A bar gaining unit representativ e who objectively performs the safety as sessment using the SRM pr ocess. on: An SRM panel m ember represents t Functional Descripti he program, facility, organization, or constituency potentially affected by the safety r isk , the safety r equirements associated with the proposed NAS c hange, and/or the existing safety issue . Among other r esponsibilities, SRM panel m aluate the c hange / existing safety i ssue objectively, thoroughly, and fairly; embers ev make determinations on safety risk i n the NAS ; and review the document describing the safety findings of the SRM pan el m eeting. takeholder w arty s rd-p ployee or thi FAA em An xpert: ho serves as a technical Subject M atter E expert on the NAS c oposed solution oftware system, or pr hange, procedure, hardware or s ng SRM. undergoi escription: SME Functional D hare data, detailed information, and experience on the topic s s being discussed during the SRM pan el m eeting. An SM E is n ot a panel m ember and does not s not plications i m on the safety i ensus ons , and his or her c sessment pate in the safety as partici sought. atter Ex bjec t M ed generically. s us pert” i eas of the SMS Manual, the term “Su Note: In other ar SR M panel m ember i s ex pected to have technical knowledge in a subject area that would Each suggest his or her participation in the panel m eeting is appropriate. SR M Panel Observer: eeting An individual present during the proceedings of the SRM pan el m who is not part of the SRM panel . pting to gain a better understanding of the someone attem Functional Descripti on: An observer i s hange being as AS c ocess, not the specific N SRM pr he is not an active member sessed. He or s eeting, does not provide input during the del iberations, and may not use of the SRM panel m anel obs eeting. The presence of p electronic recording devices during the panel m ervers is permitted at the discretion of the change proponent. 5.2_SMSM_201904  9 Originally published April 201 Uncontrolled copy when downloaded

75 Section 5 Preparing, Performing, and Documenting a Safety Analysis Skills and Backgrounds for SRM Panel Members 5.2.2.6.5 Examples of should select and involve SRM -facilitator /co and SRM panel facilitator The change proponent with varying levels of experience and knowledge to promote a comprehensive panel members They and balanced consideration of the safety issue. should obtain information on the knowledge, experiences, positions, and thoughts of each member. The following list, though not all -inclusive, provides types of experts to consider for participation on an SRM panel: or managing the existing • Employees directly responsible for developing the NAS c hange safety issue, Employees with current knowledge of and experience with the system or NAS change, • Hardware/software engineering and/or automation experts (to provide knowledge on • , equipment performance) • , Human factors specialists • Systems specialists , • System operators , • Employees skilled in collecting and analyzing hazard and error data and using , specialized tools and techniques (e.g., operations research, data, human factors) • Quality Control / Quality Assurance employees (to help ensure that the safety performance target is measurable and auditable or to help develop an alternate means , to verify predicted residual risk) • Air traffic procedures specialists , Information/cyber , • -security specialists • , Third-party stakeholders , Air traffic controllers • , • Maintenance technicians Traffic management specialists, and • . • Bargaining unit representatives The 5M Model, described in Section 3.3.3.2, is useful for identifying potential SRM panel embers. Note that i m t may be necessary to elevate a request for participation to an appropriate management level to ensure participation by al l affected stakeholders. 5.2_SMSM_201904 6 8 9 Originally published April 201 Uncontrolled copy when downloaded

76 Section 5 Preparing, Performing, and Documenting a Safety Analysis Performing a Safety Analysis 5.3 experts and stakeholders, the Safety Following the identification and invitation of subject matter ) panel is convened. During the SRM RM panel, the facilitator will lead Risk Management (S participants in objectively examining, identifying, and mitigating potential safety hazards and AS) change effects associated with the N . ational Airspace System (N or existing safety issue First Day of the SRM Panel 5.3.1 he SRM panel meeting, the facilitator or a designee must present an SRM On the first day of t panel orientation that includes: • on the agenda for the meeting; A briefing A summary of the goals and objectives for the SRM panel; • A brief review of the SRM process; • • SRM panel ground rules; if known) ; and The assessment method(s) by which the SRM panel will identify hazards ( • A draft of the “Current System” and “Description of Change” sections of the SRM • document 3.1 ). , if available, provided by the change proponent (see Section 5.4. 5.3.2 Administ ering the SRM Panel Meeting The SRM panel facilitator may perform or delegate the function of time keeper in order to manage start times and breaks. The facilitator may also delegate the recording of meeting notes, document , and the provision of audio/visual support. In some the writing of the SRM cases, a co -facilitator may assist . A co -facilitator is especially helpful when the panel size exceeds 12 members and/or the subject matter is complex. -person meetings , if possible; however, The SRM panel should be conducted using in stakeholders can participate in SRM panel meetings via other methods, such as web meetings or teleconferences. In the event that the invited stakeholders cannot participate in an SRM panel, consult with the change proponent and, if feasible, continue the safety assessment as scheduled. The findings should then be forwarded to the absent stakeholders to gather additional input, comments, or concerns. 5.3.3 Factors that Jeopardize Safety Assessment Results Failure to adequately describe the system and scope of the safety analysis can negatively affect the fidelity of the risk analysis and potentially hinder the implementation of a NAS change. Change proponents, facilitators, and SRM panel members should adhere to the following guidelines to help ensure that SRM panel deliberations support the goals of the change proposal: • Sufficiently define the scope. • Involve relevant stakeholders. • Identify drivers and constraints. • De fine product boundaries and external interfaces. • Baseline the scope before writing requirements. 5.3_SMSM_2016 07  Originally published July 2016 Uncontrolled copy when downloaded

77 Section 5 Preparing, Performing, and Documenting a Safety Analysis SRM Panel Deliberations 5.3.4 SRM panels should strive to reach consensus, but there may be instances in which not all SRM panel members agree on the results of the safety analysis. In those cases, document the results of the analysis, record the opinions of the dissenters, and deliver the results to the decision- maker. Safety and Technical Training encourages dissenting SRM panel members to provide their own rationale and data for why their severity and/or likelihood determination differs from that of the other SRM panel participants . The SRM panel facilitator must mediate and assist SRM panel members in working through differences of opinion. The facilitator should be able to recognize, acknowledge, and use differences of opinion to help the SRM panel consider different points of view. 5.3_SMSM_2016 07  Originally published July 2016 Uncontrolled copy when downloaded

78 Section 5 Preparing, Performing, and Documenting a Safety Analysis Safety Risk Management Documentation 5.4 Hazard Analysis Worksheet 5.4.1 U Safety Risk Management (SRM ) azard Analysis Worksheet (HAW) to organize the se the H panel’s deliberations into 16 key categories. The HAW provides a snapshot of the SRM panel 1 hazard identified. conclusions and will be included in the SRM document for each HAW Table 5.1: 2. 1. 4. 3. Hazard ID Cause System State Hazard Description An expression of The origin of a hazard Alpha-numeric the various Any real or potential condition identifier that can cause injury, illness, conditions, characterized by or death to people; damage to quantities or qualities, in (under 10 or loss of a system, which a system can exist characters) equipment, or property; or damage to the environment Controls: 5. 6. Controls Control Justification Any means currently reducing a hazard’s A justification for each control, indicating its effect on the causes or effects identified hazard's causes or effects Initial Risk: 10. 7. 11. 12. 8. 9. Lik Severity elihood Effect Initial Risk Likelihood Severity Rationale Rationale The estimated Explanation of The real or The composite of Explanation of The probability or credible how severity how likelihood consequences the severity and likelihood of a or impact of a frequency, in harmful was determined was determined hazard’s effect hazard, quantitative or outcome that considering only or outcome in has occurred qualitative controls and terms of degree terms, o f a or can be hazard’s effect expected if the documented of loss or harm hazard occurs assumptions for a or outcome in a defined given system state system state Safety Requirements: 13b . 14a. 13a . 14b. n Organizatio Responsible for Planned for Safety Requirement Point of Contact Implementation? Description (POC) Implementing Safety Requirement A planned or proposed means to Denotes whether the POC’s name and The organization’s name / reduce a hazard’s causes or safety requirement is telephone number routing code planned for effects implementation (Yes/No) 1. All of the SRM documentation detailed in the Safety Risk Management Guidance for System Acquisitions parative and the Com (OSA) (with the exception of the Operational Safety Assessment (SRMGSA) (CSA)) require the use of a HAW, as they follow the basic methodology of a Preliminary Hazard Safety Assessment in the SRMGSA; refer to the SRMGSA Analysis. Worksheets specific to the OSA and CSA are contained to determine when a HAW is required. 1 7 5.4_SMSM_201904 Originally published April 2019 Uncontrolled copy when downloaded

79 Section 5 Preparing, Performing, and Documenting a Safety Analysis Predicted Residual Risk: 15a. 15b . Predicted Pre dicted Residual Risk Rationale Residual Risk If necessary, any additional explanation needed to help the reader understand how the The risk that is predicted residual risk was determined estimated to exist after the safety requirements are implemented or after all avenues of risk mitigation have been explored Safety Performance Target: 16. Performance Target Safety The measureable goals that will be used to verify the predicted residual risk of a hazard 5.4.2 Monitoring Plan Use a monitoring plan table to organize the SRM panel’s plan for monitoring the safety hazard identified. performance target and verifying the predicted residual risk for each Table 5.2: Monitoring Plan Safety Performance Provide a safety performance target (as documented in the HAW ) that can be 1. used to verify the predicted residual risk for the hazard. Target associated with this safety from the HAW that is ID(s) Provide the hazard Hazard ID(s) 2. performance target . from the HAW. Include the initial risk Initial Risk 3. hazard’s causes or means that will be implemented to reduce the the Include Requirements Safety 4. effects f rom the HAW . Organization Responsible for Include information on the responsible organization / POC documented in the 5. HAW. Implementing Safety Requirements Predicted Residual Include the p from the HAW. residual risk redicted 6. Risk Enter the name and contact information of the person who will be responsible for 7. Monitoring POC(s) conducting the monitoring of this target. led by the monitoring POC to collect and analyze Describe the tasks that will be Monitoring Activities 8. data to verify the predicted residual risk . Monitoring Start activities should begin. the monitoring Enter the date when 9. Date Specify how often the monitoring activities will be reported. Reporting Frequency 10. Specify the total length of time for the monitoring effort. 11. Reporting Duration 5.4_SMSM_201904 7 2 Originally published April 2019 Uncontrolled copy when downloaded

80 Section 5 Preparing, Performing, and Documenting a Safety Analysis SRM Documents 5.4.3 ocument is used to record the SRM panel determinations for National Airspace An SRM d existing safety issues. The SRM d ocument presents evidence System (NAS) changes and accepted by supporting whether the NAS change and/or risk management strategies should be TO Air Traffic Organization (A Federal Aviation Administration (FAA) management officials ) or There are two types of SRM documents: safety findings with from a safety risk perspective. 2 hazards hazards and safety findings without . 5.4.3.1 Safety Finding With Hazards When an determ S panel RM a NAS change or existing safety issue could introduce ines that hazards or increase safety risk , the panel must complete each phase of the DIAAT process. Typically , this results in new means to reduce risk (i.e., safety requirements ) being devised and monitored proposed for implementation. Safety risk and overall safety performance must be after implementation of the NAS change and/or safety requirements to address the identified 3 This information should be contained in an SRM document with hazards. hazards. Finding Without H 5.4.3.2 Safety azards that no SRM panel determination ument without hazards is used to record an SRM doc An increase with the implementation of the hazards will be introduced or that safety risk will not he SRM document should include a description of the system being assessed. T change NAS and change and a rationale explaining why the change does not introduce hazards or NAS increase safety risk. 5.4.4 Writing the SRM D ocu ment The change proponent, the SRM panel facilitator, or a designated individual should begin drafting the SRM document immediately after the SRM panel meetings. The draft SRM should be presented to the SRM panel document to verify that the SRM panel members’ discussions have been correctly recorded and concurrence has been achieved. In the event SRM panel member does not concur with a determination made during the risk analysis that an or risk assessment phases of the process , he or she may submit a dissent in writing. Such dissents are included in the SRM document for evaluation by the risk acceptance official. The change proponent, SRM panel facilitator, or designated individual must enter the SRM document into the Safety Management Tracking System ( SMTS ), as per Section 5.4.5. ts the applicable sections and criteria for SRM documents : The following reflec list • Executive Summary • SRM Document Signatures • Executive Summary • Current System • Description of Change / Existing Safety Issue • Rationale for a Safety Finding Without Hazards (if no hazard are identified) The purpose of SRM is not to record all modifications to elements of the NAS but rather to assess the 2. risk potentially caused by proposed changes to or existing safety issues in the NAS. SRM documentation should strictly consider and document safety concerns and safety findings. Certain modifications m ay not necessarily be considered NAS changes under the purview of this Safety Management System (SMS) Manual. The change prop onent must consider potential safety ramifications when making any m odification to the NAS (see Section 3.2.1). Modifications t and Technical hat do not relate to safety will not require SRM and do not need to be documented. Contact a Safety assistance, if necessary. Training safety case lead for 3. issues, the approach for safety findings with hazards is the most appropriate. When addressing existing safety 5.4_SMSM_201904 7 3 Originally published April 2019 Uncontrolled copy when downloaded

81 Section 5 Preparing, Performing, and Documenting a Safety Analysis • Dissention ( when applicable) • Hazard and Risk Analysis (i f hazards are identified) • Monitoring Plan (if hazards are i dentified) • when applicable) Dissention ( • SRM Panel Attendees • Appendices See Section 5.4.4.1 thro ugh Section 5.4.4. 10 for more information about the content of each SRM document s ection. For additional guidance writing either type of document, consider using nagement Sy stem ument templates available in SMTS and on the ATO Safety Ma the SRM doc oolbox. (SMS) T 5.4.4.1 Executive Summary for Use the Executive Summary to provide only the substantive information necessary akers to understand the current system, NAS change / decision-m existing safety issue, and, if applicable, t he associated safety r isk and proposed ways to address the hazards and safety risk . Provide detailed information and supporting nar rative on these items i n the body of the SRM doc ument. : Include the following administrative information regarding the s afety anal ysis which the document’s subject clear, concise name of the document with Include a Title. • can be easily understood. Provide the organization that is initiating the NAS change or Initiating Organization. • addressing the existing safety issue. Include the that has taken responsibility for ’s name and FAA routing code. organization • Safety Analysis Type. Indicate the type of safety analysis by choosing one of the following: o Operations and Second-L evel Engineering : o For acquisition cases only Hazard Preliminary Analysis o o System Hazard Analysis o Sub-System Hazard Analysis o Operating and Support Hazard Assessment o System Safety Assessment Report o OSA CSA o with SRM documents hazards, use the tables below to summarize the hazards identified For and proposed means of mitigation/monitoring: Summary : Hazard Table 5.3 Predicted Initial Risk Hazard ID Hazard Description Residual Risk 5.4_SMSM_201904 7 4 Originally published April 2019 Uncontrolled copy when downloaded

82 Section 5 Preparing, Performing, and Documenting a Safety Analysis 5.4: Table Safety Requirements Associated Organization POC Safety Requirement Responsible Hazard ID(s) Signature Table 5.5 : Monitoring Plan Summary Associated Safety Performance Target Hazard ID above If no hazard tables. Instead, provide a brief rationale s are identified, do not include the for a safety finding without hazards . 5.4.4.2 SRM Document Signatures For each the SRM document signature page. required on the signatures List ed below are signatory, include the printed name, signature ( handwritten or electronic), organization, and date. Signatures should be obtained, and must be listed, in th e following order: concurrence (where appropriate), approver, risk accepto r, and ATO Chief Safety Engineer (when necessary). 1. Concurrence. This sig nature is used to represent a technical review of the safety that the rationale used throughout is consistent with the overall analysis and to confirm comes from The concurrence signature risk assessment. an SRM expert who is well versed in the SMS Manual and familiar with the terminology and processes therein. Refer to Section 6.5. 2. Approval. Include an approval signature from an official representing the organization responsible for implementing the NAS change (and from the ATO Chief Safety Engineer , An approver provides a if required). technical and administrative quality control review of the safety analysis, its findings, and the identified results. Refer to Section 6.6. 3. Risk Acceptance. Include a risk acceptance signature from an appropriate official representing the organization that will be using the safety-assessed NAS equipment, policy, or procedure. This signature indicates acknowledgment of the identified safety risk(s) and denotes its acceptance into the NAS upon implementation of the NAS change. Refer to Section 6.4. signatures from the responsible organization(s) and associated POC (s) The safety requirements are contained within the Executive Summary. Current System 5.4.4.3 Provi cri de a detailed des ption of the hardware/software system, operation, or procedure that constitutes the NAS change or the environment in which the existing safety issue has Refer to Section 3.3 for information on manifested. Phase 1 of the DIAAT process, “Describe System.” Include the following information when applicable: ed the need for a NAS change or the evaluation of an • A brief background on what trigger issue existing safety . If there is an associated SRM document, compliance issue, or it here and include the that necessitated this NAS change, briefly summarize 5 issue Top . associated reference or documentation as attachments • The current hardware or software system or existing procedures/operations and the corresponding (operational) system states. 5.4_SMSM_201904 7 5 Originally published April 2019 Uncontrolled copy when downloaded

83 Section 5 Preparing, Performing, and Documenting a Safety Analysis The current procedure and its operational environment and, when applicable, a • discussion about elements of this issue that make it particularly unique or challenging. • Equipm ent or procedures needed to accommodate the implementation of the NAS change. • Future configuration, system, or procedural changes that might affect the proposed change/procedure or existing safety issue. 5.4.4.4 Description of Change / Existing S afety Issue Provide a description of the proposed NAS change or the existing safety issue being addressed. 3.3 Refer to Section . Include the following process for information on Phase 1 of the DIAAT when applicable: information, safety • A description of the pro posed NAS change/procedure or existing issue and any critical safety parameters that are involved (e.g., prohibited/restricted airspace, noise abatement area, operational limitation) . When applicable, discuss the types of verifications that will be performed throughout the • development process to review whether the finalized proposed NAS change will be safe, operational, and effective once implemented. Evaluation can consist of simulator modeling, live testing, or a combination thereof. (if visual • A depiction of the proposed NAS change/procedure or existi ng safety issue illustration is beneficial ). issue more Assumptions that make evaluating the NAS change or existing safety • manageable or that better scope the analysis. • A summary of the relevant results of any related or preceding safety analyses ( i.e., an Include any references and/or associated acquisition program or operational change). Section 5.4.4. 10. documentation mentioned in The traceability between the proposed change and the NAS Enterprise Architecture. • Hazards (If No Hazards Are Identified) 5.4.4.5 ithout Finding W Rationale for a Safety There may be cases in which, through performing elements of the SRM process (i.e., describing the system /change and identifying hazards), hazards associated with the implementation of the not identified, or i t is determined that the NAS change does not increase the NAS change are current risk level. In such cases, include a detailed rationale that explains how the SRM panel or team came to that con clusion. When the provisions of this section apply, the SRM document is nearly complete. Follow the guidance for Dissentions (as applicable), SRM Attendees, and 6 for information on Appendices and prepare the SRM document for signatures ( see Section signatures). isk A re Identified) (If Hazards A 5.4.4.6 H azard and R nalysis a detailed explanation of each hazard identified. Provide the completed HAW for each Provide and the information hazard (see Section 5.4.1 for instructions on creating a HAW) necessary to support the risk overview in the Executive Summary. 5.4.4.7 Monitoring Plan (If Hazards Are Identified) The SR a monitoring plan for each hazard or for the existing identified M panel must complete for instructions on creating a monitoring plan) . being safety issue assessed (see Section 5.4.2 7 5.4_SMSM_201904 6 Originally published April 2019 Uncontrolled copy when downloaded

84 Section 5 Preparing, Performing, and Documenting a Safety Analysis Provide the compl eted monitoring plan table for eac h hazard and the information necessary to support the monitori ng overview in t he Executiv e Summary. 5.4.4.8 Dissention ith the official fi sagrees w ember di ndings of the SRM panel, the nature and If any SR M panel m summary of the complaint must be documented in this part of the SRM do cument. 5.4.4.9 SRM Panel Attendees l attendees name and relevant information including his or Include a table with each SRM pane her pos he participated as part hether he or s ition, fac ility, and FAA r outing code. Make clear w of the facilitation team or as a change proponent, SRM panel member, subject matter expert, or S RM panel observer. Refer to Section 5.2.2.6 for more information. 5.4.4.10 Appendices Use appendices to include the following: ; Supporting documentation such as simulations, modeling, and other technical analyses • Relevant references ; and • Acronyms, terms • , and definitions . MTS 5.4.5 S 4 The change proponent is the official repository f or all completed ATO SRM documents . SMTS or organization accepting safety risk must enter the SRM document before the into SMTS implementation of the NAS change, or the initiation of monitoring activities, the completed achievement of the FAA Acquisition Management System decision point. See the SRMGSA for a more detailed description of mandatory entry requirements for acquisition programs. ion Dates in SMTS 5.4.5.1 Implementat Once the SRM document has been completed and all required signatures have been obtained, the change proponent is responsible for providing a monitoring start date (i.e., the date after all safety requirem are implemented) ents . This date must be entered into SMTS to trigger the automated email notification process for the monitoring plan. 4. A complete SRM document includes all required signatures (both ink and digital signatures are accepted). 5.4_SMSM_201904 7 7 Originally published April 2019 Uncontrolled copy when downloaded

85 Section 5 Preparing, Performing, and Documenting a Safety Analysis Special SRM Efforts/Considerations 5.5 Some Safety Risk Management (SRM) efforts may be in response to atypical National Airspace AS) System (N changes. Other efforts need to address safety issues or support decisions expediently to circumvent existing policies and processes. While the requirement to perform TO ) acknowledges the need to truncate, deviate SRM still applies, the A ir Traffic Organization (A from, or add to the safety analysis process in some cases. In other cases, the requirement for SRM must be reiterated. Use this section for information related to the specific SRM documentation/process requirements and considerations for the following: • Deactivation, removal, or decommissioning of NAS equipment • Emergency modifications Existing high -risk hazards • Waivers • Deactivation, Removal, or Decommissioning of NAS Equipment 5.5.1 Over time, NAS equipment, procedures, systems, and services must be removed due to limited parts, obsolete services, funding constraints, facility relocation, or a function no longer being needed. If NAS equipment, procedures, systems, or services are removed, discontinued, deactivated, or decommissioned from the NAS, then a safety risk assessment must be performed in accordance with this Safety Management System (S MS ) Manual. The safety risk assessment must be completed before the equipment, procedures, systems, or services are removed from the NAS. The results of the safety risk assessment must be uploaded to the Safety Management Tracking System (SMTS) provided by Safety and Technical Training (AJI) (see Section 5.4.4 ), and any identified means to reduce risk and safety performance targets must be monitored by the equipment, procedure, system, or service owner. Emergency Modifications 5.5.2 When an emergency modification is necessary, a memorandum must be sent to the ATO Chief Safety Engineer within two days of the implementation of the modification. The memorandum must: • State what system was modified, • Provide a summary of the emergency modification, • Identify why the modification was made, and • Indicate when the safety risk assessment will be conducted. The official who authorized the emergency modification must ensure that a safety risk assessment is performed in accordance with this SMS Manual within 30 days of the implementation of the modification. After the safety assessment is completed, a follow -up memorandum must be sent to the ATO Chief Safety Engineer stating that the safety assessment has been completed and uploaded to SMTS . The ATO Chief Safety Engin eer must inform the Air Traffic Safety Oversight Service (AOV) hief Operating Officer and the ATO C ). (COO Existing High -Risk Hazards 5.5.3 When an existing hazard is determined to be a high -risk hazard, the ATO Chief Safety Engineer must notify the ATO COO and AOV of the high risk and any interim actions to mitigate the risk. The ATO COO must either approve the interim action and accept the associated risk or require that the operation be stopped. 5.5_SMSM_2016 78 07 Originally published July 2016 Uncontrolled copy when downloaded

86 Section 5 Preparing, Performing, and Documenting a Safety Analysis Thirty days after the notification is sent to the COO and AOV, the responsible Service Unit must coordinate with the ATO Chief Safety Engineer to develop a permanent plan that will eliminate the hazard or reduce the risk to an acceptable level and provide that plan to AJI. The plan must include: • A description of the hazard and system state, • The severity and likelihood of the high risk, • Data or empirical evidence that justifies the determination that a high- risk hazard exists, or a decision to cease • Safety requirements the operation, A schedule to complete an SRM d ocument in accordance with this SMS Manual, and • • An approval signature by the Vice President of each responsible/affected Service Unit. Cessation is viable if the prescribed means are inadequate to reduce the risk to an acceptable level. In some cases, though, cessation of the operation may not be the safest means to mitigate the risk. There could be unintended consequences that result in more potential harm or increase system safety risk. The Service Unit must forward the plan with a memorandum via its Vice President to the Vice President of AJI for approval and copy the ATO Chief Safety Engineer, who will then forward the memorandum to AOV. AJI will notify AOV of any subsequent changes to the approved plan. The hazard must be documented in an SRM d ocument that is written in accordance with this SMS Manual and uploaded to SMTS within 30 calendar days of the implementation of the final equirements . The responsible Service Unit must adhere to the SRM documentation safety r approval and risk acceptance requirements documented in this SMS Manual. Refer to Section 6.3 for information on the review and approval of the SRM d ocument . 5.5.4 Document ation, Review, and Approval Process for Waivers to Separation Minima A waiver to separation minima can result in aircraft being allowed closer than approved separation from terrain, obstacles on the surface of the earth, airspace, or other aircraft. The ATO -SG) on separation minima lists the requirements in Federal current Safety Guidance (ATO Aviation Administration (FAA) Order JO 7110.65, Air Traffic Control , that pertain to separation The ATO -S G also details which NAS changes related to separation minima minima. rements need approval from AOV. requi Any new waiver request or waiver renewal request that pertains to separation minima requires a new SRM d ocument d ocument on file that is developed in accordance with this SMS or an SRM Manual. The safety analysis should include a quantitative analysis (e.g., scientific study, Flight Standards Service report, detailed modeling, or Monte Carlo simulation) to support the information documented in the SRM d ocument . 5.5.4.1 Ini tiate the Request for a New Waiver or Waiver Renewal Waivers must be kept to a minimum, as they contribute to a nonstandard NAS and make all future changes more difficult to assess. Therefore, before developing or renewing a waiver, coordinate with the appropriate Service Area and the Service Unit to obtain their commitment to the effort . The Service Unit will initiate coordination with AJI to determine what level of safety risk analysis is warranted to support the request and SRM d ocument . 5.5_SMSM_2016 07 79 Originally published July 2016 Uncontrolled copy when downloaded

87 Section 5 Preparing, Performing, and Documenting a Safety Analysis Waiver Development Guidance: Identify Appropriate Hazards 5.5.4.2 Attempt to identify Most paragraphs in FAA and ATO orders mitigate a potential safety hazard. the hazard that the relevant order intends to mitigate to determine the appropriate hazard(s) to address in the safety analysis. If the waiver request is intended to reduce safety risk , make the document case in the in the SRM and show the waived procedures as a means to reduce risk Analysis Worksheet (HAW) . Hazard Relationship between the Waiver Request and the SRM D ocument 5.5.4.3 When an analysis is done correctly, all of the waiver requirements should be covered in the d ocument : SRM • The “Affected Directive” and “Operations Authorized” sections of the waiver should match the “ Description of Change” section of the SRM . document • The “Special Provisions, Conditions, and Limitations” section of the waiver should flow out of the HAW section of the document , specifically from the control s and/or the SRM safety requirements. • Remember to include any new safety requirements in the SRM d ocument . 5.5.4.3 .1 Waiver Renewals Waivers must be renewed every two years. When submitting a waiver renewal request, read the current SRM to determine whether any updates are necessary. Remember, an document document SRM must be updated to reflect the current operational environment. All required means to reduce risk , including the publication of information and any refresher training requirements, as delineated in the original SRM document , must be in place. For each waiver renewal request: Determine whether the level of safety risk that was introduced with the initial waiver • remains acceptable, • Use the monitoring plan developed in accordance with the information in Section 3 to allow the responsible organization to determine whether the waiver is working as intended, and • Determine whether the provisions of the waiver have matured sufficiently that they should be made available to all others in the NAS through inclusion in FAA Order JO 7110.65. Before submitting a er renewal request, ensure the monitoring information pertaining to the waiv . All proposed modifications to any provision of the current existing waiver is up to date in SMTS waiver will require a new waiver to be developed with a new SRM (refer to Section d ocument 5.5.4 ). 5.5.4.3.2 Waiver Approval All new waivers and waiver renewal requests will be approved by AJI. AJI will coordinate the approved waiver with AOV, if necessary. Ensure that new waivers and information pertaining to waiver renewals are entered in SMTS . 5.5_SMSM_2016 07  Originally published July 2016 Uncontrolled copy when downloaded

88 Section 6 Risk Acceptance and Safety Documentation Review 6.1 Risk Acceptance and Approval and Overview Ri sk M anagement (SRM ) documents and acceptance of any The review and approval of Safety rganization (ATO c O ) risk safety r isk i s des igned to m aintain and assure the qual ity of Air Tr affi management activities. There are key v ariables that affect safety risk ac c eptance and SRM documentation review and signature r equirements. They i nclude the organization(s) affected by the proposed National Airspace System (NAS ) change, the organization that developed the ssociated with the NAS c hange, and whether the NAS c hange is document, the risk(s) a c onsidered national or l ocal in scope. There are several s ignature authorities as sociated with ceptance, and safety requirements SRM doc umentation: concurrence, approval, risk ac on. Refer to Se ly scoped ly and local nformation regarding national implementati ction 6.2 for i changes. guidance on specific signature types, refer to Sec For tion s 6.3 through 6.6. Tables 6.1 through 6.4 summarize the SRM document signature requirements. The terms “affected facilities” and rganizations e Units ” refer to the facilities or o that are impacted by the safety risk “affec ted Servic ed with the NAS change or existing safety issue. associat h) predicted Table 6.1 is not to be used for safety analyses with an unacceptable (hig Note: residual risk (see Table 6.4). RM D ignatures for S Table 6.1: S ocument Approval and Risk Acceptance (Use with Section 5.4.3.1, S afety Fi nding With Hazards) (1) (2) (13) (14) Initial Required Safety Risk Required SRM Requires AOV Type of Acceptance Document Approval Predicted Approval/Acceptance? Change Risk Level Signatures Signatures (4) (3) Support Managers or ATMs or Technical Low/Medium System Support Center Operations Managers No (5) Managers of the affected of the affected facilities ( 6) facilities (6) Support Managers or Technical ATMs or S ystem Support Low/Medium Operations Managers ATO Center Managers, (5) of the affected Local Chief Safety Engineer facilities (8 ) (9) Yes Headquarters (s) or Technical Director Operations Service Vice President of the High (7) Area Director, ATO Service Unit affected Chief Safety Engineer (8) 81 6.1_SMSM_201904 201 Originally published April 9 Uncontrolled copy when downloaded

89 Section 6 Risk Acceptance and Safety Documentation Review Required Safety Risk Required SRM Initial Requires AOV Type of Document Approval Approval/Acceptance? Acceptance Predicted Change Signatures Risk Level (3) Signatures (4) Headquarters Group Headquarters Low/Medium Manager of the change Director(s) of the (5) ATO Chief proponent, affected Service Safety Engineer (8) Unit(s) National Yes/No Headquarters Vice President(s) of Director(s) of the (7) High the affected Service , affected Service Unit(s) ATO Chief Safety Unit(s) (8) Engineer Director of Operational Concepts, Validation & ; Director Requirements Headquarters Low/Medium of P rogram Director(s) of the (5) Management affected Service Organization; ATO 2) Unit(s) (1 Chief Safety Engineer 1) (10) (1 Yes/No Acquisitions Operational Director of Concepts, Validation & Requirements ; Director Vice President(s) of of P rogram the affected Service (7) High Management 2) Unit(s) (1 Organization; ATO Chief Safety Engineer (10) (1 1) Notes: (1) The change proponent must ensure that the SRM documents are entered into the ATO Safety Management Tracking System (SMTS) for tracking and monitoring the status of NAS changes. (2) Signature responsibility may only be delegated from a Director to a Deputy Director. The changes that require Air Traffic Safety Oversight Service (AOV) approval are listed in (3) . If there is -risk an initially identified h FAA Order 1100.161, Air Traffic Safety Oversight igh eans to reduce safety risk and the Headquarters Director hazard, AOV must approve the m of Operations or Technical Operations Service Area Director and the ATO Chief Safety Engineer (8) must sign the document. The proponent of an air traffic chang (4) e must send an informational copy of the SRM document to the Director of Air Traffic Operations (Service Area) before submitting the SRM document to the ATO Chief Safety Engineer for approval. (5) In cases where medium or low safety risk and/or controls go outside of the ATO, the mitigations must be approved by the designated management officials within the other Business of (LOBs) Lines and accepted by AOV. General Manager or If a facility does not have a Support Manager, the District (6) Manager of facility shall designate an SRM document approver. the affected 6.1_SMSM_201904 8 2 Originally published April 201 9 Uncontrolled copy when downloaded

90 Section 6 Risk Acceptance and Safety Documentation Review risk ns to reduce safety t submit safety cases with mea The ATO Chief Safety Engineer mus (7) sk safety hazards to AOV for approval. of any initially identified high-ri afety issue meets the criteria for AOV approval, the ATO Chief If the change or existing s (8) Safety AOV accordingly. Engineer must submit it to SRM documents that accompany air traffic waiver requests must also be signed by the (9) Headquarters Director of Operations. Safety (10) t undergo a peer review documentation developed for acquisition programs mus before signature, as described in the Safety Ris k Management Gui dance for Sys tem the SRMGSA for more information. uisitions (SRMGSA). Refer to Section 8 of Acq (11) The Director of ATO Operational Concepts, Validation & Requirements or his/her designee must provide their approval when the safety requirements are not already documented in an approved Program Requirements Document (PRD). (12) Risk acc eptance must be obtained for safety analyses in which risk is identified, except for the Operational Safety Assessment and the Comparative Safety Assessment. For approval and/or risk acceptanc (13) ide of the ATO, AOV may facilitate signatures on e outs behalf of the ATO. However, the Service Unit change proponent should obtain signatures from the affected organization (user) participating on the SRM panel. (14) . vel Engineering should start with Table 6.2 for their signature requirements Second-Le 6.1_SMSM_201904 8 3 Originally published April 201 9 Uncontrolled copy when downloaded

91 Section 6 Risk Acceptance and Safety Documentation Review Table 6.2 : Signatures for Second-Level Engineering SRM Document Approval and Risk (3) Acceptance (1) (2) Does a Previous Facilitated by Proposed Safety Required AJI or Modification to Required SRM Assessment Requires AOV Hazard Safety Risk Document Approved Document the Identified? Approval or System -Level Acceptance Approval Safety Signatures Acceptance? Signatures Requirements? Implications of (4) ) (7) (5) (6 the Proposed Modification? Yes No additional assessment required Headquarters Headquarters Director(s) of Group Manager Yes affected the of the change Service proponent Unit(s) No Headquarters Group Manager No None of the change No proponent No See signature requirements in Yes Table 6.1 Headquarters Group Manager Yes of the change No None , ATO proponent Chief Safety Engineer See signature requirements in Yes Table 6.1 Headquarters Yes Group Manager of the change No None , ATO proponent Chief Safety Engineer Yes/No Yes See signature requirements in Yes Table 6.1 Headquarters No Group Manager of the change No None , ATO proponent Chief Safety Engineer 8 4 6.1_SMSM_201904 Originally published April 201 9 Uncontrolled copy when downloaded

92 Section 6 Risk Acceptance and Safety Documentation Review Notes : -level NAS changes only. For local changes, refer to (1) Thi e applies to national s tabl - and local -level NAS changes. Table 6.1. Refer to Section 6.2 for a discussion on national The change proponent must ensure that the SRM s are entered into SMTS for document (2) tracking and monitoring the status of NAS changes. (3) Signature responsibility may only be delegated from a Director to a Deputy Director. . System Level Requirements refer to the requirements listed in the Final PRD (4) (5) The changes that require AOV approval are listed in FAA Order 1100.161. the ATO, the (6) In cases where medium or low safety risk and/or controls go outside of means to reduce safety risk m ust be approved by the designated management officials within the other LOBs and accepted by AOV. eans to reduce safety risk (7) any of The ATO Chief Safety Engineer must submit the m initially identified high-risk safety hazards to AOV for approval. : Signatures for SRM Document Approval Table 6.3 .2, Safety Finding Without Hazards) (1) (2) (4) (Use with Section 5.4.3 SRM Document Required Type of Change Approval Signatures Director of Air Traffic Operations (Service Area), Manager or Terminal District Local (3) General Manager, or Technical Operations District Manager or General Manager Headquarters Director(s) of Service Unit(s) affected National , ATO Chief Safety Engineer Director(s) of Headquarters affected Service Unit(s) , Acquisitions ATO Chief Safety Engineer Notes: (1) The change proponent must ensure that the SRM d ocument is entered into SM TS for tracking and monitoring the status of NAS changes. (2) Signature responsibility may only be delegated from a Director to a Deputy Director. ir T (3) For local changes, the SRM document is signed one level above the A raffic Manager (ATM) oute at the facility completing the SRM document. For Air R Traffic Center ATMs, this is the Service Area Director of Operations; for (ARTCC) Control Manager; and for Technical this is the District Manager or General Terminal ATMs, 6.1_SMSM_201904 8 5 Originally published April 201 9 Uncontrolled copy when downloaded

93 Section 6 Risk Acceptance and Safety Documentation Review Operations Managers, this is the Technical Operations District Manager or General Manager. (4) This table does not apply to Second-L evel Engineering. NAS Changes Only Proposed for Table 6.4: Signatures for SRM Document Approval unacceptable redicted (2) (3) (Use with s afety analyses with risk) (1) [high] p residual SRM Document Required Type of Change Approval Signatures ATMs or Technical Operations Local ( affected Managers of the 4) facilities Headquarters Director(s) of the , affected Service Unit(s) National ATO Chief Safety Engineer Notes: When the pr ted res (1) idual risk is unacceptable (high), AOV approval is not required. edic policy, a high predicted residual risk is unacceptable and System Management (2) Per Safety is the NAS change in question must not be implemented. The SMTS submitter closing out the safety analysis. (See responsible for notating this in SMTS and the SMTS User Manual .) responsibility may only be delegated from a Director to a Deputy Director. Signature (3) SRM document is signed one level above the ATM at the facility (4) For local changes, the is the Service Area Director of completing the SRM document. For ARTCC ATMs, this for Terminal ATMs, this is the District Manager; and for General Operations; Manager or Technical Operations Managers, this is the Technical Operations District Manager or General Manager. 6.1_SMSM_201904 8 6 Originally published April 201 9 Uncontrolled copy when downloaded

94 Section 6 Risk Acceptance and Safety Documentation Review Scope of NAS Changes 6.2 changes are considered either local or national. A national AS) National Airspace System (N is one fo r which a Safety and Technical Training (AJI) safety case lead facilitates NAS change RM ) effort or that meets at least one of the following or leads the S afety Risk Management (S criteria: The NAS change has high visibility or a potential political, economic, or financial impact • 1 to the Federal Aviation Administration (FAA) , the NAS, or the flying public. • S change is the result of financial or operational decisions made by FAA The NA executive management, Cabinet -level executives, or Congress. The NAS change includes means to reduce • safety risk identified as part of the Top 5 any Program. The NAS change modifies safety policy that must be incorporated into a directive. • • The NAS change could or does present operational or technical conflicts to multiple affected Service Units or FAA Lines of Business . • The NAS change will be implemented on a national level, affecting multiple facilities. Note: There may be cases in which an AJI safety case lead facilitates a local SRM panel and none of the aforementioned criteria apply. These changes will be considered local. A NAS change is considered to be local if : It does not meet any of the preceding criteria and it affects three or fewer Service • Delivery or Points within a single Service Area • It is a change proposed by Technical Operations that involves a single piece of equipment that is restricted to one district. In cases where a NAS change affects two adjacent Service Delivery Points in different Service Air Route Traffic Control Center with e Areas or a single Terminal Radar Approach Control / mor than two underlying Airport Traffic Control Towers, the change proponent has the authority to determine if the change will be considered local or national in scope. Note: Many systems and facilities that provide service in the NAS are not procured, owned, or federal entity. The FAA has the authority and responsibility maintained by the FAA or another to assur safety of these services in accordance with Title 49 of the United States Code § e the Systems, procedures, facilities, and devices , and Title 14 of the Code of Federal 44505, Regulations Part 171, Non -Federal Navigation Facilities . Although a system/service may not be procured by the FAA, implementation into the NAS is considered a NAS change and requires safety assessment, approval, and risk acceptance as if the FAA were acquiring the appropriate system/service. 6.2.1 Local Implementation of National NAS Changes RM document cannot follow the national When the local implementation of a nationally scoped S standard, local SRM is required to assess and accept any risk for local deviations. If formal waivers are required in such cases, local SRM does not eliminate the waiver requirement. AJI will typically identify t hese types of changes. 1. 6.2_SMSM_2016 07 87 Originally published July 2016 Uncontrolled copy when downloaded

95 Section 6 Risk Acceptance and Safety Documentation Review 6.3 Approving Safety Requirements An organization’s safety requirement approval signature represents its commitment to implementing the safety requirement in accordance with the associated Safety Risk Management (SRM) document. For acquisition systems, if the approved Program contains the safety requirements referenced in the safety Requirements Document (PRD) ontact (POC) of C analysis, no Point If the requirements are not listed in signature is required. the approved PRD, the safety analysis must include a POC signature for each additional safety requirement. 6.3.1 Appropriate Signatories evel with the ability to fund ial l Safety requirement signature authority must be at the manager and ensure the implementation of the safety requirement. The appropriate signing official may be determined by the Federal Aviation Administration (FAA) organization. When multiple officials are responsible for providing safety requirements signatures in an SRM document, they must share similar managerial status or responsibility. When an organiz F AA is responsible for a safety requirement, a signature on ation outside of the file is required. This requirement may be met through a memorandum or an SRM document. The change proponent is responsible for following up on the status of the implementation of safety requirements identified in the SRM document. Endorsing Implementation of Safety Requirements 6.3.2 All safety requirements that the SRM panel identifies must be accounted for in the SRM Safety Management Tracking System (SMTS). document and the The change proponent and POC (s) must collaborate to determine which safety appropriate safety requirement(s) requirements will be implemented and notate that decision in SMTS and in the SRM document. The risk acceptor is accountable for ensuring that all safety requirements are implemented and all monitoring activities are recorded in SMTS. 6.3.2.1 Safety Requirements Not Planned for Implementation If a safety requirement is not going to be implemented: 1. The rationale for not implementing the safety requirement must be entered in the SRM document and recorded in SMTS. In addition, if any of the SRM p anel members dissent the removal of the safety requirement, the dissention must be recorded in the SRM with document. risk, predicted SRM panel must be contacted to verify that the 2. The residual safety the se must be plan have not been affected. If m onitoring performance target, and/or changed as a result a safety requirement not being planned for implementation, the SRM panel’s revised analysis must be documented, along with any dissenting opinions. 6.3.2.2 Safety Requirements Planned for Implementation Worksheet Analysis All safety requirements included in the Hazard of the signed SRM ational document System Airspace must be implemented before or in conjunction with the N change, even when the risk is classified as medium or low. All organizations responsible for implementing a safety requirement must: for the safety requirement approval, document Sign the SRM 1. Document the status of the safety requirement (e.g., implemented, not implemented, or 2. in progress), and 88 6.3_SMSM_201904 Originally published April 201 9 Uncontrolled copy when downloaded

96 Section 6 Risk Acceptance and Safety Documentation Review Record objective evidence supporting the safety requirement’s implementation. 3. Only safety requirements that are to be implemented must have an accompanying signature. Safety Recommendations 6.3.2.3 to SMTS in a doc Safety recommendations may be uploaded ument separate from the SRM They do not require any endorsement. document. 89 6.3_SMSM_201904 9 201 Originally published April Uncontrolled copy when downloaded

97 Section 6 Risk Acceptance and Safety Documentation Review Risk Acceptance 6.4 Risk acceptance is certification by the appropriate management official that he or she acknowledges and accepts the safety risk that is expected to remain once the National Airspace AS System (N ) change is fully implemented. Safety risk must be accepted before the implementation of a proposed NAS change and the execution of the monitoring plan. Risk acceptance is based on the predicted residual risk (see Section 3.7.4). Risk acceptance and other inputs (e.g., cost-benefit analysis) are necessary before a change to the NAS can be implemented. When an individual or organization accepts a risk, it does not mean that the risk is eliminated; some level of risk will remain. Risk acceptance requires: Signed confirmation from the appropriate management official that he or she • understands and accepts the predicted residual safety risk(s) associated with the ; hazard(s) identified in the safety analysis Signatures for the safety requirements identified in the Safety Risk Management (SRM) • document; • target(s) or alternate method(s) identified to verify Approval of the safety performance the predicted residual risk associated with each hazard, confirming that the safety performance target(s) or identified alternate method(s) can be used to measure the current risk; and • A comprehensive monitoring plan that the risk acceptor agrees to follow to verify the predicted residual risk. For nationally implemented NAS changes, risk can be accepted at the national level. However, if a facility is not able to comply with all of the safety requirements or has additional hazards and/or causes that were not identified in the national SRM d ocument, a local assessment must be completed (with local risk acceptance) prior to the implementation of the NAS change. Refer to for information on local versus national implementation of safety requirements. .1 6.2 Section Authority to Accept Safety Risk 6.4.1 The acceptance of the safety risk depends on the span of the program or NAS change and the associated risk. In most cases, the responsibility for risk acceptance ultimately lies with the organization(s) affected by the NAS change. Risk acceptance authority also depends on whether a NAS change is local or national in scope. igning the SRM document, the risk acceptor is confirming the following are understood and By s accepted: The safety analysis, including the identified safety risk(s); • • The predicte risk(s) associated with the hazard(s) identified in the safety d residual analysis; • The safety requirements that will be implemented; • The safety performance target(s) identified to measure the predicted residual risk the safety performance target(s) may associated with each hazard, thus confirming that be used to measure the current risk level; and • The information contained in the monitoring plan. 90 6.4_SMSM_201904 Originally published April 201 9 Uncontrolled copy when downloaded

98 Section 6 Risk Acceptance and Safety Documentation Review The risk acceptor is accountable for: Ensuring that all monitoring activities are being recorded in the S • Management afety System Tracking ; Ensuring that performance data needed for the monitoring activities is being collected • safety performance target(s) are being met; the and analyzed to verify that Determining the need to reconvene an SRM panel if performance data indicate that the • predicted residual risk is not the risk management strategy is proven to be met and/or if and inadequate; of the predicted residual risk, safety performance target(s), Coordinating a reassessment • and/or monitoring plan if a safety requirement identified by the SRM panel cannot be implemented. Organization raffic 6.4.2 Risk Acceptance Outside of the Air T ir T If the affected party is outside of the A raffi c Organization (ATO) (e.g., navigation or weather services), each organization responsible for establishing requirements for contracted services accepts the risk into the NAS. Lines of Business (LOBs )/organizations outside of the ATO Office of Airports, Office of NextGen, Office of Commercial Space Transportation, or (e.g., Office of Aviation Safety) are also responsible for components of the NAS and have a role in accepting safety risk. ATO vice presidents, directors, managers, and supervisors must work closely with their Bs/organizations outside of the ATO to help ensure that the appropriate party counterparts in LO or parties accept and manage any safety risk resulting from NAS changes. Again, it is not in compliance with ATO policy to implement a NAS change without having first accepted any associated safety risk. Refer to Federal Aviation Administration Order 8040.4, Safety Risk Ma -LOB risk acceptance. , for policy on cross nagement Policy 6.4_SMSM_201904 1 Originally published April 201 9 Uncontrolled copy when downloaded

99 Section 6 Risk Acceptance and Safety Documentation Review Document Concurrence 6.5 SRM is used to represent a technical review of the safety analysis Concurrence and to confirm that the rationale used throughout the S afety Risk Management (SRM) document is consistent with the overall risk assessment. The concurrence signature comes from a n S RM expert who is well versed in the S Management System Manual and familiar with the terminology and afety processes therein. The c oncurrence signature is not a required signature; however, Service Area s, District Offices, or individual facilities may require a c oncurrence signature on their respective SRM documents. 6.5_SMSM_2016 07 2 Originally published July 2016 Uncontrolled copy when downloaded

100 Section 6 Risk Acceptance and Safety Documentation Review 6.6 SRM Document Approval requires and ) document with hazards Approval of a Safety Risk Management ( SRM represents that: • The SRM d ocument was developed properly, Hazards were systematically identified, • • Risk was appropriately assessed, • If identified, safety requirements were deemed valid , • Safety performance targets or other methods to verify predicted residual risk were approved by the responsible Service Unit, and • A monitoring plan was prepared. Approval of an SRM document without hazards requires and represents that: ocument • The SRM d was developed properly, AS ) change, • No hazards were introduced by the National Airspace System (N The analysis did not address an existing safety issue, • The NAS change will not affect risk, and • of no hazards finding Sufficient justification exists to support the • . In approvi ng SRM documentation, the approval authority affirms that the aforementioned items have been performed and agrees that the underlying assumptions are reasonable and the findings are complete and accurate. SRM documentation approval does not constitute approval for implementation or acceptance of any risk associated with the NAS change or existing safety . issue 6.6.1 Service Unit SRM Documentation Approval or Concurrence ted or s Affec takeholder Service Units must assign an appropriate management official to provide approval or concurrence of the safety analysis. The person selected must be available to provide input to the management official(s) who will accept the risk associated with the NAS . change or existing safety issue If SRM documentation be sent outside the Service Unit for approval (to another Service of must AJI ), or Air Traffic Safety Unit, another Line of Business (LOB), Safety and Technical Training ( Oversight Service (AOV)), the documentation must have an approval or concurrence signature approval before it leaves the Service Unit. All identified means to reduce safety risk requiring and acceptance by AOV must first be sent through AJI. If SRM documentation requires the approval or concurrence of more than one Service Unit, discrepancies in the approval standards or processes may exist between the organizations. In these cases, the change proponent should request that AJI adjudicate the discrepancies. (see the development of a safety analysis / SRM d If an AJI safety case lead managed ocument Section 5.2.2. 3), no other Service Unit concurrence is required; however, the risk acceptor must still review and sign the ocument SRM d before the NAS change can be implemented. If an AJI SRM d develops the safety case lead ocument, the relevant/affected operational Service Unit(s)  6.6_SMSM_ 2019 04 Originally published April 201 9 Uncontrolled copy when downloaded

101 Section 6 Risk Acceptance and Safety Documentation Review must follow the issue that accepted the associated risks of the NAS change or existing safety monitoring plan documented in the SRM d ocument. 6.6.2 AJI Review and Approval AJI review and approval is a technical and non-technical assessment by AJI safety case leads to verify that the SRM process has been followed, that the safety documentation is complete, Manual tem (SMS) and that the safety documentation adheres to the Safety Management Sys principles and guidelines. Any documentation forwarded to the A ir T raffic Organization (ATO) Chief Safety Engineer for approval must first go through an AJI peer review. review, forward SRM For an AJI p eer case documentation to the AJI s afety lead in draft form and without signatures. At this point, the AJI safety case lead will facilitate the remaining steps in the review process. When the SRM hange notify the c proponent, who document is ready for signature, the AJI safety case lead will Finally, when the SRM document has all signatures obtain the appropriate signatures. will will present the SRM that of the ATO Chief Safety Engineer, the AJI safety case lead except for document to the ATO Chief Safety Engineer for signature. Other requirements ocument SRM d Section 5.4.4 that are documented in this SMS Manual remain. .2) (see hange or existing safety issue facilitated by AJI crosses Federal Aviation When a NAS c LOBs/organizations, an AJI safety case lead reviews the assessments to A) Administration (FA verify that affected LOBs/organizations have reviewed and approved the SRM documentation or existing safety issue for accuracy and correctness with regard to the NAS change . When a NAS change or existing safety issue facilitated by AJI crosses FAA LOBs, the ATO Chief Safety Engineer must approve and sign the safety analysis. 6.6.2.1 AJI Participation in System Acquisition Safety Analyses AJI safety case leads will be involved with NAS change efforts from concept development through In-Ser vice Management. In coordination with the Office of NextGen, an AJI safety case lead will be assigned to a portfolio, capture team, or program to provide safety guidance and will be familiar with the portfolio, capture team, advice, as appropriate. The AJI safety case lead on within the Enterprise or program; the program’s possible NextGen interfaces; its positi Architecture; its milestones; and its safety documentation requirements. The AJI safety case lead will stay with that portfolio, capture team, or program throughout its lifecycle. The will ensure that all required safety documentation meets the AJI safety case lead Guidance for System anagement Risk M afety requirements of this SMS Manual and the S ject Acquisitions to review the experts matter and will assemble the necessary sub documentation before it is presented to the ATO Chief Safety Engineer for approval. The ATO Chief Safety Engineer reviews SRM documentation and the associated safety assessments, analyses, reports, and plans, providing approval or comments. 6.6_SMSM_ 2019 04 94 9 Originally published April 201 Uncontrolled copy when downloaded

102 Section 6 Risk Acceptance and Safety Documentation Review AOV Approval and Acceptance 6.6.3 6.6.3.1 It ems Requiring AOV Approval or existing safety is the formal approval of a NAS change V approval submitted by a issue AO requesting organization. This approval is required before the NAS change can be implemented. or existing This is not the same as approval of the SRM document itself. All NAS changes safety issues submitted to AOV for approval first require approval and concurrence by AJI and any applicable Service Units. Refer to Section 6.6.2 for information on AJI approval. The following items require AOV approval before implementation: Controls that are defined to mitigate or eliminate initial and current high-risk hazards. • (For specific guidance regarding the AOV high-risk hazard acceptance/approval process and modeling requirements, see FAA Order 8000.365, Safety Oversight Circulars AOV Concurrence/Approval at Various Phases of Safety Risk -02, AOV SOC 07 ; (SOC) Management Documentation and Mitigations for Initial High- ; and AOV Risk Hazards Guidance on Safety Risk Modeling and Simulation of Hazards and -05A, SOC 07 ) . Mitigations • Changes or waivers to provisions of handbooks, orders, and documents that pertain to FAA Order JO 7110.65, Air Traffic Control (see the current separation minima, including Safety Guidance (ATO edition of the ATO ) on separation minima) -SG • Waiver renewals pertaining to separation standards • Changes to NAS equipment availability and any changes to the program Specific ATO -SGs pertaining to the SMS, as explained in FAA Order JO 1030.1, • Air Traffic Organization Safety Guidance (ATO-SG) 6.6.3.2 Item s Requiring AOV Acceptance The following require acceptance by AOV: and • Means to reduce risk that have lowered safety risk to medium or low span FAA LOBs Exclusions to SMS requirements granted by AJI • • United Order 8200.1, Changes to the criteria in FAA Inspection States Standard Flight Manual , including: o The flight inspector’s authority and responsibilities o Facility status classification and issuance of Notices to Airmen o Records and reports Extensions in the periodicity or interval of inspections o o items for the inspection of specific system areas Changes in required checklist Changes in established tolerances, or tolerances proposed for new equipment or o new functionality Changes in the procedures for evaluating the safety and flyability of instrument o flight procedures 6.6_SMSM_ 2019 04 95 9 Originally published April 201 Uncontrolled copy when downloaded

103 Section 6 Risk Acceptance and Safety Documentation Review Aeronautical -3 3410.2, Changes to the personnel certification requirements in Order JV • Navigation Products Career Progression and Certification Program for Aeronautical Information Specialists General Maintenance Changes to the certification criteria in FAA Order 6000.15, • Handbook for National Airspace System (NAS) Facilities Changes to the personnel certification requirements in FAA Order JO 3000.57, Air • Traffic Organization Technical Operations Training and Personnel Certification oordination of SRM Documentation C 6.6.4 c AJI will ollaborate with AOV to obtain the necessary reviews, approval, and risk acceptance ATO for all the signatures for SRM documentation with all applicable organizations outside of NAS changes. The scope of potential changes includes products, services, systems, and ederal facilities. Service Unit change proponents federal and non-f procedures associated with ATO may initiate these reviews and signatures through outside organizations. However, the case lead safety must inform the appropriate AJI Service Unit change proponent of such action. Note: There are cases when the ATO is responsible for conducting safety assessments for facilities). ederal or operated by the FAA (i.e., non-f facilities and equipment that are not owned ederal The ATO is required by law to ensure the safety of the services provided by these non-f facilities. 6.6_SMSM_ 2019 04  9 Originally published April 201 Uncontrolled copy when downloaded

104 Section 6 Risk Acceptance and Safety Documentation Review D ocument 6.7 Revising an SRM Through post -implementation monitoring, a need to modify the previously approved S afety Risk may arise (see Section 4.3.2 Management (SRM) d ocument equires a revision of the ). This r document and new SRM document approval and risk acceptance signatures. SRM Revisions Table 6.4: Signature Requirements for SRM D ocument New SRM D ocument Approval Signature and Part of SRM Type of Change Version Protocol Document Risk Acceptance Changed Required? Whole number New hazard; change to predicted Safety analysis revisions Yes residual risk assessment (e.g., 1.0 to 2.0) Adding, changing, removing, or not Safety analysis and Whole number Yes implementing new or existing safety requirements revisions safety requirements Updating charts, maps, airport layout, and approach plates, as Decimal revisions System description No long as change does not affect (e.g., 1.0 to 1.1, 1.2) hazards or risk levels Adding rationale or data for risk assessment when risk is not Risk analysis and No Decimal revisions changed and/or means to reduce assessment safety risk are not added or changed Clarification of safety requirements, including Standard Operating Procedures, Letters of Agreement, Safety requirements, letters to airmen, and No Decimal revisions monitoring plan, and implementation and monitoring appendices reports, as long as risk is not changed and means to reduce sk are not added or safety ri changed The risk acceptor(s), in coordination with the change proponent, may need to update or change an SRM document as a project progresses and decisions are modified. As discussed in Section 3.7.2 , monitoring may indicate that the National Airspace System (NAS) change does not meet the predicted residual risk, that the risk management strateg y is less effective than expected, or that additional hazards exist. In this case, additional safety requirements may be necessary. Any change to the safety analysis that may affect the assumptions, hazards, causes, or document ssitates a revision, including new signatures. A estimated risk in an SRM nece change page (containing a description of each change to the SRM document and the number of each affected page) must be included with each SRM . document If evaluations conducted by organizations external to the SRM panel indicate high residual risk for existing hazards, a revision to the SRM document is needed. These include Independent Operational Assessments, Flight Inspections, post -implementation safety assessments, Safety and Technical Training s and assessments, and the NAS Technical Evaluation Program. audit Based on the results of these assessments, the change proponent may need to modify the SRM document , which could include reopening the safety analysis for additional assessment. 07 97 6.7_SMSM_2016 Originally published July 2016 Uncontrolled copy when downloaded

105 Section 7 Audit and Assessment Programs ATO 7.1 Audit and Assessment Programs Overview 7.1.1 hnical Safety and Tec Training (AJI) Safety Assurance programs evaluate compliance with Safety Management System (SMS) requirements and Federal Aviation Administration (FAA) and/or Air Traffic Organization (ATO) orders, standards, policies, and directives. Audit and assessment programs evaluate: • The effectiveness of performance and operations in the Service Units, • The effectiveness of Air Traffic Control (ATC) facilities’ and Technical Operations districts’ inte rnal Quality Control efforts (e.g., operational skills assessment, system service review, certification, periodic maintenance, data integrity, modification, and availability), • The effectiveness of Quality Control mitigation efforts in response to identified trends and risks, Trends identified from safety data analysis, • • -related policies and procedures, and The effectiveness of safety • Compliance with SMS requirements. 7.1.2 Air Traffic Compliance Verification Evaluation Program FAA Air Traffic Organization Quality Assurance Program (QAP) , and FAA Order JO 7210.633, Order JO 7210.634, Air Traffic Organization (ATO) Quality Control , describe the current ATC facility evaluation and assessment programs, which involve assessments and audits focusing on compliance and safety. Air Traffic Service Area directors, air traffic managers, and Technical Operations districts are responsible for conducting internal evaluations of their respective facilities. The AJI Quality Assurance Office retains oversight of the ATC evaluation process and performs program assessments. 7.1.3 Difference between ATC Faci lity Audits and Assessments int of pl anc er f ca i or facility he ernal co m r i The ai e v r i traffic m i anager o t f a ons facility co his nducts in accordance AA Order J O 7210. 634 . AJI conducts audi ts base d on identified or with F suspected sa fety i ssues and non- compliance in accordance with the current v ersion of Order FAA JO 1000.37, Air Traffic Organization Safety Management System . The office determines priorities by soliciting input from the Service Areas and other FAA Lines of Business and by d risk analysis results. analyzing objective criteria from sources such as occurrence reports an In addition, AJI conducts no-n otice spot inspections of ATC facilities and Technical Operations activities, including the Aviation System Standards group. 7.1.4 National Airspace System Technical Eval uation Program FAA Order 6000.15, General Maintenance Handbook for National Airspace System (NAS) Facilities ; FAA and Order JO 6040.6, National Airspace System Technical Evaluation Program ; FAA Order 8200.1, United States Standard Flight Inspection Manual , describe the equipment evaluation and auditing programs that are part of the National Airspace System (NAS) Technical Evaluation Program. 7.1_SMSM_201509 98 Originally published September 2015 Uncontrolled copy when downloaded

106 Section 7 Audit and Assessment Programs ATO The NAS Technical Evaluation Program provides AJI, asset management, and safety decision-m aking information based on an independent review of: es and services meet their intended objectives: • How well faciliti Evaluators check key performance parameters and certification parameters at o selected facilities. o Evaluators review NAS Performance Analysis and NAS Performance Index data. How well the maintenance program is executed: • o Evaluators review facility logs to verify certification, periodic maintenance accomplishments, and documentation of corrective and scheduled maintenance activities. o Evaluators review the completion of required modifications. Evaluators review facility documentation such as Technical Performance o Records and required reference data. • How well customer needs are being met: o Evaluators solicit customer feedback through interviews and surveys. o Evaluators review the outage coordination process. Evaluators may also review specialist certification records and credentials. These reviews are either part of a special inspection or are random spot checks of documentation in a location that is geographically convenient to the routine evaluation. 7.1.5 Independent Operational Assessments AJI supports the agency’s commitment to field safe and operationally -ready solutions by conducting Independent Operational Assessments (IOAs) on designated new or modified systems or capabilities before the In tem - or -Service Management phase. An IOA is a full sys -level evaluation conducted in an operational environment. An IOA’s purpose is to capability confirm the readiness of a system from an operational and safety perspective. IOAs are independent of the Program Management Organization implementing the solution. IOAs determined critical operational issues. evaluate systems against pre- The Vice President of AJI directs the commencement of an IOA after the acceptance of an IOA Readiness Declaration from the Vice President of Program Management Organizati on. To assess the system/capability, AJI collaborates with Subject Matter Experts from the organizations that will operate, maintain, or otherwise be operationally affected by the solution. AJI reports any new or previously identified hazards, as well as operational concerns, based on data observed and collected during the IOA. At the conclusion of an IOA, the team assesses the solution’s operational readiness based on the identified hazards and any observed operational concerns. The team reports and bri efs the results of the IOA to affected stakeholders, including the Vice President of AJI, the Program Management Organization, the affected operating service(s), and any other affected organizations. The results are also provided to the In -Service Decisio n authority. The change proponent is responsible for the treatment and monitoring phases of Safety Risk Management (SRM) for the hazards identified during the IOA. Hazards identified by IOA must still undergo all necessary phases of the SRM process by the change proponent. 7.1_SMSM_201509 99 Originally published September 2015 Uncontrolled copy when downloaded

107 Section 7 Audit and Assessment Programs ATO Independent Assessments 7.1.6 AJI performs independent assessments to evaluate operational procedures, order compliance, fielded systems, and safety benefits. An AJI independent assessment is independent of the program office or operating service responsible for the program or operation. Independent assessments are post -implementation evaluations of NAS changes that assess actual performance. During independent assessments, the teams verify that any previously documented hazards were rated accurately (based on observed data) and that no unacceptable safety risks exist. In addition, teams may identify operational issues and other findings. Independent assessments may involve several facility or program assessments over a long period of time, one assessment that lasts for an extended period of time, or multiple brief assessments. The processes and procedures for an independent assessment are tailored according to its duration and the complexity of the operation or program being assessed. The assessment may be conducted at one or multiple sites, and data may be collected on site or remotely. Results and/or recommendations are based on the assessment team’s analysis of data collected during and, if applicable, before the assessment. The conclusions and recommendations are independent from external sources. 100 7.1_SMSM_201509 Originally published September 2015 Uncontrolled copy when downloaded

108 Audit and Assessment Programs ATO Section 7 Safety Data Reporting, 7.2 Analysis Tracking, and stems (SMS s) require the collection and analysis of data from different Safety Management Sy sources and various vantage points to determine if hazards exist. The key to safety data analysis is developing the capability to sort and analyze a vast array of data and transform the data into information that permits the identification and mitigation of hazards, preventing f uture incidents and accidents. Purpose of Safety Data Collection and Evaluation 7.2.1 The tracking and anal ) yzing of safety data to enhance the Air Traffic Organization’s (ATO of potentially hazardous situations is a critical aspect of the SMS. Safety and awareness -wide safety data and Technical Training ( AJI ) assists with the collection and analysis of agency ational Ai supports sharing the data to continually improve the safety of the N rspace System (NAS). Safety data are used to: • Identify risks, trends, and vulnerabilities in the system; Determine the effects of a NAS change on the operation as a whole; • in managing risk; safety requirements Assess the performance of • Identify areas where safety could be improved; • Contribute to accident and incident prevention; and • Assess the effectiveness of training. • In most cases, if the analysis of safety data leads to the identification of issues or hazards, the afety Risk resolution or corrective action constitutes a NAS change, which requires S Management (SRM ). This is an example of the continuous, closed- loop process for managing -loop process between SRM and Safety Assurance. depicts the closed Figure 7.1 safety risk. Safety Risk Management Safety Assurance No No Conduct post- Was predicted Describe the Assess risk level and devise Yes Yes implementation Implement the Hazards residual risk safety requirements for system and NAS monitoring NAS change identified? implementation, if necessary achieved? change activities Perform periodic Data audits and Sources assessments Were any hazards identified or risk introduced? Yes No Figure 7.1: SRM and Safety Assurance Closed-L oop Process 7.2_SMSM_201607 1 10 Originally published July 2016 Uncontrolled copy when downloaded

109 Section 7 ATO Audit and Assessment Programs 7.2.2 AJI’s Role in Safety Data Collection and Evaluation data through various sources within and outside the Federal Aviation AJI obtains safety Administration (FAA) assesses safety by tracking safety metrics to produce reports on . AJI NAS safety, which are shared with appropriate Lines of Business and/or external stakeholders. and Reporting Processes Safety Data Collection 7.2.3 The FAA collects and r lists eports on safety data from various sources in the NAS. Section 8 many of the existing FAA and ATO orders, processes, and databases related to safety data collection and reporting. FAA Order J • O 7210.632, , provides Air Traffic Organization Occurrence Reporting specific direction regarding the recording, reporting, and investigation of air traffic incidents. National Airspace Performance Reporting System , and FAA Order JO 6040.15, FAA • , cover reporting on the r 6000.30, Orde National Airspace System Maintenance Policy serviceability of ATO facilities and systems, such as failures and degradations of and equipment that affect safety. , surveillance, and other systems communications Maintenanc e guidelines, directives, checklists, configuration management, and NAS Technical Evaluation Program all contribute to the periodic review and maintenance of and procedures. equipment • iation safety inspectors The Safety Recommendation Reporting System provides FAA av method to develop and submit s afety recommendations directly to the Office of with a Investigation and Prevention. (See FAA Order 8020.16, Air Traffic Accident Organization Aircraft Accident and Incident Notification, Investigation, and Reporting .) Risk Analysis Process quantifies the level of risk present in any air traffic incident. It The • provides a method for consistent and coherent identification of risk elements and allows s to prioritize actions designed to reduce the effect of those elements. The process user uses the Risk Analysis Tool developed by EUROCONTROL to analyze each Risk perts bjec Analysis Event. Risk Analysis Events are assessed by a panel of su t ma tter ex from air traffic and flight operations (e.g., controllers and air-transport rated pilots). This responsible for conducting the analysis of Risk Analysis Events and panel is coordinating Tool ssessment reporting, mitigating, and tracking. The Risk Analysis he post-a t . The Risk produces a numerical value of severity and repeatability on a risk matrix Analysis Tool also c aptures any associated c ausal, systemic, and contributing factors. Sev punitive, voluntary reporting programs allow pilots and ATO personnel to report an eral non- incident or event without reprisal. These programs include the Aviation Safety Action Program (refer to AOV SOC 07 -04, Aviation Safety Action Program (ASAP) for Credentialed ATO Personnel Technical Operations Safety Action ), the Aviation Safety Reporting Program, Program , and Air Traffic Safety Action Program. They are designed to foster consistent reporting and higher quality data. Other mechanisms employed by the FAA for employees to report issues include the Unsatisfactory Condition Report program, the Aviation Safety Hotline, and the Administrator’s -1111. Hotline. Both hotlines can be reached by calling 1 -800- 255 7.2_SMSM_201607 10 2 Originally published July 2016 Uncontrolled copy when downloaded

110 Section 7 Audit and Assessment Programs ATO 7.3 Safety Incident and Accident Reporting and Analysis Evidence has shown that for every accident, there are many precursor events. Therefore, accident prevention programs focus on the collection, analysis, and investigation of incident data. Incident investigation is valuable because real -world occurrences are analyzed to prevent or eliminate future occurrences. The Air Traffic Organization (ATO ) fine -tunes incident prevention measures by analyzing low -level indicators that may contribute to an incident or accident. Incident reporting prompts the ATO to conduct investigations. ATO employees who conduct the investigations reconstruct and analyze the event. They identify contributing factors and categorize them as either direct or indirect. They also identify factors that may have lessened the effect of the occurrence. ATO employees use the information gained from an investigation as input to recommend risk mitigation strategies and safety -enhancing measures to preclude similar events in the future. Corrective actions can enhance safety at all levels, from national to local. They can include: • Airspace and airport improvements, • Additional communication, navigation, and surveillance systems and/or automation systems, • Additional staffing, or • Other safety -enhancing changes. 1 03 7._SMSM_201509 Originally published September 2015 Uncontrolled copy when downloaded

111 Section 7 Audit and Assessment Programs ATO Reported Safety D ata about S erviceability of E quipment, S ystem s, an d Facilities 7.4 , significant ev ajority o aintenance logs ca ent reports, and general m f pture t he m eports Outage r daily system pe rformance metrics, i ncludi ng incidents. (Refer to Federal A viati on Administration (FAA) Order JO 6030.41, Notification of Facility Service Interruptions and Other Significant ents.) Air Traffic Organization Events , for more information about reporting significant ev ts i n t he form o employees m ake addi tional r epor ccident reporting, and irmen and a f Notices t o A they co data via a formal hot line and t he U nsatisfactory C ondition Report pr ogram. llect Hardware and software in t he National Airspace System (NAS) that are used for aircraft separation have established performance standards necessary for system safety. Overall trends and performance levels are monitored systematically, and requirements are documented during the certification process. Certification is a Quality Control method used to help ensure that NAS systems and services are performing as designed. Refer to FAA Order 6000.30 , , for more information. National Airspace System Maintenance Policy The NAS Technical Evaluation Program and Unsatisfactory Condition Report program require written documentation and management involvement in the review, mitigation, and analysis of trends. Through the NAS Technical Evaluation Program, personnel conduct periodic independent technical reviews of services provided by systems, sub -systems, and equipment. These reviews also address how well the services match customer needs. The Unsatisfactory Condition Report program allows employees to file reports on identified deficiencies in the safety or efficiency of procedures, equipment, working environment operations, or services. Refer to FAA Order 1800.6, Unsatisfactory Condition Report , and FAA Order JO 6040.6 , , for detailed information. National Airspace System Technical Evaluation Program  1 9 7.4_SMSM_20150 Originally published September 2015 Uncontrolled copy when downloaded

112 Section 7 ATO Audit and Assessment Programs Reporting 7.5 Voluntary Data ve describe the r eporting of sp ecific he pr ocesses l f sa fety dat a. However, isted abo types o T f these dat a, it i s i mportant t hat ea ch employee repor over and abov e the r eporting o ts any National s or c ould become a haz ard within the ion that he or sh occurrence or situat e thinks i lture depends on this type of v oluntary r eporting. Airspace System ( NAS) . A positive safety cu he F ederal A viation Administration (FAA) has f ormal m echanisms f or employees to report T including the U issues, ogram, the Aviation Safety H otline, and nsatisfactory C ondition Report pr Other r eporting systems a re listed i n Section 8. the Administrator’s H otline. 7.5.1 Unsatisfactory Condi tion Report The Unsatisfactory Condition Report program is a means to advise management of an existing unsatisfactory condition. The Unsatisfactory Condition Report process has a defined feedback loop that requires the responsible organization to complete the review cycle and respond to the submitter within 30 calendar days. An Unsatisfactory Condition Report cannot be closed based on planned actions; it can only be closed once the condition described in the report is resolved, unless it is equipment related. 7.5.2 Aviation Safety Hotline he A viation Safety H otline (1 -800- 255-1111) i s i ntended for r eporting po ssible violations of T Federal R f the Code of egulations o Title 14 o r other av iation safety i ssues, su ch as i mp roper recordkeeping, non-adherence to procedures, and unsafe aviation practices. The hotline is If a caller in FAA Order FAA Hotline Program . , requests described further 1070.1 and information in the report concerning an individual are confidentiality, caller identity from release under the Privacy Act. If the caller requests protected and has provided feedback his or her name and address, he receives a written response after the issue is closed. he or s Administ 7.5.3 rator’s Hotline The Administrator’s Hotline operates in the same fashion as the Aviation Safety Hotline. It can also be reached at 1 -800-255- 1111. After dialing the hotline number, a menu directs callers in the appropriate direction. The main operational difference between the two hotlines is that issues reported to the Administrator’s Hotline are closed within 14 calendar days of the report. A / Technical Operations Safety Action Program 7.5.4 ir Traffic Safety Action Program In cooperation with its employee labor organizations, the A ir Traffic Organization (A TO) has established voluntary safety reporting programs for air traffic and Technical Operations Air Traffic Safety Action Program ( ATSAP) and Technical Operations Safety Action employees. Program ( T- SAP) are modeled after the Aviation Safety Action Program. They allow employees to voluntarily identify and report safety and operational concerns as part of the FAA’s overall safety goals. The collected information is reviewed and analyzed to facilitate early detection and improved awareness of operational deficiencies and adverse trends. The primary purpose of A TSAP and T -SAP is to identify safety events and implement skill enhancements and system -wide corrective actions to reduce the opportunity for safety to be compromised. Information obtained from ATSAP and T- SAP will provide stakeholders a mechanism to identify actual and potential risks throughout the NAS. The programs foster a voluntary, cooperative, non- punitive environment for open reporting of safety concerns. ATSAP and T -SAP reports allow all parties to access valuable safety information that may otherwise be unavailable. SAP are brought to an Event Review Committee Reports submitted through ATSAP and T- which reviews and analyzes the submitted reports, determines whether reports require further 1 05 .5_SMSM_201509 Originally published September 2015 Uncontrolled copy when downloaded

113 Section 7 ATO Audit and Assessment Programs investigation, and identifies actual or potential problems from the information contained in the reports and proposed solutions. All Event Review Committee determinations are made by consensus. The Event Review Committee may direct skill enhancement or system corrective action and is responsible for follow -u p to determine that the assigned actions are completed in a satisfactory manner. Safety Risk Management may be required for corrective actions. 106 .5_SMSM_201509 Originally published September 2015 Uncontrolled copy when downloaded

114 Data and Information Repositories Section 8 Safety 8.1 Overview ion Administration (FAA) employees populate several aviation safety databases Federal Aviat with information regarding National Airspace System (NAS) safety events and serviceability. Many professionals use aviation safety data and information as input for the development of NAS safety enhancements. Sources for gathering safety data and information include: • National Transportation Safety Board recommendations, mmendations, FAA reco • • Air Traffic Safety Oversight Service compliance issues, The Risk Analysis Process, • • Requirements for new communication, navigation, surveillance, and automation services to enhance or expand airspace management, , • Unsatisfactory Condition Reports Employee suggestions, • Applications for procedural changes, • Research and development, • • Acquisition of new systems and equipment, Industry advocacy, • Participation in international forums, • • The Safety Risk Management process documented in the Safety Management System Manual, and • Runway Safety Database. provides an overview of various safety databases and recording systems used by the Table 8.1 FAA. 8.1_SMSM_201509  1 Originally published September 2015 Uncontrolled copy when downloaded

115 Section 8 Safety Data and Information Repositories Table 8.1: Safety Databases and Reporting Systems Safety Databases and Reporting Systems Overview System Name Mandatory Reporting Data The Aviation Safety Information Analysis and Sharing System is a data warehouse Aviation Safety and integrated database system. It enables users to perform queries across multiple Information Analysis and databases and display queries in useful formats. It includes accidents, incidents, Sharing System and pilot reports of near mid- air collisions. The Accident/Incident Data System contains data records for all general aviation and Accident/Incident Data System commercial air carrier incidents since 1978. National Transportation The National Transportation Safety Board accident and incident database is the accident Safety Board official repository of aviation accident data and causal factors. In this database, and incident database personnel categorize events as accidents or incidents. Formerly known as the National Airspace Incidents Monitoring System, the Air Traffic Quality Assurance database is a collection of databases specific to the following subjects: near -midair collisions, pilot deviations, vehicle/pedestrian deviations, Area Navigation / Required Navigation Performance deviations. The flight incidents where two near-midair collision database contains reports of in- Air Traffic Quality aircraft have closed to an unsafe distance but avoided an actual collision. The pilot Assurance database deviation database contains incident reports in which the actions of a pilot violated a Federal Aviation Regulation or a North American Aerospace Defense Command Air Defense Identification Zone tolerance. The vehicle/pedestrian deviation database contains incident reports of pedestrians, vehicles, or other objects interfering with aircraft operations on runways or taxiways. Safety Assessment System is a national database that contains The Facility Facility Safety historical information related to the facility safety assessment process. This Assessment System information includes evaluation checklists, reports, facility information, tracking information, and response dat a. This national database contains reports, findings, and mitigation plans from NAS Integrated NAS Technical aintained by the NAS Technical Evaluation Program audits and assessments. It is m Evaluation Program and Performance Group in the Technical Operations Services Quality Assurance Application . Management Office Comprehensive Electronic Data Analysis and Reporting provides an electronic Comprehensive means of assessing employee performance, managing resources, and capturing Electronic Data Analysis -related information and metrics. The tool provides a standard interface for the safety and Reporting collection, retrieval, and reporting of data from multiple sources. It also automates the creation, management, and storage of facility activities, events, briefing items, Reviews, Technical Training discussions, and FAA forms. Quality Assurance The Compliance Verification Tool replaces the Facility Safety Assessment System. Facilities conduct internal compliance verifications and enter the information in the Quality Control groups in the Service Units conduct external compliance tool. The Compliance Verification verif ications and enter the information in the tool. Service delivery points also Tool develop risk mitigation plans that communicate how specific risks will be mitigated for all checklist items contained in the Compliance Verification Tool determined to be non- compliant. 8.1_SMSM_201509 1 Originally published September 2015 Uncontrolled copy when downloaded

116 Section 8 Safety Data and Information Repositories Systems Safety Databases and Reporting System Name Overview The Performance Data Analysis and Reporting System calculates a range of performance measures, including traffic counts, travel times, travel distances, traf fic trail separations. It turns these measurement data into information flows, and in- P erformance Data useful to FAA facilities through an architecture that features: Analysis and Reporting • Automatic collection and analysis of radar tracks and flight plans, System Automatic generation and distribution of daily morning reports, • Sharing of data and reports among facilities, and • Support for exploratory and causal analysis. • quantify the level The Risk Analysis Tool is used during the Risk Analysis Process to of risk present in any air traffic incident. The Risk Analysis Tool is used to capture any associated causal, systemic, and contributing factors. The Risk Analysis Tool produces a numerical value of severity and repeatability on a risk matrix. Risk Analysis Tool Using the Risk Analysis Tool, the Risk Analysis Process provides a method for onsistent and coherent identification of risk elements and allows users to prioritize c actions designed to reduce the effect of those elements. The Operations Network is the official source of NAS air traffic operations and delay data. The data collected through the Operations Network are used to analyze the Operations Network performance of the FAA’s air traffic control facilities traffic count and delay information, air traffic control tower and Terminal Radar Approach Control operations, etc. This database contains Letters of Agreement, Standard Operating Procedures, and Facility Directives facility orders for all facilities nationwide. Repository Voluntary Reporting The Aviation Safety Reporting System collects voluntarily submitted aviation safety Aviation Safety Reporting incident/situation reports from pilots, controllers, and other personnel. It identifies system deficiencies, and issues messages to alert individuals in a position to corr System ect the identified issues. The Aviation Safety Action Program promotes voluntary reporting of safety issues and events that come to the attention of employees of certain certificate holders. It Aviation Safety Action -related incentives to encourage employees to voluntarily includes enforcement Program report safety issues, even though the issues may involve an alleged violation of Title . 14 of the Code of Federal Regulations The Air Traffic Safety Action Program is a non - punitive, voluntary reporting program modeled after the Aviation Safety Action Program for employees delivering air traffic Action Air Traffic Safety services. It allows for employees to submit safety concerns and deficiencies so Program issues can be resolved before a major error occurs. This voluntary reporting helps promote a strong safety culture within the ATO. The TechNet website provides a means for expediently distributing NAS operational information within the FAA. It contains information such as NAS delay information TechNet by service (e.g., automation, surveillance, navigation, communication) and active ages (i.e., full interruptions to service). equipment out - The Technical Operations Safety Action Program is a voluntary, non punitive safety reporting program for ATO Technical Operations Services personnel. Employees at Technical Operations the point of service have a unique understanding of safety and can better identify Safety Action Program threats and risks to their particular operations. By studying the data gained from voluntary reports, safety issues c an be more efficiently identified and mitigated. 8.1_SMSM_201509 109 Originally published September 2015 Uncontrolled copy when downloaded

117 Safety Data and Information Repositories Section 8 Systems Safety Databases and Reporting Overview System Name Reporting Tools ATO manages and databases that will facilitate formal, structured information Lessons Learned sharing within the ATO. Lessons Learned Repositories allow ATO employees to Repositories access and contribute lessons learned and best practices derived from successes and challenges. Table 8.2: Data Types and Applicable Reporting Requirements References Overview Data This order mandates that personnel collect Air Traffic FAA Order JO 7210.632, Mandatory Occurrence and analyze data concerning air traffic Org anization Occurrence Reporting Reports incidents. FAA Order JO 8020.16, Air Traffic This order contains reporting requirements Organization Aircraft Accident and Aircraft incident or regarding safety issues, concerns, Incident Notification, Investigation, and accident incidents, and accidents. Reporting This order mandates that outage reports FAA Order JO 6040.15, National Airspace be filed and contributes to daily system System outages Performance Reporting System performance and incident reporting. FAA Order 6030.41, Notification of Facility This order mandates that significant events Significant system and Service Interruptions and Other be reported and contributes to daily system events Significant Events performance and incident reporting. This order provides FAA employees with a Unsatisfactory FAA Order 1800.6, ng management of means of informi Unsatisfactory condition Condition Report unsatisfactory conditions. This order establishes procedures for anic Reporting Oce FAA Order 7110.82, Oceanic altitude and processing reports and for collecting Errors navigation errors system data for analysis. FAA Order 8020.16, Air Traffic This order establishes procedures for Organization Aircraft Accident and Safety Aviation Safety Inspectors to report safety Incident Notification, Investigation, and recommendations Office of recommendations directly to the Reporting Accident Investigation and Prevention. This order defines the policy and procedures for ATO Voluntary Safety Reports. It identifies the responsibilities of Voluntary Safety FAA Order JO 7200.20, Voluntary Safety individuals and organizations and the Reporting Programs (VSRP) Reports requirements, expectations, and policy under which the identified programs operate. 8.1_SMSM_201509 110 Originally published September 2015 Uncontrolled copy when downloaded

118 Section 9 and Acronyms Definitions 9.1 Definitions In cases where both Federal Aviation Administration (FAA) and Air Traffic Organization Note: (ATO ) definitions are provided for the same term, the ATO definition is provided as an expansion of the FAA definition to facilitate understanding when communicating within the ATO. In those cases where terms and resultant effects are communicated outside the ATO, the FAA definition will be the standard of reference. Acceptable Level of Safety Risk. Medium or low safety risk. An unplanned event or series of events that results in death, injury, or damage to, or Accident. loss of, equipment or property. Active Failure. An error of omission or commission that is made in the course of a particular operation. An active failure can also be a known problem or a known mechanical deficiency or fault. FAA policy dealing with any aspect of lifecycle Acquisition Management System (AMS). acquisition management and related disciplines. The AMS also serves as the FAA’s Capital Planning and Investment Control process. The process whereby the regulating Acceptance. Air Traffic Safety Oversight Service (AOV) organization has delegated the authority to the service provider to make changes within the confines of approved standards and only requires the service provider to notify the regulator of those changes within 30 days. Changes made by the service provider in accordance with their delegated authority can be made without prior approval by the regulator. AOV Approval. The formal act of approving a National Airspace System (NAS ) change AS submitted by a requesting organization. This action is required prior to the proposed N change being implemented. A process of measuring or judging the value or level of something. Assessment. Assumptions. Conclusions based on the presumed condition of a system or system state – not documented facts, desired outcomes, or mitigations. A review of an organization’s safety programs or initiatives to verify completion of tasks Audit. and determine an organization’s compliance with FAA directives and procedures. Baseline. The written processes, procedures, specifications, and other conditions of the system that were accepted as the starting point for oversight of safety in the NAS on March 14, 2005. The ATO must maintain the NAS at a safety level that is at least equal to that state, in compliance with current policies, processes, and procedures that are documented in its orders, “Acceptance of the baseline did not imply or state that the handbooks, and manuals. ( Note: NAS was or was not inherently safe as configured on that date, nor did it imply that the NAS had -01, no existing high risks,” Acceptance of the Air Traffic Organization (ATO) C 07 AOV SO .) Baseline Bounding. A process of limiting the analysis of the NAS change or system to only the elements that affect or interact with each other to accomplish the central function. 11 9.1_SMSM_201904 1 Originally published April 201 9 Uncontrolled copy when downloaded

119 Section 9 and Acronyms Definitions Cause. The origin of a hazard. Change Proponent. The individual, program office, facility, or organization within the FAA that is proposing or sponsoring a NAS change or means to address an identified existing safety e. members are selected at the discretion of the anel ) p RM The Safety Risk Management (S issu facilitator. anel change proponent and/or SRM p Common Cause Failure. A failure that occurs when a single fault results in the corresponding failure of multiple system components or functions. Compliance Audit. An audit that evaluates or assesses conformance to established criteria, processes, and work practices. The objective of a compliance audit is to determine if employees and processes have followed established policies and procedures. M processes are repeated until the safety risk associated with each Continuous Loop. SR hazard is acceptable and has met its predicted residual risk. Concurrence. is used to represent a technical review of the safety The concurrence signature analysis and to confirm the rationale used throughout is consistent with the overall risk assessment. an SRM expert who is well versed in the The concurrence signature comes from System (SMS) Manual and familiar with the terminology and processes Management Safety therein. Configuration Management. A process for establishing and maintaining consistency of a product’s performance, functional and physical attributes with its requirements, design, and operational information throughout its life. Confirmation. The act of using a written response from a third party to confirm the integrity of a specific item or assertion. Control. • FAA Definition. Safety Risk Control: A means to reduce or eliminate the effects of hazards. currently means • ATO Definition. Any . (See a hazard’s causes or effects reducing “Mitigation.”) Credible. It is reasonable to expect that the assumed combination of conditions that define the ) TC system state will occur within the operational lifetime of a typical Air Traffic Control (A system. Critical NAS System. A system that provides functions or services that, if lost, would prevent users of the NAS from exercising safe separation and control over aircraft. Current Risk. FAA Definition. The predicted severity and likelihood at the current time. • efinition. The assessed severity and frequency of a hazard’s effects in the ATO D • present state. 9.1_SMSM_201904 11 2 Originally published April 201 9 Uncontrolled copy when downloaded

120 Section 9 and Acronyms Definitions All the planned and systematic actions used to substantiate, at an Development Assurance. adequate level of confidence, that errors in requirements, design, and implementation have been identified and corrected such that the system satisfies the applicable approval or certification basis. The real or credible harmful outcome that has occurred or can be expected if the hazard Effect. occurs in the defined system state. —operating either independently or within a system/sub- Equipment. A complete assembly system —that performs a specific function. A system that is designed and implemented in such a way that, to the Error-Tolerant System. maximum extent possible, errors and equipment failures do not result in an incident or accident. fault-tolerant design An error-tolerant design is the human equivalent of a . Generally, any installation of equipment designated to aid in the navigation, Facility. communication, or control of air traffic. Specifically, the term denotes the total electronic equipment, power generation, or distribution systems and any structure used to house, support, sub- and/or protect these equipment and systems. A facility may include a number of systems, systems, and equipment. A system designed such that if it sustains a fault, it still provides a subset of Fail Operational. its specified behavior. A system designed such that if it fails, it fails in a way that will cause no harm to Fail Safe. other devices or present a danger to personnel. Fault Tolerance. The ability of a system to respond without interruption or loss of capabilities in the event of an unexpected hardware or software failure. An expression of how often a given effect occurs. Frequency. Hazard. • FAA Definition. A condition that could foreseeably cause or contribute to an accident. • ATO Definition. Any real or potential condition that can cause injury, illness, or death to people; damage to or loss of a system, equipment, or property; or damage to the environment. A hazard is a prerequisite to an accident or incident. s A tool used to provide an initial overview of the hazard’ Hazard Analysis Worksheet (HAW). presen ce in the overall flow of the operation. The determination of the hazard scenarios and associated Hazard Identification. consequences (undesired events) as a consequence of introducing a new system into the NAS. This provides an intermediate product that expresses the hazards that will be used during risk analysis. High-Risk Hazard. A hazard with an unacceptable level of safety risk; the NAS change cannot be implemented unless the hazard’s associated risk is mitigated and reduced to medium or low. 9.1_SMSM_201904 11 3 Originally published April 201 9 Uncontrolled copy when downloaded

121 Section 9 and Acronyms Definitions substantially damaged beyond economic repair, missing, or aircraft destroyed / An Hull Loss. completely inaccessible. The structured process during concept and requirement definition, design, Human-Centered. development, and implementation that identifies the user as the focal point of the effort for which apabilities procedures, equipment, facilities, and other components serve to support human c and compensate for human limitations; sometimes also called “user -centered.” A multidisciplinary effort to generate and compile information about human Human Factors. capabilities and limitations and apply that information to equipment, systems, facilities, procedures, jobs, environments, training, staffing, and personnel management for safe, Human Factors comfortable, and effective human performance. (See FAA Order 9550.8, .) Policy Incident. An occurrence other than an accident that affects or could affect the safety of operations. Initial Risk. FAA Definition. The predicted severity and likelihood of a hazard’s effect or outcomes • when it is first identified and assessed; includes the effects of preexisting risk controls in the current environment. • ATO Definition. The composite of the severity and likelihood of a hazard, considering only controls and documented assumptions for a given system state. It describes the risk before any of the proposed mitigations are implemented. The technique of asking questions and recording responses. Inquiry. t of critically examining documents to determine the content and quality of a Inspection. The ac transaction, such as inspecting leases, contracts, meeting minutes, requirements, and organization policy. Latent Failure. An error or failure whose adverse consequences may lie dormant within a system for a long time, becoming evident when combined with other factors. The estimated probability or frequency, in quantitative or qualitative terms, of a Likelihood. hazard’s effect or outcome. Maintenance. Any repair, adaptation, upgrade, or modification of NAS equipment or facilities. This includes preventive maintenance. Management Strategy. Actions designed to reduce or manage the ri sk associated with a NAS change or operation. Mitigation. means to reduce the risk of a hazard. Any A complex system that is composed of airspace, airports, AS ). National Airspace System (N communication, navigation, and facilities; aircraft, pilots, air navigation facilities, and ATC surveillance services and supporting technologies and systems; operating rules, regulations, policies, and procedures; and people who implement, sustain, or operate the system components. 9.1_SMSM_201904 11 4 Originally published April 201 9 Uncontrolled copy when downloaded

122 Section 9 Definitions and Acronyms A modification to any element of the NAS that pertains to, or could affect the NAS Change. provision of air traffic management, communication, navigation, or surveillance services. Air Collision Categories. FAA Order 8900.1, Flight Standards Information Near Mid- nagement System , Volume 7, Chapter 4, identifies the following definitions of Critical, Ma Potential , and Low categories : “A” – A situation in which collision avoidance was due to chance, rather than a 1) Critical. Situations where large evasive maneuvers are necessary to pilot’s evasive act or action. avoid collision and/or situations where little or no time is available to recognize the threat and react appropriately. Encounters of less than 100 feet separation are considered to be critical ris k. 2) “B” – Potential. A situation which would probably have resulted in a collision if no action had been taken by the pilot; a situation in which a Traffic Alert and Collision (RA ) was received and followed; or Resolution Advisory Avoidance System (TCAS) ) alert or (TIS Traffic Information Services where a pilot sighting the traffic without the which a electronic aid caused pilot evasive action. A “potential” risk is a situation in collision would probably occur eventually if Situations no action is taken by either pilot. of encounters of less than 500 feet separation may be considered potential risk. 3) “C” – Low Potential. A situation in which a collision is unlikely, however, one or both surprised by the proximity of the other; one in which the course of the aircraft pilots was bring them closer than required standard separation; a situation where whether or not the pilot took evasive action a collision probably would not occur; or a situation in which there is ample time to take action to avoid a collision. TCAS traffic A TIS alert or may cause the pilot to take action after sighting the traffic either with or without advisory the aid of an electronic alert system; situations of encounters of 500 feet or greater; slowly converging flightpaths may be considered low potential collision risks. Objective Evidence. Documented proof; the evidence must not be circumstantial and must be vation, measurement, test, or other means. obtained through obser Observation. The process of witnessing an organization’s process. It differs from a physical examination in that the auditor only observes the process; no physical evidence is obtained. Operational Assessments. An assessment to address the effectiveness and efficiency of the organization. The objective of an operational assessment is to determine the organization’s ability to achieve its goals and accomplish its mission. Regulatory supervision to validate the development of a defined system and verify Oversight. compliance to a pre-defined set of standards. The act of gathering physical evidence. It is a substantive test Physical Examination. involving the counting, inspecting, gathering, and inventorying of physical and tangible assets, such as cash, plants, equipment, and parameters. Preconditions. The system states or variables that must exist for a hazard or an accident to occ -tolerant system. ur in an error Predicted Residual Risk. The risk that is estimated to exist after the safety requirements are mented, or after all avenues of risk mitigation have been explored. imple 9.1_SMSM_201904 11 5 Originally published April 201 9 Uncontrolled copy when downloaded

123 Section 9 and Acronyms Definitions A hazard identification tool used to list all potential hazards in Preliminary Hazard List (PHL). the overall operation. Development of a PHL typically begins with a brainstorming session among the individuals performing the safety analysis. Process. A set of interrelated or interacting activities that transforms inputs into outputs. A Safety Assessment’ s review of an organization’s safety programs or Program Assessment. initiativ es. Programs and initiatives include, but are not limited to, Service Area Quality Assurance, Air Traffic Facility Quality Control, Runway Incursion Prevention Plans, Equipment Availability Programs, and Contractor Quality Assurance programs for FAA contract towers. Qualitative Data. Subjective data that is expressed as a measure of quality; nominal data. A program for the systematic monitoring and evaluation of the various Quality Assurance. aspects of a project, service, or facility to ensure that standards of quality are being met. It is a process to assess and review the processes and systems that are used to provide outputs (whether services or products) and to identify risks and trends that can be used to improve these systems and processes. Quality Control. A process that assesses the output (whether a product or service) of a particular process or function and identifies any deficiencies or problems that need to be addressed. Objective data expressed as a quantity, number, or amount, allowing for a Quantitative Data. more rational analysis and substantiation of findings. Recording. The process of documenting the identified hazards and the associated safety analysis information. A design attribute in a system that ensures duplication or repetition of elements Redundancy. to provide alternative functional channels in case of failure. Redundancy allows the service to be provided by more than one path to maximize the availability of the service. An essential attribute or characteristic of a system. It is a condition or capability Requirement. that must be met or passed by a system to satisfy a contract, standard, specification, or other formally imposed document or need. Residual Risk. • FAA Definition. The remaining predicted severity and likelihood that exist after all selected risk control techniques have been implemented. ATO Definition. The level of risk that has been verified by completing a thorough • monitoring plan with achieved measurable safety performance target(s). Residual risk is the assessed severity of a hazard’s effects and the frequency of the effect’s occurrence. Risk. The composite of predicted severity and likelihood of the potential effect of a hazard. Risk Accept The confirmation by the appropriate management official that he or she ance. understands the safety risk associated with the NAS change and that he or she accepts that safety risk into the NAS. Risk acceptance requires that signatures have been obtained for the 9.1_SMSM_201904 11 6 Originally published April 201 9 Uncontrolled copy when downloaded

124 Section 9 and Acronyms Definitions d ocument and that a comprehensive monitoring plan SRM safety requirements identified in the has been developed and will be followed to verify the predicted residual risk. Risk Analysis Event. A loss of standard separation between two aircraft in a radar environment that results in less than 66 percent of the applicable separation minima maintained. Risk Assumption Strategy. A risk management strategy used to accept the risk. A risk management strategy used to avert the potential occurrence Risk Avoidance Strategy. and/or consequence of a hazard by either selecting a different approach or not implementing a specific proposal. A risk management strategy used to develop options and take actions Risk Control Strategy. to lower the risk. Risk Mitigation. Refer to “Mitigation.” A risk management strategy used to shift the ownership of a risk to Risk Transfer Strategy. another party. Safety. The state in which the risk of harm to persons or property damage is acceptable. Processes within the SMS that function systematically to measure safety Safety Assurance. performance and determine whether an organization meets or exceeds its safety objectives through the collection, analysis, and assessment of information. The way safety is perceived and valued in an organization. It represents the Safety Culture. priority given to safety at all levels in the organization and reflects the real commitment to safety. A mandate from AOV to ATO to take immediate corrective action to address Safety Directive. a significant unsafe condition. a noncompliance issue that creates An integrated collection of processes, procedures, ). Safety Management System (S MS policies, and programs that are used to assess, define, and manage the safety risk in the provision of ATC and navigation services. Safety -level requirement and the limit of the The buffer between the actual minimum Margin. hardware or software system. Metrics identified to determine how risk mitigations are Safety Performance Indicators. performing. Safety Performance Monitoring. The act of observing the safety performance of the NAS to ensure an acceptable level of safety risk. Measurable goals used to verify the predicted residual risk of a Safety Performance Targets. hazard. They should quantifiably define the predicted residual risk. The documented organizational policy that defines management’s commitment, Safety Policy. responsibility, and accountability for safety. Safety Policy identifies and assigns responsibilities to key safety personnel. 9.1_SMSM_201904 11 7 Originally published April 201 9 Uncontrolled copy when downloaded

125 Section 9 Definitions and Acronyms Safety Promotion. The communication and distribution of information to improve the safety culture and support the integration and continuous improvement of the SMS within ATO. Safety Promotion allows ATO to share successes and lessons learned. Safety Requirement. A planned or proposed means to reduce a hazard’s causes or effects. Certification that the safety requirements can and will be Safety Requirement Approval. implemented. ). Safety Risk Management (SRM • FAA Definition. A process within the SMS composed of describing the system; identifying the hazards; and analyzing, assessing, and controlling risk. • ATO Definition. The processes and practices used to assess safety risk within the NAS, document NAS changes, and define strategies for monitoring the safety risk of the NAS. SRM complements Safety Assurance. SRM Document. A documented safety analysis for a proposed NAS change or an existing safety . It documents the evidence to support whether or not the proposed NAS change / issue is mitigated to an from a safety risk perspective. existing safety issue level acceptable that was developed properly; Indication that the SRM document Approval. SRM Document hazards were systematically looked for and identified if applicable; and if a hazard was identified valid safety or safety risk was negatively impacted, that: 1) risk was appropriately assigned, 2) requirements were proposed, and an effective implementation and monitoring plan was prepared. SRM document approval does not constitute acceptance of the risk associated with the NAS change or approval to implement the NAS change. Panel. SRM A diverse group of representatives, stakeholders, and subject matter experts from the various organizations affected by the NAS change. They conduct an objective safety analysis and pr ovide findings and recommendations to decision-makers in an SRM d ocument. SRM Panel Co-Facilitator. A person who shares responsibilities with the SRM facilitator in supporting the SRM panel. Panel Facilitator. SRM A trained expert on the SRM process who moderates the deliberations of the SRM panel members from a neutral position. He or she captures the decisions of the and remains panel members, mediates any disagreements, documents any dissenting opinions, neutral throughout the process without advocating for a specific outcome. The facilitator/co -facilitator (or his or her designee) may write the safety document describing the safety findings of the SRM panel meeting. SRM Panel Member . An SRM panel member is a stakeholder who represents the program, potentially affected by the safety risk and/or potential facility, organization, or constituency safety requirements associated with the NAS change and/or identified existing safety risk. The SRM panel members are selected at the discretion of the change proponent and/or panel facilitator. SRM Panel Observer. An SRM panel observer is someone attempting to gain a better observer understanding of the SRM process, not the specific NAS change being assessed. An is not an active member of the SRM panel meeting and does not provide input during the anel observers are permitted at the discretion of the change proponent. deliberations. SRM p 9.1_SMSM_201904 11 8 Originally published April 201 9 Uncontrolled copy when downloaded

126 Section 9 and Acronyms Definitions Any person trained on ATO SMS policy that uses any ATO process to Practitioner. SRM identify safety hazards, evaluate safety risk, and/or recommend activities that can affect safety of the provision of air traffic management and/or communication, navigation, and surveillance services. Safety Risk Tracking. A closed-loop means of ensuring that the requirements and mitigations associated with each hazard that has associated medium or high risk are implemented. Risk tracking is the process of defining safety requirements, verifying implementation, and reassessing the risk to make sure the hazard meets its risk level requirement before being accepted. Severity. The consequence or impact of a hazard’s effect or outcome in terms of degree of loss or harm. The failure of an item that would result in the failure of the system and is Single Point Failure. not compensated for by redundancy or an alternative operational procedure. The plan that specifies the activities required for SMS Continuous Improvement Plan. individual ATO Service Units to allocate sufficient resources toward the integration and maturation of ATO SMS. Any real or potential origin of system failure, including equipment, Source (of a hazard). operating environment, human factors, human-machine interface, procedures, and external services. that is affected by or is in some way accountable for the A group or individual Stakeholder. outcome of a undertaking; an interested party having a right, share, or claim in a product safety or service, or in its success in possessing qualities that meet that party’s needs and/or expectations. A technical expert on the NAS change, hardware or software Subject Matter Expert (SME). gency system, or proposed solution undergoing safety assessment. An SME is typically an a employee; however, when the a gency does not have the expertise in-house, a vendor or industry representative may be invited to the SRM p anel as an SME. System. An integrated set of constituent elements that are combined in an operational or support environment to accomplish a defined objective. These elements include people, hardware, software, firmware, information, procedures, facilities, services, and other support facets. System State. An expression of the various conditions, characterized by quantities or qualities, in which a system can exist. The continued process of documenting the results of monitoring activities and the Tracking. change’s safety effect on the NAS. A high-risk hazard or a combination of medium/low risks Unacceptable Level of Safety Risk. that collectively increase risk to a high level. Worst Credible Effect. The most unfavorable, yet believable and possible, condition given the system state. 9.1_SMSM_201904 11 9 Originally published April 201 9 Uncontrolled copy when downloaded

127 Section 9 Definitions and Acronyms 9.2 Acronyms ADS -B Automatic Dependent Surveillance – Broadcast Management Services AJG AJI Safety and Technical Training AMASS Airport Movement Area Safety System Acquisition Management System AMS Air Traffic Safety Oversight Service AOV Air Route Surveillance Radar ARSR ARTCC Air Route Traffic Control Center ARTS Automated Radar Terminal System Airport Surface Detection Equipment ASDE ASR Airport Surveillance Radar ATC Air Traffic Control ATCRBS Air Traffic Control Radar Beacon System Air Traffic Management Air Traffic Manager / ATM ATO Air Traffic Organization Air Traffic Organization Safety Guidance ATO-SG Air Traffic Safety Action Program ATSAP Corrective Action Plan CAP CAT Category COMM Communications COO Chief Operating Offi cer CSA Comparative Safety Assessment FAA Federal Aviation Administration HAW Hazard Analysis Worksheet HMI Hazardously Misleading Information ICAO International Civil Aviation Organization IMC Instrument Meteorological Conditions Independent Operational Assessment IOA Line of Business LOB Mode Select Beacon System MODES NAS National Airspace System NATCA National Air Traffic Controllers Association Navigation NAV OCS Obstacle Clearance Surface OHA Operational Hazard Assessment OSA Operational Safety Assessment Preliminary Hazard List PHL POC Point of Contact PRD Program Requirements Document Resolution Advisory RA 20 9.2_SMSM_201904 1 Originally published April 201 9 Uncontrolled copy when downloaded

128 Section 9 Definitions and Acronyms Subject Matter Expert SME SMS Safety Management System SMTS Safety Management Tracking System ar Safety Oversight Circul SOC Safety Risk Management SRM Safety Risk Management Guidance for System Acquisitions SRMGSA STARS Standard Terminal Automation Replacement System T-SAP Technical Operations Safety Action Program Traffic Alert and Collision Avoidance System TCAS TIS Traffic Information Services Terminal Radar Approach Control TRACON Visual Meteorological Conditions VMC lateration Wide Area Multi WAM 1 1 2 9.2_SMSM_201904 Originally published April 201 9 Uncontrolled copy when downloaded

Related documents

JP 3 30, Command and Control of Joint Air Operations

JP 3 30, Command and Control of Joint Air Operations

Joint Publication 3-30 Command and Control of Joint Air Operations 10 February 2014

More info »
DODF 082 03 05 2019 INTEGRA

DODF 082 03 05 2019 INTEGRA

o - 82 ANO XLVIII EDIÇÃO N BRASÍLIA - DF, SEXTA-FEIRA, 3 DE MAIO DE 2019 PODER EXECUTIVO SUMÁRIO SEÇÃO II SEÇÃO III SEÇÃO I PÁG. PÁG. PÁG. Poder Legislativo ... 44 . 1 DECRETO Nº 39.797, DE 02 DE MAIO...

More info »
JO 7210.3Y   Facility Operation and Administration

JO 7210.3Y Facility Operation and Administration

ORDER Y JO 7210.3 Air Traffic Organization Policy Effective Date: April 3, 2014 Facility Operation and Administration SUBJ: This order provides direction and guidance for the day- to-day operation of ...

More info »
Top100FictionK12

Top100FictionK12

Top 100 Fiction Books Read by ATOS Book Level Range With the creation and adoption of the Common Core State Standards (CCSS), the notion of text complexity— the level of challenge presented by a given...

More info »
Message Aggregation

Message Aggregation

Message Aggregation Danny Auble (SchedMD), Matthieu Hautreux (CEA) Yiannis Georgiou, Martin Perry (Bull) Slurm User Group 2015 Copyright 2015 SchedMD LLC Copyright 2015 Atos Copyright 2015 CEA

More info »
ADS Chapter 545   Information Systems Security

ADS Chapter 545 Information Systems Security

ADS Chapter 545 Information Systems Security Partial 07/02/2018 Revision Date: Responsible Office: M/CIO/IA File Name: 545_ 070218

More info »
2019 05 02 bg083

2019 05 02 bg083

GOVERNO DO ESTADO DO PARÁ SECRETARIA DE ESTADO DE SEGURANÇA PÚBLICA E DEFESA SOCIAL POLÍCIA MILITAR DO PARÁ AJUDÂNCIA GERAL BOLETIM GERAL Nº 083 02 DE MAIO DE 2019 Para conhecimento dos Órgãos subordi...

More info »
Generations Personal Super and Personal Pension Additional information booklet

Generations Personal Super and Personal Pension Additional information booklet

Generations® Pension and Personal Super Personal Additional booklet information Throughout this booklet Contents References to: To be read as: can 1. Wha of contributions t type 2 be made? Inrespectof...

More info »
ayout 1

ayout 1

0465039146-FM:FM 12/5/06 12:25 AM Page i C O D E

More info »
MOD SNC CODIGO DE CONDUTA ETICA DO BRB BANCO DE BRASILIA A.GOV.1.007.0004

MOD SNC CODIGO DE CONDUTA ETICA DO BRB BANCO DE BRASILIA A.GOV.1.007.0004

#10 Interna Código A.GOV.1.007/0004 Responsável SUGEP/GEDEP CÓDIGO DE CONDUTA ÉTICA DO BRB – BANCO DE BRASÍLIA Vigência 18/10/2018 - 17/10/2020 / 18 Página 1 TÍTULO: CÓDIGO DE CONDUTA ÉTICA DO BRB – B...

More info »
spr86434n71121

spr86434n71121

Instructions super fund members for Deduction for personal super contributions Notice of intent to claim or vary a deduction for How to complete your personal super contributions This form should only...

More info »
06 iran strategy

06 iran strategy

BROOKINGS 1775 Massachusetts Ave., NW A N A L Y S I S P A P E R Washington, D.C. 20036-2103 www.brookings.edu N u m b e r 2 0 , J u n e 2 0 0 9 WHICH PATH TO PERSIA? Options for a New American Strateg...

More info »
Microsoft Word   belief.doc

Microsoft Word belief.doc

istemology of Phenomenal Be lief Th e Content and Ep Davi d J. Chalmers am hy Progr Philosop Research School of Social Sciences Nation al University Australian 1 Introd uction riences and beliefs are ...

More info »
scr 2012 kozinski nguyen

scr 2012 kozinski nguyen

Has Technology Killed the Fourth Amendment? Alex Kozinski* and Eric S. Nguyen** We’ve been trying to protect our privacy ever since Adam went off looking for a fig leaf. But, according to conventional...

More info »
Notice of intent to claim or vary a deduction for personal super contributions

Notice of intent to claim or vary a deduction for personal super contributions

Notice of intent to claim or vary a deduction for personal super contributions instructions We have provided some information below to assist you Will I be eligible for a deduction? with the completio...

More info »
Microsoft Word   IT Modernization Report FINAL to StaffSec.docx

Microsoft Word IT Modernization Report FINAL to StaffSec.docx

2017 Report to the President on Federal IT Modernization

More info »
7340.2H  Bsc dtd 3 29 18

7340.2H Bsc dtd 3 29 18

ORDER JO 7340.2H Air Traffic Organization Policy Effective Date: March 29, 2018 Contractions SUBJ: contractions used by ed word and phrase This handbook contains the approv personnel of the Federal Av...

More info »
SM5100B AT Command Set

SM5100B AT Command Set

SM5100B-D AT Command SM5100B-D AT Command Shanghai Sendtrue Technologies Co.,Ltd Page 1 of 141

More info »
Systems Thinking Tools: A User's Guide

Systems Thinking Tools: A User's Guide

TOOLBOX PR INT S ER IES RE SYSTEMS THINKINGTOOLS  AUser’sReferenceGuide  DANIELH.KIM BY

More info »