The art of measuring nothing: The paradox of measuring safety in a changing civil aviation industry using traditional safety metrics

Abstract

Measuring safety as an outcome variable within the ultra-safe civil aviation industry during periods of deliberate organizational change is a difficult, and often fruitless, task. Anticipating eroding safety processes, based on measuring nothing happening over time, does not adequately capture the true state of an evolving safe system, and this is particularly relevant for leaders and managers in a civil aviation industry responsible for maintaining and improving ultra-safe performance while simultaneously managing demanding strategic business goals. In this paper, I will look at the difficulties of measuring safety as an outcome measure in high reliability organizations (HROs) using the traditional measures of incident and accident reporting during periods of deliberate organizational change inspired by the desults from a three-year longitudinal case study of the Norwegian Air Navigation Services provider - Avinor. I will first review the current safety literature relating to Safety Management Systems (SMSs) used in the civil aviation industry. I will then propose a more holistic model that shifts the focus from the traditional safety monitoring mechanisms of risk analysis and trial and error learning, to the natural interactivity within socio-technical systems as found in High Reliability Organizations. And finally, I will present a summary of the empirical results of an alternate methodology for measuring perceived changes in safety at the operational level as leading indicators of evolving safety at at the organizational level.