Can believing in SMS make us more Vulnerable?

When the FAA embraced Safety Management System (SMS), we all joined in with an understanding of shared responsibility, we did not consider social drivers that would ultimately result in drift. I was one of the eager adopters and brought SMS to the US Forest Service aviation community. We attended classes given by the FAA and quickly included SMS in our arsenal of defenses. We even required contract aviation companies to develop their own SMS programs. We considered the risk management processes to be a principle defensive barrier and it worked, to a point. SMS set the stage for drift by requiring the organizations who provided services to be their own watchdogs within their prescribed SMS programs.  

The reliance on SMS and similar risk management systems may, unintentionally, contribute to an overall system vulnerability. 

When the FAA provided us with the basic tenants of SMS, they let us know that we could rely on our contractors to follow the SMS processes as outlined in their plan and that all we had to do is verify that they had complied with their SMS documentation. We moved from hands-on inspection of aircraft and pilots, to inspecting SMS documentation. As long as the relationship between the inspectors and the companies remained close, the system seemed to work. We weren’t thinking about the relationship as a possible vulnerability, rather it was the basis for open self-reporting and ultimately the promise of compliance. 

This makes sense when you consider that organizations cannot afford to have accidents. Correspondingly, the regulator or inspector, believes that the work has been done correctly, because of the alignment of goals and processes described in the SMS. After all, the inspector/regulator always has the option of verification through hands-on inspection, should they want to do so. Essentially, the entire aviation community was involved in creating safety, so what could go wrong?

You might say that efficiency began to trump thoroughness and what seems to be rather predictable family of shortcuts emerged. Shortcuts in themselves do not equate to a lack of safety in the system; however, they can contribute to reduced margins. Aviation safety is built on embedded margins. Margins are created in four ways: engineering design, system design, regulation/guidance and social design. 

Engineering design creates a structure that is stronger than required for the application. System design limits the cycles and stresses to a set amount, which then requires maintenance or inspection. Regulation/guidance place operational limits that are below design limits to add to the margin. Finally, social design is created through programs like Crew Resource Management, which helps operators to recognize anomalies and hazards, make sense of this information, learn in the moment and then to devise innovations to meet the challenges. This last mode is called human performance. All these modalities have variability and that variability describes the operational envelope. When boundaries are reached the system will fail. The problem is that the boundaries are fuzzy and complex and often go unseen by those closest to the risks.

SMS was not designed to consider the importance of human performance variability. One glaring aspect of human variability is the normalization of risk. Risk is normalized when people are exposed to hazards and nothing bad happens. The result can be an erosion of compliance with guidance (drift). It is important to recognize that normalization is a common human attribute that we all experience. 

The longer a system remains safe, the more we, humans, come to believe that our defensive systems are working. Normalization of risk happens when we no longer recognize or give value to the risks inherent in our operations, instead, we become accustomed to operating with them. We begin to believe in our own system of controls, specifically that we have mitigated, controlled, removed or transferred the risks and we forget the risks that we have accepted as necessary to complete our goals.

We often demand risk assessment and management to force margin back into the system. However, risk assessments are commonly subjective, or really hard to do. “The human mind has difficulty coping with complicated probabilistic relationships, so people tend to employ simple rules of thumb that reduce the burden of processing such information. In processing information of uncertain accuracy or reliability, tends to result in simple yes or no decisions” (Amos Tversky and Daniel Kahneman, “Judgment under Uncertainty: Heuristics and Biases,” Science, 27 September 1974, Vol. 185, pp. 1124-1131).The natural tendency to oversimplify makes us more vulnerable, but that vulnerability lives under the surface in seemingly safe systems.

Goal conflicts add to the drift and they begin to emerge over time. The very idea of making processes leaner, ultimately creates space for the exploitation of the system. The desire to become more efficient, leaner and more profitable begins to erode the margins we carefully constructed in our systems. Quite literally everyone who benefits from the trust relationship that exists between the regulator and the manufacturer can become complicit in drift. In the airline industry this includes passengers who demand on-time operation and discount fares; demands for high performance engines; airlines; the regulator; and even the national GDP, all of which unintentionally place pressure on the system to become more efficient. One glaring example of this drift likely contributed to 737 Max catastrophic crashes.

The presumption of safety is prevalent in many stories that end in catastrophic outcomes. One big question is, ‘Would stricter oversight make a difference?’ It is likely that SMS is structured in such a way that many risks will remain unseen by those closest to the work and will therefore continue to be normalized. Risk that is not recognized cannot be controlled, mitigated, transferred, managed or avoided. Socially many risks are simply accepted and rationalized. 

Risk awareness and management is a messy and complex issue that defies simplification. High Reliability Organizing points to the need for a pre-occupation with failure and a reluctance to simplify. One answer may be found in the creation of interdisciplinary approaches to these complex issues. Although more time consuming, the idea of approaching problems from multiple perspectives is appealing. Certainly, adding social psychologists to the SMS process would be helpful in the identification of the human contributions to drift.

No company produces safety as a product. So, it is unlikely that we will remove all goal conflicts. Perhaps the best we can do is to actively create margin systemically. This can be done through design, social recognition and communication of risk. We should focus on developing the capacity to become aware of internal and external pressures, develop our ability to recognize anomalies and hazards, and more importantly, create the willingness to discuss the insidious risks inherent in all human systems.