Can believing in SMS make us more Vulnerable?

When the FAA embraced Safety Management System (SMS), we all joined in with an understanding of shared responsibility, we did not consider social drivers that would ultimately result in drift. I was one of the eager adopters and brought SMS to the US Forest Service aviation community. We attended classes given by the FAA and quickly included SMS in our arsenal of defenses. We even required contract aviation companies to develop their own SMS programs. We considered the risk management processes to be a principle defensive barrier and it worked, to a point. SMS set the stage for drift by requiring the organizations who provided services to be their own watchdogs within their prescribed SMS programs.  

The reliance on SMS and similar risk management systems may, unintentionally, contribute to an overall system vulnerability. 

When the FAA provided us with the basic tenants of SMS, they let us know that we could rely on our contractors to follow the SMS processes as outlined in their plan and that all we had to do is verify that they had complied with their SMS documentation. We moved from hands-on inspection of aircraft and pilots, to inspecting SMS documentation. As long as the relationship between the inspectors and the companies remained close, the system seemed to work. We weren’t thinking about the relationship as a possible vulnerability, rather it was the basis for open self-reporting and ultimately the promise of compliance. 

This makes sense when you consider that organizations cannot afford to have accidents. Correspondingly, the regulator or inspector, believes that the work has been done correctly, because of the alignment of goals and processes described in the SMS. After all, the inspector/regulator always has the option of verification through hands-on inspection, should they want to do so. Essentially, the entire aviation community was involved in creating safety, so what could go wrong?

You might say that efficiency began to trump thoroughness and what seems to be rather predictable family of shortcuts emerged. Shortcuts in themselves do not equate to a lack of safety in the system; however, they can contribute to reduced margins. Aviation safety is built on embedded margins. Margins are created in four ways: engineering design, system design, regulation/guidance and social design. 

Engineering design creates a structure that is stronger than required for the application. System design limits the cycles and stresses to a set amount, which then requires maintenance or inspection. Regulation/guidance place operational limits that are below design limits to add to the margin. Finally, social design is created through programs like Crew Resource Management, which helps operators to recognize anomalies and hazards, make sense of this information, learn in the moment and then to devise innovations to meet the challenges. This last mode is called human performance. All these modalities have variability and that variability describes the operational envelope. When boundaries are reached the system will fail. The problem is that the boundaries are fuzzy and complex and often go unseen by those closest to the risks.

SMS was not designed to consider the importance of human performance variability. One glaring aspect of human variability is the normalization of risk. Risk is normalized when people are exposed to hazards and nothing bad happens. The result can be an erosion of compliance with guidance (drift). It is important to recognize that normalization is a common human attribute that we all experience. 

The longer a system remains safe, the more we, humans, come to believe that our defensive systems are working. Normalization of risk happens when we no longer recognize or give value to the risks inherent in our operations, instead, we become accustomed to operating with them. We begin to believe in our own system of controls, specifically that we have mitigated, controlled, removed or transferred the risks and we forget the risks that we have accepted as necessary to complete our goals.

We often demand risk assessment and management to force margin back into the system. However, risk assessments are commonly subjective, or really hard to do. “The human mind has difficulty coping with complicated probabilistic relationships, so people tend to employ simple rules of thumb that reduce the burden of processing such information. In processing information of uncertain accuracy or reliability, tends to result in simple yes or no decisions” (Amos Tversky and Daniel Kahneman, “Judgment under Uncertainty: Heuristics and Biases,” Science, 27 September 1974, Vol. 185, pp. 1124-1131).The natural tendency to oversimplify makes us more vulnerable, but that vulnerability lives under the surface in seemingly safe systems.

Goal conflicts add to the drift and they begin to emerge over time. The very idea of making processes leaner, ultimately creates space for the exploitation of the system. The desire to become more efficient, leaner and more profitable begins to erode the margins we carefully constructed in our systems. Quite literally everyone who benefits from the trust relationship that exists between the regulator and the manufacturer can become complicit in drift. In the airline industry this includes passengers who demand on-time operation and discount fares; demands for high performance engines; airlines; the regulator; and even the national GDP, all of which unintentionally place pressure on the system to become more efficient. One glaring example of this drift likely contributed to 737 Max catastrophic crashes.

The presumption of safety is prevalent in many stories that end in catastrophic outcomes. One big question is, ‘Would stricter oversight make a difference?’ It is likely that SMS is structured in such a way that many risks will remain unseen by those closest to the work and will therefore continue to be normalized. Risk that is not recognized cannot be controlled, mitigated, transferred, managed or avoided. Socially many risks are simply accepted and rationalized. 

Risk awareness and management is a messy and complex issue that defies simplification. High Reliability Organizing points to the need for a pre-occupation with failure and a reluctance to simplify. One answer may be found in the creation of interdisciplinary approaches to these complex issues. Although more time consuming, the idea of approaching problems from multiple perspectives is appealing. Certainly, adding social psychologists to the SMS process would be helpful in the identification of the human contributions to drift.

No company produces safety as a product. So, it is unlikely that we will remove all goal conflicts. Perhaps the best we can do is to actively create margin systemically. This can be done through design, social recognition and communication of risk. We should focus on developing the capacity to become aware of internal and external pressures, develop our ability to recognize anomalies and hazards, and more importantly, create the willingness to discuss the insidious risks inherent in all human systems.

7 Comments

  1. Ron GanttRon Gantt Reply

    Great post Ivan. I’m glad you’re discussing margins, although I would say that rather than these things eroding margins, they are shifting them. When we make our processes more lean, we are not eliminating things, but moving them and transforming them into margins in other areas. A shortcut turns thoroughness into efficiency in one task, which then gives us margins to be more thorough in another task.

    I think this is important to note because it forces us to see the goal of risk assessment not merely as ensuring margins aren’t eliminated, but rather that they are allocated appropriately for the context given the goals we are trading off.

    1. Ivan PupulidyIvan Pupulidy Post author Reply

      Thank you Ron – our resident expert on all things Drift. I would have to say when we lean processes you are spot on; however, leaning manufacturing, supply and logistics may be another story. Also, I don’t think that I have considered process leaning to be shortcutting – that is a nice connection.

  2. dickrobinson1 Reply

    Really interesting post Ivan. On the theme of risks becoming normalised when nothing untoward happens for a long period of time, I recall Trevor Kletz’s warning (Lessons from Disaster, Institute of Chemical Engineers), ironically almost 30 years ago, about the necessity of not only learning lessons from the past, but also to find a way for organisations to remember them. It seems as though workplaces continue to find new and imaginative ways to defeat the effectiveness of controls, but as your article clearly articulates, there are steps that can be taken to recognise and mitigate drift.

    1. Ivan PupulidyIvan Pupulidy Post author Reply

      Hi Dick,
      I will admit that I have long struggled with the concept of keeping learning relevant and in the front of our minds. A big problem with this has been that we move to other issues, concerns and opportunity very rapidly in our complex work environments. Adding to this is the normal organizational response to create more guidance, regulation, training and procedure. I have faced many procedures that did not make sense until I understood the history of their development.
      The answer that began to emerge for us was that we had to develop the capacity of our workforce to adapt through sensemaking, real-time learning and improvisation. This was a move away from prescription and toward doctrine and field learning in the moment.

  3. Jim Whiting Reply

    Congratulations – A great article on the role of an SMS
    Most of the issues and your concerns re an SMS, however, are related to the implementation of an SMS not the SMS itself
    EXTRACT QUOTE 1: When the FAA provided us with the basic tenants of SMS, they let us know that we could rely on our contractors to follow the SMS processes as outlined in their plan and that all we had to do is verify that they had complied with their SMS documentation. We moved from hands-on inspection of aircraft and pilots, to inspecting SMS documentation. END OF QUOTE 1
    COMMENT 1 Desktop Documentation-only Audits without Physical Verify & Validate Inspections give very little assurance of intended risk control and hence are always weak.
    EXTRACT QUOTE 2: SMS was not designed to consider the importance of human performance variability. END OF QUOTE 2
    COMMENT 2 Most SMS I have had contact with from AS 4804, AS4801, BS8800, OHSAS 18000 before ISO45001, always proposed risk based thinking even if they didn’t call it that and hence more importantly steered one’s thinking to consider the most significant and common risk factor for any risk, namely human performance variability.

    1. Ivan PupulidyIvan Pupulidy Post author Reply

      Jim,
      Thank you so much for your thoughtful response. I agree that implementation of any regulation has to be considered. To your first point – As Erik Hollnagel points out, if the regulator or inspector is short staffed, the result will be an increase in efficiency and a corresponding decrease in thoroughness. Your second point is interesting – I teach for a university in an engineering masters degree program. The over reliance on risk-based decision aids is profound and with it comes a reliance, almost a religious belief, in the processes. What is forgotten is that in most cases safety research does not supply enough data to support the forecasts and then we have to consider the complex nature of our work and how complex systems deliver the unexpected. These are aspects that are poorly understood by many and not well represented in the SMS programs I have seen.

  4. Paul OORTMAN Reply

    Highly interesting article. I too was an early enthusiast of SMS’s and made a (intellectual and practical) journey over the years and have come to similar conclusions. SMS, or broader: all management systems are constructs, tools to achieve a goal. As all tools, they can be used rightly or wrongly, effectively or wasteful. I welcome international standards for MS’s because they standardise and harmonize the constructs and this greatly improves efficiency and learning across organizations.

    Thanks for sharing your thoughts.

Leave a Reply to Paul OORTMAN Cancel reply

Your email address will not be published. Required fields are marked *