A client approached us about a recent incident in a process involving three organizations. They asked for an independent investigation into the incident. During the initial meeting it became clear that there was some relevant history to the process in which the incident happened.
Extending the scope of the investigation
After discussing this history and using contemporary knowledge in Human Factors and System Safety, it became apparent to us that we needed a different approach to analyze this complex socio-technical system. We suggested to the client that we would like to focus on the process as a whole instead of just focusing on the incidents. So we would look at daily operational practice including the incidents in our investigation. They agreed with this approach and we started the project.
Execution of the investigation
It was our intention to use the Functional Resonance Analysis Method (FRAM) for this project to map the processes and the interactions that were present. The execution thus focused on the normal workings of the operational process and other relevant processes (work as done). The information gathering was done in three ways:
- Individual interviews with several people involved in the process (operators, planners, work preparators, etc.);
- Driving/walking along with operators to see and feel the way the work gets done (both ways aimed at looking for the local rationality of the people involved);
- Group sessions with operational personnel. Three sessions in total were executed with different emphasis:
- One to gather and share information and to discuss and verify the FRAM model of the processes (sharing of perspectives);
- One to present the findings and to talk about possible solutions;
- One to analyze the solutions in more details to come up with an action plan.
The information provided a rich picture of work as done and enabled the creation of the FRAM models and an analysis of the processes involved.
Use of systemic models
During the investigation it became apparent that the Ten system principles as developed by Eurocontrol would help us to go even further ‘up and out’ (Dekker) in the investigation. The investigation eventually resulted in the following uses of the methods and models:
- A FRAM model for the two operational processes involved (it involved two separate locations with their own characteristics and thus their own model);
- A FRAM model of the planning process. The planning process had a large influence on the execution and was too large to include in the other two models;
- The ten principles for system thinking were used to elaborate on the findings in a different and complementary way.
The theoretical underpinnings of FRAM and the system principles were used as the perspective to make sense of what we saw during the information gathering phase. Below are some examples of normal, daily and necessary adjustments (performance variability) in practice:
- Formal software registration of activities becomes too cumbersome during high workload situations, the operators then switched to (unofficial) paper registration. So the software became a constraint and adjustment was necessary to facilitate the flow of the process;
- Working according to plan is a spear point for the organizations involved but in some situations, operators deviated from this to ensure that they accommodate small delays in an efficient way. In that case they didn’t fulfill the requirement of working to their (personal) plan but they did fulfill the requirement of keeping up with the overall schedule. So in essence they used their insight in the process and situation to accommodate small delays in a resilient manner;
- A safety constraint related to the maximum number of parallel activities was applicable for the main activities. Sometimes, especially during high workload situations, this constraint was stretched to accommodate all activities. From an organizational perspective, the overall goal of ensuring flow to continue was fulfilled but the margin toward the (unknown) performance boundary was probably and, in some cases, definitely reduced.
The investigation resulted in several systemic conclusions.
- Work as imagined varied for different actors involved and a lot of operational agreements were not supported by the workings of the system, which undermined the effectiveness. This also supported the view that work as done was the basis of the investigation;
- Daily practice is characterized by interdependencies and complexity which have a large impact on daily practice, especially during high workload situations;
- Daily practice is also characterized by inherently present goal conflicts (i.e. punctuality and safety/quality);
- Local optimization is done on a day to day basis to accommodate small and larger changes and to solve goal conflicts;
- Punctuality and working to plan are organizational spear points but last minute changes to accommodate for delays and malfunctions ask for flexibility and room to maneuver. This conflict is always present and has to be resolved at the operational level.
The investigation also revealed that most of the underlying systemic problems were already present before the relocation of part of the personnel but they could be dealt with locally before something happened that needed recording. These local adjustments were normal work for the operational personnel and never resulted in a bad outcome. After the organizational change, these locally efficient adjustments were no longer possible and that changed the whole system and also its outcomes.
The road to improvement
The presentation to the operational personnel was done in a group session, which was also aimed at discussing the improvement measures. The investigation itself already revealed several opportunities for improvements. And those were the starting point for the session. The session resulted in an elaborate set of measures with the following topics.
- Improvement of the communication systems;
- Temporarily limiting the maximum number of parallel activities (making the safety constraint more strict);
- Temporary local supervision;
- Several planning related measures aimed at increasing the flexibility especially during peak moments;
- Alignment of understanding of rules, agreements and procedures between and within the organizations;
- Realignment of responsibilities of two specific roles related to the day to day adjustment of the planning;
- Training and education;
- Increasing the amount of automation in the system.
After finishing the full report including improvement measures, we were also asked to assist with the implementation. Senior management agreed with the full set of recommendations and the improvement process is currently underway.
This project provided for a very interesting case to make use of systems thinking and systemic models. Below are general reflections on the execution of this project and the use of theory in practice.
- Analysis of complex socio-technical systems can benefit from a combination of perspectives on Human Factors and System Safety. The use of multiple models and methods can provide a rich systemic insight into the system under investigation;
- The use of different labels of the theories (i.e. Safety I and II, old and new view, etc.) can be interpreted as an OR situation instead of an AND situation. Instead, the dialogue should focus on applicability of the underlying assumptions and a combination of perspectives;
- The investigation revealed problem areas, room for improvement as well as strengths of the system. This rich picture translates in a broad set of specific improvement measures that are aligned with all of these outcomes;
- Because we investigated the whole system, we were able to make the interactions between measures explicit in the context of the system so that decisions to continue could be based on that.
Reflections on the use of models
- FRAM is a very good method to facilitate discussion on a process in a systemic way (focused on interactions). The visual representation is a way of communicating the model and facilitating the discussion. The underlying written result of the discussions and the findings is the core of the result;
- FRAM is very helpful to align understandings of work as done, especially when multiple organizations are involved. It also makes the gap between work as imagined and work as done explicit which calibrates different understandings of the process;
- The Ten principles provide a very promising framework to help focus attention during an investigation of a complex socio-technical system. Again the underlying written result is the core of the result.
The approach that we took during this interesting project also highlights the fact that process improvement is not only about safety. Safety is just one of the goals that an organization strives for besides production, quality, environmental requirements, etc. The fact that multiple requirements have to be met at the same time, results in inherently present goal conflicts. The use of systems thinking, local rationality and multiple theories allows you to make these inherently present goal conflicts explicit. And as Todd Conklin also points out in his podcast (safety moment of April 1st), the requirements are dealt with at the operational level and we need to know how they do that to understand why that works so well most of the time.
In the end, I think that using these concepts and models provides perspectives that deals with reality in a very humanistic, systemic and locally rational way. I think that we should aim for that in everything we do!