The Nuances of Risk and Developing the Risk Threshold
A “Deeper Look at the FRAT” is a three part series follow-up to the article, “Is the FRAT Undermining your Safety Initiatives?“. If you haven’t already read the article, you may wish to do so.
The purpose of the FRAT is to quantify and evaluate the compounding risk (loss exposure) associated with the threat of multiple hazards, for a given flight or flight segment.
A risk value on your FRAT is the value, you have determined through the analysis of exposure to a given hazard, in terms of probability of a consequence and the resulting severity thereof.
Risk Analysis and Assessment
Before we discuss risk threshold, lets step back for a moment and talk about the nuances of risk analysis and assessment.
|The risk assessment is a common point of failure for many operators. Most have not identified their organizational risk tolerance.|
The hazard and risk analysis process is beyond the scope of this article. Contact us for a copy of our animated PowerPoint, titled: “Understanding the Risk Analysis.” We will be happy to send you a copy.
The risk analysis studies the potential consequences, of the exposure to a hazard, in an attempt to determine the severity AND the probability of an outcome.
The risk analysis consists of three steps:
- Identify potential consequences of an individual hazard.
- Determine the potential severity of each identified consequence(s).
- Determine the probability of occurrence of each identified consequence(s).
A consequence is the potential outcome or outcomes realized by exposure to a hazard. Exposure to a hazard rarely presents a single potential consequence. There are generally multiple consequences, all which need to be taken into account.
Probability and severity
Determining the probability and severity of a consequence can be somewhat complex. As noted above, hazards typically contain multiple consequences. Extending this concept, each potential consequence may have multiple probability/severity combinations.
The existence and effectiveness of risk controls (policies, procedures, equipment, training, etc.) influence the probability and severity of the consequences.
Risk controls exist in two forms:
- Avoidance Barriers – controls intended to reduce probability of occurrence.
- Recovery Barriers – controls intended to reduce severity of an occurrence in absence or failure of avoidance barriers.
Risk assessment is the process used to determine if the risk is acceptable and in alignment with your company risk strategy. This is the point you either accept or reject risk.
The risk assessment is a common point of failure for many operators. Most have not identified their organizational risk tolerance.
For a corporate flight department, the flight department management’s role in the risk assessment process is providing the probability and severity data for each hazard. Determining the acceptable level of risk should go beyond the flight department and involve the accountable executive, risk management department, and/or other corporate group responsible for managing organizational risks.
Now that we’re armed with the basic understanding of risk analysis and assessment, lets revisit a hazard on the FRAT templates provided by the FAA and IBAC.
|Stopping distance greater than 80% of available runway||5|
|Usable Runway Length < 5000 ft||5|
|Usable Runway Length > 5000 ft but < 5500 ft||3|
In the simplest term, the hazard identified in the table is a short runway. How did the respective analysts determine the risk values? What are the consequences and risks the analysts considered to arrive at their score?
Remembering “Risk of what” is a three-part question:
- What are the potential consequences (outcomes) of operations utilizing a short runway?
- How bad will the consequences be (severity) if they come to fruition?
- How often will the consequences occur (probability)?
What consequences did the analysts consider during operations utilizing a short runway?
A list might include:
- No adverse consequence
- Remain on the runway with adverse consequences (i.e., brake fire, blown tire, et. al.).
What level of severity did the analysts consider? The criterion used to measure severity may have a significant influence on the risk value:
- Actual outcome(s) of past event(s):
- Requires an abundance of data typically not available to corporate and small/mid-size commercial operators.
- Historical data does not consider the application of post occurrence risk controls.
- Actual outcome may cause of the demise of the typical corporate flight department.
- Worst-case scenario:
- Worst-case scenario in aviation mishaps may be multiple fatalities and/or aircraft destroyed.
- May be applicable to specific consequences, such as a high-speed overrun, but not necessarily to a blown tire.
- Most probable outcome:
- Most probable outcome is typically no adverse consequence.
- If it assumed zero accidents is the same as zero risk the analysis would not consider the actual risk the hazard carries.
- Most credible outcome:
- It speaks to the consequence(s) you are trying to avoid.
- Most credible outcome acknowledges the consequence(s) that is of most concern when exposed to the hazard.
Air Safety Group recommends to our clients that they conduct risk analysis around the most credible outcome.
How did the analysts calculate probability? Did they consider:
- Specific A/C type or location
- Each of the potential consequences, individually
- Actual number of occurrences of each consequence (historical)
- Potential for occurrences of each consequence (predictive)
In the analysis process of the short runway, did the analysts consider risk controls? If yes, what risk controls and how was their effectiveness applied to the results?
Up to now, we’ve been talking about the risk values assigned to individual hazards. The risk threshold is the sum of compounding risk values that triggers an administrative action. The action is dependent upon the purpose of the tool within individual organizations.
Consider the following statements closely.
- If a hazard is on your FRAT, it implies that it has been determined to be within your organization’s acceptable level of risk.
- If a score on your FRAT, above a specified value, establishes an unacceptable level of risk, it is reasonable to conclude that a score below that value would establish an acceptable level of risk.
To illustrate the operational implications of using the FRAT as a go/no-go tool, revisit the FRAT templates one last time. Notice the respective risk value remains below the value that would trigger additional management scrutiny. Clearly, if you plan to use the FRAT as a go/no-go tool, it would be prudent to ensure that a score accurately represents significant risk when it is present.
The FAA FRAT
The FAA FRAT template uses a risk threshold of 15 to trigger additional management scrutiny.
|22||Stopping distance greater than 80% of available runway||5|
|34||Crosswinds greater than 15 knots||4|
|35||Runway braking action less than good||5|
|Total Factor Score||14|
The IBAC Template
The IBAC provided template uses a risk threshold of 18 as the management scrutiny threshold.
|35||Sleet – Moderate||3|
|37||Icing – Severe Really?||5|
|38||Surface winds greater than 30 knots||3|
|39||Runway braking action – Nil Really?||5|
Does your organization authorize flight into known severe icing or operations on runways with braking action reported as Nil? If not, why in the world would you put them on your FRAT?!
Setting the risk threshold
The risk threshold is not used to trigger the need to mitigate the risk of individual hazards. Mitigation of hazards has, or should have been, long hence completed.
Upon reaching the risk threshold, you have four strategies to consider:
- Risk Acceptance – Accept the trip with the elevated risk.
- Risk Avoidance – Cancel/reschedule the trip.
- Risk Limitation – Eliminate one or more of the hazards to reduce the risk value.
- Risk Transference – Hand-off the trip to an organization with greater capabilities.
When setting your risk threshold, consider a threshold that requires additional management scrutiny for continued dispatch, well before hitting an unacceptable level of risk. This strategy would allow managers an advanced opportunity to review and/or change details, as appropriate.
As you design your FRAT, keep in mind that it needs to be easy to use in the day-to-day operating environment.
That is not to say its development should be haphazard. Developing an effective FRAT requires effort. Understanding the derivation of the risk values and the organizational tolerance for risk is critical to an effective FRAT.
Look at your FRAT carefully and consider the following:
- Do the risk values represent your organization’s actual tolerance for risk?
- What message is it sending to your operating personnel?
- How will an administrative law judge rule in the event of a regulatory violation?
- What will the plaintiff’s attorney sell to a jury in the aftermath of an accident?
Many organizations lack the time and/ expertise proper development. Don’t be afraid to get a little help from a qualified safety specialist.