Risk Matrices

[Fred Lewis]

In order to quantify risk, Safety Management Systems are fond of using risk matrices. These matrices have no solid statistical foundation and their use will lead to almost certainly false and misleading determinations of risk. In any human endeavour and in aviation in particular, use of faulty information is hazardous.

 

While there are many proponents of risk matrices, Wikipedia provides a discussion of them in which their numerous shortcomings are detailed (http://en.wikipedia.org/wiki/Risk_Matrix). Transport Canada promotes risk matrices. See the aforementioned web site to view their effort.

 

The weakness of the TC method will be emphasized using an empirically determined probability of being involved in a fatal helicopter accident in Canada. The Transportation Safety Board of Canada has published the number of fatal helicopter accidents and the total number of hours flown for the years 1998 to 2007 inclusive (http://www.bst-tsb.gc.ca/eng/stats/aviation/2007/ss07.asp#table_1). From these numbers, the average fatal accident rate per hour for these years can be estimated. It is 1.06599E-05 or 0.0000107.

 

Some things must be said about this statistic. It is determined empirically, or by observation. This is in contrast to drawing a heart from a deck of cards the odds of which can be immediately written down as 0.25 (P(heart) = 0.25). Empirically determined probabilities are estimates, not withstanding the fact in theory a precise probability for involvement in a fatal helicopter accident could be computed. P(fatal accident) for any particular instance is affect by a long list of variables. Some of these are weather, geography, aircraft mechanical integrity, pilot skill and the nature of the operation. Some pilots flying under some circumstances will enjoy a P(fatal accident) lower and in some case much lower than that stated above. Conversely, some pilots in some circumstances will suffer a P(fa) that is much higher.

 

Consider a helicopter pilot embarking on a 5,000 hour career. While P(fa) is 0.0000107 for one hour, in 5,000 hours is considerably greater than that. It is in fact 0.0505 or about 5 in 100. One way of looking at this is that if 100 pilots each flew 5,000 hours, about 5 of them would be involved in a fatal accident.

 

P(fa 5,000 hrs) = 0.0505 which on TC’s scale qualifies as frequent or a probability index of 4. The exposure index is also 4 because the pilot is continuously exposed to the opportunity for the event to occur. The severity index is likewise 4, or catastrophic as fatal injuries are involved.

The product of these factors is 64 which is more than double the number at which TC recommends that the activity be halted. TC has determined that helicopters are too dangerous to fly.

 

Of course they are not too dangerous. It is TC’s methodology which is dangerous. Risk matrices should never be used to determine risk. They are just plain wrong.

 

Will any pilot use statistical methods to determine risk? He will not. If he did, there would be no time left to fly and his head would probably explode.

 

Aviation is more than 100 years old. Just about every conceivable dangerous situation has been experienced by someone, somewhere, sometime. “Learn from the mistakes of others as you will not live long enough to make them all yourself.” is sage advice and a far better way to identify risky practices than some arbitrary, nonsensical device.

 

[helicopterjim]

Aviation is more than 100 years old. Just about every conceivable dangerous situation has been experienced by someone, somewhere, sometime. “Learn from the mistakes of others as you will not live long enough to make them all yourself.” is sage advice and a far better way to identify risky practices than some arbitrary, nonsensical device.

 

Not helicopters but this video sums up the statement above!

 

 

[splitpin]

In order to quantify risk, Safety Management Systems are fond of using risk matrices. These matrices have no solid statistical foundation and their use will lead to almost certainly false and misleading determinations of risk. In any human endeavour and in aviation in particular, use of faulty information is hazardous.

 

While there are many proponents of risk matrices, Wikipedia provides a discussion of them in which their numerous shortcomings are detailed (http://en.wikipedia.org/wiki/Risk_Matrix). Transport Canada promotes risk matrices. See the aforementioned web site to view their effort.

 

The weakness of the TC method will be emphasized using an empirically determined probability of being involved in a fatal helicopter accident in Canada. The Transportation Safety Board of Canada has published the number of fatal helicopter accidents and the total number of hours flown for the years 1998 to 2007 inclusive (http://www.bst-tsb.gc.ca/eng/stats/aviation/2007/ss07.asp#table_1). From these numbers, the average fatal accident rate per hour for these years can be estimated. It is 1.06599E-05 or 0.0000107.

 

Some things must be said about this statistic. It is determined empirically, or by observation. This is in contrast to drawing a heart from a deck of cards the odds of which can be immediately written down as 0.25 (P(heart) = 0.25). Empirically determined probabilities are estimates, not withstanding the fact in theory a precise probability for involvement in a fatal helicopter accident could be computed. P(fatal accident) for any particular instance is affect by a long list of variables. Some of these are weather, geography, aircraft mechanical integrity, pilot skill and the nature of the operation. Some pilots flying under some circumstances will enjoy a P(fatal accident) lower and in some case much lower than that stated above. Conversely, some pilots in some circumstances will suffer a P(fa) that is much higher.

 

Consider a helicopter pilot embarking on a 5,000 hour career. While P(fa) is 0.0000107 for one hour, in 5,000 hours is considerably greater than that. It is in fact 0.0505 or about 5 in 100. One way of looking at this is that if 100 pilots each flew 5,000 hours, about 5 of them would be involved in a fatal accident.

 

P(fa 5,000 hrs) = 0.0505 which on TC’s scale qualifies as frequent or a probability index of 4. The exposure index is also 4 because the pilot is continuously exposed to the opportunity for the event to occur. The severity index is likewise 4, or catastrophic as fatal injuries are involved.

The product of these factors is 64 which is more than double the number at which TC recommends that the activity be halted. TC has determined that helicopters are too dangerous to fly.

 

Of course they are not too dangerous. It is TC’s methodology which is dangerous. Risk matrices should never be used to determine risk. They are just plain wrong.

 

Will any pilot use statistical methods to determine risk? He will not. If he did, there would be no time left to fly and his head would probably explode.

 

Aviation is more than 100 years old. Just about every conceivable dangerous situation has been experienced by someone, somewhere, sometime. “Learn from the mistakes of others as you will not live long enough to make them all yourself.” is sage advice and a far better way to identify risky practices than some arbitrary, nonsensical device.

 

I just Googled "risk matrices". What a bunch of gobblygook crap. I think I just figured out one of the reasons why our industry is so screwed up. How about this for a "risk matrice" If you fu-k up, you die!! Soooo Simple.

[Auto Relight]

What a bunch of gobblygook crap.

 

Couldn't agree more SP.

 

I'm pretty certain this is what it looked like in Rome circa 400AD.... just with flying machines. We are going down a very flawed path, wonder if anyone will actually notice?

 

AR

[twinstar_ca]

risk = probability x consequence.... just saying...

[helicopterjim]

risk = probability x consequence.... just saying...

 

Is that a guarantee?

 

I should be dead now......