A critical appraisal of the key deficiencies in risk workshop methodology that may lead to them producing of misleading or useless data

Tuesday 2 December 2008

The risk workshop methodology is deeply entrenched in the UK risk management process and has been for many years, nowhere more so than in project risk and safety risk. If you examine the international use of risk workshops you will note that they are far less and even minimally used as a means of eliciting risk knowledge.

An examination of risk British literature over the years has not revealed any critical analysis of the performance of risk workshops in delivering accurate risk quantification (an examination of major UK projects from infrastructure to IT will demonstrate the gross cost, performance and schedule deficiencies).

Careers are built on facilitation of risk workshops and consultancies charge substantial fees for undertaking them. I have sat in on many risk workshops and discussed the results with many risk managers who expressed concern and dissatisfaction with the results.

A critical appraisal is long overdue on the effectiveness of the risk workshop process and to this effect I drew up the list of points given in the 'blog' post. All these points are logically obvious from allied research in the socio-psychological fields. One particular area I haven't addressed is that of "Successful sociopathy" which has significant import to risk analysis e.g. Enron, and may be the contributing factor in the 'over-confidence' identified by Bent Flyvberg (Megaprojects and Risk) also cf. Network Rail West Coast Mainline upgrade, NHS Data base, Battlespace, CSA, Olympic Games 2012 etc. etc..

A number of risk managers have suggested to me that the camaraderie and "group hug" are important aspects of the risk workshop. It is my contention that this is total nonsense. If you want to build team spirit go to the pub or go paint-balling. A risk workshop has only one function and that is to deliver identification of risks and accurate estimates of their probability and impact. If it fails on this single measure it was a waste of time and money.

The solution I recommended but am not locked into (if anyone has a better solution let me know) is that of a soft systems methodology known as SODA, there are a wide variety of problem structuring methods for complexity, ambiguity and uncertainty (see Rational Analysis for a Problematic World, Ed. Jonathan Roeshead, Wiley). Some of these were developed by engineers in the aircraft industry for coping with complex engineering problems such as Concorde but have found applications in wide and disparate fields.

With regard to issues of consensus and 'buy-in: what exactly are these issues. In the first instance the purpose of risk analysis is to produce an accurate qualitative or quantitative assessment of the risks (a calculus of possibilities or probabilities) for management to make decisions under uncertainty. I contend that attendees of risk workshops may be keen to reach consensus for a wide variety of reasons that may have nothing to do with the importance of the risk assessments and for this reason the risk assessment process fails. Central to "buy-in" is the concept of sharing of responsibility and all that that entails - I addressed this point in the 'blog''.

I would be very happy to receive a point by point critique of my posting and particularly on how facilitators account for or overcome the issues raised.

N.B. Successful Sociopathy:- “Successful Sociopathy and corporations:- Robertson et al (Medical J. of Australia 1996) describe a pathological condition which they call "successful sociopathy". They suggest that many successful businessmen and entrepreneurs show some of the features of this condition.

More on this can be found on Brian Martin's web site at:- http://www.uow.edu.au/arts/sts/bmartin/dissent/documents/health/sociopathy.html#Background

Gavin Lawrence

____________________________________________________

No comments: