Reasons for Risk Workshops failing to deliver accurate quantitative information

In the UK risk workshops play a central role in the elicitation of risks and their probabilities and impacts. Below I give 19 reasons for risk workshops failing to deliver accurate and meaningful results. The list is not exhaustive and the reasons are not mutually exclusive. I would welcome some feedback on how these should be best dealt with.

i. Participants believe that their view is 'out on a limb' from the rest of the group.

ii. Participants may believe that they will be subject to ridicule for expressing an alternative view.

iii. Participants may think that others have already said 'it', or thought about it because it seemed so obvious, and that the idea must have been rejected for "good" reasons.

iv. They may have 'trading agreements' with the others in the group that would be broken if they expressed a view that opposes that of their trading partners - to do so would have consequences for support on other issues.

v. To dissent from the view of the group may put team cohesiveness at risk - threatening established order.

vi. It damages the camaraderie of the team.

vii. Participants may be frightened of reprisals for expressing a particular view that is thought to be counter to the prevailing view of those in power.

viii. May demonstrate weaknesses or knowledge/experience etc., deficiencies of the participant.

ix. Drawing on research in cognitive sciences on pattern recognition and visual perception, the practice of plotting, on a white-board, of probability (y-axis) and £-impact - log scaled (x-axis) may influence the risk assessments vis a vis relative positioning and density of the plot points for reasons such as symmetry and balance.

x. Major infrastructure cost over-runs are regularly in the range of 25 - 600%, Flyvberg identifies over-optimism as the single greatest because of over-estimation; this may flow from the points listed above.

xi. Group assessments tend to give extreme values greater credence i.e. approaching 0 or 1.

xii. Social norms of the group can discourage the extent to which the thinking of each of individuals in the group is used in the group decision making.

xiii. Existing social processes and social relationships encourage only shallow thinking to surface.

xiv. Group decision making often results in higher risk decisions being taken because of the shared responsibility and hence, less liability, than that of the individual decision maker.

xv. There is research to indicate that workshops and group brain-storming sessions do not produce any more creative thinking or better results than that produced by the individuals alone.

xvi. Participant selection is a key issue to be explored further in the context of management v. Front-line executive. Also how expert are the experts?

xvii. Over-emphasis placed on the value and out-put of workshops by managers and decision makers.

xviii. Risk consultants over-emphasise the value of risk workshops because they make money from running them!

XiX. Attendees of risk workshops are keen to achieve a consensus, right or wrong, in order to get back to work that they feel is more important i.e. risk workshops are perceived time wasting etc...

I have presented 18 points to a wide variety of risk academics and risk practitioners around the world and none have disputed the issues raised with regard to the 'risk workshop's' deficiencies. In fact most have offered total agreement that these workshops, no matter how well facilitated, do not deliver the quantitative assessments that are comprehensive and accurate enough. Aside from the personal anecdotal evidence there is more than enough research evidence to draw a conclusion that workshops do not deliver.

Assuming that the risk workshop has attendees that have been appropriately selected, something like a dozen key members of staff are taken out of work, effectively for a day - perhaps representing a loss of US$ 15,000 add to this the facilitators cost, plus everybody’s travel expenses to the workshop – perhaps totalling as much as US$25,000.

I would like to suggest that the most effective approach to resolving the risk workshop deficiencies would be that known as "Cognitve Mapping for Strategic Options and Development Analysis" (SODA). The concept and practice of SODA is extensively described in documents on http://www.banxia.com/ , and In the book, Making Strategy: The Journey of Strategic Management, Eden C and Ackermann F., so I won't dwell on the detail.

The advantage of SODA is multi-fold:- the risk analyst can interview the "experts" individually in their own time, revisit them for additional information. The experts are not, therefore, taken out of their work situation for more than an hour at a time. The individual's risk perceptions are explored in depth without any influence from colleagues. Cognitive maps are prepared of each experts perspectives on risk with regard to the project.

There is no limit to the number of experts that can be interviewed as is the case with attendees of a workshop where 12 is pretty much the maximum.

The benefits:-
1. The individual interview process has removed the points raised in points i to viii and xii.
2. The cognitive mapping process removes issues in points ix and xi.
3. Larger number of interviews reduces selection problems, point xvi.
4. More creative and deeper thinking may be encouraged, points xiii and xv.

I submit that the risk analyst will have achieved far more at this stage using SODA, more efficiently and cost effectively than any risk workshop would have done. Not only will a wider range of problems have been described along with the quantitative assessments but insights in to mitigation strategies and value management issues would be becoming apparent.

If you use the Decision Explorer software package each interviewee's cognitive map can be stored, in addition the interviewee's individual maps can be merged in to one. The aggregation of the interviews cognitive maps is very important:-

Consider the risky events that have been identified and individual assessments of probability P(E) x likely outcome distribution from a wide variety of individuals of varying expertise. How to combine these different risk assessments, I have suggested a method in the Bayesian Statistics Forum; it is also important to check out issues and methodologies cited in Roger Cooke;s book, Experts in Uncertainty (see Recommended Books forum). Certainly from the combined risk weighted estimates of the interviews a comprehensive and detailed risk handling matrix can be developed.

Prioritisation of risky events normally flows from a straight forward cost ranking of the simulated risk register. The development of risk mitigation strategies follows from this. What can a risk workshop contribute to this – answer nothing!

Yet, not only do the aggregated cognitive maps contain the embryonic strategic solutions to risk handling and mitigation and are worthy of further detailed analysis and development, they are also fundamental in the analysis and understanding of complex problems.

It is only after the SODA process has been undertaken should a review 'risk workshop' be held at which all the issues raised can be discussed and analysed.