From:David Harris
Date:December 2, 2008
To: Gavin Lawrence
Group:Risk Economics
Status:Pending
I have looked at this and all of these things are well understood social phenomenon that a qualified facillitator should be able to avoid or at least should plan for. I have found that risk based scheduling and risk based PERT systems work best if facillitated well from the beginning. Rather than seek to minimize expected costs they seek to minimize variance. The methods are nice because they can be rank based or subjective, as long as raters are consistent with themselves. The mechanism kills projects early by forcing failure early rather than delaying an inevitable failure late. Incentive neutral expected costs are higher because you are not minimizing against expectation, but rather against expected or in some cases median risk, but it is rare that the cost difference is large. Indeed a project killed early is far less costly than one that is permitted to continue so people keep their jobs. It puts the incentives on the table up front. Project planning is NEVER incentive neutral. One real risk is "who is going to get fired for telling the truth." That sets up powerful incentives to bias estimates, "rework" solutions and dig for further capital to keep the process running.
Some of your problems are not "problems," but rather incentive failures. Consider your post:
v. To dissent from the view of the group may put team cohesiveness at risk - threatening established order.
There is substantial research that shows that groups only learn through dissent. Overcoming organizational defenses and the associated incentives is the true problem and must be addressed prior to any risk workshop. If the facillitator has not addressed "the unspoken," the group process itself is a danger to the risk management process. Your list can be overcome by good process. Your list is a description of process failure, but not inevitable process failure. If an element of a process has known faults, like over valuing extreme outcomes, then the process should include that knowledge. Failure to include such knowledge is a failure to learn and implement learning.
I am also extremely aware that this is very hard to do. Group equilibriums are incentive equilibriums and these equilibriums are very powerful. Only the marginal decision makers can move them, which may not be members of the management chain you are working with and may not be members of management. Finding these leverage points requires substantial work that would appear to the customer as non-work because you are looking for which social contract(s) is driving all other social contracts, which is both threatening and does not directly appear to bear on the "work at hand."
All of this gets lost in discussions of "misaligned" incentives. Incentives are never misaligned, the subjective goals of the marginal deciders will in expectation be reached, or everyone will go down. The problem isn't "misalignment," but rather poor contracting system wide and system controls sitting somewhere other than in the self-interest of the principals who made themselves "non-marginal."
I disagree that "no facillitator, however good, can resolve," because that is more than a strong statement of certain process failure, it is a statement that learning is impossible. In some groups and work systems this is very true, but only because people are paying them to not learn. If you pay people to be stupid they will gladly be stupid for you.
I am concerned about the emphasis on "cognitive maps." They are tools and very good tools, but you should always question the assumptions behind a tool before using it. In the case of physical tools, like a Phillip's head screw driver, this is easy. Does your problem contain a screw with a Phillip's head? In the case of social tools whether individual tools like cognitive maps or whole systems of tools like Six Sigma, they contain assumptions that must be met in whole. Are they met?
____________________________________________________
Tuesday, 2 December 2008
The risk workshop methodology is deeply entrenched in the UK risk management process and has been for many years, nowhere more so than in project risk and safety risk. If you examine the international use of risk workshops you will note that they are far less and even minimally used as a means of eliciting risk knowledge.
An examination of risk British literature over the years has not revealed any critical analysis of the performance of risk workshops in delivering accurate risk quantification (an examination of major UK projects from infrastructure to IT will demonstrate the gross cost, performance and schedule deficiencies).
Careers are built on facilitation of risk workshops and consultancies charge substantial fees for undertaking them. I have sat in on many risk workshops and discussed the results with many risk managers who expressed concern and dissatisfaction with the results.
A critical appraisal is long overdue on the effectiveness of the risk workshop process and to this effect I drew up the list of points given in the 'blog' post. All these points are logically obvious from allied research in the socio-psychological fields. One particular area I haven't addressed is that of "Successful sociopathy" which has significant import to risk analysis e.g. Enron, and may be the contributing factor in the 'over-confidence' identified by Bent Flyvberg (Megaprojects and Risk) also cf. Network Rail West Coast Mainline upgrade, NHS Data base, Battlespace, CSA, Olympic Games 2012 etc. etc..
A number of risk managers have suggested to me that the camaraderie and "group hug" are important aspects of the risk workshop. It is my contention that this is total nonsense. If you want to build team spirit go to the pub or go paint-balling. A risk workshop has only one function and that is to deliver identification of risks and accurate estimates of their probability and impact. If it fails on this single measure it was a waste of time and money.
The solution I recommended but am not locked into (if anyone has a better solution let me know) is that of a soft systems methodology known as SODA, there are a wide variety of problem structuring methods for complexity, ambiguity and uncertainty (see Rational Analysis for a Problematic World, Ed. Jonathan Roeshead, Wiley). Some of these were developed by engineers in the aircraft industry for coping with complex engineering problems such as Concorde but have found applications in wide and disparate fields.
With regard to issues of consensus and 'buy-in: what exactly are these issues. In the first instance the purpose of risk analysis is to produce an accurate qualitative or quantitative assessment of the risks (a calculus of possibilities or probabilities) for management to make decisions under uncertainty. I contend that attendees of risk workshops may be keen to reach consensus for a wide variety of reasons that may have nothing to do with the importance of the risk assessments and for this reason the risk assessment process fails. Central to "buy-in" is the concept of sharing of responsibility and all that that entails - I addressed this point in the 'blog''.
I would be very happy to receive a point by point critique of my posting and particularly on how facilitators account for or overcome the issues raised.
N.B. Successful Sociopathy:- “Successful Sociopathy and corporations:- Robertson et al (Medical J. of Australia 1996) describe a pathological condition which they call "successful sociopathy". They suggest that many successful businessmen and entrepreneurs show some of the features of this condition.
More on this can be found on Brian Martin's web site at:- http://www.uow.edu.au/arts/sts/bmartin/dissent/documents/health/sociopathy.html#Background
Gavin Lawrence
____________________________________________________
An examination of risk British literature over the years has not revealed any critical analysis of the performance of risk workshops in delivering accurate risk quantification (an examination of major UK projects from infrastructure to IT will demonstrate the gross cost, performance and schedule deficiencies).
Careers are built on facilitation of risk workshops and consultancies charge substantial fees for undertaking them. I have sat in on many risk workshops and discussed the results with many risk managers who expressed concern and dissatisfaction with the results.
A critical appraisal is long overdue on the effectiveness of the risk workshop process and to this effect I drew up the list of points given in the 'blog' post. All these points are logically obvious from allied research in the socio-psychological fields. One particular area I haven't addressed is that of "Successful sociopathy" which has significant import to risk analysis e.g. Enron, and may be the contributing factor in the 'over-confidence' identified by Bent Flyvberg (Megaprojects and Risk) also cf. Network Rail West Coast Mainline upgrade, NHS Data base, Battlespace, CSA, Olympic Games 2012 etc. etc..
A number of risk managers have suggested to me that the camaraderie and "group hug" are important aspects of the risk workshop. It is my contention that this is total nonsense. If you want to build team spirit go to the pub or go paint-balling. A risk workshop has only one function and that is to deliver identification of risks and accurate estimates of their probability and impact. If it fails on this single measure it was a waste of time and money.
The solution I recommended but am not locked into (if anyone has a better solution let me know) is that of a soft systems methodology known as SODA, there are a wide variety of problem structuring methods for complexity, ambiguity and uncertainty (see Rational Analysis for a Problematic World, Ed. Jonathan Roeshead, Wiley). Some of these were developed by engineers in the aircraft industry for coping with complex engineering problems such as Concorde but have found applications in wide and disparate fields.
With regard to issues of consensus and 'buy-in: what exactly are these issues. In the first instance the purpose of risk analysis is to produce an accurate qualitative or quantitative assessment of the risks (a calculus of possibilities or probabilities) for management to make decisions under uncertainty. I contend that attendees of risk workshops may be keen to reach consensus for a wide variety of reasons that may have nothing to do with the importance of the risk assessments and for this reason the risk assessment process fails. Central to "buy-in" is the concept of sharing of responsibility and all that that entails - I addressed this point in the 'blog''.
I would be very happy to receive a point by point critique of my posting and particularly on how facilitators account for or overcome the issues raised.
N.B. Successful Sociopathy:- “Successful Sociopathy and corporations:- Robertson et al (Medical J. of Australia 1996) describe a pathological condition which they call "successful sociopathy". They suggest that many successful businessmen and entrepreneurs show some of the features of this condition.
More on this can be found on Brian Martin's web site at:- http://www.uow.edu.au/arts/sts/bmartin/dissent/documents/health/sociopathy.html#Background
Gavin Lawrence
____________________________________________________
LinkedIn
Date: 11/29/2008
Subject RE: Reasons for Risk Workshops failing to deliver accurate quantitative information
Group: Global EHS Online Software Solutions
Andy Evans wrote:
Gavin Some of the '19 possible defects' are hazards to the risk assessment process. Perhaps you might care to risk assess each one!
Certainly the potential for 'group-think', 'reluctance to challenge the status quo (or plan)', 'a desire to get the right answer (i.e all is ok) and 'complacency' can be problems.
Looking at your 19 though I struggle to see how a workshop ' damages the camaraderie of the team' so much thatt is better to see the team individually.
Equally you solution seems to me to involve an even greater requirement for a skilled facilitator and probably will involve more facilitator time (as they are working 1-1 with each participant) which would suggest more consultants should be favouring your technique on revenue grounds!
And you state: "There is research to indicate that workshops and group brain-storming sessions do not produce any more creative thinking or better results than that produced by the individuals alone." But what does the research say on the level of agreement with the results? To my mind one reason for a workshop is so that there will be buy-in where action is required to reduce risk.
Andy Evans
Senior Aviation Advisor at AviateQ International
Date: 11/29/2008
Subject RE: Reasons for Risk Workshops failing to deliver accurate quantitative information
Group: Global EHS Online Software Solutions
Andy Evans wrote:
Gavin Some of the '19 possible defects' are hazards to the risk assessment process. Perhaps you might care to risk assess each one!
Certainly the potential for 'group-think', 'reluctance to challenge the status quo (or plan)', 'a desire to get the right answer (i.e all is ok) and 'complacency' can be problems.
Looking at your 19 though I struggle to see how a workshop ' damages the camaraderie of the team' so much thatt is better to see the team individually.
Equally you solution seems to me to involve an even greater requirement for a skilled facilitator and probably will involve more facilitator time (as they are working 1-1 with each participant) which would suggest more consultants should be favouring your technique on revenue grounds!
And you state: "There is research to indicate that workshops and group brain-storming sessions do not produce any more creative thinking or better results than that produced by the individuals alone." But what does the research say on the level of agreement with the results? To my mind one reason for a workshop is so that there will be buy-in where action is required to reduce risk.
Andy Evans
Senior Aviation Advisor at AviateQ International
I agree with the blogspot comments, they are risks or defects to a successful risk workshop. As a risk consultant I never approach the process as if the risk workshop is the only tool. Usually I allow all participants to participate in an anonymous survey to identify Strategic, Operational, Compliance and Reporting risks using a very detailed magnitude and vulnerability scale that they helped develop in prior sessions. The Magnitude and Vulenrability (aka Likelhood) scale take a long time to create and I used workshops to do so. In the end if a 5 point scale is being used one can end up with 50 paragraphs of text for the Magnitude scale (5 x 10 factors) and 60 paragraphs of text for the 5 x 12 vulnerability factors. Its a little hard later to dispute the results since the whole group participated in describing the Magnitude and Vulnerability paragraph descriptions, but 110 paragraphs is alot of work. After the Magnitude and Vulnerability scale is created by the client/organization, an anonymous survey can be sent out where they used the rating scale, I usually use SurveyMonkey or some other tool and then summarize results. Only after this is done is a workshop planned.
During the actual workshops usually 20 persons participate from every department in the global company. I believe my risk workshops have been successful because I start out with a statement to the effect that the participants need to be wary of a false sense of precision that some workshop facilitators and participants may believe can occur. I emphasize that a workshop is not precise and therefore is only one tool in the toolkit to prioritize risk response investments by the entity.
Other tools are OECD bribe payer indecies, corrupt practices indecises and software piracy indecis available from OECD which help with country risk assessments which are again only one tool. I used about twenty different sources of data for each top 20 risk statements and about 25 for business process risk by country when I do a global risk assessment. Prior audit experience, change, complexity etc.
Unfortunately I can't give more details because a US process patent is pending.
Bernice Lemaire
President Lemaire Consulting at Lemaire Consulting LLC
bernicecpa@aol.com
____________________________________________________
During the actual workshops usually 20 persons participate from every department in the global company. I believe my risk workshops have been successful because I start out with a statement to the effect that the participants need to be wary of a false sense of precision that some workshop facilitators and participants may believe can occur. I emphasize that a workshop is not precise and therefore is only one tool in the toolkit to prioritize risk response investments by the entity.
Other tools are OECD bribe payer indecies, corrupt practices indecises and software piracy indecis available from OECD which help with country risk assessments which are again only one tool. I used about twenty different sources of data for each top 20 risk statements and about 25 for business process risk by country when I do a global risk assessment. Prior audit experience, change, complexity etc.
Unfortunately I can't give more details because a US process patent is pending.
Bernice Lemaire
President Lemaire Consulting at Lemaire Consulting LLC
bernicecpa@aol.com
____________________________________________________
LinkedIn
Date: 11/28/2008
Subject RE: Reasons for Risk Workshops failing to deliver accurate quantitative information
Group: Risk Management and Compliance Experts
Roger Miles wrote:
Good blog post - looks like a largely successful trawl for empirical evidence to back the social-behavioural effects modelled by our academic colleagues in Political Science and cognitive psychology. Such real-world manifestations of the effects are sadly familiar but not generally well documented beyond a stack of "anecdotals".
I strongly agree that the absence of cognitive and group-behavioural components from risk assessment (and indeed risk management as a whole... and most particularly from the public-policy end of it) is responsible for much of the markets' current difficulty. In fact I felt strongly enough about this to have spent three years writing it up as a PhD thesis - ironically enough, relating these effects to fragility in the banking system. Hate to have to say to them "I told you so", but...
Interested to hear more about what you plan to do with this.
Regards
Roger
Roger Miles
Risk analysis, Centre for Risk Mgmt.
roger.t.miles@kcl.ac.uk
_________________________________________________
Date: 11/28/2008
Subject RE: Reasons for Risk Workshops failing to deliver accurate quantitative information
Group: Risk Management and Compliance Experts
Roger Miles wrote:
Good blog post - looks like a largely successful trawl for empirical evidence to back the social-behavioural effects modelled by our academic colleagues in Political Science and cognitive psychology. Such real-world manifestations of the effects are sadly familiar but not generally well documented beyond a stack of "anecdotals".
I strongly agree that the absence of cognitive and group-behavioural components from risk assessment (and indeed risk management as a whole... and most particularly from the public-policy end of it) is responsible for much of the markets' current difficulty. In fact I felt strongly enough about this to have spent three years writing it up as a PhD thesis - ironically enough, relating these effects to fragility in the banking system. Hate to have to say to them "I told you so", but...
Interested to hear more about what you plan to do with this.
Regards
Roger
Roger Miles
Risk analysis, Centre for Risk Mgmt.
roger.t.miles@kcl.ac.uk
_________________________________________________
LinkedIn
Date: 11/27/2008
Subject RE: Reasons for Risk Workshops failing to deliver accurate quantitative information
Group: Risk Management Executives & Professionals
Franco Oboni wrote:
Dear Gavin,
Add me to the list of people who agree on the 18 points.
Ar Riskope we are totally against workshops, but unfortunately we get sometimes forced into them by clients.
We have expressed this view in our book, suggesting that one on one interviews are fare more constructive than workshop, defined an interview format, and a set of rules to determine probabilities.
To complete this discussion I would like to add that we generally develop tolerability criteria simultanously with the risk assessment, there too using specific methodologies and applications.
Visit our web iste to know more http://www.riskope.com
Let me know if you would like to join my network on LinkedIN.
Thank you for the list of 18 points. I will certainly use it, and reference you!
Bye
Franco
Franco Oboni
Oboni Riskope Associates Inc.,
foboni@riskope.com
_________________________________________________
Date: 11/27/2008
Subject RE: Reasons for Risk Workshops failing to deliver accurate quantitative information
Group: Risk Management Executives & Professionals
Franco Oboni wrote:
Dear Gavin,
Add me to the list of people who agree on the 18 points.
Ar Riskope we are totally against workshops, but unfortunately we get sometimes forced into them by clients.
We have expressed this view in our book, suggesting that one on one interviews are fare more constructive than workshop, defined an interview format, and a set of rules to determine probabilities.
To complete this discussion I would like to add that we generally develop tolerability criteria simultanously with the risk assessment, there too using specific methodologies and applications.
Visit our web iste to know more http://www.riskope.com
Let me know if you would like to join my network on LinkedIN.
Thank you for the list of 18 points. I will certainly use it, and reference you!
Bye
Franco
Franco Oboni
Oboni Riskope Associates Inc.,
foboni@riskope.com
_________________________________________________
Thursday, 27 November 2008
Reasons for Risk Workshops failing to deliver accurate quantitative information
In the UK risk workshops play a central role in the elicitation of risks and their probabilities and impacts. Below I give 19 reasons for risk workshops failing to deliver accurate and meaningful results. The list is not exhaustive and the reasons are not mutually exclusive. I would welcome some feedback on how these should be best dealt with.
i. Participants believe that their view is 'out on a limb' from the rest of the group.
ii. Participants may believe that they will be subject to ridicule for expressing an alternative view.
iii. Participants may think that others have already said 'it', or thought about it because it seemed so obvious, and that the idea must have been rejected for "good" reasons.
iv. They may have 'trading agreements' with the others in the group that would be broken if they expressed a view that opposes that of their trading partners - to do so would have consequences for support on other issues.
v. To dissent from the view of the group may put team cohesiveness at risk - threatening established order.
vi. It damages the camaraderie of the team.
vii. Participants may be frightened of reprisals for expressing a particular view that is thought to be counter to the prevailing view of those in power.
viii. May demonstrate weaknesses or knowledge/experience etc., deficiencies of the participant.
ix. Drawing on research in cognitive sciences on pattern recognition and visual perception, the practice of plotting, on a white-board, of probability (y-axis) and £-impact - log scaled (x-axis) may influence the risk assessments vis a vis relative positioning and density of the plot points for reasons such as symmetry and balance.
x. Major infrastructure cost over-runs are regularly in the range of 25 - 600%, Flyvberg identifies over-optimism as the single greatest because of over-estimation; this may flow from the points listed above.
xi. Group assessments tend to give extreme values greater credence i.e. approaching 0 or 1.
xii. Social norms of the group can discourage the extent to which the thinking of each of individuals in the group is used in the group decision making.
xiii. Existing social processes and social relationships encourage only shallow thinking to surface.
xiv. Group decision making often results in higher risk decisions being taken because of the shared responsibility and hence, less liability, than that of the individual decision maker.
xv. There is research to indicate that workshops and group brain-storming sessions do not produce any more creative thinking or better results than that produced by the individuals alone.
xvi. Participant selection is a key issue to be explored further in the context of management v. Front-line executive. Also how expert are the experts?
xvii. Over-emphasis placed on the value and out-put of workshops by managers and decision makers.
xviii. Risk consultants over-emphasise the value of risk workshops because they make money from running them!
XiX. Attendees of risk workshops are keen to achieve a consensus, right or wrong, in order to get back to work that they feel is more important i.e. risk workshops are perceived time wasting etc...
I have presented 18 points to a wide variety of risk academics and risk practitioners around the world and none have disputed the issues raised with regard to the 'risk workshop's' deficiencies. In fact most have offered total agreement that these workshops, no matter how well facilitated, do not deliver the quantitative assessments that are comprehensive and accurate enough. Aside from the personal anecdotal evidence there is more than enough research evidence to draw a conclusion that workshops do not deliver.
Assuming that the risk workshop has attendees that have been appropriately selected, something like a dozen key members of staff are taken out of work, effectively for a day - perhaps representing a loss of US$ 15,000 add to this the facilitators cost, plus everybody’s travel expenses to the workshop – perhaps totalling as much as US$25,000.
I would like to suggest that the most effective approach to resolving the risk workshop deficiencies would be that known as "Cognitve Mapping for Strategic Options and Development Analysis" (SODA). The concept and practice of SODA is extensively described in documents on http://www.banxia.com/ , and In the book, Making Strategy: The Journey of Strategic Management, Eden C and Ackermann F., so I won't dwell on the detail.
The advantage of SODA is multi-fold:- the risk analyst can interview the "experts" individually in their own time, revisit them for additional information. The experts are not, therefore, taken out of their work situation for more than an hour at a time. The individual's risk perceptions are explored in depth without any influence from colleagues. Cognitive maps are prepared of each experts perspectives on risk with regard to the project.
There is no limit to the number of experts that can be interviewed as is the case with attendees of a workshop where 12 is pretty much the maximum.
The benefits:-
1. The individual interview process has removed the points raised in points i to viii and xii.
2. The cognitive mapping process removes issues in points ix and xi.
3. Larger number of interviews reduces selection problems, point xvi.
4. More creative and deeper thinking may be encouraged, points xiii and xv.
I submit that the risk analyst will have achieved far more at this stage using SODA, more efficiently and cost effectively than any risk workshop would have done. Not only will a wider range of problems have been described along with the quantitative assessments but insights in to mitigation strategies and value management issues would be becoming apparent.
If you use the Decision Explorer software package each interviewee's cognitive map can be stored, in addition the interviewee's individual maps can be merged in to one. The aggregation of the interviews cognitive maps is very important:-
Consider the risky events that have been identified and individual assessments of probability P(E) x likely outcome distribution from a wide variety of individuals of varying expertise. How to combine these different risk assessments, I have suggested a method in the Bayesian Statistics Forum; it is also important to check out issues and methodologies cited in Roger Cooke;s book, Experts in Uncertainty (see Recommended Books forum). Certainly from the combined risk weighted estimates of the interviews a comprehensive and detailed risk handling matrix can be developed.
Prioritisation of risky events normally flows from a straight forward cost ranking of the simulated risk register. The development of risk mitigation strategies follows from this. What can a risk workshop contribute to this – answer nothing!
Yet, not only do the aggregated cognitive maps contain the embryonic strategic solutions to risk handling and mitigation and are worthy of further detailed analysis and development, they are also fundamental in the analysis and understanding of complex problems.
It is only after the SODA process has been undertaken should a review 'risk workshop' be held at which all the issues raised can be discussed and analysed.
i. Participants believe that their view is 'out on a limb' from the rest of the group.
ii. Participants may believe that they will be subject to ridicule for expressing an alternative view.
iii. Participants may think that others have already said 'it', or thought about it because it seemed so obvious, and that the idea must have been rejected for "good" reasons.
iv. They may have 'trading agreements' with the others in the group that would be broken if they expressed a view that opposes that of their trading partners - to do so would have consequences for support on other issues.
v. To dissent from the view of the group may put team cohesiveness at risk - threatening established order.
vi. It damages the camaraderie of the team.
vii. Participants may be frightened of reprisals for expressing a particular view that is thought to be counter to the prevailing view of those in power.
viii. May demonstrate weaknesses or knowledge/experience etc., deficiencies of the participant.
ix. Drawing on research in cognitive sciences on pattern recognition and visual perception, the practice of plotting, on a white-board, of probability (y-axis) and £-impact - log scaled (x-axis) may influence the risk assessments vis a vis relative positioning and density of the plot points for reasons such as symmetry and balance.
x. Major infrastructure cost over-runs are regularly in the range of 25 - 600%, Flyvberg identifies over-optimism as the single greatest because of over-estimation; this may flow from the points listed above.
xi. Group assessments tend to give extreme values greater credence i.e. approaching 0 or 1.
xii. Social norms of the group can discourage the extent to which the thinking of each of individuals in the group is used in the group decision making.
xiii. Existing social processes and social relationships encourage only shallow thinking to surface.
xiv. Group decision making often results in higher risk decisions being taken because of the shared responsibility and hence, less liability, than that of the individual decision maker.
xv. There is research to indicate that workshops and group brain-storming sessions do not produce any more creative thinking or better results than that produced by the individuals alone.
xvi. Participant selection is a key issue to be explored further in the context of management v. Front-line executive. Also how expert are the experts?
xvii. Over-emphasis placed on the value and out-put of workshops by managers and decision makers.
xviii. Risk consultants over-emphasise the value of risk workshops because they make money from running them!
XiX. Attendees of risk workshops are keen to achieve a consensus, right or wrong, in order to get back to work that they feel is more important i.e. risk workshops are perceived time wasting etc...
I have presented 18 points to a wide variety of risk academics and risk practitioners around the world and none have disputed the issues raised with regard to the 'risk workshop's' deficiencies. In fact most have offered total agreement that these workshops, no matter how well facilitated, do not deliver the quantitative assessments that are comprehensive and accurate enough. Aside from the personal anecdotal evidence there is more than enough research evidence to draw a conclusion that workshops do not deliver.
Assuming that the risk workshop has attendees that have been appropriately selected, something like a dozen key members of staff are taken out of work, effectively for a day - perhaps representing a loss of US$ 15,000 add to this the facilitators cost, plus everybody’s travel expenses to the workshop – perhaps totalling as much as US$25,000.
I would like to suggest that the most effective approach to resolving the risk workshop deficiencies would be that known as "Cognitve Mapping for Strategic Options and Development Analysis" (SODA). The concept and practice of SODA is extensively described in documents on http://www.banxia.com/ , and In the book, Making Strategy: The Journey of Strategic Management, Eden C and Ackermann F., so I won't dwell on the detail.
The advantage of SODA is multi-fold:- the risk analyst can interview the "experts" individually in their own time, revisit them for additional information. The experts are not, therefore, taken out of their work situation for more than an hour at a time. The individual's risk perceptions are explored in depth without any influence from colleagues. Cognitive maps are prepared of each experts perspectives on risk with regard to the project.
There is no limit to the number of experts that can be interviewed as is the case with attendees of a workshop where 12 is pretty much the maximum.
The benefits:-
1. The individual interview process has removed the points raised in points i to viii and xii.
2. The cognitive mapping process removes issues in points ix and xi.
3. Larger number of interviews reduces selection problems, point xvi.
4. More creative and deeper thinking may be encouraged, points xiii and xv.
I submit that the risk analyst will have achieved far more at this stage using SODA, more efficiently and cost effectively than any risk workshop would have done. Not only will a wider range of problems have been described along with the quantitative assessments but insights in to mitigation strategies and value management issues would be becoming apparent.
If you use the Decision Explorer software package each interviewee's cognitive map can be stored, in addition the interviewee's individual maps can be merged in to one. The aggregation of the interviews cognitive maps is very important:-
Consider the risky events that have been identified and individual assessments of probability P(E) x likely outcome distribution from a wide variety of individuals of varying expertise. How to combine these different risk assessments, I have suggested a method in the Bayesian Statistics Forum; it is also important to check out issues and methodologies cited in Roger Cooke;s book, Experts in Uncertainty (see Recommended Books forum). Certainly from the combined risk weighted estimates of the interviews a comprehensive and detailed risk handling matrix can be developed.
Prioritisation of risky events normally flows from a straight forward cost ranking of the simulated risk register. The development of risk mitigation strategies follows from this. What can a risk workshop contribute to this – answer nothing!
Yet, not only do the aggregated cognitive maps contain the embryonic strategic solutions to risk handling and mitigation and are worthy of further detailed analysis and development, they are also fundamental in the analysis and understanding of complex problems.
It is only after the SODA process has been undertaken should a review 'risk workshop' be held at which all the issues raised can be discussed and analysed.
Subscribe to:
Posts (Atom)
