Conducted by Dr. Lawrence D. Pohlmann, Strategics Consulting for the International Council on Systems Engineering
Copyright © by INCOSE 2003, 2005. All Rights Reserved
Posted with Permission of INCOSE
This e-Interview was originally conducted in the fall of 2003. It was developed in concert with a review of the book Effective Risk Management: Some Keys to Success, Second Edition by Dr. Conrow. The book review was published in the journal of the International Council on Systems Engineering (INCOSE) Insight, Volume 6, Issue 2, January 2004, p. 44.
The review is also posted here with special permission from INCOSE: Conrow_Review_Incose_Insight_January_2004_Distribute.pdf.
(Minor updates and additions were made in January 2005.)
1) Dr. Conrow, why did you write this book? What specific objectives did you have in mind?
Basically, I was not satisfied with the project risk management guidance that was available. My overall goal was to provide a resource that was not only easier to use, but that provided a more thorough and advanced treatment of risk management technology and processes.
Most project risk management books published in the last 5 to 10 years that I've read include a limited amount of original and useful information. They also contain a fair amount of material that is either erroneous from a mathematical and probabilistic perspective, overly simplistic, inconsistent with risk management best practices, or speculative. For example, there is often little guidance provided for implementing risk management on actual programs. (By and large, tools and techniques are the easy part of project risk management--implementation considerations, including organizational and behavioral factors are the hard part.) The material is typically written at an academic level, difficult for the reader to understand and apply, and assertions are often made without the author providing any justification. In addition, discussions of risk management associated with technical/performance are often very weak or absent. Yet it can be shown that many systems are developed and procured with three primary variables specified--cost, performance, and schedule.
In my case I've written a book that is easy to understand, builds upon introductory material, provides examples and/or theoretical considerations to substantiate key points, and is easy to apply to either a new or existing program.
2) Who is your intended audience? Who do you think should use your book?
The book was written for project managers as well as systems engineers, risk managers, and other practitioners across a wide variety of industries. I've tried to strike a balance between too much versus too little detail, and between information that is too technical\ versus non-technical. Although a lot of the material is applicable to DoD and NASA programs, I made special efforts to make sure that the guidance provided was also directly applicable to a wide variety of commercial programs across industries. For example, risk management process steps, risk analysis methods, a structured approach to developing risk handling strategies, and barriers to successfully implementing risk management and methods to overcome them are almost universally applicable. They can be used across a wide range of programs and industries, and across a wide range of program sizes. While many of the lessons learned apply to "high tech" programs, I've also used the material in an industry that has absolutely zero technology and technical risk, and on very small programs.
3) When people ask you “what is your book about?,” what do you say?
The book provides an introduction to project risk management then moves to implementation topics for each risk management process step as well as across process steps. It includes more than 700 tips to succeed and traps to avoid for implementing project risk management. Many of these tips and traps have been derived from work on actual programs. The book is formulated in such a way that a one or two sentence summary is provided after many tips and traps to help managers and others with limited time to encapsulate the key message of a topic and get to the "bottom line." At the same time substantial levels of are routinely provided. These discussions help the systems engineer, risk manager, and other practitioners who either want to or need to, because of their position, achieve a deeper understanding of the material.
4) Essentially all good books on systems or software engineering include discussions to various degrees on risk management. What do you consider unique or different or important about your book?
First, the tips to succeed and traps to avoid are organized by risk management process step rather than being intermixed. [In cases where the item spans more than a single process step it is included in a separate risk management process-level chapter (Chapter 3).]
Second, the items are written in such a way to enable managers to extract key information as well as systems engineers, risk managers, and other practitioners to focus on more extensive details.
Third, a number of implementation examples are given, and key organizational and behavioral considerations are discussed. These topics are often weakly covered or absent from a number of project risk management books.
Fourth, many of the tips and traps are gleaned from actual programs. I've also attempted to write each tip and trap presented in the book in such a way that it could be considered by the reader for their specific program.
Fifth, the concepts presented are mathematically and probabilistically sound, yet approachable to the reader to permit easy implementation.
Sixth, an extensive survey and statistical analysis associated with words of estimative probability (e.g., high) is included. Often times authors or analysts discuss such words and ascribe probability levels to them (e.g., high probability = 0.8) without having any basis for the specified probability levels associated with the words. This can lead to substantial errors when applied to a risk analysis.
Seventh, there is a large body of literature that incorrectly performs mathematical operations on results from ordinal "probability" and consequence of occurrence scales. I have included an extensive discussion of why such practices are flawed and lead to erroneous results, and also provided sound methods for estimating risk levels when risk scales are used.
Eighth, while many in the project management community intermix opportunity and risk, my book includes a carefully written Appendix by Dr. Robert N. Charette on why opportunity should not be merged into the definition of risk, and why opportunity management should not be pursued as a mirror to risk management.
5) I was pleased to see the expanded table of contents and index in the second edition. What else do you view as the most significant changes in this second addition?
First, I've provided top-level guidance for implementing project risk management in Chapter 2, Sec. III, "Some Risk Management Implementation Guidelines." This section also serves as a roadmap to the remainder of the book, as it points to specific sections containing key information.
Second, an important discussion authored by Dr. Robert N. Charette on whether/not to include opportunity as part of risk management is contained in a new Appendix to the book (Appendix E). Dr. Charette points out numerous flaws in the arguments advanced by proponents of such positions, and argues convincingly that including opportunity in the definition of risk is unwarranted as is developing "opportunity management" as a parallel to risk management.
Third, I've added more than 450 additional tips to succeed and traps to avoid for implementing risk management on an actual project.
6) How would you suggest people in the systems engineering community use your book?
My goal was to structure the book so that it can serve three major types of use. First, it contains introductory materials that are designed to be useful for those who are just learning about risk management. Second, the book contains detailed and advanced treatment of all phases of the risk management process. These are intended to be used by those who have direct responsibilities in the area of project risk management – especially on complex programs. Thirdly, I’ve included a variety of navigation aids to make it a good reference book – that is a book in which it is easy to locate the risk related topic of current interest.
The book can be used in the traditional arenas of systems analysis and control (e.g., establishing, implementing, and verifying the risk management process) and requirements flowdown (e.g., risk identification followed by risk analysis, handling, and monitoring as warranted). The guidelines for risk planning can also be used to write a risk management plan and/or a risk management section in the corresponding systems engineering plans (e.g., SEMP). The material is also suited for function, matrix and project implementations for systems engineering. Here, the risk management process fundamentals (e.g., process steps, their order, and functions per step) are often the same, but what organization owns the process and how it is implemented may vary widely.
Because there are a large number of tips and traps (more than 700), I would expect users to benefit from repeatedly browsing sections of the book that may be relevant to their task at hand.
7) Your background includes extensive experience in the aerospace projects sector. How do you see project risk management as having evolved in this sector over the last five to seven years?
Let me first state that project risk management processes and techniques and tools have been evolving for decades. More recently on the DoD side there was a major evaluation of risk management performed in 1996-1997. This was an outgrowth of Dr. Paul Kaminski's (Under Secretary of Defense for Acquisition and Technology) 4 December 1995 memo that established Cost As an Independent Variable (CAIV). As part of that memo Dr. Kaminski encouraged the DoD to examine and enhance its risk management process as warranted. A major effort involving roughly 20 to 30 individuals from the Office of the Secretary of Defense, Air Force, Army, Navy, Defense Acquisition University (DAU), Defense Systems Management College (DSMC) and other organizations led to an enhanced risk management process, most readily available to the public as the "Risk Management Guide for DoD Acquisition," through the DAU web site. I served as an unpaid technical advisor and consultant to both the Air Force and OSD on this study, and subsequently to DAU and DSMC to assist in updating the "Risk Management Guide for DoD Acquisition," now in the Fifth Edition, Version 2 (2003). The resulting document can be downloaded free of charge by anyone at:
This document is widely used in government and industry, and often far outside of DoD programs. My book can be viewed as an adjunct to this DoD guide, in terms of tips and traps for each process step, implementation guidelines, and especially advanced topics.
In terms of individual programs, the average implementation effectiveness has likely increased (at least somewhat). However, there is still a substantial standard deviation of implementation effectiveness across programs. It is not uncommon that some programs exposed to the same reference material will embrace risk management much more so than others even though they are in the same organization.
8) How is your crystal ball working? What kinds of evolution in project risk management processes do you see in the next five to seven years?
On the positive side, enlightened organizations will do three things:
· First, they will use a refined project risk management process for most if not all programs (albeit the specific details may be different on a case by case basis).
· Secondly, they will place increased emphasis (as warranted) on explicitly evaluating and managing integration risks (e.g., hardware/hardware, hardware/software, and software/software) and systems level risks (both for a single system and across systems).
· Thirdly, they will more consciously focus on implementation considerations throughout the development process.
On the negative side, there is the distinct possibility that, as Dr. Robert N. Charette has observed, risk management may go the way of disco. This is because risk management is too frequently overhyped and oversold by consultants and trainers with little or no experience and accountability on making risk management work on an actual program. This can lead to havoc on actual programs where the opportunity cost in terms of un-recoverable schedule and dollars may be very high. For example, in one case I began working on a program that was in substantial trouble. The very first day I was confronted by program management and told that the last project risk management consultant primarily focused on developing a schedule risk analysis rather than risk management. Within a short period of time I had documented more than 25 risk management-related deficiencies with recommended changes—many of which existed when the prior consultant was engaged. Here the opportunity cost was huge—the wasted time and ineffective treatment of both risks and problems could never be recovered.
Also, it is commonplace that project risk management trainers have never held a long-term risk manager position on an actual program, and the courses they offer have little/no implementation insights. Caveat Emperor!
9) Briefly, how would you compare and contrast project risk management challenges and processes in the DoD, other Government agency, and commercial sectors?
In some respects the project risk management process and implementation has been stronger in DoD programs than many other government agencies and commercial programs. This may seem counter-intuitive to some given that $500 toilet seats may come to their minds. But the perception that anything done by DoD can’t be “cutting edge” or is overly bloated is incorrect. The DoD focus on risk management is the result of both requirements (e.g., specification in Directives, Instructions, and source selection), good best practices (e.g., “Risk Management Guide for DoD Acquisition”), and desired advances in the state of the art sought for many development activities.
While you may "get away" with a relatively simple, unstructured risk management process on a program far below the state of the art, doing so on a program that stretches the state of the art will likely lead to substantial cost, performance, and schedule issues, if not program cancellation without good risk management. Some government agencies have either developed their own risk management processes or borrowed from existing processes. In some cases, I have found these risk management processes are relatively weak compared to the DoD process, both in terms of process characteristics and implementation considerations. In addition, with one agency the "not invented here" syndrome is routinely practiced despite the fact that the organization that developed their risk management process was disbanded several years ago. I've also looked at project risk management material offered by large and reputable commercial firms, ranging from industry leaders to traditional consulting firms. By and large, this material is overly simplistic, and far less comprehensive than the readily available DoD material that is not copyrighted and available free of charge. There is also typically little/no information provided that assists the reader in implementing risk management on their program--this often appears to be the "hook" from those peddling their services.
10) During the last several years, we have experienced continued and increased emphasis on process maturity models in software engineering, systems engineering, acquisition, and other areas. How has this helped risk management technology? How has it hurt?
I don't believe that process maturity models have had much impact on project risk management to date. Risk management is subject to the same concerns as with process maturity concerns for other disciplines. On the plus side they may cause some organizations to take a hard look at different risk management processes implemented on various programs (e.g., are the process variations across programs reasonable) and standardize on a given process even if implementation differences may exist. On the minus side the criteria for assessing maturity, in the worst case: 1) may be subjective, counterintuitive, or erroneous; 2) may be oriented or tied to a specific risk management process; 3) may be biased towards the experience of those that developed the criteria (and not how risk management is broadly practiced on actual programs); 4) may not suitably capture organizational and behavioral characteristics; 5) are not reflective of potential contractual requirements external to the organization that may dictate a specific implementation or methodology for a given program; 6) may not scale with the scope of a program, etc. For example, in one risk management model at the highest level of maturity there is mention of project risk management being top-down from upper management. I agree that this is highly desirable in a mature risk management process. But it is insufficient, thus the model is incomplete (or incorrect). There is no mention that project risk management also needs to be bottom-up from the workers. Without this the process will be ineffective.
I view risk management maturity models at the present time as a work in progress--worth observing but not accurate or mature enough to mandate or apply on actual programs else the results may be erroneous.
11) You have provided a large number of heuristics in your book, literally hundreds. This will be intimidating to many. How can people start to use your guidance without being overwhelmed?
My goal was to provide a thorough treatment of project risk management. I clearly recognize that good risk management can be complex in that lots of things may need to be considered. Thus, I provided a large number of heuristics – or tips and traps. Naturally, I do not expect readers to try to understand and practice these all at once.
Rather, I suggest the reader first examine the guidance for implementing risk management in Chapter 2, Sec. III, "Some Risk Management Implementation Guidelines." This section also serves as a roadmap to the remainder of the book, as it points to specific sections containing key information. Also see Chapter 3, Secs. IX ("Some Risk Management Implementation Considerations), XII ("Organizational Considerations and Risk Management Process Implementation"), XIII ("Support for Risk Management Within the Program"), and XIV ("Some Behavioral Issues Associated with Implementation"). Additional, and more detailed information can then be found by consulting the Table of Contents and Index.
12) What do you see as the most common “errors” in project risk management? And why?
First, there is often blind faith that the existing risk management process is "good enough," when it may, at least in some cases, be substantially flawed. Continued use of a risk management process does not verify or validate it. These same organizations are often opposed to performing an independent review of their process or incorporating enhancements that will improve the process (balanced against resources available).
Second, there is generally a much greater focus on tools and techniques (including advanced tools and techniques) than implementation considerations. Yet, tools and techniques are often not valuable if there are top-level issues associated with process implementation (e.g., limited upper management participation). And attempting to implement advanced tools and techniques when the process is missing a step or if you have fundamental organizational and behavioral issues is generally problematic and may lead to substantial opportunity cost. Similarly, there is often a much greater emphasis on risk analysis than all the remaining process steps combined, yet it is only one of five project risk management process steps (risk planning, identification, analysis, handling, and monitoring).
Third, it is common that mathematical operations are performed on values from ordinal probability and consequence of occurrence scales. This will almost always lead to erroneous results. This flawed practice was promulgated by a major contractor through DoD and grew considerably for almost 10 years (1980s) before I challenged it in the early 1990s. Another 10 years have since passed and only now has this erroneous practice been curtailed. (Unfortunately, once it appeared in DoD documents it began to take on a life of its own.) I've included both a proof of why such mathematical operations are erroneous and an irrefutable example that has an average error of more than 600% in my book (Chapter 6, Section IV.B)!
Fourth, there are mathematical forms of risk that are inherently wrong, yet widely used without recognizing the errors present. Simply stated, if you must perform mathematical operations, use Risk = Probability * Consequence, and not another representation.
13) If I tell you that I have been recently assigned to be the risk manager in a new project, what kinds of advice would you give me?
I'd first recommend examining contractual and other requirements for project risk management, organizational best practices, applicable "command media" (if any) and the (organizational and behavioral) implementation environment.
As mentioned in question 12 (above), use the top-level guidance for implementing project risk management contained in Chapter 2, Sec. III, "Some Risk Management Implementation Guidelines," as well as the references to other sections of the book (as warranted) to outline the desired process. Also examine Chapter 3, Secs. IX ("Some Risk Management Implementation Considerations), XII ("Organizational Considerations and Risk Management Process Implementation"), XIII ("Support for Risk Management Within the Program"), and XIV ("Some Behavioral Issues Associated with Implementation") to assist with roles and responsibilities, and organizational and behavioral considerations.
Beyond the references I'd say that it's important to realize that risk management is not a "silver bullet" to save a program and the process itself is not "rocket science." However, it must be logical; structured; repeatable; well integrated with higher, similar, and lower-level processes; and continuously practiced.
A substantial key to success is for the project manager, deputy project manager and other key managers to use risk management principles in his decision making; else working level personnel will question the commitment to risk management and not truly support it. However, even with upper management support it is important to realize that "progress will take time" – effective risk management is something that doesn't happen overnight. Fortunately, and effective risk management process can be built up over time. Continuing success leads to increased interest and practice on the part of other project personnel.
14) A term we are seeing more of is “systems of systems.” What kinds of risk management challenges do you see in this context?
One key consideration is the integration of often a diverse set of systems of varying types and levels of maturity. This often points to a series of integration risks and system (or “top level”) risks not typically seen on a single program, or if present much smaller in magnitude. It may also present the challenge of integrating legacy systems with those that are presently under development and in some cases entirely new risk categories than what existed on individual programs. From the implementation perspective there will also likely be more than one risk management process in place and/or under consideration, and widely varying views of risk management across key managers and stakeholders. This is a potentially challenging environment for implementing risk management, but the risk management principles discussed in my book also can also be tailored and applied to “systems of systems.” I specifically discuss systems of systems considerations in greater length in my February 2005 article contained in CrossTalk, The Journal of Defense Software Engineering (“Risk Management for Systems of Systems”). The reader can view and download this article beginning in February 2005 from the CrossTalk web site:
CrossTalk Web Site
or directly from this site:
15) Do you foresee a third edition or writing another book?
I'm on a long vacation from book writing and not contemplating another edition of this book or writing another book beyond updates to the project risk management chapter I’ve written in Harold Kerzner’s best selling project reference book, “Project Management: A Systems Approach to Planning, Scheduling, and Controlling,” Wiley, and related collaborations.
16) In closing, what final comments would you like to add about your book?
The book serves as both a guide for developing and implementing a risk management process, as well as a comprehensive reference source to use during the course of the program. I encourage the reader to apply the material to their programs, and leave a legacy that builds upon the information provided for current and future project managers, systems engineers, risk managers, and other practitioners.
Lastly, I would like to thank INCOSE both for publishing a review of my book and for sponsoring this e-Interview.
And thank you Dr. Conrow. As our systems and processes continue to increase in complexity and interdependence, we can view good project risk management as more important than ever. Lastly, let me reiterate that I found your book to be, in my opinion, the most thorough and advanced treatment of risk management that I am aware of. It is a journeyman’s reference book. I highly recommend it to anyone with direct responsibilities in that area of risk management. Thank you again!
Links to Project Risk Management Services
Risk Management e-Interview by International Council on Systems Engineering
Links to Other Services
This web site owned and operated by:
Dr. Edmund H. Conrow, CMC, CRM, PMP
Reproduction or any other use of this material is prohibited without written permission.
This material is Copyright © 2000-2014 by Edmund H. Conrow, All Rights Reserved