Stop Talking about Risk, Get Serious about Developing Effective Risk Management Tools

We need to stop just talking about risk, and get serious about actually developing effective risk management tools.

The pharmaceutical industry and its regulatory agencies view risk through the very narrow lens of ICH Q9 – Quality Risk Management (QRM).1 ICH Q9 barely scratches the surface of a much larger field of risk analysis that spans almost every area of human endeavor from financial decision making and gambling to personnel safety and protecting patients. Risk analysis dates back to antiquity with more formal approaches being developed in the eighteenth century for games of chance.2 ^, 3  Since then, a great deal of work has been done to quantify the combination of the risk’s impact value or severity of its consequence and the uncertainty or probability of the risk consequence occurring. The combination of severity and uncertainty combine to create a risk’s utility in the form of a probabilistic weighted value. Much of this work is mathematical and somewhat theoretical making it difficult to understand and apply to patient, operating, and business risks. However, none of these concepts and approaches appear in ICH Q9. To make matters worse, I see no evidence that anyone, either in any regulatory agencies or industry forums are aware that this valuable literature exists.

The pharmaceutical industry desperately needs usable approaches to identify, analyze, evaluate, and mitigate a wide variety of business and operating risks to minimize product quality risks for our patients. The tools listed in ICH Q9 are process focused. None of them properly treat the uncertainty component of a risk’s utility by using probabilistic concepts. These tools have been largely discredited by the greater risk community, with some stating that failure mode and effect analysis (FMEA) and similar risk matrix approaches may be “worse than nothing” because they are as likely to self-deceive as illuminate.4 ^, 5 ^, 6 ^, 7 ^, 8  So, how can we develop better methods of managing risks?

The greater risk literature is full of important fundamental concepts and approaches, much of it buried in obscure mathematical definitions, equations, and axioms. This literature needs careful review and analysis to harvest ideas for how to quantify, to the extent possible, a risk’s severity, and most importantly its uncertainty as measured by estimating probabilities. Using these greater concepts, important classes of risks can be addressed by making critical assumptions and applying simple, theoretically sound approaches to properly identify and manage important risks for eventual acceptance and understanding.

Overlapping and interacting areas of study include, but should not be limited to:

• Probability – Understanding the mathematics of probability for providing straightforward methods for measuring and manipulating probabilities and likelihoods is critical to determining a risk’s utility.9 ^,10
• Subjective Uncertainty – How people make choices under uncertainty has been studied extensively. Much can be learned by understanding how people use and process information and prior experience to make subjective judgment calls about rating the uncertainty of future risks.11 ^, 12 ^, 13 ^, 14 ^, 15
• Reliability – Understanding and describing how processes and systems perform and fail under a variety of situations is important. Reliability concepts provide important tools, including important probability distribution concepts for different situations and system types.16 ^, 17
• Bayesian Networks (BN) and Probabilistic Graphical Models (PGM) – BN and PGM can be used on execution (operating) risks to model the flow of threat-induced failures through extended process flow diagrams (PFD) to model the failure risks in a wide variety of manufacturing and their support processes both during development and during product manufacturing.18 ^, 19
• Causal Analysis – Provides an understanding of how probability impacts the flow of threats through process networks to result in the realization of risk consequences. Aids in separating association and causality to provide better methods of both retrospectively and prospectively estimate risk using statistical and probability modeling of uncertainties are very important. 20 ^, 21 ^, 22
• Causal Mechanisms – Understanding how system mechanisms work can provide valuable insights into their probability of performing as they are designed. Improving a system’s mechanism of action can significantly improve the likelihood that it will be more consistently successful and reliable. 23 ^, 24 ^, 25 ^, 26
• Decision and Game Theory – Provides insights into how a business or decision alternative’s impact rating can be combined with its estimated uncertainty to provide a true measure of a decision risk’s value or utility for reaching the best result from a complex decision exercise. 27 ^, 28 ^, 29 ^, 30 ^,31
• Risk Analysis Literature – Considerable insights can be gained on how risks can be handled by reviewing many references. 32 ^, 33 ^, 34 ^, 35

The importance of highly effective QRM cannot be overstated. The most effective methods for managing risks will be synthesized by combining the above topics, along with others, to reach a more complete understanding of all the underlying concepts behind risk and their management. These concepts can be used to develop approaches and tools for examining a wide variety of operating and business risks. The essence of developing and manufacturing pharmaceuticals is controlling risks by developing effective control strategies. While I have perhaps dug the scratch a little deeper, it will take a collective effort of many people, much smarter than I am, to fully develop and communicate all the risk management concepts necessary for providing the complete toolbox for managing pharmaceutical risks on behalf of the patients. 36 ^, 37 ^, 38 ^, 39

• 1FDA (CDER/CBER) – Guidance for Industry: Q9 Quality Risk Management; June 2006. ICH
• 2Covello, V. T. & J. Mumpower. Risk Analysis and Risk Management: An Historical Perspective. Risk Analysis, Vol. 5, No. 2, 1985
• 3Bernstein, P. L., Against the Gods: The Remarkable Story of Risk; Wiley & Sons 1998
• 4Hubbard, D. W.; The Failure of Risk Management: Why It’s Broken How to Fix It; Wiley & Sons 2009
• 5Cox, L. A., D. Babayev, & W. Huber; “Some limitations of Qualitative Risk Rating Systems;” Risk Analysis; Vol. 25, No. 3, 2005
• 6Cox, L. A.; “What is wrong with risk matrices?” Risk Analysis, Vol. 28, No.2, 2008
• 7Ball, D. J., and J. Wait; “Further Thoughts on the Utility of Risk Matrices,” Risk Analysis, Vol. 33, No. 11, 2013
• 8Gulsum, K., et. al., A Review of Risk Matrices Used in Acute Hospitals in England, Risk Analysis, Vol. 70, No. 5, pp1060-1070, 2019
• 9De Finetti, B. Theory of Probability: A Critical Introductory Treatment, Wiley 2017
• 10Jeffreys, H., Theory of Probability, 3rd ed., Oxford University Press 1961
• 11Lindley, D.V., Understanding Uncertainty, Wiley 2014
• 12Halpern, J., Reasoning about Uncertainty (2nd Ed.), MIT Press, 2017
• 13Kahneman, D., Thinking Fast and Slow, Farrar, Straus and Giroux, 2011
• 14Kahneman, D. and A. Tversky, ed., Choices, Values and Frames, Cambridge University Press, 2000
• 15Kahneman, D, P. Slovic, A. Tversky, ed. Judgement Under Uncertainty: Heuristics and Biases, Cambridge University Press 1982
• 16Tobias, P.A., D. C. Trindade, Applied Reliability, 3rd ed. CRC Press 2012
• 17Singpurwalla, N., Reliability and Risk, A Bayesian Perspective, Wiley 2006
• 18Kjaerulff, U., A. Madsen, Bayesian Networks and Influence Diagrams: A Guide to Construction and Analysis, 2nd ed., Springer 2013
• 19Sucar, L., Probabilistic Graphical Models, Principles and Applications, Springer 2015
• 20Pearl, J. & D. Mackenzie, The Book of Why, Basic Books 2018
• 21Pearl, J., Causality, 2nd ed., Cambridge University Press 2009
• 22Salmon, W., Scientific Explanation and the Causal Structure of the World, Princeton University Press 1984
• 23Chao, H., et. al., ed., Mechanisms and Causality in Biology and Economics, Springer 2013
• 24Machamer, P., L. Darden, C. Craver; “Thinking About Mechanisms,” Philosophy of Science, 67 (March 2000) pp. 1-25.
• 25Glennan, S., “Rethinking Mechanistic Explanation,” Philosophy of Science 69.3 (2002)
• 26Glennan, S., “Mechanisms and the Nature of Causation,” Erkenntnis, 44: 49-71 (1996)
• 27Lindley, D.V., Making Decisions, 2nd Ed., Wiley & Sons, 1985
• 28Holloway, C., Decision Making Under Uncertainty, Models and Choices, Prentice-Hall, Inc. 1979
• 29Gilboa, I., Theory of Decisions under Uncertainty, Cambridge University Press 2009
• 30Savage, L., The Foundations of Statistics, 2nd ed., Dover Publications, Inc. 1972
• 31Von Neumann, J, O. Morgenstern, Theory of Games and Economic Behavior, 60th Anniversary Ed., Princeton Press 2004
• 32Cox, L. ed., Breakthroughs in Decision Science and Risk Analysis, Wiley 2015
• 33Aven, T., Quantitative risk assessment: the scientific platform; Cambridge University Press 2011
• 34Cox, L., Improving Risk Analysis, Springer 2012
• 35Sunstein, C., Risk and Reason – Safety, Law, and the Environment, Cambridge University Press 2002
• 36Witcher M., Analyzing and managing biopharmaceutical risks by building a system risk structure (SRS) for modeling the flow of threats through a network of manufacturing processes. BioProcess J, 2017; 16. https://doi.org/10.12665/J16OA.Witcher
• 37Witcher M., Integrating development tools into the process validation lifecycle to achieve six sigma pharmaceutical quality. BioProcess J, 2018; 17. https://doi.org/10.12665/J17OA.Witcher.0416
• 38Witcher, M., “Quality Risk Management (QRM): Part I – Identifying, Evaluating, and Mitigating Threat Risks to Biopharmaceutical Enterprises” BioProcess J, Vol. 15, No. 3, Fall 2016. https://doi.org/10.12665/J153.Witcher
• 39Witcher, M. F. and H. Silver “Comparing Facility Layout Options for Managing Business and Operation Risks” BioPharm Intl., June 2019