A Better Way to Make Risky Decisions

From predicting the weather to developing pharmaceuticals, we can measure risk using decision analysis

Based on the research of Jim Dyer

These days, when someone shops for a car, that person may be borrowing techniques from a discipline that oil companies use in choosing where to drill a well and pharmaceutical companies use to pick what drugs to develop.

This discipline is called decision analysis, and Jim Dyer, professor in the Department of Information, Risk, and Operations Management at Texas McCombs, has been involved in research related to it since 1970.

“Decision analysis is a field that focuses on decision making involving risk and multiple criteria,” Dyer says. He was recently tapped by Management Science to survey 25 years of the journal’s articles on the discipline and to assess how it has progressed.

We recently spoke with Dyer about major developments of the past quarter century, particularly in the practical uses of decision analysis. They are as diverse as medical policymaking, weather forecasting, and improving the safety of offshore drilling platforms. But the concepts can apply to almost any business decision, he says.

Decision analysis has been around since the 1960s, but the average person might have a hard time telling you what it means. How do you explain it?

Really, it’s focused on two things: First, what are the risks associated with a decision, and how do we measure those, using probability estimates? Second, what are the outcomes that might occur, and how do we value those? Then, we multiply these probabilities times the value measures of the outcomes, often measured in dollars. We get risk-adjusted values that we can use to rank alternatives.

Why were oil and pharmaceutical companies among its first adopters?

Oil companies and pharmaceutical companies have a lot in common, in the sense that one has to make a large investment in a new oil well or in a new drug. The success or failure of that investment is difficult to forecast, and the uncertainty may be resolved over long periods of time. In the oil business, you drill a well, and you might have a dry hole. In the pharmaceutical industry, a new drug may go through years of testing, and it may fail. Furthermore, the returns from these investments, if there are returns, may not be realized for another five or 10 years.

So, both businesses require you to bet your money and take your chances. But decision analysis helps decision makers quantify these chances and their outcomes in a way that can rationalize their decisions.

You recently were on a committee of the National Academy of Sciences, Engineering and Medicine that made recommendations to the federal Bureau of Safety and Environmental Enforcement (BSEE) on modernizing the inspection of offshore oil and gas platforms. How did decision analysis apply?

The legislation in place basically says that BSEE will inspect every platform once a year. There are approximately 1,700 of these platforms in the Gulf of Mexico. In the past, the BSEE inspectors went to each rig and used a protocol similar to a checklist of items and operations to inspect it.

These inspections have provided a wealth of data that can now be analyzed to determine the history of how well these platforms have performed, both individually and as groups categorized by size. Our committee supported the increased use of tools of decision and risk analysis to emphasize the use of “risk based” inspections. The objective is to identify high-risk offshore platforms that should be subject to inspections that are more focused and perhaps more frequent.

Another interesting trend in your review is that people are using decision analysis in everyday life.

People are much more inclined to use probabilities in casual conversations these days. One of the examples is weather forecasting. The United States is one of the few countries where the weather forecasters give you probabilities of rain, rather than just saying it is likely or not likely. We conjecture that public familiarity with these forecasts has contributed to a broader acceptance of probabilistic estimates of outcomes to support decision making.

You cited one study that calibrated forecasts against historical weather data. How did that work?

Let’s say a weather forecaster said, “I think there’s a 50% chance of rain on a particular day.” Now, let’s look back at all of those days where there was a forecast of a 50% chance of rain and use historical data to determine the percentage of time it actually did rain. It turns out that when forecasters say there is a 50% chance of rain, about half the time it rains, and about half the time it does not. The study validates that the weather forecasters do a good job of providing useful forecasts.

Similar studies in pharmaceutical companies have shown that well-trained specialists do a very accurate job in predicting the probability of success of a new drug as it moves through the three stages of the Food and Drug Administration drug testing process.

How does decision analysis help with another everyday activity: car shopping?

In buying a car, people have multiple criteria or objectives in mind. What is the purchase price? What are the operating costs? How does it drive and how does it look?

It is possible to help people to think about their choices by talking about these different criteria and suggesting weights for them that, loosely speaking, reflect their relative importance. In addition, different cars can be rated on each criterion on a numerical scale. The ratings can then be multiplied by the weights and added up to rank the alternatives.

An individual could develop his or her own weights and ratings, but it may be easier to use ratings that have been developed by people who have similar preferences to yours. A magazine such as Consumer Reports may provide ratings with more weight on safety and affordability. One may get very different ratings in a magazine for automobile enthusiasts like Car and Driver with higher weights on acceleration and handling. Decision analysis theories provide the underlying methodologies for developing these ranking systems in a logical and reasonable way.

“Innovations in the Science and Practice of Decision Analysis: The Role of Management Science” is published in Management Science.

Story by Steve Brooks