You can’t just “follow the science”, but don’t ignore scientific method
Decision making in times of unusual uncertainty is one of the greatest, perhaps the single greatest, challenges of leadership. Navigating the current COVID-19 crisis and crafting appropriate responses has been a challenge for leaders around the world. And whatever happens, we are likely to have years of second-guessing as to what we could have done differently. To some extent this is inevitable – with so many unknowns, it is a certainty that we either over-reacted or under-reacted, deploying certain measures either too early or too late; and we may never know which, as we cannot go back in time and try something different.
As we slowly return to some kind of new normal, what lessons can business leaders draw from this situation and its evolving challenges? One piece of advice that has emerged recently is the idea that we just need to ‘follow the science.’ That sound bite must test well, but it is based on the false choice that we must either follow the science or ignore the science and ‘go with our gut.’ In truth, no right-thinking person is in favor of ignoring ‘the science.’ But deciding which science to follow and what actions are indicated remains difficult, as it is during any rapidly evolving crisis in business or elsewhere, for a number of reasons:
- First, scientific certainty takes a long time. In a quickly evolving crisis, action needs to be taken before all the facts are in. This means that we are not dealing with ‘settled’ science, rather we are dealing with models and projections. And while some of those models have improved dramatically, many were initially fabulously wrong, as models almost always are.
- Second, when it comes to policy, not all scientists agree. At the present time, there are respected doctors arguing that we should extend shut-down orders until a vaccine is found and others suggesting that the health impact of the shutdown (depression, suicide, domestic violence, postponing medical treatment) may actually be worse than the impact of the virus. Further, many individual scientists have changed their minds over time. In the world of science, this is a good thing – when presented with new information, science ‘revises its hypotheses.’ But this makes it difficult to rely on scientific opinion alone to make long-range decisions.
- Third, scientists are human beings and as such, they are biased. It would be nice to think that because someone has an advanced degree in a scientific discipline, they can look at a set of facts and draw an objective conclusion, but that is not how our brains work. Every one of us, regardless of background or intelligence, makes conclusions based on emotional factors and then ‘tells ourselves a story’ with selected facts. Selective interpretation of data is part of why well-meaning people disagree on so many things (see “The Righteous Mind” by Jonathan Haidt, for a thorough explanation of this phenomenon and its implications). At a minimum, this means even the ‘experts’ favor solutions that leverage their particular expertise – so in a complex business situation, no single expert is likely to have the complete perspective.
- Lastly, and closely related to the previous reason, for complicated issues, no single branch of science has all the answers. Complex decisions require trade-offs. So, a decision-maker has to balance contradictory and sometimes seemingly irreconcilable advice. As the current situation highlights, it can be tempting to focus only on the most visible aspects of the crisis. But every decision reflects a cost or a compromise, and sometimes there are second and third order effects that are not immediately evident. For example, as already mentioned, keeping the economy shut down has a personal as well as financial cost. How and when to open up is a challenging question, and no wonder that you might get different recommendations from those who specialize in different aspects of the problem. The same is true in business, leaders need to clearly define objectives, align incentives and coordinate goals to make these dynamic trade-offs over time. Further, they need to recognize that in a crisis, specific goals and targets may need to change as the assumptions underlying them are no longer valid.
So, if you can’t just ‘follow the science,’ what is a leader to do? We have long counseled our clients to follow a hypothesis-driven approach and that advice hasn’t changed. In short, just because individual scientists may be wrong, don’t give up on the scientific method.
Scientific method, as you may or may not remember from middle school, is what makes the idea of scientific progress possible. One forms a hypothesis consistent with observable facts, designs an experiment to test that hypothesis and determines if it can be accepted or rejected. After multiple tests, accepted hypotheses become theories (note they are not ‘facts’ as counter-examples might still surface) and rejected hypotheses get re-formulated for further testing.
In a simple example, if you have a hypothesis that water will boil at a lower temperature at a higher altitude because of the lower atmospheric pressure, you can stash some water and a Bunsen burner into your backpack and climb a mountain to run a test and measure the result.
In business, we have seen broad adoption of the word ‘hypothesis,’ probably something for which we can thank consultants. But, as one of our clients pointed out, just saying “my hypothesis is…” to mean “well, I think that…” may make you sound smarter but doesn’t really change anything. So how should business leaders use this type of thinking, and why is it particularly relevant during a crisis? Here are a few pointers:
- Remember hypotheses are not the same as assumptions. Good hypotheses are consistent with all known facts, so in a rapidly evolving crisis, as the facts change, so should our hypotheses. Many business leaders have been taught to value confidence and hence may state things as known when they are in fact still uncertain. While in normal times, this may be inspiring, the danger is that it shuts down inquiry and may slow a company’s response when their assumptions turn out to be wrong. This may be a really good time to revisit some of your own “everybody knows…” statements, if events haven’t already forced you to do so.
- Beware of the naive hypothesis. A naïve hypothesis is an oversimplified explanation based on too little data to be meaningful, often just personal experience. A recent example was seen with spring breakers on the Florida beaches in early March – when interviewed, they said essentially, “I don’t know anyone who is sick, therefore this virus must not be a big deal.” Just as this turned out to be wrong, when data are changing daily, it is a mistake for leaders to base decisions solely on their personal experience.
- Understand that you can’t prove a negative. This is a subtle, but critical, distinction that is broadly relevant in the business world. We remember distinctly a client general manager whose position was, “I know our exclusive distribution strategy is right, we just need to fine-tune it.” Our consulting team found out the hard way that it was impossible to prove that the exclusive distribution strategy would not work – every piece of evidence we presented that it was not working (and there was a lot of evidence), was met with a response along the lines of, “we just need to improve execution.” Since we could never prove that we had exhaustively tried every possible improvement idea, we could never ‘prove’ that it wouldn’t work. As a footnote, this division president retired shortly after our work and his successor broadened distribution and presided over a three-fold increase in sales. Even this doesn’t ‘prove’ that the exclusive distribution strategy might not have worked, but it seems like a strong indication.
- Know when to stop asking for more data. In a crisis, there are likely lots of things that you would like to know that you cannot know exactly. But not knowing something exactly doesn’t mean you lack insight into the problem. Even with a lot of unknowns, you can often form good hypotheses. For example, one of our clients sold maintenance management software designed to optimize process uptime. While it would take a detailed study to understand how much a particular customer would benefit from their offering, they could make a pretty good estimate based on just a couple of pieces of data (e.g., cost of downtime, value of customer’s output, are they running at capacity?) and use that estimate to dramatically improve targeting for their salesforce. Business leaders have to strike a balance, using the data that are available, but resisting the temptation to instinctively ask for more data before making a decision.
- Recognize the option value of flexibility. When you realize that hypotheses may turn out to be false and need to be re-visited, it should be clear that you wouldn’t want to lock yourself into positions that are difficult to undo when things change. We experienced this vividly in 2001 as part of a major consulting firm. The consulting industry was already experiencing a downturn by the middle of 2001 and then after September 11 of that year went into a full-fledged crisis: many of our clients froze discretionary spending while they studied the impact on their businesses, and for a while we didn’t even know if it would be possible to fly to see our clients. Unfortunately, just prior to that series of events, our firm had implemented a new partner compensation program where pay cuts and demotions were no longer permitted among the partnership. As such, the only way to reduce partner compensation to weather this crisis was to reduce the number of partners, which is exactly what happened. To a certain extent, this made sense, as partners are the most highly compensated individuals in a consulting firm. But partners are also the salesforce, so as the consulting market recovered, our firm was at a disadvantage and lost market share in many critical sectors. In a time of crisis, you should strive to have more degrees of freedom, not fewer.
- Acknowledge the importance of running ‘tests’. One of the problems with using scientific method to set policy is that sometimes there is no easy way to run a controlled experiment – for example, when the Fed lowers interest rates, there is no way to know what would have happened if they hadn’t lowered rates. One way that business leaders can at least partially overcome this is to allow different facilities or regions to try different things, enabling what scientists call “natural experiments.’ While not a substitute for a real controlled study, if viewed objectively, this can be a great way to rapidly test multiple responses. But it requires measurement, feedback and a way to share the results. This type of delegation is not weakness, in fact it can be a source of strength. One of the secrets of Wal-Mart’s success in the 80’s and 90’s and Whole Foods Market’s success in the 90’s and 00’s, for example, was a high degree of local store manager autonomy relative to other retailers.
Incorporating scientific method into how you think about your business can be powerful, but it is not easy. Like scientific progress, it starts with recognizing that not everything is known, and further that some of what we think we ‘know’ may be wrong. Acknowledging the incomplete state of our knowledge, while still taking decisive actions incorporating what is known is the central challenge of crisis management. It will be interesting to see what lesson business leaders learn from the current situation.
