The Innovation Ingredients

Nothing is more important than innovation for the long-term success of an organization. You can get lucky once or twice, but today’s success isn’t a guarantee of anything tomorrow. The world is in constant flux, so you can’t just keep doing what you’re doing and expect the same results. Businesses (and people) need to constantly re-master the new environments the world serves up.

The trouble with lots of innovation advice is that it boils down to some variation of “get good at fortune telling:”

I skate to where the puck is going to be, not where it has been.

Wayne Gretzky was an incredible hockey player. His record of 92 goals in a season may never be broken. His famous quote is great advice for hockey centers, but applied to business innovation, it’s bullshit.¹

The Trendwatching Myth

“Where the puck is going to be” is reasonably predictable, especially if you’ve seen lots of pucks in lots of games (Gretzky started studying hockey on TV at age 4). In business and life, we usually have questions that are much harder to predict: Will customers love feature X when we launch it? Will housing costs around our headquarters keep rising? How will that affect engineering salaries? Will WebAssembly make JavaScript irrelevant? These kinds of predictions can be made, but their quality will be low and their usefulness will be brief. The movements of a business are exponentially harder to predict than a block of rubber on ice.

The fundamental error in applying the Gretzky quote, and in lots of innovation strategy generally, is ignoring the importance of time horizons. Gretzky is predicting puck movements a few seconds into the future. I’m no meterologist, but even I can predict the weather that far out. If Gretzky were a business, he’d need to be capable of seeing years or even decades into the future.

Despite the fact that it’s obviously impossible to make meaningful predictions over longer time horizons, we often fool ourselves into thinking some people are capable of it. In 2019, the innovation gurus at McKinsey called just-in-time production “a trend to watch"; the next logical step after several other just-in-time supply chain innovations. Oops. The waves of the COVID-19 lockdowns, plant closures and widespread illness turned highly-tuned supply chains into a libability, and they now recommend the opposite strategy.

It’s time to put the crystal ball away. Better innovation isn’t a matter of seeing further into the future or extrapolating every trend to its logical conclusion. As lots of businesses found out in 2021, trends can reverse without warning.

This doesn’t mean trends are worthless. As we’ll see in a minute, trends are relevant to innovation, just not in the way Gretzky’s quote describes.

Nature’s Innovation Map

If Gretzky is the G.O.A.T. in hockey, then Nature is certainly the G.O.A.T. innovator. The greatest hits list include marvels like the Welwitschia, a plant that can survive for thousands of years in the harshest deserts, and tardigrades, which can survive being radiated or boiled, dehydrated. Even exposure to the vacuum of outer space, or going without food or water for 30 years won’t kill a tardigrade.

Your high school education probably had a primer on nature’s approach to innovation: evolution through variation and natural selection. Each generation of offspring includes variations from the ancestors, and this variation is hereditable. The best-suited offspring survive to reproduce. Rinse and repeat.

Nature’s approach is the polar opposite of trendwatching. Nature seems to know prediction is hopeless. Instead of even trying to guess what will work tomorrow, it just makes random tweaks on what’s working today, and hopes one of them will be right for the future. Then it unceremoniously kills off the unsuccessful approaches (just like Google does 😉 ²).

Unfortunately, we can’t mimic nature’s approach directly. We have another time horizon mismatch; but in the opposite direction. It takes millenia for evolution to produce organisms that are resilient. And even a planet-scale company like Google doesn’t have the resources to try completely random variations and wait to see how they work out. Nature is patient in ways that would scare off even the friendliest investors.

The Ingredients of Innovation

We can’t directly implement variation and natural selection as the basis for an innovation philosophy. But we can leverage the underlying dynamics that make them work. The English economist Tim Harford succinctly summarized the approach in his 2011 book Adapt:

first, seek out new ideas and try new things; second, when trying something new, do it on a scale where failure is survivable; third, seek out feedback and learn from your mistakes as you go along.

Harford calls these the “Palchinsky Principles” in honor of Peter Palchinsky, the outspoken Russian engineer executed for questioning the narrow-minded Stalinist approach to industrial planning. His biographer describes him as the “ghost” that haunts Chernobyl and other Soviet engineering catastrophes³.

I like labelling these ingredients because each is a crucial component, but each needs to be balanced with the others to ensure a harmonious result.

In the same way any good dish needs Salt, Fat, Acid, and Heat all effective innovation attempts are defined by Humble Experimentation, Survivable Failure, and Accurate Feedback.

Ingredient 1 - Humble Experimentation

The first ingredient is a mechanism for introducing novelty (a.k.a. variation). We know nature’s random variation is too extreme, though. How can we take a more measured approach, while still remaining open to the unexpected?

Palchinsky’s life experience is instructive here. While serving under both Tsarist and Stalinist regimes, he saw firsthand the errors of ego-driven policy. When Stalin wanted to build the world’s largest dam over the Dnieper River, he saw through the vanity of the project and recommended a network of smaller dams instead, each with a varied design. Inspired by the scientific method, Palchinsky treated every project like a hypothesis to test rather than an opportunity to flaunt his expertise. Each small dam project would allow testing of different designs, and could benefit from local expertise and unique approaches appropriate the site.

Both the humility and the experimentation are important. The ego is a surprising source of resistance to good experimentation practice. With Palchinsky’s dam proposal, having small projects led by different teams of engineers means the “top” engineer won’t be doing all the work. This seems suboptimal at first, but it’s not. We often forget that the most dramatic innovations in history–from CRISPR to penicillin to dynamite–happened mostly be accident. Just because you’re the most experienced person in the room doesn’t mean you have all the good ideas. Listen to new ideas, even when they come from “unqualified” people, and recognize that experience on one well-worn path is useless to a team that’s trailblazing.

In the words of Kendrick Lamar:

Be humble. Sit down.

Ingredient 2 - Survivable Failure

Seeking out new ideas and trying new things necessarily raises the probability of failure. Nobody likes to fail, but it’s crucial to recognize its role as the engine of evolution. We can’t expect to innovate without changing our attitudes toward failure. It needs to be celebrated rather than feared.

With one exception: We can’t fail so dramatically that we lose our ability to keep experimenting. Failure is great, ruin is not.

The Dnieper Dam was built in 1927 despite the opposition of Palchinsky and other engineers. Instead of the the spectacular achievement Stalin hoped it would be, it was a spectacular failure. In Winter, the water level in the river dropped too low to generate any electricity at all. To make room for the dam’s reservoir, 10,000 farmers were forced from their homes and acres of fertile farmland was destroyed. A russian hydrologist later estimated that they could have generated the same amount of electricity by burning the hay from the farms that were lost.

After Palchinsky’s death, the Soviet Union continued to ignore his advice, pursuing grand endeavors in engineering rather than the small-scale experiments he championed. Mikhail Gorbachev later blamed the Soviet Union’s demise on the failure of one of those projects, Chernobyl.

Ingredient 3 - Accurate Feedback

It’s not enough to allow some experiments to fail, we need a reliable way to determine which are succeeding.

Accurate feedback provides the crucial selection component of evolution.

Palchinsky was obsessed with data. He would often stubbornly refuse to give an evaluation without sufficient data, even if it meant collecting it himself. In most of the world today, accurate feedback is the most uncontroversial ingredient, but it’s the one that got Palchinsky killed. According to the files on him discovered by Loren Graham, his “crime” was “publishing detailed statistics on the mining and petroleum industries”³.

Feedback may be uncontroversial today, but I’m convinced this ingredient is the hardest to get right. And once again its the ego that we have to fear.

In the real world, projects aren’t abstract “things” being dispassionately observed to see if they succeed or fail. They’re created by people. And they generally don’t succeed without the passionate involvement of people invested in their success. When you’re sufficiently committed to a project, its failure can be personal, and it can be easy to let confirmation bias steer one away from truly accurate assessments of results.

You Need All Three

It’s remarkably common to see companies neglect one or more of these ingredients.

Facebook’s early motto “Move fast and break things” went all-in on experimentation without much attention to feedback or survivability.

John Doerr’s Measure What Matters has convinced thousands of companies to start collecting massive amounts feedback (OKRs) on “what matters” today. For some, this has resulted in a fear of trying new things that might reflect poorly in their OKRs. What matters today might not matter tomorrow. In other words, what if the OKRs themselves need to evolve?