I care deeply about innovation. It is literally where the future comes from, but it is a curious thing. Innovation and its more powerful partner, invention proceed at a snail’s pace. “In a world where change is the only constant” is a lie. It took this world 20 years to invent a decent can opener. And as there are sufficient problems waiting to be solved (affordable healthcare for all, energy that does not deplete our resources and poison our atmosphere, transportation that does not clog up infrastructure to name but a few), we need to do better.
Yet most most recent material on innovation is vapid and focuses on what I’d like to call zeroth order invention: rearranging well known pieces of technology into new shapes. Great examples include such startup darlings as Uber, AirBNB, Snapchat & Slack. While some of these companies enable us to do wonderful things we couldn’t do before, almost all of their innovation is restricted to the (non?)business model. Nothing new is invented as these startups were built on a rearrangement of classic and trusted technologies: GPS, Web, Databases, Reviews.. and that 1988 invention called IRC.
What I care about is actual innovation where new stuff gets invented and we move beyond serving up known things in a novel shape. To see what this means, imagine Uber starting up in 2001 where they would have had to deliver reliable GPS, be able to run on REALLY slow mobile networks and invent feedback algorithms to rank drivers and customers. But with sufficient funding and energy, it could have been done. And then “Uber2001” would have helped invent the future. Instead, they launched in 2010 when apps, mobile networks, precise positioning and review technology were all well established, and they “only” had to actually roll it out.
Whole bookshelves are written about startups that bang out Minimum Viable Prototypes within months and then pivot and iterate themselves to greatness. And don’t get me wrong: this is already hard enough to do. But the most famous of these books, The Lean Startup literally only contains the word “invent” 5 times — and only once in the context of a startup inventing something.
Sadly, I know of only a few relevant books on what I like to call “first order” and “second order” innovation: Skunk Works, The Idea Factory and The Right Kind of Crazy (which I review below after discussing how hard innovation is).
In first order innovation, there is actually something left to be invented beyond how to get traction with our users. Your new idea is right now not possible with reliable & available technology. You’ll have to invent stuff or have to take research-grade ideas and productize them into something that actually works.
A great example of first order innovation is Google back in the late 1990s. At the time, search technology was simply not able to index the internet at sufficient speed or to find useful links because no one had ever needed to index such stupendous amounts of widely distributed & unstructured information that also changed quickly. The few search engines that came before (HotBot, Altavista) simply could not do it. To make it happen, Google had to actually invent something new. But even they could innovate out of a garage by applying their considerable smarts to existing technologies you could buy more or less off the shelf.
The scariest kind of innovation however is where you have to invent things so you can invent the things you need. This is truly is frightening but this is also where the leaps and bounds happen that deliver our future. The best documented example I know of this ‘second order innovation’ comes from Bell Labs, as elaborated extremely well in the fascinating book “The Idea Factory”:
“Bell Labs was behind many of the innovations that have come to define modern life, including the transistor, the laser, the silicon solar cell and the computer operating system called Unix. Bell Labs developed the first communications satellites, the first cellular telephone systems and the first fiber-optic cable systems.
The Bell Labs scientist Claude Shannon effectively founded the field of information theory, which would revolutionize thinking about communications; other Bell Labs researchers helped push the boundaries of physics, chemistry and mathematics, while defining new industrial processes like quality control.” — New York Times
To achieve their goal of telephone calls between New York and San Francisco, first Bell Labs had to invent really good electrical conductors of voice, attempting iron wires, copper wires and finally what we now know as twisted copper pair.
This got them to around 1700 miles, but not yet to San Francisco. To do that, they picked a poorly understood amplifying device called the Audion, in which hid a predecessor of the vacuum tube (without an actual vacuum). Once they did the research and got it working in the lab and could fix what was wrong with it, they were able to scale it up to to production quality and managed in 1915 to deliver the first voice call between New York and San Francisco. In this process they had created the first practical vacuum tube, the predecessor of the transistor that powers all current computers.
In another great example, Bell labs then actually invented that transistor, but when they tried them out for real, they didn’t work well enough because of impurities in the materials used. Early transistors were so sensitive they could stop working if someone waved a hand or slammed a door, even simply touching the device broke it. The causes were traced to impurities, so eventually Bell Labs innovated in metallurgy too and invented Zone Melting which finally enabled production grade semiconductors.
An additional example of inventing things for the things you want to invent: Skunk Works at Lockheed did research on supersonic plane materials and discovered titanium would fit the bill. Then they had to invent the tooling that could actually machine titanium! Even more far out was the (unrelated) Manhattan project which had to invent new techniques in order to enrich Uranium in order to make an atom bomb.
So in short — they had to invent things to invent things to invent things. And now we finally get to the point of this post — while literature abounds on how to ‘lean startup’ yourself into business, there is almost no guidance how to truly operate in the terra incognita of real invention. And guidance is sorely needed as in this territory everything is stupendously difficult.
The Curse of Dimensionality
First, how hard is even basic innovation? Let’s say we want to build a new product out of existing technologies. Something mundane, maybe a car, maybe a slightly novel electrical toothbrush or a cloud service. So what are we going to build? A high-end product with loads of features? Or one with few features but very solid quality?
Or neither, but it is going to be super stylish? What price point will we pick? This is 4 dimensions of choice: features, quality, styling, price. And these are of course not unrelated — we’d be hard pressed to make a device with lots of features, quality, styling AND a low price. A fifth dimension then pops out, how much effort and/or time are we going to spend reaching this goal?
Note that the just announced Tesla Model 3 intends to be at that sweet spot of great features, high quality, looking good with attractive pricing, and therefore we’ll only see it in production at least two years from now.
The mind boggles at five dimensions, so let’s just look at two for now. Assuming a fixed setting for quality and styling, where should we position our new product in terms of price and features?
If we deliver loads of features for a small price, we’ll surely be very attractive. But the question is, can we actually do that?
If we deliver fewer features for the same amount of money, our margins will be healthier, but perhaps we won’t sell anything because we end up being seen as too expensive.
And even if we stick closely to a “fair” pricing per feature, we might still end up with a product that is either too basic (“good price, but doesn’t solve my problem”), or one that only appeals to the high end of the market (“I can understand the price, but I can’t afford it” or as we say in Dutch “Not expensive, but still a lot of money”).
In the middle there is of course a safe ground, the “Goldilocks zone”, medium features, medium quality, medium price. But if you put your product there, you’ll probably share that spot with a dozen competitors who got there first. Divining where to go among five dimensions is as much an art as it is a science.
As an aside, I heard a lovely story from AMD engineers way back when in 2002 at the Ottawa Linux Symposium where the first low priced 64-bit processor was unveiled to kernel developers. AMD had sent lots of their engineers there and over drinks they told us how they had been positioning CPUs in terms of capabilities, performance, power-use and price, since you have to think that through before you start the multi-year process of actually making a new CPU.
And they described how one CPU candidate among many (Opteron, Duron, Athlon etc) had come quite a way in this process, but that people had been adding and removing features and changing resource envelopes for so long that at one point it became clear it was no longer a product that made any kind of sense. The CPU then acquired the nickname ‘Moron’ and never did see the light of day.
So picking out what to do among even a few dimensions is stupendously hard. But if you are on the leading edge of innovation, this is only where your problems start. Not only do you have to pick what your stuff should do, you have to include all new dimensions on how it should actually happen.
So let’s say we want to go to Mars and land a huge rover there to do a science. The money isn’t committed, but we are hoping to launch it within a decade. It should carry a thousand kilos in instruments and last for 5 years.
A quick inventory shows that no landing device exists that meets the criteria — there are landing systems that can land in an empty plane devoid of anything interesting (Viking style). There are landing systems that can deliver a smaller sized rover safely to exciting territory. But nothing that gets your car sized robot anywhere interesting.
The sheer amount of choice is overwhelming, but anything you investigate is going to take years to come to fruition! Bigger airbags? Larger parachutes? Flying the rover like an airplane with lift? Landing on a “crushable base” and then drive off from it? Choices abound!
The Right Kind of Crazy
This is true innovation as I like it. It requires invention, doing things that have never been done before. And in the process, inventing stuff so you can invent your stuff. Adam Steltzner has delivered (with William Patrick) an important book that on one level is about how Adam progressed from a drifting musician to the guy in charge of landing the Curiosity rover on Mars. And an entertaining and exciting story it is too, by the way.
But Adam also makes it quite clear that the book has a bigger message. How do you navigate through all these dimensions of a problem to something that lands on Mars.. at the first try? And he has a few surprising messages on this front.
“The Right Kind of Crazy: A true story of teamwork, leadership and high-stakes innovation” delivers a somewhat surprising toolbox for inventing the future. And much like The Lean Startup has given us a “startup lexicon” of MVP, Pivot, Traction, Adam provides us with terms that describe high stakes invention.
Your spidey sense
For me the most surprising thing in the book, something that I have never seen written down before in credible literature is the absolute need to trust.. your spidey sense. Yes.
Recall from above the search among just two dimensions to find the right direction of our innovation. Any real innovative project will have far far more dimensions to consider. Huge hypervolumes within that space don’t deliver working solutions, or solutions that work but please no one.
And much like how a chess grandmaster sees many moves into the future, but can’t consciously recall actually evaluating billions of positions, we need to bring an unseen side of our consciousness to the table that will enable us to navigate our huge problem space and find solutions that smell right: the spidey sense. And I absolutely know this to be true.
So what does that spidey sense deliver? It allows us to pick the things that are going to work as well but also enables us to smell it when things go awry. Now, is this magic? There is no need for that. This sense only comes to people that have been around the block.. numerous times, much like the chess grandmaster that has needed to play and understand thousands of games to get to that level of brilliance.
And in a similar way one can develop this sense of what will work and what won’t. And it had better work — because from even a cursory calculation it becomes clear that even if we could try and build one Mars lander configuration per second, we would still not be able to scan all possibilities within the lifetime of the universe.
In sharing Adam’s belief in the spidey sense I have to admit a personal bias. Even though my software powers a significant fraction of the internet, I know I am not the greatest programmer in the world. In fact, many better programmers than I work on PowerDNS. But my spidey sense ranks up there.
It is not just the facts that count
Adam describes how he discovered the world of physics and hard sciences and immediately fell in love: finally here was something that was just true. No discussion needed, no ifs and buts: just truth on how the universe works. Many aspiring engineers have fallen in love with this extremely clear truth, and it shows in how we operate and converse. It works well in some circles, but it only gets you so far.
Adam tells us to break off that love affair and look beyond just being right. Being right all the time does not actually get your robot on Mars if you were so right you were denied funding by people that have better things to do than being told they are wrong.
There is a need for soft skills because great things only happen when teams of people give it their all. Love your team. If you can’t love the whole person, try to find at least one aspect in them to love. Make the magic happen.
Because greatness has never come from process. Greatness may be facilitated by procedure, but it will never deliver the kind of passion that makes someone drive to the lab in the middle of the night to try out a hunch that ends up saving the mission.
Holding on to the doubt
Innovation is not on the human brain. When confronted with something new, it is in our nature to reject it as newfangled stuff that surely is worse than what we know already. All innovators have painstakingly trained themselves out of this reflex (although later in life they may sadly regain it).
Allowing yourself to believe in new things does carry a downside. Once you have allowed something new in.. you may fall in love with it. And this is why Adam cautions us to Hold On To the Doubt.
If you don’t Hold On To the Doubt (HOTTD) you will miss the signals you picked the wrong solution. As an example, if during the development of your project the great algorithm you invented fails to work just once unexpectedly, it is easy to dismiss that as a glitch. “Must have been bad data”. But this is not how you land rovers on Mars. That one glitch needs to trigger the doubt on if you are on the right track. It should feed your spidey sense & make it tingle.
HOTTD is what keeps your love of new innovative things in check and makes sure you don’t follow your favorite invention into a smoking crater on Mars.
The Dark Room
As a concept, this is golden. Within any truly innovative project, you will hit a wall and get locked up in the truly scary place that Adam has named “The Dark Room”. Or, as told by Tom Wolfe in The Right Stuff:
Sometimes at Edwards they used to play the tapes of pilots going into the final dive, the one that killed them, and the man would be tumbling, going end over end in a fifteen-ton length of pipe, and he knew it, and he would be screaming into the microphone, but not for Mother or for God or the nameless spirit of Ahor, but for one last hopeless crumb of information about the loop: “I’ve tried A! I’ve tried B! I’ve tried C! I’ve tried D! Tell me what else I can try!”
This is the Dark Room, except less deadly. The project can’t use SQL database X, NoSQL database Y could do it but is not reliable enough, the supposed magic from vendor Z kills the business model. We’ve hit a fundamental dead end. Nothing goes. To a more junior person this can feel like the end of the world.
Adam teaches us that 1) any self-respecting innovative project WILL end up in The Dark Room one day and 2) there is always a way out, as his fascinating Mars landing examples show.
The role of leadership is paramount here however. Even if you don’t yourself as a leader know how to get out, telling people stories about how you did so in the past will liberate minds and make them find solutions. The Dark Room is not something to fear, and sometimes recognizing that you are heading for one and accepting it is the best way to get out.
Summing up
High-stakes innovation that actually invents new things has received far less press than typical startup innovation. Reading The Idea Factory and Skunk Works will reset your brain about what actual innovation and invention are. And The Right Kind of Crazy will do likewise, but also hand you a mental toolbox to help you invent the future. Enjoy!
