The black box under the hood

The recent scandal at Volkswagen presents a fascinating story about the insidious use of software.

In case you missed it, the story is this: it was recently discovered that Volkswagen had installed software inside at least 11 million vehicles that was designed to fake out the US emissions control test. Supposedly the cars, when running normally, would fail this test, but the software could detect when it was being tested and would essentially force the car into compliance only for the duration of the test. (The technical term for falsifying test results is “cheating”.) Volkswagen has of course been publically humiliated by the revelation and the CEO has resigned in disgrace (though he might not have personally know about the software). All the cars are being recalled to have this fixed.

We all know that today’s autos are heavily loaded with control and diagnostic software, and this makes them susceptible to both design flaws and failures on the one hand, and external intrusion and hacking on the other. That cars can be hacked by a determined attacker has been well known and talked about for a long time. Google “automobile hacking” and you’ll see what I mean. From cars to air traffic control to pacemakers, the rule is this: if it has software and can be accessed externally, it’s vulnerable to a hack. The prospects of someone hacking into a pacemaker (yes it’s possible and you can find out how on the internet) are dismaying in the extreme – but that’s not the subject of this blog.

What’s disturbing about the Volkswagen story is that a major corporation deliberately used software to deceive its customers and its oversight bodies (US regulators who set emissions standards). The Volkswagen story is less akin to automobile hacking and more like Bernie Madoff’s doctored computer output that he used to hoodwink his clients.

Except that Bernie Madoff was just a sleazy Ponzi operator in a $3000 suit, abetted by two software developers who slid down a slippery slope from corporate cubicles to prison cells. VW on the other hand, is a $220 billion a year multinational corporation. Whatever they did was done on a massive scale, by a large number of people and with far-reaching results – including both increased pollution from VW autos, and an immeasurable amount of brand damage to a major company.

The story also tells us something about the increasing complexity of systems, and the difficulty that customers and regulators face in understanding or policing them. With systems like this routinely running to millions of lines of compiled code, who can tell what is happening under the hood? (The NY Times estimates that high-end cars today contain over 100 million lines of code.) Flaws and deceptions have likewise been discovered – sometimes tragically – in medical systems and factory control systems. Many more may be lurking undetected, whether introduced by mistake or malicious intent. It almost doesn’t matter (although of course it does).

One proposed solution to this challenge has been the idea of forcing auto makers (for example) to make all their code open source. Although this would devalue or destroy what they undoubtedly consider a competitive advantage, it would have the benefit of transparency: public scrutiny brings a lot of eyeballs of oversight to the game.

The term “black box” has long been used in the computer industry to describe a piece of hardware or software that performs an important function, but the inner workings of which are unknown and impenetrable. We are today surrounded by countless black boxes. We don’t know how they work, but we have no choice but to rely on them. The software under the hood of the Volkswagen was just one more in which we had to place our trust. When that trust is betrayed, we’re all at risk.

---------------------------------------------------------------------------------------------------
The VW scandal was well-covered by the NY Times, particularly here.