Billion Dollar Bugs
15 February, 2002
So you don't write code for NASA? Your code isn't running on a life support system or medical equipment? If there's a bug, QA will find it, and besides, you're behind schedule and you needed to get this out the door last week! Right?
Wrong. Your bugs, design flaws, and security holes could cost your customers billions of dollars. If the business environment ever changed so that software vendors were liable for defective products just like any other vendor, your company could be bankrupted by the lawsuits.
Today my boss sent around this CERT advisory to our group. I responded to the group, saying that the root causes of these vulnerabilities are mostly software bugs. I included two [dead link] links to secure programming FAQs. One of my co-workers responded to me, asking "I don't understand the relationship between the CERT SNMP advisory and the links that you specified." Ok, so that's not exactly a question.
The point is that we shouldn't just be looking at the SNMP stack we bought. We should also look at code that we've developed in-house for bugs. "Well, duh", I can hear you saying. "That's why you have a QA group." Of course we have a QA group. We have excellent testers! But do we really expect them to write special software that attempts to exploit every possible nook and cranny in our system? Not exactly.
Testing groups are more concerned with black box testing. They don't usually get involved with examining the source code, so they aren't exactly in a position to know where the nooks and crannies are hidden. (Hmm, funny thing. I have a hankering for an English muffin right now.)
I've heard people remark, "We don't need to operate at that level. We're not writing code for the space shuttle." Implicit in this remark is the idea that NASA programmers are very methodical and cautious. Some might say they're the best in the business. Part of the reason for being careful on code for, say, the space shuttle, is the enormous cost and the potential for loss of life. According to the NASA website, the Space Shuttle Endeavour cost $1.7 billion.
If you are writing code for a widely deployed piece of software and there is a serious flaw, the costs imposed by that bug can be enormous. Case in point: Outlook Express is installed on millions of computers around the world. This software was obviously not scrutinized (neither the design nor the code would be my guess) for security flaws. But this was obviously ok at the time, because the software was a) largely being distributed for "free", and b) there are no lives at stake -- it's a mail client. Right?
Wrong. According to this article at cnet (and other sources), the "Love Bug" virus caused an estimated $8.75 billion in damages. That's more than five space shuttles! What would it have cost the programmers at Microsoft to think a little bit more carefully about their design for Outlook Express? Two or three million, tops? I guess it doesn't matter now that Microsoft has made a strong commitment to creating secure software. "SirCam", which also propagates through email, cost $1 billion. "Nimda" was a little more clever, attempting to replicate through multiple methods. One expert estimated that Nimda cost $500 million in one week! Hey, wait, I'm not finished: "Code Red" exploited multiple flaws in Microsoft's IIS, costing an estimated $2.6 billion.
That's over $12 billion in costs to users and those are just Microsoft's programs. I could talk about bugs in sendmail and other widely deployed open source software, but I won't. (Because you should be using qmail.)
I haven't heard any stories of anyone being killed or injured because of these viruses, but the situation could easily arise: imagine that a worm like Code Red (but with a more vicious payload) is highly active at the same time a major earthquake hits on the US West Coast. People are calling 911 to report emergencies, but the emergency dispatch centers are under attack by the worm. Or people are trying to make calls or send emails to let family members know that they are ok, but major networks are swamped because of the extra traffic introduced by the worm. Or even something as basic as hospitals' information systems getting swamped because of internal or external traffic caused by worm infections. A ten minute delay in getting an ambulance to a victim can be the difference between life and death. Even though you're just writing a web server, you have the potential to either save lives or kill people. That's scary stuff, from where I sit.
The economic costs related to viruses and virus-like software were $17.1 and $13.2 billion in 2000 and 2001, respectively. I'm not sure if those estimates take into account the tremendous amounts we spend on anti-virus (AV) software (the AV market is $2.8B (source: http://infotech.indiatimes.com/enterprise/emergintech/monster.html, this link is now dead). This wouldn't be necessary if systems were designed for virus resistance in the first place.
The same goes for firewall software. According to this report [dead link], firewall software is fastest growing software market, with AV software right behind. These systems shouldn't be as important as they are! Instead of putting layers of armor over all of these defective products, why not just make the products bullet-proof to begin with? Because after all, firewall software and AV software is, well, software -- and is not immune to bugs and security flaws.
If we got into the habit of creating software that was immune to viruses like Code Red and Love Bug, we'd gain from a reduction in regular bugs (the ones that make software crash without malicious intervention). Those cost money too: lost productivity, lost data, inaccurate data, lost business, etc.
I'm even going to go out on a limb: bulletproofing all software, not just the stuff that NASA uses, would cost nothing. The reason that I say this is that the techniques that are often used to produce "zero defect" software would provide offsetting gains. We'd gain from programmers' increased productivity, better schedule predictability, and decreased testing requirements. Not to mention that our systems would be far less vulnerable to viruses and malicious intruders.