Microsoft's "The Security Development Lifecycle" - Chapter Four

Chapter 4: SDL for Management
Chapter 3 here

Chapter 4 is about ensuring the success of the SDL through management support. As such its really the first chapter that I think starts to address some of the core issues of the SDL. This chapter makes a lot of sense in relation to Dave Ladd's post from the other day about culture change.

There are a few key takeaways for this Chapter:

• Management support is critical for implementing a software security effort
• Continuous messaging, reinforcement, recognition for a job well done, and training are all a part of the management support and culture change
• The SDL can add 15-20% overhead when it is first implemented but considerably less thereafter.

Chapter four again includes some background on what drove Microsoft to implement the SDL. Without throwing cold water on the four main reasons outlined:

• Customer complaints
• Actual attacks
• Press coverage - bad PR
• Frequent patching required diverting developers from new products to maintenance

I think another concern is liability. Microsoft's position as a monopoly in the desktop space makes them much more liable than other software companies might be. Regardless of the specifics of the list, all of these point to a purely economic analysis for why Microsoft chose to go down the SDL route. The SDL isn't in place merely because a bunch of smart developers thought Microsoft should develop more secure software. The SDL is in place because Microsoft management made the decision that the benefits of the SDL outweighed the costs of its implementation. This is a key point in gaining and sustaining management support.

Now a review of a few specifics of the chapter where I think they got it right and/or I have a few small comments.

On page-44 they discuss the vulnerability rate in Microsoft products post-SDL and the improvements seen. Similar to earlier chapters they claim that the discovered vulnerability rate has gone down and try to correlate that remaining vulnerabilities in the codebase. I'm not sure I agree with this assessment methodology. The jury is still out on 0day attacks, people not releasing vulnerabilities in certain key pieces of software, etc. Forensics data isn't widespread either as it relates to attack vectors. Most companies aren't very public about exactly how they were attacked, what vector was used, etc. So, we're a little blind on exactly how many attackers there have been against SQL-Server 2000sp3. Perhaps Microsoft has better forensics data than other folks because people actually report to them hacks?

Another point made is that as Microsoft's software has gotten harder to attack the attackers have focused their energy elsewhere. Its like the old joke....

Two guys are in the woods and they come across an angry grizzly bear. The bear starts to chase after them. One guy says "I'm sure glad I wore my running shoes today". The other guy says "It doesn't matter what shoes you are wearing, you can't outrun the bear." The first guy responds "I don't have to outrun the bear - I just have to outrun you."

On page-48 the section "Factors That Affect the Cost of SDL" they discuss the costs for existing vs. new development. One assumption that seems to be baked into their analysis is that implementing the SDL is an all-or-nothing sort of endeavor, and that roughly speaking it happens all at once. While it doesn't change the costs to implement over a long term or short-term, much of the language of the book seems to speak as if the SDL springs into being all at once.

On page-50 in the section "Rules of Thumb" they give the estimate that the SDL can cost 15-20% in initial implementation costs but less thereafter as a product matures. I'd be interested to know how many positive side effects they see from SDL implementation that aren't directly security related. Many elements of the SDL enforce a certain rigor on the development process that might not exist otherwise. You can't do proper threat analysis without good architecture documentation. And, the architecture documentation has to be accurate or the value of the threat analysis is going to be lower. In organizations that don't have a robust software development process to begin with I'm guessing that the artifacts of the SDL provide considerable value outside of security.

My only complaint so far about chapter four is that they don't spend more time discussing metrics. Key to gaining and retaining management support is demonstrating the effectiveness of the program. Chapter four has a limited section on SDL metrics and I can see that some chapters have explicit sections on metrics while other don't. The metrics provided in Chapter four aren't very extensive and they are sort of meta-metrics about the SDL - so we'll see whether this is covered in more detail later.