Banning Programming Languages to Fix Problems?

Michael Howard had an interesting piece the other day on the SDL blog and also gave an interview about some similar topics. The subject I'd like to address is the banning of certain things during the development process and the theoretical and practical aspects of it. I'd like to show that banning function calls and enforcing annotations as a coding practice is reasonably far down the path to saying that C/C++ isn't such a good programming language from a security perspective and that to achieve higher levels of assurance we need to go further down that road.

I'd like to touch on a few points in this piece:

• The SDL vs. Legacy Code
  
• Banning Programming Constructs, Functions, and/or Whole Languages
• A Sliding Scale of Assurance

The SDL vs. Legacy Code

One of the things I'm struck by in Microsoft's SDL book is how pragmatic it is with respect to making software more secure. They point out repeatedly in the SDL book that products have been delayed and features scrapped because they either weren't secure, or couln't be made secure. While I can't argue with them about whether they've done this, there are at least a few places that we can point out a less than stellar track record of reducing feature sets to improve security. Web Browsers and how Windows does file associates come to mind as big security vs. feature wars that the feature folks seem to have won.

Michael and the other folks at Microsoft have done a ton of good work in creating, refining, and implementing their secure software development methodology, SDL. The SDL however reflects the realities of the Microsoft situation - millions of lines of legacy code. No one with that much legacy code can afford to start over from scratch. Anyone who suggested writing the next version of Word in All C# would probably get a swift kick. So, the Microsoft security folks are left with taking the practical approach. Given the fact that there is a lot of legacy code, and given that they aren't going to make a wholesale switch to another programming language, what is the best they can do in C++.

I think this is a fair approach but I don't really believe for a minute that if you asked one of the security guys there what programming language they want new things written in they'd pick C++ over C# for anything other than things like a kernel.

Banning Programming Constructs, Functions, and/or Whole Languages

Michael said in his recent piece:

In all of our SDL education we stress the point that .NET code is not a security cure-all, and we make sure that developers understand that if you choose to ignore the golden rule about never trusting input no language or tool will save you. Using safe string functions can help mitigate risks of buffer overruns in C/C++, but won’t help with integer arithmetic, XSS, SQL injection, canonicalization or crypto issues.

The key point is that languages are just tools; anyone using a tool needs to understand the strengths and limitations of any given tool in order to make informed use of the tool.

One final thought; in my opinion, well-educated software developers using C/C++ who follow secure programming practices and use appropriate tools will deliver more secure software than developers using Java or C# who do not follow sound security discipline.

I can't argue with Michael's point. I also don't know a lot of people in the security world that believe eliminating C/C++ automatically makes you secure. I do know quite a number of people though that make the good case that programming in C# or Java significantly reduces the exposure to certain classes of vulnerabilities and overall results in more secure software when using a develop trained in security.

WhiteHat Security's numbers of the general presence of web application security vulnerabilities per language seems to bear this out. In general applications written in ASP.NET and Java have fewer security vulnerabilities in them due to language constructs and secure coding frameworks.

I think you only have to look at what Microsoft has done with banning certain functions and requiring annotations to see that they have already taken some steps in the - C++ is bad - direction.

Why do they ban certain function calls such as strcpy, etc? Is it because the function inherently cannot be used safely? No. The reason is that its tremendously hard to use safely, and by removing it from the programmers options and replacing it with something that is easier to use, they improve the security of their software.

Why do they do annotations? They do them so that their static analyzers have an easier time in ferreting out certain classes of security defects.

If we could train developers to use all of the features of the programming language correctly, we wouldn't have to worry about either of these things. We'd simply do training and get on our way. The reality is that we cannot rely on developers to use certain portions of the language properly. We've shown repeatedly that certain function calls are the root of the vast majority of the security vulnerabilities out there. Most of the function calls that Microsoft has banned are ones that have been found to result in buffer overflows.

If we take as our starting position that developers can and will make mistakes, and we're wiling to enforce certain rules on them to constrain how they can use a programming language, why not take the next step and ask what else we can do to improve the security of the code we deliver?

There are several other areas of C++ that are problematic from a security perspective. Memory allocation comes to mind. The number of flaws we see from misuse of memory allocation is huge. Why not switch to a language that makes it fundamentally harder or impossible to have these kinds of issues?

Take the annotations Microsoft is using to hint their static analyzers. What these amount to are lightweight programming-by-contract constructs that we use for after-the-fact analysis rather than just forcing them through the programming language and calling violations errors. why not switch to a programming language that forces annotations rather than treating them as an after-the-fact construct?

A Sliding Scale of Assurance

In the end what we have is a sliding scale of assurance and functionality. Processes such as the SDL that mandate processes but don't require us to switch tools only get us so far down the assurance path. It is further than many/most companies are willing to go and credit to Microsoft for making such an effort. At the same time there are things the SDL doesn't necessarily do:

• Enforce the use of tools and programming languages that are more likely to reduce security vulnerabilities
• Eliminate software features that are truly problematic from a security standpoint and yet users have come to expect

If we want the SDL to be about delivering fundamentally more secure software at higher assurance levels, then it must mandate certain programming methodologies that result in higher quality code. Without these sorts of mandates we're still just putting band-aids on our existing software development processes.

Assurance comes at a cost. Development costs, testing costs, etc. When we switch tools that eliminate certain classes of vulnerabilities both remove the effort that developers would spend on securing the code, and also on the time the testers spend looking for certain classes of defects. In the end this is what Microsoft has done by removing certain programming constructs, they just haven't made the slightly bigger jump to another programming language for reasons I've already explained above.

Upshot

What I think this comes down to is use C++ if you must to muck with legacy code, but please don't do new development there. From a security perspective its going to be a lot more expensive if you do.

Microsoft's "The Security Development Lifecycle" - Chapter Four

Chapter 4: SDL for Management
Chapter 3 here

Chapter 4 is about ensuring the success of the SDL through management support. As such its really the first chapter that I think starts to address some of the core issues of the SDL. This chapter makes a lot of sense in relation to Dave Ladd's post from the other day about culture change.

There are a few key takeaways for this Chapter:

• Management support is critical for implementing a software security effort
• Continuous messaging, reinforcement, recognition for a job well done, and training are all a part of the management support and culture change
• The SDL can add 15-20% overhead when it is first implemented but considerably less thereafter.

Chapter four again includes some background on what drove Microsoft to implement the SDL. Without throwing cold water on the four main reasons outlined:

• Customer complaints
• Actual attacks
• Press coverage - bad PR
• Frequent patching required diverting developers from new products to maintenance

I think another concern is liability. Microsoft's position as a monopoly in the desktop space makes them much more liable than other software companies might be. Regardless of the specifics of the list, all of these point to a purely economic analysis for why Microsoft chose to go down the SDL route. The SDL isn't in place merely because a bunch of smart developers thought Microsoft should develop more secure software. The SDL is in place because Microsoft management made the decision that the benefits of the SDL outweighed the costs of its implementation. This is a key point in gaining and sustaining management support.

Now a review of a few specifics of the chapter where I think they got it right and/or I have a few small comments.

On page-44 they discuss the vulnerability rate in Microsoft products post-SDL and the improvements seen. Similar to earlier chapters they claim that the discovered vulnerability rate has gone down and try to correlate that remaining vulnerabilities in the codebase. I'm not sure I agree with this assessment methodology. The jury is still out on 0day attacks, people not releasing vulnerabilities in certain key pieces of software, etc. Forensics data isn't widespread either as it relates to attack vectors. Most companies aren't very public about exactly how they were attacked, what vector was used, etc. So, we're a little blind on exactly how many attackers there have been against SQL-Server 2000sp3. Perhaps Microsoft has better forensics data than other folks because people actually report to them hacks?

Another point made is that as Microsoft's software has gotten harder to attack the attackers have focused their energy elsewhere. Its like the old joke....

Two guys are in the woods and they come across an angry grizzly bear. The bear starts to chase after them. One guy says "I'm sure glad I wore my running shoes today". The other guy says "It doesn't matter what shoes you are wearing, you can't outrun the bear." The first guy responds "I don't have to outrun the bear - I just have to outrun you."

On page-48 the section "Factors That Affect the Cost of SDL" they discuss the costs for existing vs. new development. One assumption that seems to be baked into their analysis is that implementing the SDL is an all-or-nothing sort of endeavor, and that roughly speaking it happens all at once. While it doesn't change the costs to implement over a long term or short-term, much of the language of the book seems to speak as if the SDL springs into being all at once.

On page-50 in the section "Rules of Thumb" they give the estimate that the SDL can cost 15-20% in initial implementation costs but less thereafter as a product matures. I'd be interested to know how many positive side effects they see from SDL implementation that aren't directly security related. Many elements of the SDL enforce a certain rigor on the development process that might not exist otherwise. You can't do proper threat analysis without good architecture documentation. And, the architecture documentation has to be accurate or the value of the threat analysis is going to be lower. In organizations that don't have a robust software development process to begin with I'm guessing that the artifacts of the SDL provide considerable value outside of security.

My only complaint so far about chapter four is that they don't spend more time discussing metrics. Key to gaining and retaining management support is demonstrating the effectiveness of the program. Chapter four has a limited section on SDL metrics and I can see that some chapters have explicit sections on metrics while other don't. The metrics provided in Chapter four aren't very extensive and they are sort of meta-metrics about the SDL - so we'll see whether this is covered in more detail later.

More thoughts on training

My wife pointed out a very interesting article to me yesterday that she'd come across while reading Richard Dawkins's site. The article was about a lack of training/education in medical schools and how this lack of a basic understanding of evolution is at least partially to blame for drug-resistant bacteria.

I also read Dave Ladd's excellent piece, "Oil Change or Culture Change". Dave says:

Furthermore, many of the processes used by SDL (and other methodologies) are generally acknowledged as effective in identifying and mitigating security threats. Pondering this notion is what lead me to my realization about culture change, and prompted a question: If this stuff has been around for awhile in some shape or form, why aren’t more people doing it?

Dave's point is that we're not dealing with new knowledge here for the most part. We're dealing with a failure in education. In my previous posting about security training and what training is important I mentioned I'd had a conversation with Gary McGraw. Reading the pieces on Dawkin's site and Dave's piece made me remember something else Gary said that I think is very appropriate. I'm paraphrasing here so I hope I'll be forgiven but when I asked about the source of the problem Gary responded with an answer about your first engineering vs. first CS class.

Engineering Class: Professor opens class showing video of horrible engineering accident. Maybe something like the Tacoma Narrows bridge or the Challenger accident. In ominous voiceover - "If you don't study hard and do a good job, you could build something like this. People could die!!!!! Don't mess up, this is serious stuff."

Computer Science Class: Hello, look at this cool stuff you can do. Let's write a program that prints "Hello, World".

I think Gary is right. The culture we're trying to change is corporate culture, but it is equally computer science and programming culture. In a sense we have a chicken and egg problem. Until we have more companies demanding to treat software development as an engineering discipline, our universities won't be motivated to turn out students that treat it as such. And until schools start turning out software engineers rather than software developers we aren't going to have the talent necessary to get corporate culture change.

I don't know that we're at some sort of crisis point for software development education, I'd hate to be that melodramatic. At the same time I think what we're seeing is a disconnect between the realities of what it means for software engineering to exist as a true discipline, and our capability of achieving it. If we started with the mindset that it is engineering we're doing rather than "development" then we might stand a chance at making some progress.

I'll be interested in knowing the differences that exist at different university programs in CS and or CSE to see whether I'm wrong. Maybe there is broad support for a CS curriculum based in engineering rather than development.

Time for a bit of research on different CS programs and their focus.