Laws of Supply and Demand Still Apply in Software Development

I read Gunnar's post the other day "Still Waiting to Meet a Developer Who Wants to Write Insecure Code" and he echoed something I've also been saying for a long time. In all of the training events I've ever done, times I've worked with developers, I've rarely met a developer who wants to actively write insecure code.

At the same time, there is a lot of bad code out there, and as much as we'd like to just blame managers, users, short timelines, that isn't necessarily the whole story.

Let's take a look at another industry - the home construction business. We see a vast difference in the quality of home construction based sometimes on how much the buyer is willing to pay, but also based on the supply and demand elements of qualified builders, and workers themselves. During regular non-boom times, the construction quality of homes while not excellent, is generally consistently "decent." Regulatory systems in many states (throw out the recent Texas fiascos if you will) are reasonably good at ensuring building code are followed, worker safety isn't compromised, and that the consumer ends up with a decent product.

Compare that to the construction of the last 5-7 years. Especially out here in California where the housing boom was vast, and demand for new housing, and consequently people to build them, far outstripped the supply of workers with experience and skills. The results in terms of quality are quite striking. Looking around at much new construction even when it was brand-new shoed all sorts of shortcuts. Baseboards glued on instead of done with nails. Cheap flooring put down onto uneven subflooring. Carpets coming loose because of shody installation. Paint jobs that don't match-up across a big wall. Showers that crack after 6 months because they we're installed level or with the right bracing. You name it, it happened. Shortcuts were the norm as housing was put up faster than reasoable but unqualified laborers.

How does this happen? The demand for new housing vastly outstripped the supply of those qualified to build them. It was a problem across the whole country, and conseuently we ended up with a lot of poorly bult houses.

Why didn't buyers enforce quality standards? Why didn't their inspectors? Again, a problem of supply and demand. A good home inspector has knowledge in most areas of home construction. They have the same skills you'd expect out of a general contractor. They understand electrical wiring standards, plumbing standards, general carpentry standards, etc. They are also experienced, and know where to sniff out the problems in a house they are inspecting.

So, what happened during the housing boom? We had a shortage of qualified home inspectors. In jurisdictions that don't mandate particular skills, we ended up with lots of unqualified housing inspectors. We probably ended up with a lot of kickbacks and bribery too, though admittedly I haven't seen any direct evidence for that, its just a guess. On the buyer front, we ended up with a bunch of first-time house buyers, who didn't know the kinds of problems to look for in a new home.

So, based on more demand than could be supplied with quality construction, we ended up with substandard product.

Hopefully you see where I'm going with this as it relates to software supply and demand.

In software our demand for software vastly outstrips the supply of qualified workers to build it. Software is worse though because at least in the housing case, in a normall regulated market, we have regulations and inspectors we have to satisfy. When the electrical inspector shows up on your job site, he (or she) is going to ask first for the plans and drawings. If you don't even have those, then you do't pass Go, and you don't collect your $200. You go right back to start, and have to get that fixed before you've even allowed to keep working, whether the work is being done right at all. If you don't have documentation, you're dead in the water. Sure the electrical work might be getting done right. And maybe you had the customer give you feedback on every outlet (agile?) but if you do't have documentation about what you're building, you stop what you're doing, and you on't get to start again until you do.

This isn't to say that I think the way we regulate buildings is the same way we should regulate software or computing. It is merely to point out that when you have a large mismatch between supply and demand, you're probably going to make sacrifices somewhere, and quality is one of those places.

So, maybe I've never met a develoepr who wanted to develop insecure software, but I guess that doesn't always imply that they wanted to, or were acpable of, developing quality software. In fact, given that much software is full of stupid and simple bugs, and that most development processes aren't structured to even make sure those don't slip through (make the subfloor level, make sure 2x4's are spaced right and at 90 degree angles) is it any surprise that we end up with software with lots of quality defects, with lots of security defects?

I remain optimistic that we can solve ths problem through training/education. But it isn't going to happen overnight, and it isn't going to happen until some of our attitudes about building software change from hiring amateur contractors who haven't ever built anything before, to hiring professionals who really know what they are doing.

Headed South

Starting June 19th I'll be headed to the ICANN meeting in Sydney, and then also off for a little side trip to Auckland, NZ. Its all part of my plans to fix Internet governance, or at the least, stop things from getting worse.

If you're also in Sydney or Auckland and have any good restaurant or tourist recommendations, I'm all ears.

Or, if anyone wants to meet up, that's cool - drop me a line.

Quick Thoughts on Safari-4 Final

A few quick thoughts on Safari-4 on my Macbook.

1. Wow is this thing ridiculously fast. I'm usually a Firefox user and I'm not bogged down with a bunch of extensions/add-ons. Safari-4 feels about 2-3 times faster for 2 major apps - Gmail, and Google Reader.
2. I still don't like the EV-Certificate support. I wish I had access to Apple's user testing of the EV-certificate user interface to see about the security usability factors. I don't find it nearly as obvious or useful as the interfaces in IE7, IE8, or Firefox-3. I don't know Chrome that well though, so who knows whether I like it better than than.
3. The user interface seems a lot cleaner than the beta releases. Additionally, the hidden close x-marks on tabs that don't reveal until you highlight a tab is a nice feature.
4. First impressions of the web-inspector is a lot like the little bit of Chrome I've played with - pretty sure this is a webkit feature. Very positively impressed so far.

So, other than a relatively dismal security track record, I may stick with Safari-4 for a few "trusted" sites just to get some better impressions.

A Little Shameless Self Promotion

I was recently interview by Gary McGraw for his Reality Check podcast. Check it out here -

http://www.cigital.com/realitycheck/

Don't blame the judges

Two California court decisions came down last week that have people making all sorts of crazy claims about our legal system.

The first was the decision by the California Supreme Court affirming Proposition-8, the amendment/revision to the California Constitution.

The second was a decision by a California court ruling that the service offered by LifeLock is illegal under California law. An article on that decision is here.

In the Wired article about the LifeLock decision, we have the following passage:

But Chris Hoofnagle, director of information privacy programs for the Berkeley Center for Law and Technology, says the ruling is a disappointment.

“The idea that we could some day see a market where we pay $10 a month to a company to opt us out of junk mail, to monitor our credit, to do all sorts of privacy-enhancing steps that we don’t have time to take … for that market to emerge, LifeLock’s business model and similar ones have to be legal,” Hoofnagle says.

I find this comment puzzling. All the judge did was interpret the law, he didn't make it. Nowhere does Mr. Hofnagle say the judges ruling was wrong on the facts, but he criticizes it anyway.

There was also rather a lot of this same type of commentary about the California Supreme Court's decision ruling that Proposition-8 is legal and will stand. Much of the complaining was along the lines of "but how can they take away our rights like that?" It isn't that I'm not fundamentally sympathetic to this position, but like it or not, the California constitution is a complete mess, the direct democracy system we have is a complete mess, and it all needs reworking.

This all points me to a couple of conclusions.

1. If you're not winning under the current rules of the game, change the rules. This can be through getting a law passed, or even through amending the constitution.
2. The California direct democracy experiment has gone horrible wrong, and its time to write a new constitution that eliminates this nonsense.
3. Richard Friedlander had it right in this perspective piece on KQED.

And lastly, all of this complaining about judges reminds me of a famous Richard J. Daley quote, which I'll tweak here for my own purpose:

The Judges aren't here to cause disorder, they're here to preserve disorder.