Laws of Supply and Demand Still Apply in Software Development

I read Gunnar's post the other day "Still Waiting to Meet a Developer Who Wants to Write Insecure Code" and he echoed something I've also been saying for a long time. In all of the training events I've ever done, times I've worked with developers, I've rarely met a developer who wants to actively write insecure code.

At the same time, there is a lot of bad code out there, and as much as we'd like to just blame managers, users, short timelines, that isn't necessarily the whole story.

Let's take a look at another industry - the home construction business. We see a vast difference in the quality of home construction based sometimes on how much the buyer is willing to pay, but also based on the supply and demand elements of qualified builders, and workers themselves. During regular non-boom times, the construction quality of homes while not excellent, is generally consistently "decent." Regulatory systems in many states (throw out the recent Texas fiascos if you will) are reasonably good at ensuring building code are followed, worker safety isn't compromised, and that the consumer ends up with a decent product.

Compare that to the construction of the last 5-7 years. Especially out here in California where the housing boom was vast, and demand for new housing, and consequently people to build them, far outstripped the supply of workers with experience and skills. The results in terms of quality are quite striking. Looking around at much new construction even when it was brand-new shoed all sorts of shortcuts. Baseboards glued on instead of done with nails. Cheap flooring put down onto uneven subflooring. Carpets coming loose because of shody installation. Paint jobs that don't match-up across a big wall. Showers that crack after 6 months because they we're installed level or with the right bracing. You name it, it happened. Shortcuts were the norm as housing was put up faster than reasoable but unqualified laborers.

How does this happen? The demand for new housing vastly outstripped the supply of those qualified to build them. It was a problem across the whole country, and conseuently we ended up with a lot of poorly bult houses.

Why didn't buyers enforce quality standards? Why didn't their inspectors? Again, a problem of supply and demand. A good home inspector has knowledge in most areas of home construction. They have the same skills you'd expect out of a general contractor. They understand electrical wiring standards, plumbing standards, general carpentry standards, etc. They are also experienced, and know where to sniff out the problems in a house they are inspecting.

So, what happened during the housing boom? We had a shortage of qualified home inspectors. In jurisdictions that don't mandate particular skills, we ended up with lots of unqualified housing inspectors. We probably ended up with a lot of kickbacks and bribery too, though admittedly I haven't seen any direct evidence for that, its just a guess. On the buyer front, we ended up with a bunch of first-time house buyers, who didn't know the kinds of problems to look for in a new home.

So, based on more demand than could be supplied with quality construction, we ended up with substandard product.

Hopefully you see where I'm going with this as it relates to software supply and demand.

In software our demand for software vastly outstrips the supply of qualified workers to build it. Software is worse though because at least in the housing case, in a normall regulated market, we have regulations and inspectors we have to satisfy. When the electrical inspector shows up on your job site, he (or she) is going to ask first for the plans and drawings. If you don't even have those, then you do't pass Go, and you don't collect your $200. You go right back to start, and have to get that fixed before you've even allowed to keep working, whether the work is being done right at all. If you don't have documentation, you're dead in the water. Sure the electrical work might be getting done right. And maybe you had the customer give you feedback on every outlet (agile?) but if you do't have documentation about what you're building, you stop what you're doing, and you on't get to start again until you do.

This isn't to say that I think the way we regulate buildings is the same way we should regulate software or computing. It is merely to point out that when you have a large mismatch between supply and demand, you're probably going to make sacrifices somewhere, and quality is one of those places.

So, maybe I've never met a develoepr who wanted to develop insecure software, but I guess that doesn't always imply that they wanted to, or were acpable of, developing quality software. In fact, given that much software is full of stupid and simple bugs, and that most development processes aren't structured to even make sure those don't slip through (make the subfloor level, make sure 2x4's are spaced right and at 90 degree angles) is it any surprise that we end up with software with lots of quality defects, with lots of security defects?

I remain optimistic that we can solve ths problem through training/education. But it isn't going to happen overnight, and it isn't going to happen until some of our attitudes about building software change from hiring amateur contractors who haven't ever built anything before, to hiring professionals who really know what they are doing.