One of the problems, as was pointed out before, is that software and computers don't have a fixed use that can be anticipated during the development cycle, and consequently saying that software isn't "fit for purpose" is a really tough judgment call.
I started thinking of other products where bad outcomes happen even during correct use,where the flaws aren't necessarily the fault of the manufacturer. Pharmaceuticals come to mind as a product that have:
- Large safety concerns
- Potentially large benefits (antibiotics sure are nice, aren't they?)
- Per-individual side effects that are tricky to predict
- Computer testing of toxicity
- Animal testing of toxicity
- Stage-1 trials in humans (small group) to test toxicity and effects
- Stage-2 trials (larger number of people) to determine drug efficacy
- Stage-3 clinical trials (hundreds to thousands of people over 1-3 years) to determine efficacy, adverse effects, etc.
- Drug interaction trials and labeling
- Extensive documentation trail
- Get FDA Approval
- Post Approval
- Adverse event reporting capability
- Updates to labeling
- Constant quality checks
- Individual "allergic" reaction
- Complicated or unforeseen drug interaction
- Unsafe Product
- Long-term safety issues that didn't surface during clinical trials.
A pharmaceutical company can no more anticipate individual allergic reactions than a software vendor can guess at how someone is going to use their software. What matters most in determining liability is the level of due diligence and proper process that went into the product development, not the outcome itself.
All of this costs money. Current estimates are that developing a drug and bringing it to market costs approximately $800-million dollars. Individual manufacturing costs are generally low such that the first pill that comes off the production line costs $800-million and each additional pill costs 5-cents.
There is a big discussion going on right now about flaws in the process on the legal, FDA, and Pharmaceutical side. Right now its tricky to bring smaller targeted drugs to market because the costs are prohibitive to develop and gain approval for a new medication. The Economist had a few recent pieces of how drug companies are trying to develop targeted medications and how FDA regulation may be doing more harm than good in some cases.
If you've read this far you may be asking yourself what this has to do with software liability and software security. The points are:
- Other products are subject to heavy regulation but still manage to turn a profit
- The quality of the process doesn't always guarantee a quality outcome - especially in the face of uncertain product use
- If we impose too much liability on software manufacturers we could drastically raise prices and/or reduce the amount of software available
- Sometimes regulation does more harm than good