How are folks approaching regression testing for web app security bugs, especially in cases where you may have remediated a small problem via mod_security or mod_rewrite?
In many cases where you have a code-related issue it is relatively straightforward to write new test cases in your software testing frameworks to test for recurrence and/or correct behavior.
In deployed web applications though you might choose to fix a simple hole via a webserver hack, config change, etc.
Most of the scanners out there could be trained to look for the hole in question and detect whether it recurs. Or, I could use something perl-mechanize to write up some test cases against the potentially vulnerable app.
Anyone have any recommendations for doing this?
I'm open to product ideas and/or toolkits. Ideally all fixes would be done to the originally vulnerable code-base, but in cases where that isn't the right approach, or isn't the initial approach, you still want continuous monitoring for issues.