Ryan Barnett of Breach Security has a great post up on how to think about outcome-based metrics in a web application security world, instead of always being input-centric.
“We are focusing too much on whether a web application's code was either manually or automatically reviewed or if it was scanned with vendor X's scanner, rather than focusing on what is really important - did these activities actually prevent someone from breaking into the web application? If the answer is No, then who really cares what process you followed. More specifically, the fact that your site was PCI compliant at the time of the hack is going to be of little consequence.”
Spoken like a man who’s actually had to defend a website before, the U.S. federal ATF website incidentally. I bet he has some great stories he can never tell either. :) Ryan’s NFL analogies are borrowed from Richard Bejtlich, but I loved how he expounded upon them with his own.
“…vulnerability scanning in dev environments is akin to running an Intra-squad scrimmage.”
“Running actual zero-knowledge penetration tests is like Pre-season games in the NFL.”
“Web application firewalls, that are running in Detection Only modes, are like trying to have a real football game but only doing two-hand touch.”