Tuesday, August 03, 2010

Website Vulnerability Assessments: Good, Fast, or Cheap - Pick Two

Website vulnerability assessments are part of any mature secure software development lifecycle. Today software, especially web-based software, is being updated at ever increasing speeds with the adoption of agile and other iterative development methodologies. To fit security into that fast paced life-cycle, application security must find a way to fit within the delivery requirements AND constraints. Upon immediately reading the headline, software managers will be well familiar with what I'm getting at. So website vulnerability assessments, ideally you want them good, fast and cheap.

= comprehensive assessments focusing on finding as many of the vulnerabilities as possible that bad guys really exploit. This requires an experienced pen-tester, a top-tier scanner, and a thorough threat-based testing methodology.

= assessments are those completed within a couple days or more specifically within a given QA testing window where preferably any outstanding issues can be addressed before production release.

Cheap = assessments are those that can be routinely performed with each code change without exceeding the allocated budget.

The challenge is, as illustrated by the Project Triangle, that unfortunately you can’t have it all. Choices and tradeoffs must be made. As wikipedia elegantly puts it (Project Triangle): Like any human undertaking, projects need to be performed and delivered under certain constraints. Traditionally, these constraints have been listed as "scope," "time," and "cost.”

When it comes to website vulnerability assessments, enterprises are faced with a similar choice:
  • Performed comprehensively and quickly, but it will not be cheap.
  • Performed quickly and cheaply, but will likely lead to missed vulnerabilities and potentially a security incident.
  • Performed comprehensively and cheaply, but it will take a long time.
Good, Fast, or Cheap -- Pick Two.


Calandale said...

I think you may be overoptimistic in allowing to pick TWO. Fast enough to meet the QA terms that you elucidate isn't enough time to learn the application sufficiently. And, no matter what, a top-tier scanner isn't cheap, in my book.

You can definitely manage fast and cheap, so that's where the industry settled.

Anonymous said...

Really? You think it's possible to get both "good" and "cheap"? I'm skeptical.

I suspect it's not "good, fast, cheap, pick any two"; I suspect it's "good, cheap, pick any one".

Jeremiah Grossman said...

@Anonymous: The great thing about having a blog is that I can start off with one opinion and collect insight from others.

"good, cheap, pick any one".... you might in fact be right upon further reflection. :)

alan shimel said...

Jeremiah there was a good song once that was titled "two out of three ain't bad", the same applies here I think.

Sherif Koussa said...

I agree, getting down to business requirements and fast pace rhythm, it DOES boil down to fast, cheap and good. It is funny because I was thinking about a similar topic a couple of month ago and I ended up with a conclusion that in order for security to really penetrate into software development life cycle the three has to be available? Would we get there one day? :)

Jeremiah Grossman said...

@Sherif: can't have all three, simply impossible -- unless you define the terms to your benefit. When it comes down to it, the business really has to decide precisely HOW good, HOW fast, and HOW cheap they need to be. Or least we need to provide them their options.

RobbReck said...

You're right in summarizing that those three elements are the essential tension. But I believe there's a key component missing from that summary. One important reason for having on-staff InfoSec practitioners is to create a situation where you CAN have all three.

As we refine our processes and learn more about our customers the objective should be to get faster, better and cheaper. My goal, as a InfoSec provider to my employer, is to give them all three legs as well as possible.

While the tension between those three aspects won't go away, by getting better at our processes we can deliver high quality products in a short time for an affordable cost.

Just my two cents...

Robb Reck

Unknown said...

My 2*10^-2 cents...

Good = Breadth * Depth
We can gain in breadth either by allocating more time or more money.
But. We cannot gain in depth by allocating more time. We need to allocate money. I.e. a tool or a man that cannot perform specific kind of analysis wont be able to overcome its limitations just because of a longer run :)