The future of web application vulnerability assessment is about scale

Recently Alan Shimel (StillSecure) went out on a tiny twig and said, “vulnerability assessment (VA) is dead”. Of course Alan’s speaking about network security not web applications. His remarks are about VA's convergence with NAC’s. Fair enough. When I spoke with him he said, “Actually VA for web apps is one of the few bright spots in the VA space these days.” I'd like to think so. :) This topic is always on my mind since this is exactly what my company does. “What is the future of web application vulnerability assessment?” is a question that doesn’t get asked a lot. Personally I think we’re at the point where network VA was a few years ago, solving the challenge of scaling.

Where are we now?

• 105 million sites are on the Web with 4 million new ones each month.
• Perhaps hundreds (?) of thousands of websites collect or distribute personal information, financial and healthcare data, credit card numbers, intellectual property, trade secrets, etc.
• Web application issues top every major Top-X vulnerability list.
• 8 out 10 websites are full of holes and most of the attacks are targeting the web application layer.
• Assessments should be performed after each code change or "major" release and require about a week or two of human-time to complete.

We need to get our arms around the problem.

Analyzing the scope using some assumptions:

• 500,000 “important” websites (roughly 1/2 of 1% of the total population)
• Assessments 2-times a year per website. (Vary on change rate)
• An expert can perform 40 assessments per year with base salary of $100,000 (US).
• Retail cost per assessment $5,000 (US). (Normally higher ranging between $8,000 - $15,000)

Granted my numbers could be off and may vary a great deal from enterprise to enterprise. However, this exercise helps estimate the relative needs of the market. Let's see what kind of resources we need if we're trying to assess all these websites for vulnerabilities twice per year.

Today we'd need:

• 1 million total vulnerability assessments
• 25,000 experienced experts in web application VA
• $2,500,000,000 (US) in salary for web application experts
• $5,000,000,000 (US) retail assessment cost

Even though the assumptions were way the conservative side, it’s immediately apparent that this scenario is completely fictitious. There are probably only 3,000 experts (a guess) in the world qualified to perform assessments relative to the 25,000 required. And much as I’d wish they would, enterprises are simply not going to spend multi-billions on web application security in 2007.

Of course as the awareness of web application security builds the numbers will climb, but for now we have to face facts. And the fact is unless we can vastly improve the web application VA process, most websites will not be assessed for security and remain insecure. That’s what’s going on today. And that’s why I’m saying the future of web application vulnerability assessment is about scale.

While we certainly can’t reduce the number of “important” websites, can reduce the number of man-hours and expertise required to perform an assessment using technology and a modern processes. Modern assessment processes need to be highly streamlined, repeatable, thousands running concurrently and performable by less than top-tier webappsec experts. This is what it truly means to “scale”.

How much improve can be made near term is a subject of much debate, but we’re working on it. For fun, let’s try a few more guesses at how certain efficiencies will help.

Future improvements:

• 500,000 “important” websites (roughly 1/2 of 1% of the total population)
• Assessments 2-times a year per website. (Vary on change rate)
• An expert can perform 40 200 assessments per year with base salary of $100,000 $80,000 (US).
• Retail cost per assessment $5,000 $2,000 (US).
  

Adjusted requirements:

• 1 million total vulnerability assessments
• 5,000 experienced experts in web application VA
• $2,000,000,000 (US) in salary for web application experts
• $400,000,000 (US) retail assessment cost

These numbers are much more palatable in the grand scheme of things and gives us our benchmarks for where technology and process must bring us to. How long will it take to get there is anyone's guess.