Web Application Security Professionals Survey (Dec. 2006)

Update 2: It looks like our little survey here is reaching some critical mass. Kelly Jackson Higgins from Dark Reading posted 'Not Much Resistance at the Door' covering the results. During the article one of the areas we haven't asked questions about came up, intranets:

"The scary unknown is intranet Website vulnerability, however, which the survey did not address. "There are no good metrics for how many intranet Websites there are, or how vulnerable they are. That's a big unknown in the industry," Grossman says. "It's a whole other world inside the firewall."

Update: Once again a great survey turn out. A total of 63 respondents. Thank you everyone whom responded and those who helped me out with the questions. We didn't reach my 100 prediction, but that's OK because I'm not very good at those anyway. :) We'll try to get there in January. The problem is its getting difficult to manage this by email, I'll have to figure out some way to remedy that. The data collected certainly did not disappoint and some interesting things bubbled to the top.

My Observations
Good representation from both security vendors and enterprise professionals. Most of which have several years of experience, a significant percentage of their time dedicated to web application security, and performed 1 - 40 assessments in 2006. An experienced bunch I’d say.

About half of organizations have vulnerability assessments performed see security measurement as the primary security driver and about a quarter say compliance. I would have figured measurement would have scored higher and compliance lower. Maybe we’re seeing a shift in the industry.

The vast majority of webappsec professionals believe assessments should be performed after each code change or "major" release, takes a week or two to complete, and rarely encounter multi-factor authentication or web application firewalls. About half of people using commercial scanners say scanner complete about half or less of their workload. The other half of people who don’t say assessment are faster to do by hand, have too many false positives, or too expensive. There is much more to talk about here, but that’ll come in another post.

Question 12a on disclosure yielded some interesting results. There majority of people are evenly split between “responsible” and “non” disclosure. Think about that. Just as many as are disclosing as those who don’t because there is inherent risk. As I’ve said before, discovery is going to be a big issue moving forward. We’re going to loose the check and balance we’ve relied upon with traditional commercial and open source software.

Description
Its been a month already since the last survey. In November we got a great turn out doubling the response from October. Maybe this time we'll reach 100 respondents. Anyway...

If you perform web application vulnerability assessments, whether personally or professionally, this survey is for you. 15 multiple choice questions designed to help us understand more about the industry in which we work. Most of us in InfoSec dislike taking surveys, however the more people who respond the more informative the data will be. So far the information collected has been really popular and insightful. And a lot of people helped out with the formation of these questions.

================================================================
Guidelines

• Open to those who perform web application vulnerability assessments/pen-tests
• Email your answers to jeremiah __at__ whitehatsec.com
• To curb fake submissions please use your real name, preferably from your employers domain.
• Submissions must be received by December 14.

Notice: Results based on data collected will be published.

Privacy: Absolutely no names or contact information will be released to anyone. Though feel free to self publish your answers (blogs).
================================================================

Questions

1) What type of organization do you work for?

a) Security vendor / consultant (63%)
b) Enterprise (23%)
e) Other (please specify) (10%)
c) Government (5%)
d) Educational institution (0%)




2) What portion of your job is dedicated to web application security (as opposed to development, general security, incident response, etc)?


a) All or almost all (53%)
b) About half (28%)
c) Some (20%)
d) None (0%)




3) How many years have you been working in the web application security field?

c) 2 - 4 (33%)
e) 6+ (25%)
d) 4 - 6 (20%)
b) 1 - 2 (13%)
a) Less than a year (10%)




4) In your experience, what's the primary reason why organizations have web application vulnerability assessments performed?

a) To measure how secure they are, or not (53%)
b) Industry regulation and/or compliance (25%)
c) Customers or partners ask for independent third-party validation (10%)
e) Other (please specify) (10%)
d) No idea (3%)




5) How often should web applications be assessed for vulnerabilities?

a) After every code change (65%)
e) Other (please specify) (20%) - Answers mostly revolved around "major" releases.
c) Quarterly (10%)
b) Annually (5%)
d) Before the auditors arrive (0%)




6) How many web application vulnerability assessments have you personally conducted this year (2006)?

b) 1 - 20 (50%)
c) 20 - 40 (23%)
d) 40 - 60 (13%)
e) 60+ (10%)
a) None (5%)




7) How many man-hours does it take you to complete a web application vulnerability assessment on the average website?

c) 20 - 40 (50%)
b) 0 - 20 (23%)
d) 60 - 80 (23%)
e) 80+ (5%)
a) None (0%)




Please ONLY answer ONE of the two following questions (#8 and #9)
Commercial Vulnerability Scanners: (Acunetix, Cenzic, Fortify, NTOBJECTives, Ounce Labs, Secure Software, SPI Dynamic, Watchfire, etc.)

8) If commercial vulnerability scanners ARE part of your tool chest, how much of your preferred assessment methodology do they complete? 36 respondents (57%)

c) About half (58%)
d) A little bit (33%)
e) Not much (4%)
b) Most of it (4%)
a) All or almost all (0%)




9) If commercial vulnerability scanners are NOT part of your tool chest, why not? 27 respondents (43%)

d) Some combination of a, b, and c (61%)
a) Too many false positives (11%)
c) Faster to do assessments by hand (11%)
b) Too expensive (6%)
e) Haven't tried any of them (6%)
f) Other (please specify) (6%)




10) How often do you encounter web application firewalls blocking your attacks during a vulnerability assessment?

d) Never, or almost never (73%)
c) Sometimes (10%)
e) Hard to tell (10%)
b) About half of the time (5%)
a) A lot (3%)





11) While performing web application vulnerability assessment, how often do you encounter websites requiring multi-factor authentication?
(Hardware token, software token, secret questions, one-time passwords, etc.)


d) Never, or almost never (50%)
c) Sometimes (35%)
b) About half of the time (8%)
a) A lot (5%)
e) Hard to tell (3%)



12a) If you find a vulnerability in a website you don't have written permission to test, what do you do with the data MOST of the time?

b) Inform the website administrators (responsible disclosure) (36%)
c) Keep it to yourself, no sense risking jail or lawsuits (36%)
e) Other (please specify) (18%)
a) Post it sla.ckers.org (full-disclosure) (8%)
d) Sell it (3%)

Daniel Cuthbert: "WALK AWAY! spending 1 year fighting the british government over this exact thing made me realise this lone cowboy approach will never work :0)"


12b) How has the security of the average website changed this year (2006) vs. last year (2005)?

c) Same (50%)
b) Slightly more secure (28%)
d) Worse (20%)
a) Way more secure (3%)
e) No idea (0%)





13) What do you think of RSnake's XSS cheat sheet.
http://ha.ckers.org/xss.html


b) I like it (55%)
a) It rocks! (28%)
c) It has the basics, but there are more options (13%)
e) Never heard of it (5%)
d) Lame (0%)




14) Do you surf the Web with JavaScript turned off?


c) No (38%)
b) Sometimes (33%)
a) Yes (18%)
d) Only when clicking on links from Jeremiah (10%)




15) What operating system are you using to answer this question?

a) Windows (68%)
b) OS X (15%)
c) Linux (15%)
d) BSD (3%)
e) Other (please specify) (0%)





BONUS

16) The most valuable web application security tip/trick/idea/concept/hack/etc you learned this year (2006)? List just 1 thing. *Full list will be published*

When forms convert lower case to uppercase, use VBScript to test for XSS
since it is not case sensitive like JavaScript <* script type=text/vbscript>alert(DOCUMENT.COOKIE)<* /script>

When spidering a website use your standard USER AGENT, then crawl it agai
Using JavaScript to other than just stealing cookies :-P

The research and disclosure being done on sla.ckers.org and gnucitizen.org.
JavaScript Malware Intranet Hacking

I don't remember.

combination of XSS and XHR. (My next PoC will show you why)

Javascript Scanning

Blind SQL injection in MSSQL and MySQL, complex XSS injection (using the
great http://ha.ckers.org/xss.html)

I can tell you, but then I will have to sign you on an NDA :-)

XSS Shell

XSRF

mhtml: vulnerability - complete read access to the Internet on Internet
Explorer. Scary!

Learned none.

This might not be web app. related until after Vista is released (yeah, right).
I found interesting the concept of how to discover whether Visual Studio binaries have been /GS compiled; Used to mitigate local stack variable overflows.

Sorry, I'm restricted from saying. :/ I guess my best/most valuable tip that I use every time is don't become dependent on any one tip/trick/idea/concept/hack. :)

What cross domain restrictions? The web security model was completely smashed up this year and I don't pretend to claim to be smart enough to fix it. But what we got isn't working the way we thought it did.

I've found some new tools to try out, such as TamperIE, which I picked up from the "Hacking Web Applications Exposed 2nd Edition" book I purchased, and I believe you wrote the foreword. Plus I have to hand it to RSnake, id, maluc and the other people on sla.ckers.org, they just keep coming up with new attack vectors. Their disclosures can be a little frightening. As you said in the foreword, sometimes we just want to bury our head in the sand because we know most of the sites out there have vulnerabilities.

CSRF (Just after I tested a bloody forum too!)

impoving my XSS knowledge with the awesome help of ha.ckers.org and sla.ckers.org

Learning more about web servicves, SOAP, XML, etc.

That demonstrating issues to a vendor/customer is much less effective
than expressing the business liabilty and risk expressed in $s :-)

Fully understanding AJAX, which is important, even if all I learned was that it wasn't as big a deal as I expected it to be (from a security standpoint it is a much bigger deal from developer and user viewpoints).

There is nothing new under the sun. People still do dumb things.

XSS + Ajax avoids the same-domain security sandbox.

Bypassing filtering mechanims by UTF-16 encoding URLs even when there's no need to

Using POST content as a query string in most cases won't effect the way the receiving application reacts. Attacks are more portable and easier to demonstrate in link format.

Implications of Flash 9 crossdomain.xml, including flaws in the implementation and severe lack of best practice standards. The floodgates may be open on client-side code with cross-domain privileges (Quicktime, etc.), but it's good to see a misstep at least happening in the right direction.

This is my statement for the web application security year 2006:
Everyone can find a XSS vulnerability but fortunately only a few people can imagine what this really means.

Automating viewstate injection. Maybe I'll release some notes about it.

Nothing can replace experience!

session riding

XSS

Watch out for that stupid UTF-7 encoding.

Search myspace for answers to secret questions :)

Intranet IP scanning really opened up my thinking of what XSS could be used to accomplish.