Web Application Security Professionals Survey (May 2007)

Update (5/14/2007)
The results are in and the webappsec professionals once again have spoken! 62 respondents shared their opinions and in doing so presented interesting perspectives and insights of a larger world. Thank you to everyone who took the time to submit and please let me know if you have any burning questions that needs to be asked next time.

My Observations
1) According to Q7 and Q10, respondents are under no illusions about the current state of web security. It’s a sobering reality that no one is under the impression that things have improved significantly. Instead 71% believe web security has either stayed the same or perhaps only improved slightly during the last 12 months. And nearly 1/3 think the current state may have actually gotten worse as defensive measures have not kept pace with new attack techniques. This makes one wonder if the term “web security” has become an oxymoron. Fortunately for them about half of the respondents reached the Acceptance stage of (47%) of their physiological grief, while most of those who remain are fighting through Bargaining (27%) or Depression (14%).

2) Judging from Q11 and Q12 nearly 1/2 of all respondents believe financial institutions represent the most secure industry vertical (encouraging), or generally websites developed in either .Net (31%) or Java (28%). Virtually no love for PHP or my favorite language Perl. :( We need to be careful with drawing conclusions from these statistics. A significant numbers of peopled admitted they were just guessing. Personal experience was limited to only one or two verticals and platform languages. They hadn’t tested in-depth across the gamut. This means what works, what doesn’t work, and who’s doing the best job continues to be a blind spot. This is unfortunate because we’d like to learn from them. As we grow the pool of websites we assess, WhiteHat might be able to shed more light on this area through our Web Security Risk Report.

3) Q2 makes it pretty clear that respondents believe most developers are clueless when it comes to web security. Developers carry a heavy burden as the vast majority of people consider them to be the first and only real line of defense. They routinely have unrelenting deadlines to meet, go without proper training, and lack technology assistance. Then we expect developers to know all the issues and not make mistakes. But if you take a step back, even we the experts have trouble keeping up with all there is to know. Unfortunately for them until web security improves dramatically, developers are likely to continue being the whipping boy since there is no one else to blame. Oddly enough the things that might significantly improve web security, such as modern development frameworks, have little to do with education. But hey, maybe they can take the credit for that anyway!

4) Q3 was surprising as about 1/4 of respondents said they had a strong technical understanding of DNS-Pinning and Ant-DNS-Pinning. Further still another 60% said they possessed at least some familiarity with the concepts. Impressive. I would have thought these numbers would be lower. Looks like a few good bloggers really gotten the message out. On a similar question (Q6) I was also surprised by the relatively even split *snicker* across the Response Splitting range when it came to exploitability. There isn’t a great deal of consensus about the risks involved. I can’t help but think some 2/3 of us remain confused by the finer points of the attack. This would make sense considering recent mailing list threads.

5) As it was for the last survey a high percentage of respondents (72%) are positive about Web Application Firewalls (WAFs) and optimistic enough to give them a look. My guess is people are figuring out that there are a lot of websites out there, more bugs than we can fix in a reasonable amount of time, and security guys need to gain some control besides hoping for help from some “clueless” developer. Or I could be wrong and this is just a defense in depth mindset, but doubtful considering that even RSnake recently had a change of heart. However, WAF vendors have a lot of proving themselves to do before gaining the trust of the masses, capitalizing on current optimism, and moving beyond the early adopters.

Description
Several people have asked where the surveys have gone to in the past several months. The answer is that I've been amazingly busy the last couple of months and simply haven't had the time. The survey helps us learn more about the web application security industry and the community participants. We attempt to expose various aspects of web application security we previously didn't know, understand, or fully appreciate. From time to time I'll repeat some questions to to develop trends. And as always, the more people who submit data, the more representative the will be. Please feel free to forward this email along to anyone that might not have seen it.

Guidelines
- Survey is open to anyone working in or around the web application security field
- Answer the questions in-line and if a question doesn’t apply to you, leave it blank
- Comments in relation to any question are welcome. If they are good, they may be published
- Email results to jeremiah __at__ whitehatsec.com
- To curb fake submissions please use your real name, preferably from your employers domain
- Submissions must be received by May 14, 2007

Publishing & Privacy Policy
- Results based on aggregate data collected will be published
- Absolutely no names or contact information will be released to anyone, though feel free to self publish your answers anywhere

Last Survey Results
January 2007

Questions

1) What type of organization do you work for?
a) Security vendor / consultant (53%)
b) E-Commerce (9%)
c) Healthcare (0%)
d) Financial (9%)
e) Government (14%)
f) Educational institution (5%)
g) Other (please specify) (9%)



2) From your experience, how many web developers "get" web application security?
a) All or almost all (0%)
b) Most (0%)
c) About half (19%)
d) Some (59%)
e) None or very few (22%)


"The few that get it are the ones that have seen and played with WebGoat."


3) What is your technical understanding of DNS-Pinning and Anti-DNS-Pinning?
a) Strong (26%)
b) Some familiarity (60%)
c) I've heard of these (12%)
d) Eh? (2%)



"Some Googling and I am all fixed up here, but I had not heard of DNS-Pinning prior to taking this survey."



4) Do you click on links sent in email?
a) Never (27%)
b) Sometime (68%)
c) Always, I fear no link (5%)




"If I want to follow a link in email I copy and paste the link text and then visually make sure the link is what I think it is. I suppose this practice is susceptible to stored XSS attacks, but I never even do this much if I have the site in question is one where I can move my money around. "

"never. I check every link in status bar and copy-paste it to text file first."


5) Your recommendation about using web application firewalls?
a) Two thumbs up (13%)
b) One thumb up (59%)
c) Thumbs down (15%)
d) Profane gesture (10%)
e) No Answer (3%)



"They are a good short term measure, but there is nothing like having the application code written right in the first place to ensuring the security of any web application."

"They're not yet to the point where they help more than they hurt. A security aware development process and proper training can work much better. I'm open to the idea as they improve though."

"I do like the idea of WAF, since different independent layers of protection is what makes an application tend to be more secure. However what still is insufficient is the implementation of WAF we have at the time. I would like to point on this blog entry I wrote a few days ago, instead of re-phrasing it all again." http://christ1an.blogspot.com/2007/05/real-effectiveness-of-current-waf.html

"My exception: If inbound/outbound traffic from a hosted web application inside of a datacenter can be split so that inbound traffic (GET's, POST's) is unaffected by the WAF, and outbound traffic (Server responses) is protected - then I'm ok with implementation of a WAF. See: be conservative with what you send and liberal with what you receive. I do not believe or agree with RSnake's exception. Companies should not implement WAF's in order to buy time or because their website has 14,000 vulnerabilities. They should take down the website if it's
really that bad."

"Using a web application firewall can separate some security concerns from the business logic, but they are no substitute for good security practice. For instance, Amit Klein's algorithm to help mitigate the PDF XSS from the server is best implemented by a web application firewall, but its use is not a license to potentially open ones site to SQL injection by failing to used prepared statements on the assumption that the application firewall will provide adequate protection."

"Not as a catch all but as another synergistic layer of control. Additionally, I have recommended it to people as a stopgap measure where there site is completely riddled with obvious holes and the number of developer hours required to fix are very high."

"Despite potential shortcomings in WAF implementations, they're far more targeted towards the web application threat domain. Traditional IDS/IPS has not performed very well in this area."

"I am a purist that wants to see the code fixed but understand that that is not a viable solution for some/most clients. They could be the "savior" of web apps or as useless as your typical IDS. When implemented properly the are useful defense in depth i guess. "

"The products seem to be maturing. I think the hurdle now is now education. What they can and can not protect, and when they make sense in a deployment."


6) From your experience, what is the typical risk level of Response Splitting exploitability?
a) High (23%)
b) Medium (34%)
c) Low (43%)




"Severity ratings are highly dependent on individual factors. Rating systems (from the H/M/L basic ones to the complex CVSS, etc) are not scientific enough for my uses. In my opinion, all vulnerabilities are critical and require remediation. Prioritization during the remediation process should rarely be used because fixes should be done in parallel with as many resources as it takes to get each done as close to immediately as possible."

"On a scale of 1-10, where 10 is remote root-level compromise of a system, these would fall into the 4-6 range with things like XSS and SQL injection. It depends on the deployment. Web vulns may (or may not) have less of an impact to an Enterprise or hobbyist site than they would to a service that is used by millions of users (Amazon, MySpace, etc)."

"A related vulnerability that's probably easier to exploit is response header injection. I'd give that one a Medium."

7) How has the security of the average website changed in the last 12 months?
(Take into consideration new attack techniques and defense measures)
a) Way more secure (0%)
b) Slightly more secure (33%)
c) Same (38%)
d) Worse (29%)
e) No idea (0%)

"Many defense measures do not seem to be deployed, and many of the new attack techniques constitute a sub-set or refinement on old, known vulnerabilities which web developers should already have an inkling of an understanding about."

"The researchers are far out pacing the remediation in most sites. I also think that we haven't begun to see CSRF hit on a real scale yet. "

Worse
Considering the following:
- A new attack vector was discovered that bypass the built in .NET validation. This alone had a potentially global impact for any developer/company that relied soley on the built in validation. The patch won't be released until June 12th, just over 2 months after the public disclosure.
- PCI escalated XSS vulnerabilities to Level 4 (Critical).
- The mhtml vulnerability in IE 7 still hasn't been patched.
- The "onunload" entrapment vulnerability.
- You get the idea...



8) Do you plan to attend BlackHat Vegas of Defcon this year?
a) Yes (23%)
b) No (51%)
c) Maybe (26%)




" I'd like to attend Dinis Cruz's ASP.NET exploits training."


9) Are hacking contests, like Hack a Mac at CanSecWest, a good idea security-wise for the industry?
a) Yes (58%)
b) No (11%)
c) Somewhere in between (please describe: 1-2 sentences)




"The media attention I think is a good thing. Can be a needed reminder about security for people outside of the industry before the next real break in makes the front pages of the WSJ again."

"don't see much point other than showing off your back pocket 0days"

"It depends one the motive of the contest, sometimes it is good to give people an reason to look at a new technology. But the problem with many of these contests is how they are run in the background. The Hack a Mac contest is an example of how one of these contests can be run badly."

"They increase public awareness of an oft ignored problem, but do little to actually make things more secure in and of themselves."

"They generally don't UNCOVER new vulnerabilities - they usually just publicize existing vulnerabilities that are out there. By making an exploit public, they end up giving the exploit away for free to groups that otherwise might not have had it. (Of course, I understand the other side of the coin - full disclosure forces the vendor into fixing it.)"

"They aren't a bad thing, though there are better ways to accomplish the goal of improving security in the industry if some folks would be wiling to sacrifice the headlines and notoriety."

"The Hack a Mac at CSW served as a reality check for Mac users and pundits and so in my opinion served a greater purpose than most contests. This has a one-time value. In many cases these contests are launched as marketing campaigns to by vendors who want to prove that their product is "unhackable" and these are of no value whatsoever."

"On one hand, from my point of view it's never good if you have to break into something just for having a cool show, even if it's in an isolated environment but on the other hand sometimes it's the most efficient way to show (in this case) non-webappsec-professionals that we're not joking when we say that there are serious problems out there."

"To paraphrase Schriner it's security theater. But sometimes that's good. With any new client i use a little FUD magic tricks up front to get their interest up. That said using unsecured networks with 0-days on um isn't such a good idea. Hack a Mack you guys ROCK! :( "

"It depends on what side of the fence you are on regarding public disclosure. I can understand both sides of the argument. Yes, it makes an exploit publicly known before the vendor has a chance to create a patch for it. Yes, public disclosures to prompt vendors to be more responsive to releasing patches for discovered vulnerabilities."

"I could go on for a long time about this. I do think that public hacking both increases and decrease risk. In the very short term, it exposes issues for which fixes take time and then he consumer/customer is exposed to risk in the interim. But in the long term, 1) if the good guys don’t do it, the bad guys will and 2) vendors won’t otherwise be motivated to change/fix…this is just the way free market capitalism seems to work."

"Security by Obscurity sucks! Incentive for motivating people is always good and some people sometimes like the attention. Would Dino Dai Zovi have found the bug had it not been for the publicity? Maybe down the road but not sooner."

10) What is your stage of web application security grief?
a) Denial (5%)
b) Anger (8%)
c) Bargaining (27%)
d) Depression (14%)
e) Acceptance (46%)




"I'm far to academic in my thinking and everybody's just trying to recover their existing stuff rather than using good practice in engineering."

"Depression (if you catch me at the bar after work or a conference)"

"Acceptance. I was there from day one. Clients are always assumed to be evil. My pain point is in hiring. Most web developers I've seen couldn't construct an HTTP request/response if their life depended on it. They nearly always lack the technical knowledge to adequately defend themselves."

"Amazingly enough this the one part of my job that doesn't give me much grief. Anytime I find an issue, I open up a bug ticket and our developers make the recommended changes pretty rapidly. The anger part is in regards to the fact that simple problems continue to occur in new
applications even though they were highlighted as security issues and addressed as such in previous applications. I.e. Input validation..."

"It is not really grief anymore, it more a realisation that my job is going to be safe for sometime to come and my family will be provided for on the development mistakes of others.."

"The only stage I haven't been through is denial. I jump-started to the anger stage when my day-to-day existence became centered around trivial vulnerabilities in unused and unknown web applications. I've come all the way through to acceptance because there's a lot of smart people thinking about this space and bringing innovations to both attacks and defenses. Web application security is definitely more legit than it was 5 years ago. "


11) What is the most secure website industry vertical you encounter during vulnerability assessments?
a) Financial (47%)
b) E-Commerce (6%)
c) Healthcare (6%)
d) Government (0%)
e) Adult Entertainment (11%)
f) Gaming/Gambling (3%)
g) Don't know (14%)
h) Other (please specify) (14%)

"Overall, financial industries seem to be the most secure in my experiences. I believe that fear from regulatory compliance issues have prompted these companies to dedicate vast amounts of resources to deal with these problems. Healthcare definitely seems to be the worst, and I think that is a direct result of HIPAA lacking clarity in the spec and "teeth" in the enforcement."

"Not that they're perfect, but a few of them have really come a long way in creating mature processes. I feel like they have a head start on the other industries at this point."

"there are no most secure website industry in the web. There are a lot of sites with pdf files ;-)."

"From my point of view it has nothing to do with the industry a company is in, it has to do with popularity on the Internet. For example google.com is very secure if you compare it with other nearly unknown search engine sites. This is because that many people try to attack such big players and so the risk of a successful attack is much higher and that's why this companies have to do much more to stay secure, independent of the industry they're in."

"Those that conduct business via a verble agreement and a handshake. Other than that..."

"The pr0n industry is always ahead of the curve with tech."

"Most of the assessments I do are on finance based applications. They (the finance industry) are starting to understand (thanks to the TJX security breach), that one mistake could get them sued right out of business."

"I mostly just assess my own company's sites."

"I don't touch enough verticals to say in general"

"I really don't have much to go on here. The stuff I look at is mostly the global online services world such as what Windows Live or Google would provide. Having looked at financial applications about six years ago, I know at least they've gotten magnitudes better than those days, at least the ones I personally use."

"I've been Web app testing for coming on 7 years, back then my first web app test was a internet bank, those early adopters set a pace for the web app security race. I've found that many other industries are just playing catchup. Some are really close behind, but some are just plain lost, hell some companies took the wrong race course route and have ended up in the web app security equiv of downtown Baghdad fighting a loosing battle with the XSS insurgents.."

"Hmm...I'd like to say A because they were the first to really follow through on some security initatives (not that they already have the incentive of heading off potential exposure). However, it's possible that B can be the top because people (particularly companies) seem to be pretty aware of the fact that they need to transmit the credit card numbers securely or likely get burnt like all these companies have in recent years (i.e. TJX as the most recent victim). Then again, E and F could be the most secure (F because it really has one of the most to lose) because they tend to trailblaze new technologies (i.e. VCR and Internet). C is likely up and coming thanks to HIPAA but they are trying to go too far with leaving all patient records online. Ok this is long but I guess you could put me down as a G answer because I really don't know what is the best at the moment."


12) From your experience, what development technology is present in the most secure websites?
a) PHP (3%)
b) Java (28%)
c) ASP Classic (0%)
d) .Net (31%)
e) Cold Fusion (0%)
f) Perl (0%)
g) Don't know (19%)
h) Other (please specify) (19%)

"I simply have no clue ;) I've never spend much time with other languages than PHP and the only thing I can actually prove is that the latter is insecure. Probably all the others have very similar problems but there isn't that much on the news."

"What's about clean old HTML? These web applications are very secure as you know ;P"

"COMMON SECURITY SENSE and trust but VERIFY!"

"Java - If i had to pick one ... only because there are some tools in this space that make it hard to shoot your leg off. That said everything can be coded poorly and/or designed poorly. Garbage in Garbage out."

"Ruby or Rails"

"Ruby on Rails!!!!!!!!!!"

"A killer one again. I'd like to say B or F (although User Input without sanitization is a real killer for this language) but I really don't know...I'd have to say G."

"Some of the most secure applications I've tested have been .NET, but this doesn't mean the developer can't/didn't implement something incorrectly that could cause a major security issue. One of the sites I tested was considered by the companies developers and management to be "super" secure. It was for the most part, except the application allowed for uploading pdf files. The developer made three minor mistakes, but when put together those mistakes were huge. Mistake #1 the developer only validated the file type on the client (easy enough to bypass). Mistake #2 the directory the files were being upload to was in the application directory structure so files could be navigated to by changing the url. Mistake #3 the developer set the execute permissions in IIS on the uploads folder to match that of the application root (scripts and executables). It took about 2 minutes to discover this, upload an .asp file and dump the contents of the server's C:\ to my browser. Technology and frameworks are great, but when developers make small mistakes in implementation, they can result in huge issues."

"I know that our .Net sites are significantly more secure than our ASP Classic sites. However, I've seen an extremely secure ASP Classic site once (a bank that was using a purchased web app written in ASP Classic)."

"D - of course I'm biased and that makes up about 90% of what I see ;)"

"Python or Ruby. they are more secure for a lot of reasons, but the most strong one is that they are not as common as .NET or PHP apps."

"HTML, pure html. Even in this case it is possible to find vulnerabilities (in client-side scripts - javascript, etc.), and I had found many such holes, but pure html is the most secure variant."

"(maybe that's not fair to Java, but the .Net apps I've seen generally are not as complex so they've been a bit more secure)"

"This question sucks and is counter productive. People keep wishing to get magic answers about secure technologies. The problem is not in the technologies (at least not the decent ones... java/.net/php/etc. - I'm leaving out things like SSI) - the problem is with the coding. Saying that a certain technology is more secure is just going to make ppl to think they can slack their code security cause "java is secure" or whatever. bad, bad bad question."

"Other, I think the less well known stuff like Rails and Django have smaller, more technical, more security savvy user bases"

"h (can't say - all depends on programming approach and security awareness)"