Over the last couple of weeks I’ve been putting a lot of time into our 9th WhiteHat Website Security Statistic Report, “Which Web programming languages are most secure?” Full report is available (reg. required).
I always love doing this report to see how Web security is changing, but this time around it was even more exciting. For years the industry has been conditioned to believe that the selection of a development technology is one of the most important decisions affecting website security. However, the empirical data behind the comparison of development languages / frameworks from our latest report paints a very different picture. The bottom-line is that there just isn't a large measurable difference in the security postures from language to language or framework to framework -- specifically Microsoft ASP Classic, Microsoft .NET, Java, Cold Fusion, PHP, and Perl. Sure in theory one might be significantly more secure than the others, but when deployed on the Web it's just not the case.
IntroductionsThis type of data is likely to stir up emotion within the industry because many people are extremely attached to their development language / frameworks. They have strong convictions about the perceived security performance and opinions on why their choice is the best for others too. At the end of the day, this report shows that no one language / framework is vastly more secure than another...none are so special that they stand out. The first step to improve application security is to focus less on the technology and more on creating an executive level mandate. Unless we bridge the gap between perception vs. reality, the problem will never be properly addressed.
"Security-conscious organizations make implementing a software security development lifecycle a priority. As part of the process, they evaluate a large number of development technologies for building websites. The assumption by many is that not all development environments are created equal. So the question often asked is, “What is the most secure programming language or development framework available?”
Clearly, familiarity with a specific product, whether it is designed to be secure- by-default or must be configured properly, and whether various libraries are available, can drastically impact the outcome. Still, conventional wisdom suggests that most popular modern languages / frameworks (commercial & open source) perform relatively similarly when it comes to an overall security posture. At least in theory, none is markedly or noticeably more secure than another. Suggesting PHP, Java, C# and others are any more secure than other frameworks is sure to spark heated debate.
As has been said in the past, “In theory, there is no difference between theory and practice. But, in practice, there is.” Until now, no website security study has provided empirical research measuring how various Web programming languages / frameworks actively perform in the field. To which classes of attack are they most prone, how often and for how long; and, how do they fare against popular alternatives? Is it really true that popular modern languages / frameworks yield similar results in production websites?"
8 comments:
In my opinion, a language or framework by itself will never be able to solve the problems faced when it comes to web app security. Certain things can be done to make things easier for the developer, but nothing can really solve the problem on its own. At the end of the day, it comes down to the way in which the developer uses the language or framework, and as every developer will tell you - there are a lot of ways to accomplish the same thing.
I think this is somewhat similar to implementing safety features in cars. The manufacturer can do everything they can to make the car safe, but if I choose not to wear my seatbelt and drive 90 miles an hour down a slick road at night, nothing is going to save me when I hit a brick retaining wall.
also that xss hack works on windows 7 with firefox 3.6.3 well done :)
I haven't gotten a chance to read the report yet, but I can say one thing with certitude - at an organization where security is merely an afterthought (if that), whatever framework has the most effective default settings in place will be most secure. We have code in classic ASP, ASP.NET, and JavaEE, and far and away, the ASP.NET code holds up the best. Why? Because the developers don't do anything to it, and it's a bit more secure than when they treat code from other frameworks the same way.
Yeah, it ain't where we want to be - maybe the survey is mainly of organizations who've historically had more security awareness than mine - but a lot of organizations start out with little concern for security; once they become worth attacking, they have to go back and fix whatever is not protected by default.
Just wanted to ask for a clarification. In Figure 6 (Top Ten Vulnerability Classes) the percentage value for each vulnerability class is counted as Sum(Pj)/N, where Pj is 1 if web application has vulnerability of the specific class and 0 otherwise.
My question is about N for business logic flaws, for instance "Abuse of Functionality". Is it the total number of web sites with the specific techology or is it the total number of web sites with the specific techology with the appropriate contract (i.e. PE for business logic flaws)?
If it is the first choice then the statistics is skewed...
It would be great if this issue was covered in the report :)
@Andrew, I think I know what you are asking, but let me answer in a bit different way.
Lets assume 50 of 100 websites we assessed were found to have XSS. With this in mind figure 6 means that of the 100 websites, they have a 50% likelihood of at least 1 XSS issue.
To your question about "Abuse of Functionality", we check for it on all websites managed under the PE service, whether or not the website might have features potentially prone to the issue -- clearly some do not. This would include brochure ware websites and we do discuss this in the report. So, this percentage is of ALL websites.
That what you wanted to know?
Yes, I wanted to know exactly this: that "abuse of functionality" was measured only within PE web sites.
The reason I asked this question is that I tried to interpret the statistics in different ways and failed. One thing that I cannot understand is the variance of the likelihood of busines logic flaws, i.e. "Abuse of Functionality". It differs from 8% in ASPX and Struts to 25% in Perl.
If we assume that the distribution of "good", "average" and "bad" programmers is the same within each technology (ASPX and Perl), then one would expect that the likelihood of flaws not addressed by frameworks (i.e. Abuse of Functionality) will be around the same. But the report clearly shows that it is not so. Wrong assumption?
And yes, I understand about brochure web sites. I thought that they were not under PE contract but rather SE or BE one...
@Andrew, "Yes, I wanted to know exactly this: that "abuse of functionality" was measured only within PE web sites."
Good question, It was not. All websites were lumped together regardless of the type of Sentinel service. The breakdown of how many websites are covered under which service type is definitely a reporting blind spot. I'll take that as feedback for the next report and will publish it. Right now reasonable estimates are about 75% PE, 20% SE, and the rest BE (~5%). Will confirm in the next report.
Taking that into consideration it should be safe to assume that under PE websites only, Abuse of Functionality would measure 25% higher than their current overall numbers -- I think, but not certain.
The assumption about bad/good programmer distribution might not be accurate. While we can yet correlate it, an alternative theory is that what a website is vulnerable to has a lot to do with WHEN it what built. Developers will not code against a class of attack that they both don't know of and respect.
For instance take the average site launched in 2005 vs. 2007, we think there will be a significant difference in XSS issues between them (the Samy Worm). Or from 2007 to 2009 with regards to SQLi Injection (the mass SQL worms).
As time progresses programmers have shifted for a variety of reasons to using the more "modern" framework, like .NET and Structs over plan Perl. If the above theory is true, it would make it seem that Perl is "not as secure" as the others. Again, just another theory worth considering.
>> "Developers will not code against a class of attack that they both don't know of and respect."
Completely agree. Maybe in the futue releases of the report you will find possible to include time slices with framework slices for Perl/PHP?
For example, the usage of constantly updating Drupal, Wordpress and Joomla CMSes would be a good explanation why CSRF did not make for Top Ten for PHP...
However, thanks for your effort, which was an excellent food for thought! :)
Post a Comment