Recently on Twitter I asked why some people feel oddly compelled to rely upon the shortcomings of Web Application Firewalls (WAFs) as a means to advocate for a Secure Development Lifecycle (SDL). To me this is odd because the long-term, risk-reducing value provided by secure code is enough on its own to warrant the investment. If you can’t demonstrate that, blame directed at WAFs seems misplaced. Most importantly, we must remember our objective: protecting websites from being hacked.
The ultimate scope of this objective, 200 million websites and counting, cannot be understated. Even if we just focus on the 1.3 million websites serving up SSL certificates, the scale is still unbelievably massive. Whatever the metric, experienced industry experts and aggregated statistics reports agree, the vast majority of these websites are riddled with vulnerabilities. The exploitation of thousands of websites that is fueling headlines serves as a further proof point. To quantify vulnerabilities, let’s assume an average of six serious* vulnerabilities per website, WhiteHat Security’s published figures based on our own Sentinel assessments. This totals 7.8 million custom Web application vulnerabilities in circulation. We just don’t know exactly where they are.
The next and equally important problem, fixing the code, is a seemingly insurmountable obstacle. Imagine an extremely limited number of application security pros to convince 17 million developers (some unknown portion being Web developers) to add to their workload, learn about defensive programming, and remediate all the vulnerable code. And by the way, this will be accomplished in small increments. Vendor-supplied patches have no place here. According to Gary McGraw’s (Cigital, CTO) BSIMM studies, observations from large-scale software security initiatives, a software security group (SSG) ideally should be 1% of the size of the development team. Given that baseline, we’d need 170,000 software security experts when I doubt if more than 5,000 currently exist.
To pile on, process and staffing investments will not take place unless the stakeholders are convinced that it is worth it to risk potential revenue by refocusing developer efforts to fix security issues that may or may not be exploited (aka: risk management). This is a scenario application security practitioners are all too familiar with and frustrated by. Despite all the challenges, software security is making significant progress compared to its nearly nonexistent status of only a few years ago. However, change does not occur over night. It takes years. Unfortunately time is not on our side and the bad guys are exploiting websites and their users by the thousands every day. Waiting around for a future of software security utopia while the losses mount is ill-advised. Obviously we need more options. Just yelling for more manual code reviews, developers training, secure frameworks, etc will work for some, but certainly not all. Think about the costs!
Several years ago I was contemplating the sheer size and gravity of the aforementioned numbers, taking into account operational considerations, brainstorming possible solutions worth considering, and then it hit me. I disliked WAFs for all the same reasons as everyone else, but it became clear that even if WAFs didn’t work, practically speaking we really need them to (work). If somehow they could temporarily protect against the exploitation of a certain amount and type of vulnerability, and do so without requiring an application security person to beg for development time, allowing them to take action alone, it would have tremendous appeal. Don’t agree? Remember, WAF technology predates PCI-DSS and hundreds of millions in annual sales is not entirely driven by compliance.
Deciding to be part of a solution, or at least an option for organizations to consider, I began meeting with various WAF vendors asking how WhiteHat could help. One idea we considered was that if a WAF knew ahead of time the exact type and URL location of a website vulnerability, then perhaps WAF effectiveness would improve. Thus our exploration into virtual patching began. Let’s be clear though, WhiteHat Security does not make WAFs. We do not sell WAFs. We have no financial motivation when or if they are sold. Our main interest is assisting our customers in protecting their websites and integrating what we do into technologies they feel are beneficial, to them.
Fast forward to today. Obviously not everyone is impressed with the current state of WAF technology. Staunch critics say WAFs introduce the very vulnerabilities they are supposed to defend against and/or that they can be bypassed. We agree. No doubt, WAF technology is imperfect, but that doesn’t make them any different from every other security product segment. Anti-malware, firewalls, IDS/IPS, MFA, SSL, and so on all have vulnerabilities. All can be bypassed. Still, I’m pretty sure we’d all agree that these products provide a certain amount of value nonetheless. You don’t throw the baby out with the bath water. We regularly identify vulnerabilities and bypasses in various WAFs and results with the vendors so they may have an opportunity to improve.
So where does the anti-waf-software-security-only-zealotry really come from? I don’t believe the critics are ignorant of the problem at hand. Maybe it has something to do with the quote from the movie Vanilla Sky, “What’s the answer to 99 out of 100 questions? Money.” Technically speaking, SDL activities and WAFs are NOT mutually exclusive. They are designed to solve different problems. It is entirely reasonable to deploy one, the other, or both at the same time. The tough reality is that IT security budgets allocated towards Web security are wholly inadequate. Budget constraints force organizations to make painful choices about where to spend first. Decisions are sometimes made for compliance reasons, and other times for security. While SDL and WAFs should not be considered competing solutions, they do compete for the limited dollars. PCI-DSS 6.6 made matters that much worse.
No matter the reason, when WAFs are purchased “first,” the software security guys lose money, naturally making them resentful. They snap back saying, “the customer doesn’t get it” and “SDL is the right thing to do because WAFs suck!” Let’s be honest, if that’s how you are positioning the value of an SDL, you deserve to get back-burnered and furthermore you’ve done your client a disservice. WAF’s suckage should have zero bearing on whether or not an organization should improve its SDL. Instead, focus on the many cost-saving, risk-reducing, top-line-benefiting qualities an organization may realize by implementing a well-regulated software security program.
At the end of the day, our common enemy is really the lack of application security visibility and the allocation of necessary resources. If we come together and help address this as an industry, we’ll all be better off, and the pressure of this either or choice will be lessened.