Over the last week I’ve been inundated with people interested in WhiteHat Security’s new partnership with F5, specially the integration between Sentinel and their web application firewall (ASM). This is where we identify vulnerabilities, send custom rules to their WAF, and customers mitigate issues with the push of a button. It was actually Arian Evans (Director of Operations) who reminded me what this means for WhiteHat as a company when he recalled a conversation he had with Bill Pennington (VP of Services) and myself upon first joining the team. The question on the table was, “What is the most compelling story WhiteHat could ever tell?”After a long conversation loaded with acronyms and buzzwords the consensus was simple, “find and fix.” That is, finding vulnerabilities on websites and fixing them on an INTERNET-WIDE scale. Only a year and a half ago few outside company walls believed we could pull off the first part, let alone second. Today we’re well on our way to accomplishing exactly that and even our staunchest critics have come around. Now partnered with F5, whose #1 is performance and load balancers, we’re ambitiously taking the next step. Imagine having the time to take care of vulnerabilities in the source code when and how you choose. Imagine being a security guy with control over the security of your website(s).
Many are curious as to how we plan to succeed with the VA+WAF concept where others in the past failed. The answer is two fold. Today’s WAF products are way more technologically mature than in years past, but the most important part is we’re able to fill the biggest missing piece -- accurate vulnerability data. Commercial scanning vendors proved time and time again dumping hundreds or even thousands of unvalidated results loaded with false positives and duplicate vulnerabilities into a WAF just doesn’t work. By contrast, with people, process, and a lot of technology we’ve overcome that hurdle. WAFs can now become easy to set-up, manage, and best of all block attacks attempting to exploit vulnerabilities (a rarity).
Bill had the same impression as I did when first seeing the technology work, in a word, “amazing”. The VA+WAF combination resonates with everyone we share it with -- media, analysts, experts, IT professionals, you name it. Can you tell I’m excited? ;) The integration will also mean volumes for PCI 6.6 as a way for organizations to meet their obligations quickly and effectively. In a few short weeks the RSA Conference will be the first place we’ll have a demo on public display. Everyone is welcome to stop by the booth and see it for themselves. I can’t wait!
21 comments:
Over the last week I’ve been inundated with people interested in WhiteHat Security’s new partnership with F5, specially the integration between Sentinel and their web application firewall (ASM)
Clearly these people are living in "The Old School of Information Security" where products matter and process doesn't. I figure you have about 2 years before decision-makers that would purchase either of these products will come around to the realization that they don't work. I wonder how many F5 boxes can be sold in that time period? I wonder how long it will be before "New Schoolers" cancel the F5 contracts and remove these boxes from their networks.
This is where we identify vulnerabilities, send custom rules to their WAF, and customers mitigate issues with the push of a button.
Sounds magical. How about pushing the Continuous integration button instead?
After a long conversation loaded with acronyms and buzzwords the consensus was simple, “find and fix.”
Asset management and configuration/change management. You must be a Old School security genius to have figured this out!
Today we’re well on our way to accomplishing exactly that and even our staunchest critics have come around.
Which ones are those?
Now partnered with F5, whose #1 is performance and load balancers, we’re ambitiously taking the next step.
F5 is widely known to be one of the worst performers and most evil of charlatans by a significant majority of network engineers and information security professionals. It doesn't help that they recently had two XSS attacks in their management interface for search, as well as in the security reports. But who relies on products, especially appliances for anything measurable in risk management - and can see actual results that have statistical value?
Imagine having the time to take care of vulnerabilities in the source code when and how you choose. Imagine being a security guy with control over the security of your website(s).
Imagine spam, social engineering attacks against your application, attacks against the presentation layer, domain logic, and data tier!
Today’s WAF products are way more technologically mature than in years past
How so? Do you have any proof? According to the New School of Information Security, risk will adjust itself and there will be an increase in data breaches (especially at the web application level). You have just set up your own industry for failure.
we’re able to fill the biggest missing piece -- accurate vulnerability data
Web application security scanners such as Sentinel accurately find only 10%-30% of the software weaknesses that they are even capable of finding. I know that you claim that your have internal testers who look for other flaws manuaully, but how do you prove their value? How does one know as a WhiteHatSec customer if this manual testing is finding presentation layer, domain logic, and data tier security-related bugs?
By contrast, with people, process, and a lot of technology we’ve overcome that hurdle.
Well it's good that you know what "people and process" are. How does this overcome the hurdle of false positives and false negatives in vulnerability scanners?
Take the technology with the highest potential for false positives and false negatives then combine it with the technology that everyone agrees only solves at most a minor fraction of the potential issues...AND COMBINE THEM SO THEY NECESSARILY RELY ON ONE ANOTHER!
30% of 30% is like...9%...wow.
Bill had the same impression as I did when first seeing the technology work, in a word, “amazing”.
Actually, I read his post and it appears that he is saying that combinding VA+WAF a bad idea. Isn't he your boss?
The VA+WAF combination resonates with everyone we share it with -- media, analysts, experts, IT professionals, you name it.
Products and services are technology solutions - so of course the media, analysts, and Old School InfoSec experts love "innovations" such as VA+WAF. Technology solutions are The Old School. Welcome to The New School.
Can you tell I’m excited? ;)
Actually, it appears as if you are losing it ;>
The integration will also mean volumes for PCI 6.6 as a way for organizations to meet their obligations quickly and effectively. In a few short weeks the RSA Conference will be the first place we’ll have a demo on public display. Everyone is welcome to stop by the booth and see it for themselves. I can’t wait!
Why stop as PCI-DSS 6.6? You'll have to integrate this with SOX AS5, GLBA, PA-DSS, HIPAA, ISO27K, NIST guidelines, SafeHarbor, etc as soon as humanly possible! We need worse controls and we need them fast!
Hi Andre, Perhaps if we could keep this simple, what would convince you that this product/solution has value (if anything)? Obviously you being one of the staunchest critics, convincing you would say a lot.
@ Jeremiah:
TS/SCI Security should have a media representative at the RSA conference. I'll make sure that he gets to see your demo ;>
I would be convinced if I didn't understand the technologies so well and be already aware of what they are capable of. It would also help if F5 showed any tact or professionalism. It's less about you and it's more about them.
However, there is a lot more you can do to improve WhiteHatSec's position in the "find-and-fix" web application vulnerability industry. Do you really need suggestions?
Great. Please have them drop by the booth and make themselves known, especially if I’m around. I’d be happy to walk them through a demo and answer any questions. But if I read you correctly, you’ll remain unconvinced of the value no matter what I say or demonstrate, which sort of makes our conversation pointless.
Speaking of F5, I’m unaware of any particular instance where they’ve behaved in an unprofessional manner. Are your comments from personal experience or is there a public link to share?
As for WhiteHat, we’ve always been open to suggestion. What we do and offer as a company is the direct result of meeting the needs voiced by our customers.
Are your comments from personal experience or is there a public link to share?
Huh. Odd. The public link I wanted to show you no longer demonstrates the full idiocy of F5. I wonder why that is.
Also, I would like to note that I was a user/buyer of F5 products from 1998 until 2003. Assuming that I don't know what their products do today is somewhat of a misnomer since I have both interviewed there and dealt with their overly-aggressive salespeople for the past five years.
What we do and offer as a company is the direct result of marketing that seeks to inhibit the truth in order to promote the selling of more services.
I'm sorry to hear that. Maybe you should listen to potential customers and advocates!
Also - I honestly believe there is value in us working to resolve our differences. However, right now you see it as my will bending (no thanks, Darth!), and I see it as you/WHS providing validation and service support to secure SDLC and incident response models.
I also see SaaS as a great way to hide information about your product/services. SaaS also has extremely difficult problems measuring their successes and wins (or failures and losses). If you want to combat these, then I suggest that you
1) Share your BITS Shared Assessments SIG and AUP data with the community. Or are you not special enough to have done one of these yet?
2) Share your source code to your scanner (or components) with the community as open-source (if your magic is really in your manual testing and WAF integration, then you have nothing to worry about here!). You obviously don't need to include the integration component if this is your most valuable asset in this software.
3) Don't be so cold and marketing-like. It's really creepy. I'm not the only pundit that is working against you on this VA+WAF thing (I speak to many of them on a daily basis and some even currently work or have worked for WHS!). Wait until you get a real analyst (read: not Gartner) against you!
We might be able to use the wayback machine if you have the original URL, but hopefully the “unprofessional” comments didn’t originate for their failure to hire you. Still that should remain between you and them.
1) The BITS Shared Assessments SIG looks interesting, I had not see it before, and something I’ll look into. Though I’m not sure how when it says we’re “secure” validates our ability to measure the security of a website (let alone 1,000 weekly).
2) Share our source code!? You can’t be serious. No way our company or any other scanner vendor for that manner would do that for the most obvious of reasons. In any event this provide no means to validate the output of our results.
3) OK mr. pundit, please name the “real” analyst that’s against us. I’ve met with and talked to most of them all already and received no resistance so far. Quite the opposite in fact.
Anyway, the offer still stands that when you are ready to share the criteria for how you can be convinced of the VA+WAF combo, you know where to find me.
Jeremiah, you have to be very careful not to over-hype the concept. The integration between vulnerability scanners and web application firewalls falls into the category of things that are easy to get going, but difficult to get right.
Virtual patching is great, but it can be reliable only when one understands the context, and by this I mean understand how the vulnerability you are trying to fix is created in the application. From your point of view (the scanning vendor), you can, at best, understand how the vulnerability can be exploited (and this you can only do manually), but this is different from knowing how it should be fixed.
Without the right information you have a choice: proceed with blocking and risk disrupting site operation, which not many people will accept, or downgrade your virtual patch to only warn when it thinks the vulnerability is being exploited. The irony of the latter is that a decent web application firewall should be able to detect this sort of thing without external help.
Heya Ivan,
Personally I believe we can’t hype the concept enough as it has the ability to completely change the web security landscape. As it stands the vast majority of websites possessing vulnerabilities, few of which will fix the code anytime soon (if ever, I have stats), and few have WAFs in front of them (at least in block mode). One big reason I think WAFs are rarely deployed in block mode is because their global rule-set too easily disrupts legit traffic – a big business no no. So instead WAFs are run in alert mode where they issues warning every 15-seconds when attacks hit the website, normally in a spot where they’re not vulnerable anyway. This wastes a lot of time. With VA intelligence WAF may aggressively protect specific areas where a certain type of vulnerability is known to exist.
@ ivan:
you said,
a decent web application firewall should be able to detect this sort of thing without external help
Are you referring to WAF autolearning? For example, a WAF that learns which parameters should accept which data types? How would it deal with XML-RPC and similar scenarios?
@ Jeremiah:
We might be able to use the wayback machine if you have the original URL, but hopefully the “unprofessional” comments didn’t originate for their failure to hire you
Let's just concentrate on the positive and avoid the negative. Maybe I'll show you in person someday. Also - who said that they didn't extend me an offer?
The BITS Shared Assessments SIG looks interesting, I had not see it before, and something I’ll look into. Though I’m not sure how when it says we’re “secure” validates our ability to measure the security of a website
I know that you'll enjoy this link, because not only will you likely learn something about your business - you'll also see that sometimes I do admit that I'm wrong.
Share our source code!? You can’t be serious. No way our company or any other scanner vendor for that manner would do that for the most obvious of reasons. In any event this provide no means to validate the output of our results
I'm extremely serious. The best way to prove that your scanning code is secure/quality is to put it out there. Even in binary format, so that people like myself can reverse it.
I'm not saying your entire source code. It could be the important parsing components (the ones that need to be secured most), or even some basic architecture. Imagine how awful it would be if there was an XSS in your own product's security reporting. Or worse, an integer vulnerability in the HTTP parser library that leads to remote execution.
I know this sounds like I'm asking for a pony, but could you also include unit testing, functional testing, and code coverage tools and statistics along with the release?
OK mr. pundit, please name the “real” analyst that’s against us. I’ve met with and talked to most of them all already and received no resistance so far
This clearly shows a lack of understanding of risk on your part. Security "maturity" changes over time. In the future, even I could be a dominant industry analyst. Regardless, I think you assume way too much with respect to your allies vs. adversaries (another classic risk management mistake). Remember that some of your founders and employees are against you on the VA+WAF concept.
Anyway, the offer still stands that when you are ready to share the criteria for how you can be convinced of the VA+WAF combo
Convince me of the basics first, and then I'll let you have your pony.
I've been thinking about the Jeremiah I "used to like" recently... and while I know that everyone misses the vulnerability research and the sparkly shiny stuff... I also like it when you stick to basics.
For example, my favorite Jeremiah-isms are your posts on asset management (i.e. identification of web applications, your "first-step"). I also miss the surveys and metrics, although I think you could improve these academically a bit and try to get more peer review.
2) Share our source code!? You can’t be serious. No way our company or any other scanner vendor for that manner would do that for the most obvious of reasons. In any event this provide no means to validate the output of our results.
All of the Hailstorm tests are written in Javascript. They aren't "open source" but they are available. Anyone who downloaded an eval could see them. You love to rail about products, and compare your results to them.
1) It's apples and oranges. You don't sell a product, you sell a service.
2) Your marketing against products is largely a strawman - your real competition isn't products - it's consultancies. who use COTS scanners to drive testing by professionals. From tht perspective, your entire marketing effort is a lie, though don't take that personally - you are in no way unique in that regard in the IT industry. ;)
3) OK mr. pundit, please name the “real” analyst that’s against us. I’ve met with and talked to most of them all already and received no resistance so far. Quite the opposite in fact.
Dude. YOU wrote:
"Today we’re well on our way to accomplishing exactly that and even our staunchest critics have come around."
You tell us who they are, and the ones who "came around?" Dre asked earlier, you never answered.
Who are these staunch critics?
Pay no attention to the man behind the curtain! Sorry couldn’t resist.
#2. Cenzic can do whatever they’d like with their tests, as can the rest of the vendors, but we consider our detection criteria largely proprietary. Its taken years of battlefield testing to get the point we’re at. It does not serve our interests or our customers to reveal it to convince the inconvincible. Still if you read my posts I’ve all but begged independent reviewers to compare our results against the rest to no avail. SaaS apparently doesn’t mix well in a review scenario outside the sales cycle.
Now as you’ve pointed out product vs. SaaS is largely irrelevant. We do mostly compete against consultants these days in the wake of the acquisitions, which could be a reason why Andre and perhaps yourself as so overtly critical of the SaaS alternative. Consultants are fighting for their livelihood and I can appreciate that. Several though have realized that they can work with us instead of against with many happy customers as a result.
From your POV though, if we’re unable to demonstrate our ability to back up the marketing claims, then we’ll no doubt land in the dead pool. I wouldn’t count on it though. ;)
#3. It’s amusing for me to think about the critics hindsight, but the staunchest of those from years past we now call our best customers. Usually we convert them from scanners or annual consultant engagements to SaaS in the VA area. I know you desire names, but sorry, I can’t give you those.
@andre
No, I wasn't referring to learning. If we are going to accept that a product category called web application firewalls exists, then we should at least expect of the technology to do what it is intended to: detect attacks.
Responding to your question about XML-RPC, I don't see why a web application firewall couldn't be able to create a positive security model (learning) for it, or for any other XML content, for that matter. The data is what matters, not the transport format.
@jeremiah
There is a great deal of value to be extracted from the integration of scanners and web application firewalls. After all, each side has its own view of the system, and it we can only benefit from combining the two.
Going further, I would like to see the information flow the other way too. For example, web application firewalls have a great internal view of the applications they are protecting, detailing pages, resources, parameter types, and so on. Sharing this information with a scanning product would really take things to the next level.
P.S. I am typing this comment for the second time. My first post disappeared into the aether.
@Ivan, you might have hit the preview button by mistake, I've done it many times.
Anyway, I'm with you on the two-way communication. WAFs have a lot of visibility that could benefit VA. We could see missed functionality, maybe back doors, trends vulns next to attack data, detect application changes, ascertain behavior characteristics, ... the list goes on and on.
A brand new field of study!
It does not serve our interests or our customers to reveal it to convince the inconvincible.
So the inconvincable are not the staunchest critics?
Sorry for picking on you for this point, but your whole blog post is marketing speak - one of the reasons your blog as hone downhill imo. Even your staunchest critics are convinced! Except the ones you cleverly defined out of "staunchest critics" by relabeling them as "unconvincable."
Still if you read my posts I’ve all but begged independent reviewers to compare our results against the rest to no avail.
I can tell you why. I'm sure you know this. It's because it wouldn't be a fair test! DUH. You have an automated scanner of unknown quality and human testers of varying quality.
How can it be controlled for that your hands-on side could spend 10x s much effort on the "review" assessment as a typical one?
It' can't Any result that came from that kind of review would be more than worthless - it would be dishonest.
You have to know this and I think it's a little sad that you lament the wickedness of the presstitutes for not falling for this slight of hand. Of course a human driven service would perform better under these conditions! And no wonder you dearly want this kind of test to be done.
SaaS apparently doesn’t mix well in a review scenario outside the sales cycle.
See comments above.
Now as you’ve pointed out product vs. SaaS is largely irrelevant. We do mostly compete against consultants these days in the wake of the acquisitions, which could be a reason why Andre and perhaps yourself as so overtly critical of the SaaS alternative.
I think SaaS is overall a positive development. I am impressed with how the pros positioned your company after bringing on the VC $, but from a marketing perspective only. From a value proposition perspective, i think there's a lot of little dishonesties such as the ones I mentioned above.
Consultants are fighting for their livelihood and I can appreciate that.
Not the ones I know! They have more work than they can handle.
From your POV though, if we’re unable to demonstrate our ability to back up the marketing claims, then we’ll no doubt land in the dead pool. I wouldn’t count on it though. ;)
You don't have to not back up the marketing claims to stay out of the dead pool. Look at companies like Lifelock for example - smoke and mirrors and making money hand over fist.
Actually my opinion is that Whitehat is positioning themselves for acquisition, and I'm curious to see what happens after that. After all, your CEO sold Securityfocus to Symantec, if I recall... Not a bad exit strategy! If I were to guess it would be within the next 24 months. The VC's need their ROI.
Outside that, I'm not impressed by F5 or their technology or their organization. Never mind that there are entire classes of web app vulns which you can't really fix with a WAF. oh noes!
Marketing hype will not secure your code. There is simply only one way to win this game - whitebox testing: have an expert manually review code line-by-line in sync with experts doing pen tests. You can move as many appsec categories into the "business logic" category as you like, but it still will not change the facts around how to truly secure your web application. Jeremiah, I've been a big fan of your writings until this post. Stinky.
@Jim, I know where your coming from and can appreciate where the model you describe adds value.
Still, my writings and my work are focused at solving problems (the best I can) as I understand them rather winning audience approval. I have accepted that many of my beliefs will and have been controversial, as they were when I first started proclaiming, “webappsec is REALLY REALLY important” years ago at Defcon. Imagine doing this in a world of only network security elitists. Similar skepticism/criticism occurred when attempting the SaaS business model applied in webappsec VA. This is to be expected with any new idea.
The infosec audience is a tough crowd, as well they should be, and we know full well we have our work cut out for us proving the concept.
Please consider two things. While line-by-line code analysis no question adds value -- its also pricey, time consuming, and there simply isn’t enough qualified experts to go around to perform the work on an INTERNET-wide scale. As an industry we have to get better at reacting, even if the solutions aren’t pure or perfect. Secondly, Jeff Williams and I had some very nice conversations at SourceBoston last week. I believe he understands what I’m suggesting here where some meaning obviously got lost in the post. I don’t think you and I are as a far off philosophically as it might first appear.
Jeremiah, fair comments. Code reviews and manual testing will not solve this problem on an INTERNET-WIDE scale. That's where organizations like OWASP come into play. Free information. planetlevel solutions that address every aspect of WebAppSec.
But what still shocks me about your post is that you are tying to lump together solutions on an "INTERNET-WIDE" scale with your companies new partnership with a vendors proprietary and expensive solution.
If you want to really solve this problem on the scale you are talking about - I think you want to reach beyond vendor solutions. What about free webAppSec scanning tools? Perhaps a very detailed guide to running ApacheModSecurity in standalone mode? ModSecurity scalability guides? A free service to get up-2-the-minute free modSec rules that protect against the latest exploits? We all have to make a buck, don't get me wrong, but it's the .org that solves problem on the sale you are talking about. Maybe you care to join us talking about OSS versions of your solution paradigm at https://www.owasp.org/index.php/VA%2BWAF ?
Hmph, I’m thinking the tone of my post made it appear that this is a locked in proprietary solution and alternatives including open source will not be supported. This is not the case.
While we put significant resources into the partnership with F5, its not an exclusive arrangement. We have every intention of integrating with other WAFs as time moves on, but we needed to pick a solid partner and a place to start to demonstrate viability. The arrangement also does not preclude any OSS scanner from pumping rules into an F5’s ASM (via iControl), its a completely open API. Moving forward I fully expect other commercial competitors to offer similar features and for ModSecurity to be on the list.
Your right though, no one vendor cannot be expected to solve all these problem and community efforts are essential. Speaking of free scanners and ModSecurity, I actually developed the CIS Apache Benchmark Scoring Tool that includes ModSecurity a ways back. I released what I think was the first web server fingerprinter tool, one of the first proxies for web security experts, been (am) a vocal advocate of ModSecurity, and even wrote the forward for “Preventing Web Attacks with Apache.” I love OSS and will not leave it out in the cold. Just gimme some time. :)
The ideas you have are good ones, and I think Breach already has some of those available, the rest though will go on my list for future projects! :)
On the page you set up, what sort of questions or concerns should I begin to answer? Maybe I should just write a white paper or something.
> On the page you set up, what sort of questions or concerns should I begin to answer?
How to do VA+WAF fo' free! :)
Ahhh darn, you would have to go to the hardest question. :)
With ModSecurity the OS community has half of the equation filled, though it would be nice if the product had an API of sorts. I know its on Ivan’s list. However, it’s always been the VA part that’s proved extremely difficult for OS to compete with.
Let’s face it, no current OS product comes anywhere close to the commercial offerings, nor will they anytime soon in my estimation. It’s an extremely difficult area of software development that so far has required everyone who attempted to invest years and millions of dollars. I think the only way OS will catch up is for one of the companies to go belly up and donate the code.
What I can see happening for OS is some software tools that serve as generic connectors to the various WAF APIs and rule-sets including ModSecurity. That way however a vulnerability is found, manually or automatically, it can be utilized by a WAF. It’ll probably take all of 2008 to make this an OS reality, I mean… we’re just on the bleeding edge of it right now.
Post a Comment