Full-Disclosure, Our Turn

Vulnerabilities in websites happen, especially the ever pervasive Cross-Site Scripting (XSS). Essentially every major website has had to deal with XSS vulnerabilities published publicly or otherwise. This also includes security companies. No one is perfect, no website has proven immune, ours included. As experts in Web application security and specifically XSS, yesterday even we took our turn. We hope to learn some lessons from the experience and share the details so others may do the same.

For background, the WhiteHat Security corporate website (www.whitehatsec.com) is static brochure-ware. No Web applications, no forms, no log-in, no user-supplied input where XSS can hide. Just the every day HTML/JavaScript/CSS/Flash explaining what we do, why, and how. The website does not carry any sensitive data, although it does carry the WhiteHat Security brand and our reputation is extremely important to us. Therefore any publicized vulnerabilities or defacement would obviously be embarrassing.

Monday afternoon @thetestmanager openly posted on Twitter an XSS vulnerability that reportedly affected www.whitehatsec.com:

"It really does happen to the best of us. XSS on WhiteHatSec http://bit.ly/cIDfEA If you take a feed to your site do you check their code?"
http://twitter.com/thetestmanager/status/17283467463

“By the way, that tweet was meant as a bit of fun and certainly not a poke at @jeremiahg or any of @whitehatsec The hole isn't in their code”
http://twitter.com/thetestmanager/status/17283586987

Upon seeing the tweet a few minutes after posting I was a bit skeptical being familiar with the limited attack surface, but took the matter seriously and immediately sought to confirm or deny the validity. Fortunately/Unfortunately the particular XSS issue was very straight forward to confirm. The all to familiar alert box was plain as day. What was more worrisome was the existence of a query string, something that’s not supposed to be on the website at all! It took a few more moments to isolate out the source of the issue, a third-party supplied JavaScript block (accolo.com) located on our careers page that sourced in additional code remotely.

document.write("<* x-script src='http://members.accolo.com/a02/public/CommunityJobs_include.jsp?isJSInclude=1&" + Accolo.paramsStr + "'>");

Third-party JavaScript includes are obviously extremely common on the Web. First order of business, remove the offending JavaScript code ASAP and then perform root-cause analysis.

Remediation time: ~15min from original time of disclosure.

When embedded in a webpage Accolo’s third-party JavaScript code dynamically creates a DOM web-form where prospective interview candidates may enter their details. It is important to note that the form action points to an off-domain, but various cosmetic interface settings are pulled from a superficially created query string on the hosting website (our website). Whew, the corp website still has no Web applications. However, when Accolo’s form setting values are pulled from the query string they are document.write’ed to the DOM in an unsanitized fashion, hence the source of the XSS vulnerability. All @thetestmanager had to do was to insert a SCRIPT tag into one of the fake parameters to get a DOM-based XSS to fire. A solid find. After removing the code from our site we reported the issue to Accolo so they could remediate. Likely more of their customers are similarly affected.


Some Obvious Questions & Lessons Learned:

1) Do you consider @thetestmanager to be “irresponsible”?
Of course not. Yes, we would have preferred a more private disclosure, but whaddaya gonna do. Name calling is unproductive. Things like this happen to everyone and you deal with it as best you can. We appreciate @thetestmanager for taking the time to find the vulnerability and bringing it to our attention at all. Clearly someone less savory could have found it, not said a word, and did something worse.

2) Why was this vulnerability missed?
Among the many safeguards taken to protect the corp website, including daily network & web application vulnerability scans, we perform intensive static and dynamic security peer review by the Sentinel Operations Team on all “substantive” website changes. However, a process oversight occurred where marketing / HR requested a new website feature that did not meet the threshold for substantive change, just a simple JavaScript include, so no review was requested or performed. The Sentinel Operations Team would have caught this issue. We’ve since updated the process and double checked all third-party JavaScript code currently in use.

The Sentinel Service itself did not identify the issue due to a standard explicit-allow hostname configuration. Sentinel does not test links of forms bound to off-domain locations, such third-party JavaScript includes, for legal and liability reasons. Since Sentinel didn’t follow / fill-out the form it did not see the attack surface required to detect the vulnerability. In a Web 2.0 world this scenario is becoming far more common, is something we’re working with our customers to address who have the same challenge, and speaks to a larger industry problem worth exploring.

A) To assess a third-party (by default): If an assessment includes third-party JavaScript systems, which are technically within the domains zone of trust, there are very real legal and liability consideration when network traffic is sent their way. Getting expressed written authorization to thoroughly and properly assess a third-party is extremely difficult to obtain, but without it prosecution in many countries in a consequence one faces.

B) No control, no visibility: Even if third-party JavaScript is assessed, organizations have a very limited ability to exercise change control or even visibility when that code is updated -- almost always without their knowledge. An organizations website site might be fine one minute and literally vulnerable to XSS the next.

The bigger question then becomes, "how does an organization handle security for 3rd party includes & JS files"? For business reasons, the answer can't be "don't do it."

3) What should Accolo have performed to prevent the vulnerability?
Properly escape and perform input validation, in JavaScript space, on all the URL parameter values before writing them to the DOM. Further, use appendChild() as opposed to document.write to help prevent secondary JavaScript execution. Since the JavaScript code did not need access to the customers domain an IFRAME HTML include, rather than embedded JavaScript, would have been a better choice to achieve cross-domain separation.

4) How are you going to improve you operational website security posture?
On the average we have roughly a single XSS vulnerability in the production every 2-3 years. Remediation occurs within a couple hours at the very most in every case. These metrics are consistent with the absolute most security proactive organizations found anywhere and far exceeding industry averages (a consistent handful of XSS per year, ~50% remediation rate, and fixes taking months). Having said that, as described earlier, we’re improving our procedures to make sure things like this don’t slip by again.