How I got my start

I’ve told this story from time to time when asked, but never written about it. The inspiration to do so came from posts by Security Catalyst and Andrew Hay. Computers have been a part of my life for nearly 20 years, starting with a Commodore 64 when I was ~10, a few x286s and x386s, then a Power Macintosh 6100 my mother bought for me when I was 16. Coded a lot in between, but that’s ancient history and a story you’ve probably heard a hundred times. So let’s skip to where it gets interesting. And to borrow on line from Great Expectations: “I'm not going to tell the story the way it happened. I'm going to tell it the way I remember it.”

As a late teenager I left Maui for California to attend college, find a decent job, and seek out greater opportunity. I enrolled in a trade school for electronics engineering. At the same time my couple years of web development and Unix experience was enough to land me a job as an entry-level Unix Administrator for Amgen (a large bio-tech company). I got to work on mega-big Sun (Solaris) servers as part of a team responsible for backups, disaster recovery, and web-enabling day-to-day operation. Amgen was very wealthy, treated their employees exceedingly well, and threw lots of posh parties. Most of which I couldn’t attend because they served beer & wine and I wasn’t yet of age. Funny eh. Being only 19 or 20, this was a kickass job, learned a lot, and loved it.

Then in the autumn of 99’ something strange happened. A slew of well-publicized articles hit the Web saying someone found vulnerabilities in many of the major websites like Yahoo!, Amazon, and eBay. As a result they were “insecure”. I knew making a secure website was exceeding hard, and as far as I was concerned, impossible. So being a naive youngster, I couldn’t figure out why this was newsworthy and thought everyone already knew! What happened next was even more interesting. Weeks later the same companies issued statements that they had fixed the issues and were now completely “secure”. Amazing, naïveté kicking in, how did they do this!?!? I had to know!

After work I registered a few shiny new Yahoo! Mail accounts, and thought to myself – “how hard could this possibly be (to break into my own accounts)?” I don’t remember the actual vulnerability, but it took only a few minutes to find. Probably had something to do with an XSS / JavaScript filter-evasion or meta-character injection. Wrote up a simple minded advisory and sent it anonymously to the only internal Yahoo! corp email address I could find (specific address withheld). The message explained what the issue was, said I didn’t want any credit or press, and to let me know if they had any questions. Had my fun for the evening figuring that was the end of that.

The next morning I checked the account and lo’ and behold someone responded! Wow, Yahoo! is talking to ME! During those days there was no company bigger or more exciting than Internet darling Yahoo! The email read thank you for letting us know about the issue, we have a couple of questions, appreciate you wanting to remain anonymous, but we’d like to send you a t-shirt. WHOA HO! Not only was I able to find a vulnerability in Yahoo!, but it was important enough that they’re asking about it and want to supply me with clothing. Cool. They also said to let them know if you find anything else. Sweet, an open invitation.

Over the next week or two I’m finding bugs, reporting them, and having a casual dialog with somebody at Yahoo!. I mentioned to a co-worker what was going on and they were excited about it as well. They asked whom I was communicating with, but hadn’t thought to check. I forwarded one of the emails so they could see. They facetiously told me to have a look for myself. I took the name from the email address (withheld), and I kid you not, it was from David Filo, one of the two Yahoo! founders and so-called Chief Yahoo!. OMG! Here I am some dumb kid with a few web app vulns and I’m having an informal nonchalant conversation with a billionaire. BTW, Filo is a very cool guy and if you ever met him, he’s an amazingly smart engineer without an ounce of arrogance or superiority complex that one might expect.

A few days later a Yahoo recruiter emailed and asked if I’d like to fly up to Silicon Valley for an interview. Stunned is the word that best describes my reaction. I made sure they knew that I had no formal education or security experience to speak of. It didn’t bother them, so it didn’t bother me. At their purple and yellow laced offices, they grilled me on all sorts of technology questions for 6 hours, most of which I didn’t know. C’mon, this was Yahoo, how could I? Figuring I had no shot at all I promptly flew home. I heard back a couple days later via FedEx. YES! An offer letter from Yahoo! OMG, look at the salary! COOL, job description was nothing but security! NO WAY, stock options! YES! Wait, what the heck are those?

I took the prized yellow piece of paper to my school counselor and asked her a very simple question, “What’s the top student out of here expect to make at their next job?” Begrudgingly she told me and it was nowhere near what my offer letter said. Then asked if she had any GOOD reason why I should say. The only answer I recall was that you’ll never go back and finish your degree (she was right of course). And just like that I was gone and on my way. Took a month off back on Maui, hiding out from Y2K, then started my new life in January of 2000.

I became part of Yahoo! Engineering and felt completely out of my league around the real super brains of Web technology. My job was simple. Identify as many security vulnerabilities as possible, formulate solutions, and chase down developers to resolve them. Talk about awesome with my new "hacker yahoo" job title. It took about 6 months to get my arms around web application security and how big and important this job actually was. The scope was roughly 180 million users and 17,000 publicly facing web servers. 12+ hour days were the norm. On a system that massive, IDSs say only 1 thing that matters, everyone is attacking you with everything they got all the time. Unscientifically we thought about 1% of our user-base was malicious, a similar number we heard from our industry peers. Yes, 1.8 million bad guys.

I performed what seemed like a never-ending supply of web application security assessments. It was rare for a website to be completely free of security issues. To streamline the workload I developed an assessment methodology consisting of a few thousand security tests averaging 40 hours to complete per website. The eminent domain was over 600+ websites, enterprise-wide, in a dozen or so languages from english-to-french-to-korean. The way the math worked out assessing the security of every website would have taken over 11 years (24,000 total hours/ 2080 working hours per year) to complete if nothing ever changed. Yah, right.

Obviously we needed to reduce workload because few experts existed and we certainly weren’t going to hire a dedicated team of 10. Seeking help I talked to every expert I could find and experimented with early commercial scanners and web application firewall solutions. None of what I saw was going to solve our problem. Almost two years in what I did see was opportunity. Yahoo! was certainly not the only one in the e-commerce business or grappling with too much work, too few experts, and a lack of decent tools. I felt I could do better, which ultimately led to my founding of WhiteHat Security.

Well, that’s how I got my start. And the last 6 years have only gotten more interesting.