Follow-up: Secure (Enough) Software, Return-on-Investment Data

 

Jeremy Dallman (Senior Security Program Manager, Microsoft Security Engineering Center) kindly emailed me some information regarding my recent post entitled, “Secure (Enough) Software — Do we really know how?” Jeremy provides three resources that speak to the ROI of investing in secure code. I’ve have some follow-up thoughts at the end. In the mean time, the software security data junkies will enjoy these numbers.

“Jeremiah, I saw your blog post today and it reminded me that I hadn’t sent you a couple of recently published papers that talk specifically to the question you raise. The MidAmerican case study provides an example of the ROI companies are realizing as a result of SDL implementation. In addition, an independent Aberdeen study focuses in on the ROI and gives a larger picture derived from a study of multiple companies security practices. I also included the Forrester report I sent you previously.

MidAmerican case study that demonstrates their ROI and security improvements with data by a company who took the Microsoft SDL and built their own process: http://go.microsoft.com/?linkid=9768047

[excerpt]

The effort even took on its own brand inside MidAmerican called the Secure Development Initiative — an initiative, by all accounts, that worked. Total threats as defined by SDL and the Fortify tool did, indeed, fall below 100 by Sept. 30th, 2009. And by 2010, MidAmerican Energy was the only business unit inside the larger holding company that external auditors found to have no security vulnerabilities whatsoever.

But everyone at MidAmerican agreed there were major benefits. SDL-based planning required groups to think more efficiently not only about how they code securely, but how they code in general. “It just becomes like any scarce resource a company has to manage,” Kerber said. “But the message was, we are not throwing up barriers here in IT, we are protecting you.” On balance, the company saw a real gain in the bottom line: Increased efficiencies and fewer fixes resulting from using the SDL-inspired approach netted a productivity gain that could be as high as 20 percent.

[/excerpt]

Aberdeen Group report (which Microsoft didn’t commission or participate in) has demonstrated data behind the SDL practices. What is called “Secure at the Source” is derived from the Simplified SDL process we defined specifically to demonstrate the transferability of the process to other companies: http://go.microsoft.com/?linkid=9768149

 

 

 

The Forrester Report I sent you earlier has some interesting ROI information in it also: http://go.microsoft.com/?linkid=9762340

We’re finally getting an inflow of data that speaks to the business value of software security programs, data that better justifies the resource investment to management. One of things I probably didn’t go a good job articulating in my original post is trying to evaluate and value the various pieces of an SDL, not the SDLs effectiveness overall. To my mind there is no reason to believe that each activity in BSIMM offers the same benefit as another. For example, when the average organization deploys static analysis software testing during QA it generally costs $X and reduces the number of high risk vulnerabilities in production of Y type(s) by Z%. Something just that simple would suffice. If you could only do one right now, which would it be? That answer would and probably SHOULD be custom to each organization.

Follow-up: Secure (Enough) Software — Where Are the Requirements?

My recent post entitled, Secure (Enough) Software — Do we really know how?, sparked a very thoughtful comment by Mitja Kolsek (ACROS Security), which read more like a well-written blog post than anything else. Mitja goes onto explain one of the more fundamental challenges between the implicit and explicit (security requirements) forms of software security. He really hits the nail on the head. Such a good blog post is not something seen every day, so with Mitja’s permission, I’m republishing all readers to enjoy.

“A great article, Jeremiah, it nicely describes one of the biggest problems with application security: How do you prove that a piece of code is secure? But wait, let’s go back one step: what does “secure” (or “secure enough”) mean? To me, secure software means software that neither provides nor enables opportunities for breaking security requirements. And what are these security requirements?

In contrast to functional requirements, security requirements are usually not even mentioned in any meaningful way, much less explicitly specified by those ordering the software. So the developers have a clear understanding what the customer (or boss) wants in terms of functionalities while security is left to their own initiative and spare time.

When security experts review software products, we (consciously or less so) always have to build some set of implicit security requirements, based on our experience and our understanding of the product. So we assume that since there is user authentication in the product, it is implied that users should not be able to log in without their credentials. Authorization implies that user A is not supposed to have access to user B’s data unless where required. Presence of personal data implies that this data should be properly encrypted at all times and inaccessible to unauthorized users.

These may sound easy, but a complex product could have hundreds of such “atomic” requirements with many exceptions and conditions. Now how about the defects that allow running arbitrary code inside (or even outside) the product, such as unchecked buffers and unsanitized unparameterized SQL statements and cross-site scripting bugs?

We all understand that these are bad and implicitly forbidden in a secure product, so we add them to our list of security requirements. Finally there are unique and/or previously unknown types of vulnerabilities that one is, by definition, unable to include in any security requirements beforehand. My point is that in order to prove something (in this case security), we need to define it first.
Explicit security requirements seem to be a good way to do so. For many years we’ve been encouraging our customers to write up security requirements (or at least threat models, which can be partly translated into security requirements) and found that they help them understand their security models better, allowed them to see some design flaws in time to inexpensively fix them, and gave their developers useful guidelines for avoiding some likely security errors.

For those reviewing such products for security, these requirements provide useful information about the security model so that they know better what exactly they’re supposed to verify. Only when we define the security for any particular product can we tackle the (undoubtedly harder) process of proving. But even the “negative proof and fix” approach the industry is using today, i.e., subjecting a product to good vulnerability experts and hoping they don’t find anything or fix what they find, can be much improved with the use of explicit security requirements.“

How I got my start -- in Brazilian Jiu-Jitsu

I’ve been a UFC fan for years, even before it was acquired by Zuffa. I was fascinated by the anything goes, hand-to-hand form of combat. I suppose it reminded me of growing up in Hawaii. :) The UFC was also enjoyable because it helped answer the question, “What martial-art or fighting style was most effective?” Karate? Kickboxing? Boxing? Wrestling? Ninjutsu? What matters more, size or technique?

The UFC provided a forum, the octagon, to settle the long-standing fight-world debate. Everyone had a theory, but no one really knew for sure. What became crystal clear even today is that every fighter must have a background in Brazilian Jiu-Jitsu or they WILL lose. It’s just that simple. My background was mostly striking, so I wanted to try out this ground fighting stuff.

A co-worker, also interested in the UFC, and I found a local BJJ academy in San Jose taught by black belt instructor Tom Cissero. Tom has a passion for the martial arts and, more importantly, for his students, as he deeply feels that they are a direct reflection upon his life and value as a person. Yes, he takes his craft that seriously, and serious he is. Tom is abrasive, aggressive, and combative, attributes covering up a heart of gold. In the academy Tom will push you hard, harder than any place else, to make you good. Whether you like it or not, and he cares enough to do so. That’s why I stayed with him the better part of a decade.

Anyway, my 6’2” - 300lbs, and let’s face it, seriously fat and way out of shape frame walks in -- admittedly with a little bit of big man ego. I see Tom instantly trying to size me up. Of course he had me figured out in all of 5 seconds as you’ll read in a moment. After signing the waver, doing some drills, and learning a couple of submissions I began to familiarize myself with the basic rules and gym etiquette. Then came sparring time. Tom loves the sparring sessions more than anything else. Probably because it measures your progress in stamina and skill.

Tom pairs me up with, and I kid you not, a 150 lbs or less woman in her mid 40’s and says let’s see what you can do. She’s a purple belt with several years of BJJ experience, but I’m thinking to myself WTF!? She’s half my size! I’m going to squash her! Then of course the whole situation is running counter to my internal man moral code, never fight girls. Not being given a choice, but also not wanting to be disrespectful, I decided to go really easy as I didn’t want to hurt her or anything.

The bells sounds, I come slowly forward towards her, she quickly closes the distance, spider monkeys to my back, chokes me, and forces me to tap out inside of 10 seconds flat. I was shocked and a little upset. Here I am going light and she takes advantage of me. Clearly she’s not playing around. To hell with this, no way I’m going to let that happen again! No more Nr. Nice Guy.

We touch hands, signaling to begin again, but I go harder this time trying to put her back on the mat. She again somehow sneaks around under my arm, like an octopus, and chokes me with the same damn move! To my credit, I lasted a few more seconds that time. This scenario repeats for about 4 to 5 minutes in the session, and for the life of me, as big strong guy, I could not keep this tiny older woman off my back and robbing the oxygen from my brain. Oh, and all the while she is speaking to me in a calm instructive voice. Humiliation is the best word to describe.

At the end of class I’m thinking to myself, there is something to this Brazilian Jiu-Jitsu stuff. However, that wasn’t the most important thing to me at that particular moment. There was no way I could go on about my life happily knowing that a such a women could kick my butt so easily. Call it machoism if you like, I don’t care. It was clear to me that I had to keep training BJJ at least long enough to beat her. It only took three years. Fortunately for me by that time the motivation to simply get better and enjoy myself became my primary driver.

By the way, that woman is still training there. So if you are a big guy, and plan to drop by for a visit, don’t say I didn’t warn you. You could quickly find yourself on a journey to becoming a BJJ black belt.