The Solution to Application Security’s Biggest Challenge, Vulnerability Remediation, May Finally Arrive

The importance of vulnerability management is simple — find and fix issues before an adversary finds and exploits them. Unfortunately, the remediation rates reported by leading application security vendors average only around 50% or far less. And when vulnerabilities are fixed it takes weeks or months. The rest of the vulnerabilities? They’re often never fixed and this has been the reality for many years [1][2][3][4][5].

The underlying reason for vulnerabilities not getting fixed is basically resource constraints. When application vulnerabilities are found, typically they must be fixed by an internal software development group, not the InfoSec team. And since software development resources are always scarce, allocation between vulnerability remediation and building new features is purely a business decision. And the needs of the business largely favor revenue generating features over security issues.

At the same time, many companies have hundreds and often thousands of websites in total with an untold number of code repositories supporting them. And in my experience working in application security for ~20 years, such as WhiteHat Security and 1000+ customers, only ~20% of their websites are routinely scanned for vulnerabilities. And this essentially the same for the underlying code repositories as well.

And the reason for the lack of pervasive application scanning is understandable: if a company already can’t keep up with their current remediation challenges, they’re certainly not going to want to spend more money to identify potentially thousands more vulnerabilities that they also can’t fix any time soon.

A scalable vulnerability remediation solution is what holds back pervasive application scanning, and leaves thousands of companies at risk without viable options. Finding a way to remediate vulnerabilities faster, easier, and cheaper would be absolutely monumental and push the entire application security industry forward. That’s why I’ve been focusing and researching this problem for well over a decade.

I’ve worked with WAF technology, RASP technology, browser technology, leveraging third-party development shops, and anything else that might work. All these approaches have their pros and cons, and do work in certain scenarios, but ultimately they have so far been unsuccessful in broad market adoption. More product innovation is needed.

AI technology provides an exciting opportunity to solve vulnerability remediation. We’re already seeing how developers are able to leverage AI to automatically generate code. In the same way, what if it was possible for AI to import Static Application Security Testing (SAST) results and automatically fix the code with an AI Agent built on LLM technology. Ideally, all a developer would need to do is review the fixed code and accept it for QA testing in a single click. This allows a developer to fix an issue while it's fresh in their mind in less than a minute, much better than getting a ticket 3 months after the code was written.  

There are at least a few vendors working on this approach. Recently I was introduced to a start-up called Amplify, who is building a product based on this exact concept. Amplify provides developers with an AI-powered tool that automatically fixes vulnerabilities in a way that would be equivalent to having a Sr. Developer and Sr. Security Engineer sitting and solving the problem together. The potential of this technology is exciting and will only get better over time. I believed in the founder, the vision, and implementation enough to become an Angel investor. 

I personally want to be part of solving this problem after spending most of my career in the application security industry. Success would enable every company to finally be able to scan their entire code repositories for vulnerabilities, and when vulnerabilities are found, they can do something about it quick and easy. Remediation rates would be drastically improved, mean-time-to-fix goes way down, and application breaches become rare. This is the entire goal of the application security industry — and it could be right around the corner!

Why InfoSec Vendors Force Customers to Work with Sales

If you visit practically any enterprise InfoSec vendor’s website and are interested in trying out their products or services without speaking to a sales rep first, good luck — this is rarely allowed. Even just getting pricing info from a vendor without engaging in a sales process is next to impossible. The vast majority require customers to email or fill out an online form, schedule a meeting with a sales rep, sit through a PowerPoint presentation, and THEN they’ll let customers try the product. And all of this happens in a carefully scripted and supervised manner. For many customers, this experience is often frustrating and avoided whenever possible.

I’ve long asked why sales leaders and reps insist on connecting in person with customers before even considering allowing demos or providing pricing. One explanation they give is if trials are allowed without an initial sales meeting, customers will struggle with installation, configuration, or usage and fail to comprehend ‘full’ value. Sales leaders are concerned about potentially losing deals to competitors who require a more hands-on white-glove process. 

As for pricing, sales reps will say if the website reveals pricing upfront and competitor’s websites don’t, customers might get sticker shock and avoid contacting them. This prevents sales reps from having an opportunity to demonstrate the product and justify the value while the customer looks into another solution. For these reasons and others, is supposedly why customers must endure a people-intensive, painstaking, pressured, slow, and frustrating sales process.

While these enterprise sales philosophies may have once made sense in a previous decade, today, they feel antiquated and inferior. For example, we see the sales models of big cloud service providers such as Amazon, Google, and Microsoft. They’re capable of collectively selling hundreds of billions of dollars a year in IT services to the smallest of the small and largest of the large organizations in the world, basically friction-free. At any time, an interested customer can spin up thousands, hundreds of thousands, and even millions of dollars worth of services without ever having to speak to a sales rep or anyone in minutes. Why can’t or why isn’t every InfoSec vendor following their example?

Is the value of today's InfoSec products really too complicated for customers to understand on their own? Are customers really incapable of figuring out how to deploy products without assistance from sales? Does making pricing info readily available actually drive customers away toward competitors? If so, then my contention is we have a serious and industry-wide product deficiency problem on our hands. And every problem is an opportunity to improve.

For the average start-up I’ve worked with, the sales department generally represents 12-18% of the overall company budget. And the marketing department budgets are roughly the same. Marketing spending is an important consideration here because they have to find and push hard to convince customers to engage in a sales-led process rather than just clicking a link. Then, often because a vendor’s sales reps don’t have an existing relationship with a customer needed to get their attention, they’ll rely on the channels (i.e., VARs). For this very reason, many customers prefer to evaluate and buy through one of their ‘trusted partners.’ Tack on another 3-30% of the cost of sales in channel commissions.

All of these sales and marketing costs add up and partially explain why enterprise security products are so expensive. And contribute to why they’re out of the price range of many small and medium businesses (SMBs). In the current model, it’s just not worth a vendor’s time to sell to SMBs unless they engage on their own. Personally, I see a huge opportunity for existing vendors and start-ups alike who successfully solve this problem. 

Imagine for a moment if an InfoSec vendor found a way to cut down this sales and marketing overhead by enabling a self-provisioned sales process, and invested those dollars directly into their product that can [gasp] sell itself! The overall cost of sales goes down, customer satisfaction goes up, deals are done quicker, the vendors become more competitive, and opens up new market opportunities (eg, SMBs). The sales apparatus of the big incumbent security vendors will have a difficult time making such a shift because the entire sales department will resist. Therefore, the advantage goes to the start-ups. And we’re recently just starting to see trends of InfoSec vendors selling through Amazon’s marketplace, for example. I’m hoping this is a trend.

InfoSec Market Labor Shortage and Predictions

Observations

From my personal experience and through conversations I’ve had with many other security pros, we’ve observed that the average level of competency among enterprise InfoSec personnel is either flat or decreasing. And this has been steadily taking place for several years. This occurs despite the plethora of widely accessible educational content and professional training options. This is important to note because in order to remain effective the operational environment of InfoSec also requires professionals to learn an ever-expanding knowledge base. As an every expert will attest, this is a significant challenge for every individual and organization.

Then as businesses digitize essentially every product and service in modern life, today’s IT environments have become incredibly sprawling and more complex by the day. This level of complexity, and the associated legacy IT backlog, makes it exceptionally difficult for practitioners to comprehend, monitor, and maintain robust security of the environments they’re meant to defend.


Causes
The InfoSec market is growing rapidly ($172B annually with 10-12% CAGR), leading to a high demand for skilled professionals across the corporate spectrum. The demand and subsequent skill gap are exacerbated by new and emerging technologies such as IT/OT, cloud, virtualization, microservices, blockchain, low-code/no-code, new programming languages and frameworks, and of course, AI/ML. 

Nobody can claim expertise in all these areas or even close. Additionally, InfoSec does not have structured and widely available pathways to onboard entry-level talent. Hiring managers also commonly struggle to accurately assess the level of expertise of potential hires due to the nuanced and complex nature of InfoSec skills.


Prediction
In the near term, there are no scalable options yet on the horizon to broadly address these labor issues, and we have every indication and expectation that the skill gap will remain and likely even widen. If so, and for lack of better options, we can only expect organizations to continue placing inadequately trained and inexperienced personnel to fill vacant security roles who operate closer to program managers. This is a reasonable approach given the current constraints.

Subsequently, many organizations lack confidence in their ability to sufficiently protect their environments from breaches — and for good reason. Many practitioner surveys published over the years support this observation. While some people will suggest substantive wage increases as an immediate solution, to which I don't necessarily disagree, doing so can only help individual organizations. The larger net effect can only serve to shift labor shortages from one area of the market to another and will do little to solve the overall industry shortage.

Opportunity

1. An increasing number of organizations and their security programs will rely upon Managed Security Service Providers (MSSPs) for third-party assistance — especially MSSPs who are willing to take on contractual liability. Of course, reliance on MSSPs does not necessarily solve the core challenges; it only transfers the security problems from organizations to the MSSPs. Security product innovation remains a crucial component of the market. The MSSP market winners will be those capable of offering a comprehensive suite of security controls capable of keeping up with an evolving threat landscape. That said, no amount of technology automation in any segment of InfoSec completely removes the need for human expertise. Therefore, the MSSPs who can best hire, train, and retain top talent will have the long-term competitive edge.
2. If an increasing percentage of InfoSec budgets are going to flow through MSSPs, this becomes an increasingly attractive go-to-market strategy for both incumbent security vendors and start-ups alike. Especially for those capable of integrating seamlessly into the current MSSPs technology stack and processes.
3. Businesses are finding that cyber-insurance is becoming compulsory. And it makes sense because if you feel that you can’t protect against the breach, at least protect against the monetary loss. So we’re going to see an expansion of cyber-insurance carriers, both large and start-ups, offering insurance packages that come with a suite of security solutions bundled in — for free. The question is, will they build these technologies themselves, partner for the capability, or make acquisitions?

My prediction is: All three.

1950 Mercury Christmas Present

As a gift, or sometimes more like a curse, my dad passed down his love of classic cars to his children. Each of us has our favorites, and one of mine is a 1950 Mercury. Not just any 1950 Mercury, but a particular highly customized “led sled” hot rod. Chopped, dropped, frenched, chrome out grill, shaved door handles, bagged with black paint and red flames. It’s the kind of car most people will only see in a classic car magazine or more likely a comic book. Such as car is not really supposed to exist in real life. You’re not going to see one on the road. You’re not even going to see one at a car show. In fact, I’d never seen one like it [in person] until last year, and I’ve searched for 20 years. It’s my unicorn. 

One might ask why I just didn’t buy an old broken-down Merc and restore it. It’s a fair question. I have helped my dad restore classic cars since I was a kid. However, a 1950 Mercury project like I described would have been very different, a whole other level of cost and difficulty. Believe me, I considered it for years. The shell of body, IF you can somehow find one somewhere in any condition, will still cost $15-20K due to the rarity. Then I’d somehow have to transport it to Hawaii because they don’t exist anywhere in the state. I looked. Then the customization requires a set of skills that only master body mechanic would be capable of, with heavy fabrication skills, and a machine shop to match. With the facilities at my disposal and of Hawaii in general, it just wasn’t going to be possible. Finding and buying one in relatively close condition was the only option.

Over the last 25 years I’ve travelled a lot. 1 million miles on United Airlines alone, but who’s counting. In every state and city I’ve visited, I’d routinely fire up Craigslist and see if any of my bucket list cars were for sale in the area. The years rolled by, and while I did manage to buy pair of 19 64 Lincoln Continentals, I never ever came across my dream 1950 Mercury – that is until this time last year.

Christmas of 2019 the family travels out from Hawaii to Tennessee to spend the holiday with relatives. I’ve been to Tennessee many times before, mostly Memphis and Nashville. Nice wide-open country, friendly people, and my favorite part are the fireworks stores. Hawaii and California basically outlawed everything except sparklers, but not Tennessee. Tennessee has stores that look like Target that sell everything in all shapes, sizes, and colors. Complete with push button video demonstrations in the store so you can see what you’re buying. On this trip I filled up two grocery carts to the brim and let loose inner child pyromaniac. Let me tell you, my kids and I had a blast lighting up the sky over the river out back. All eyebrows accounted for.

Early one evening, a couple days before the 25^th, it was time to check Craigslist. To my utter astonishment, a 1950 Mercury showed up in the results located around a 1.5hr drive away. I couldn’t believe it. I was skeptical, very skeptical. No way this could be real. No way. At first I thought it must be a targeted ad or a scam or something based on my search history. It looked just like everything I’d hoped and dreamed for 20+ years. Again, I’m in a VERY rural area of Tennessee outside of Knoxville with an extremely small population.  I double-checked I was searching in the right area. Yup. Maybe the listing was outdated? Nope. Posted 2 days ago. Obvious signs of a scam? It didn’t look like it. The only way to know for sure is to call the seller. And just like that, Marvin picked up! 

On the phone I asked Marvin every question about the car I could think of. Where’d you get it? Does it really look as good as in the pictures? Who built it? How’s it run? What’s wrong with it? Rust? The price? Well, let’s be honest, I didn’t care about the price. This was literally the opportunity of a lifetime and I wasn’t about to miss even the opportunity to see it – IN REAL LIFE! So, I scheduled a time for first thing the next morning, jumped in the rental car with my son (14) and uncle Jim, and we headed out to the boonies on a classic car adventure!

After a long drive, we pulled up to the address and right there we see it parked in the driveway. A gleaming and perfect 1950 Mercury coupe. My world stopped. The three of us just stared. It looked like a full-size hot wheel. It didn’t look like real. Every detail was exactly how I’ve always imagined it. We lifted the hood, opened the doors, and crawled underneath it. E very detail was perfect right down to the 3-inch chop, push-button trunk and door locks, and electric windows. The damn thing even had pin striping on the frame. THE FRAME! The only thing that was incomplete was the interior, which had late model Acura seats. I didn’t care, I could easily replace that later. Marvin explained that’s he’s a body guy and built the car himself over the last many years. Swapped out all the old Mercury internals and replaced everything with Chevy parts. He did all the body work personally, at home, in his homemade paint booth garage. 

We took the car on a short drive around the neighborhood. This thing hadn’t seen the road in a couple of decades. It rolled, it stopped, it ran great. Because I wasn’t prepared, and typically avoid impulse purchases, I tried to find any reasonable excuse to NOT buy this car. I mean I wasn’t in Tennessee to buy a car, and this wasn’t on a work trip. I was there on a family vacation, completely unprepared. Still, I couldn’t say no, I had to have it. This opportunity was never going to come up again. Never. Ever.

Marvin wanted 4 stacks of high society and not a penny less. Not a bank wire. Not a cashier’s check. Not PayPal. Not Bitcoin. Marvin was a good ol’ country boy through and through. He wanted cash in hand. I explained to Marvin that I’m from out of town and didn’t travel prepared to make such a large cash transaction. If he could give me a little time to figure things out, and NOT sell the car to anyone else, I’d appreciate it. He agreed. I’m guessing there wasn’t going to be someone else showing up ready to buy THE car, in THIS part of the country, for THAT much in cash, 2 days out from Christmas anyway.

The next day my son and I visited a local bank branch, let them know we needed to make a large cash withdrawal to buy a car. After some identify check verification, they said the bank manager who can authorize the amount wasn’t in – holiday vacation. Dah! Back in the car and rush off to the next branch. I don’t have much time to get this done because we’ll be flying out in a couple days.

Arrived at bank branch #2, waited through another series of identity checks, manager approved the amount, but they informed me that they don’t have nearly enough cash on hand. The holiday apparently wiped out their cash reserves. They’d have to order it, which would take at least a few days. I explained that I needed it now as I’d be flying out by then and asked what my options where. So, the manager called two other branches in the area. The only one that could help was another hour drive away, and maybe they had enough. Off we go! 

Bank branch #3. By this point I learned to immediately ask to speak to the manager. I explained what I needed, and again went through the identity check procedure. Unfortunately, the bank didn’t have enough hundred-dollar bills to cover the amount, nor enough fifties, so we had to accept the remainder in twenties. Whatever. Cash is cash right!? Success!

Now, how many times does a young kid get to feast their eyes on so much money and be able to physically hold it? So, we just had to take the obligatory photo of the experience. Our next step was to contact the seller and drive back out to buy the car and get it picked up.

Let’s pause for a moment to reflect on the visual of this moment. My son and I are two out of towners from Hawaii, in rural Tennessee, driving a rented mini-van cross country, carrying a large of amount of cash in mixed denominations, and after just having visited 3 banks. You better believe I was following every single traffic law making sure to avoid getting pulled over and caught up in some kind of civil asset forfeiture situation. “No sir, I swear, we’re just trying to buy a classic car.” 

In the meantime, I’m calling around the area trying to find a tow truck driver that’s working and has the equipment to pick up a classic car in a remote region. 5 companies later, I finally found someone whose up for the job. I tell him the time and please. We’re set!

We finally arrive back at the seller’s house, and strangely his entire family is there waiting. His wife, one of his daughters, and her husband. Weird. The invite us in the house, and I get the distinct impress this affair was something far more important than just a car transaction but didn’t know what it was. My son and I sat at the kitchen table making small talk while the family divided up each brick of cash, counted it, and visually inspected each bill for counterfeits. They tell us fake bills have been a problem recently in the local area. They’d hold each bill up to the light, looking for the mag strip and watermark. The painstaking process took 2 hours. It was fine though, the family was very nice, and we got to know them and the car a little bit better.

Here’s where things become truly incredible. Once they’re nearly done counting the money, no issues, Marvin’s wife gets up from the table to make a call on the cordless phone. I hadn’t even seen one of those in years. I make out that the call is to another daughter that lives nearby. She explained that her dad just sold the Merc, and that’d be paying off her mortgage with the money as a Christmas gift. My jaw about hit the floor and my eyes are open double-wide.

They shared that Marvin had been working on the Merc for 3 years, night and day, and it was finally in good enough condition to sell. Then my son and I somehow showed up. Apparently, their daughter had recently lost her husband, leaving children behind and they’d fallen on tough times. The money was to help make sure that she and the kids would be taken care of. Like I said, this might be the ONLY way that someone would very build or part with such a car. 

My son Jaye and I are witnessing this. A pure and special moment in a family taking place right in front of us in the most impossible of circumstances. Talk about an experience. I got my dream car, they paid off their house. Merry Christmas.

 

A 1951 Ford for Dad

I wanted to get my dad a gift, but not just any gift. The perfect gift. For a diehard hot-rodder like my dad, there can only be one thing -- a car. Of course, not just any ol' thing with four wheels. He quite literally has 50 mostly junkers and clunkers already. Only THE DREAM CAR would do. What kind of car that was I really didn't know. I had to find out exactly, EXACTLY what that kind of car without letting my dad know and spoil the surprise. For this I asked my brother Zach for help. Discreetly, while both of them were watching a hotrod show on TV, Zach found out that dad's all-time favorite car is a 1951 Ford 2 door with the original flathead v8 engine. Score!

As the story goes, this is the very same car his dad, my grandfather, had bought for him at age 16 for $50. Cars have a way of making a lasting impression on people like this. True to the stereotypical ethnicity of our namesake, grandpa Hyman was worried that he might have overpaid. For reasons I still don’t know, my dad had never owned another like it since. We’re talking 50 years! This is extremely odd because over the years he’s owned essentially every other kind of car, having always been somehow connected to the car business. It probably had to do with their rarity, especially on Maui, as I came to find out to the hard way.

For months, and months, and months, Zach and I scoured Craigslist national wide and the whole rest of the Internet. We found only the trailer queen show cars selling for many tens of thousands, or on the other end of the spectrum, a pile of rusty incomplete junk. Neither option could be considered for our perfect gift. We wanted something in between. Something decent, or at least restorable, but the make and model had to be exactly right. Zach and I weren’t about to give up.

Finally, on Dec 30, 2013, a Craigslist listing came up in a place called Yantis, TX. Ever heard of it? We hadn’t either. It’s 2.5 hours East... yes... East of Dallas, TX. And remember, all of us live 4,000 miles away on the Hawaiian island of Maui. We had no idea how to get the car back to paradise. We’ll solve that problem later. Undeterred, I immediately called the seller asking if the car was available. It was! W00T! From the description, if the car was anything close to what was advertised, this was exactly what we were looking for. The price was right, perhaps even a deal. Zach and I were seriously excited!

Next, Zach calls the seller to ask a bunch of questions to make 100% certain this was everything we wanted. It was. Our search was over. Well, sort of. 24 hours later, I call the seller prepared to pay the asking price, sight unseen. I said I'd fly out immediately to get if necessary. This was more than a little shocking to the seller. “Long drive,” he says to me after revealing where I live. The seller said he was a little uncomfortable allowing me to purchase the car sight unseen, especially since he doesn't know me and I'm so far away. He didn’t want me to be disappointed upon arrival and not buy it. Obviously, a really nice guy. He did say another interested party is coming to look at the car the following day, on New Year’s Day! 

Uh, oh. At this point in the conversation, I'm extremely worried. This person might buy the car that was in my mind already MY DAD’s CAR! Who knows when I might get another chance like this?! I tried everything I could to lock in the deal over the phone, but to no avail. The seller assured me it’s more likely the other guy is just looking and won't buy it, and if they don't, it's mine. I'm asked to phone the next day to get my answer. Talk about a stressful waiting period. All I can do now is hope for the best and prepare for an immediate flight out to Dallas in case things go well.

Now, I there are two problems to sort out and less than a day to do it. 1) I have to convince my dad to take a short notice trip with me to Texas without letting him know the exact reason why. 2) The seller requires either cash in hand or a cashier’s check from a local Texas bank. 

Fortunately, my dad is always up for an adventure. So, I said asked if he'd like to take a business trip with me to visit WhiteHat in both Santa Clara and Houston. He'd never really visited the company before to see what I built. It was a fortunate coincidence the car and a WhiteHat office was in Texas. He agreed.

Next, getting a sizable amount of cash over New Year’s Day, 4,000 miles away, when I bank at a local Hawaii bank that has no out of state branches, proved to be a far more significant challenge. Western Union and Money Orders would were of no use even if the locations were open at the time. Darn holidays! Their daily limits were too small for my current needs. FYI: The dollar amount here is less than an average new car off a lot, but still something you’d not want to carry around, let alone on a plane flight.

I call the seller the following day, he green lights the deal, and I’m overjoyed. I quickly buy some plane tickets, ouch on the short notice price, and let my dad know we're leaving in 5 hours. He was a little stunned I moved so quickly, and didn't think I was serious at first, but again... he's normally game for whatever. This time proved no different. 

Note: I didn’t have a way to solve the cash problem, so was I no choice but to figure it out upon arrival in Dallas. And, we're gone, just like that.

A 7-hour red-eye flight later and we're in Dallas at 6am on Jan 2. Oh, did I mention it was friggin’ cold — like 35F. I can tolerate frigid temperature OK, but Dad's lived for 30 years in Maui and will wear sweaters when it dips below 70F. While he’s always up for an adventure, life and death circumstance and all that not being a problem, it just better not be cold. I can tell he's having second thoughts about this trip right when we step outside of the terminal. He quickly puts on every piece of clothing he packed.

We grab a rental car, check-in to a hotel, get a bite to eat, and head to the bank -- "to open a new bank account just in case”… of a zombie apocalypse is my cover story. Turns out the best way to get cash in a hurry, given my constraints, is calling your source bank, asking them to raise the daily limit on your debit account to whatever you need, and having the destination bank perform a cash advance. This essentially looks a typical debit card transaction, but instead of a 50in TV, you get cash. The process took some doing and some waiting, but I got it done. Whew!

I call the seller, tell him I'm in town and ready to go. He's quite surprised because in the same 12-hour period I'm in Maui and then in Texas. Hey, I move quick. I ask for directions. By now it's about 2pm and time to head out for a 2.5 hour drive to get the car. My dad still has no idea what I’m up to. 

We start getting noticeably WAY out in the boonies, and we have no cell phone reception for miles. That’s when dad finally asks me, “Are we meeting someone out here for business?” I reply, “yes, we're meeting someone." 

Then it happened, not 60 seconds later, I see the car, sitting perfectly out in an open driveway. It's red, shiny, gorgeously chromey, and at 1/4 mile away, completely unmistakable. 

I slyly point the car out to dad, who doesn’t see it yet, and say inquisitively, "Hey, what's that car over there?" He squints and instantly says in a more than surprised, curious, and somehow measured tone, "That's... that's a 1951 Ford!" 

“Dad,” I say, "That's why we're here.” 

“What!?” He exclaims, even more confused now than before. 

“See that man coming out of the house over there, he’s expecting us. He's selling us that car today.

We're in the driveway now and dad gets out without a word, surveys the car at 10 feet, barely acknowledging the seller. Like a little boy again, he can’t take his eyes off the car. Clearly, it’s like it's a dream, and he can't believe he's actually seeing this car, his dream car, with his own eyes. Again, you never see these cars anywhere. He mutters, “Oh my God,” obviously overwhelmed. I introduce myself to the seller then stand back quietly to take photos of the moment while the seller introduces his gem. He tells us all about the car, it's history, and on and on like only true car aficionados can appreciate.

At long last, I ask the most important question. “Dad, do you want this car?” He's not quite sure how to answer, but clearly, it's a “yes.” I pay the man and then let dad know we have to either drive or trailer his ‘new’ car back to California for shipping. He opts for the former. Obvious to anyone sane, driving an untested 60 year old car 2,000 miles cross country, is ill advised. But whatever, this car was getting back to Maui. Nothing was going to stop that from happening now. My dad tells me he would have bought this car even if the engine was missing. 

We return the rental car and set out for a LONG drive back to the California’s SF Bay area to ship out the car to Maui. I don't think I drove the car for the first 1,000 miles of the journey. Hah! We had a handful of various close calls along the way, but overall nothing major. Hundreds of people waved at us along the way. Everyone from the motorcycle gangs to others in high-end BMW's. The car performed amazingly well by any standard. We've dropped it off for shipping and it took a few weeks to get to its new home in Maui, Hawaii. 

Remember, we’re all here on earth for just a tiny moment in time. Make it count. Take the time, MAKE the opportunity, and be open to spontaneous adventure with the people you love. No matter what happens, you’ll be happy that you did.

 

Evolution of The Press

Below is a working theory on the evolution of The Press in the United States as it relates to their relationship with the government and the people. I expect to continue refining the theory as new perspectives and competing ideas are discussed.

Phase 1) TL/DR; The press’s primary value in the system is transmitting a message from the government to the people. The press’s customers are their subscribers who purchase news. 

Consider the early days of United States of America throughout the late 1700s and 1800s. As elected officials governed and managed the business of a young country, operationally it was crucial they had a way to broadly communicate with their citizens. They needed to let the everyone know that there was a strong hand was on the tiller, that the people are safe, and they can sleep well at night.

Imagine government’s options to communicate across the country. Think about the technology that was available. How ideas and thoughts were recorded and how they were transmitted. There was no radio. There was no television. There certainly wasn’t an Internet. Ink and paper was the state of the art. While the government could physically write down their message, outside of standing at podiums surrounded by small local gatherings of people or leafletting, they did not have a scalable means of transmitting their message to the masses. So, the government and the country needed assistance. This need is where an entity called “The Press” established it’s value in the larger system — transmission of the government’s messages.

The press had journalists with the necessary tools to record the government’s message down on paper, who would perform some amount of fact checking, and then package the information as a cohesive and largely transcribed story. The press also had access to a new invention called the printing press enabling them to productize the message, such as a newspaper. And most importantly, the press created channels of distribution, such as horses automobiles, and the telephone to deliver the message to a variety of locations where it could be easily purchased. Put simply, the process was the press would be invited in by the government to document their message, print a large number of copies of newspapers, and then make the materials widely available to the people where the had the opportunity to buy it. 

This predominately was the value the press provided to the system — transmission. Of course it was important for the press to be mindful about what they printed, particularly the accuracy and relevancy of the message, otherwise people might stop paying for it in favor of another newspaper. The people depended upon the credibility of the press to tell the story right. Let’s not forget this. This dynamic between the government, the press, and the people carried through until about the 40s and 50s when the radio and television began changing the paradigm.

Phase 2) TL/DR; The press’s value proposition in split between transmitting the government’s message to the people to telling them how to think about the message. The press’s customers are their subscribers and advertisers.

Over time communications technology advanced and became far more affordable. Radio became common place in society and television sets started appearing in the average U.S. household in the early 1950s. With these modern tools the government could transmit their message directly to the people across the country and cut out the middleman — the press. The government no longer exclusively needed the press to get its message out to the masses. 

And since the government could bring their message directly to the people, and the country was in a more stable position, they didn’t necessarily have to always help people sleep at night. In fact, often the opposite was true. Causing some amount of fear actually helped the government further consolidate their power. As a result, the press needed to find a new way to provide value to the system, beyond just message transmission, in order to maintain their survival. 

During this period the press began shifting their value proposition from solely message transmission to telling people how to think about the government’s message. The press would take the governments message, create a compelling narrative to help people interpret the story, and transmit their product to the masses over the television and radio airwaves. As a product, this method of news packaging and delivery was attractive to people. There had become a significant increase of information to parse from a variety of sources, too much for any one individual to decide what was important to consume. The Ted Koppel’s and Tom Brokaw’s of the television news world became the credible sources of the press and filled a void left by the government to help the country sleep well at night.

There were a couple of problems the press needed overcome though. For example, it was not possible for the press to make money with electronically broadcast news in the same way they did with print media. It was not mechanically possible to charge viewers or listeners for news transmitted electronically. The press’s solution was sponsored advertising. News content accompanied by commercials. As such, the more people that watched and listened, and the longer they did so, the more valuable their advertising slots became. Another challenge the press needed to overcome with television and radio was that the physical time available to watch or listen to content was more limited. There is far more space to pack in far more content into the pages of a daily newspaper than what’s possible in a couple of hours of daily broadcast news spots.

Collectively, the new adversing-based business model and a limited amount of space for content changed how the press covered the government’s message in two profound ways. First, it shifted the priority for the transmission and accuracy of the message as their main value proposition in favor of whatever kept people watching and listening. And secondly, the press had to be more choosey with what message and narrative filled the available time and what didn’t. Furthermore, the press had to narrowly cater to a particular demographic of person with their content than what was originally necessary with print. In television and radio the more the news captures emotions and attention, the better the press does financially.

Fast forward several decades under these conditions and the people begin to clearly see a lot of bias in the press and an agenda. And while bias and agenda is certainly present, how could there NOT be, but in this context it’s best not to think the press is taking a principled stand. They’re not. Instead think of their bias and agenda as simply the press’s way of focusing their product at a particular customer like any business would. The press is drawing a circle around a suitable demographic for their product and value proposition, which again is to both transmit the government’s message and tell people how to think about in a way that helps to maximize ears and eyeballs. For example, there effectively isn’t a left-wing or right-wind press in a truly principled manner. The exact opposite is true. There are left-wing and right-wing people where the press tailor makes a narrative based on the government’s message that is compelling to them.

Phase 3) TL/DR; The press’s value proposition is telling people how to think about the government’s  message. The press’s customers are advertisers.

Enter the Internet in the early 1990s where transmission of information had become easy and inexpensive for everyone, and not just within the United States, but the entire modern world. The government no longer needed the press to transmit their message to the people at all. The government could transmit directly to the people or the people could go directly to the government. No middleman required. Without anyone needing the press for message transmission, as a business, print media fell off a cliff in under two decades. For survival sake, the press had to complete the transition away from transmission of the government’s message as a value proposition to nearly exclusively telling people how to think about it. That’s all of value they offer and in doing so message accuracy can be sacrificed whenever necessary. And of course the press’s content is heavily layered with advertisements.

As is turns out, the best way to attract more viewers for longer is to connect on a deep emotional level. Do whatever you can to rile up your viewers and they’ll continue coming back for more, even share the content forward to others in their social group, where even more ads can be lucratively served. Press outlets such as Fox News, CNN, MSNBC, and more all cross the political spectrum have strongly adopted this approach. The press outlets that didn’t adapt, died.  

As a product, these sources offer people a compelling and packaged way to validate their worldview — and THAT’s what keep the press ultimately credible and trustworthy in their minds. As evidence notice how the Ted Koppel’s and Tom Brokaw’s of the press have been replaced by Alex Jones, Bill O’Reilly, Keith Olbermann and Don Lemon’s. Is this change of their starting lineup designed to give viewers access to more accurate news or instead get people emotionality invested? Even when the press is demonstrably biased, factually incorrect, call it ‘Fake News’ if you like, it’s extremely difficult for people to suddenly distrust the press they decided to loyally watch for so long and find another compelling source. Perception becomes reality and exists long after the occasional and quietly posted retraction.

Phase 4) TL/DR; If via the Internet people once again adopt a direct paid-for news model, the press’s primary value become providing people with an individually relevant, timely, and accurate news source of the government’s message.

Going forward into the future, many feel there is a demand for relevant, timely, and accurate news sources. News that’s devoid of the influences of advertisements and paid directly by the people. Several press outlets have set-up paywalls and the business model is showing signs of success. All people have to do is register an account on a website or mobile application and supply a credit card online to become a subscriber. Another business model is micro-payments, where viewers pay for their content a la carte — by the article. A relatively new web browser named Brave, which includes ad blocking, offers native push button micro-payment functionality which supports participating content publishers. 

Here’s the thing: If any transition back to directly paid-for news truly starts gaining enough traction to threaten to the ad-based model, fierce resistance by the advertising industry is sure to follow. Google and Facebook, which dominate the online advertising industry, who along side many others who make all their billions annually off ‘free’ content, will do everything they can to prevent the transition. Their livelihoods depend on it. Regardless, if it so happens that the paid-for model once again takes hold, many positive externalities may also come with it. Fake news goes away. Click-bait headlines go away. Online spam goes away. Privacy invading ads go away. All of these shady practices found on the Internet depend wholly on advertisements to function. The adoption of ad blockers, which now stands over 20% marketshare, indicates that people are making a choice, even if they aren’t yet paying for their content. Broad access to new technology is once again causing a shift in the press and how the government communicates it’s message.

The evolutionary waves of the penetration-testing / vulnerability assessment market

Over the last two decades the penetration-testing / vulnerability assessment market went through a series of evolutionary waves that went like this…

1st Wave: “You think we have vulnerabilities and want to hire an employee to find them? You’re out of your mind!"

The business got over it and InfoSec people were hired for the job.

2nd Wave: "You want us to contract with someone outside the company, a consultant, to come onsite and test our security? You’re out of your mind!"

The business got over it and consultant pen-testing took over.

3rd Wave: "You want us to hire a third-party company, a scanning service, to test our security and store the vulnerabilities off-site? You’re out of your mind!’

The business got over it and SaaS-based vulnerability assessments took over.

4th Wave: "You want us to allow anyone in the world to test our security, tell us about our vulnerabilities, and then reward them with money? You’re out of your mind!"

Businesses are getting over it and the crowd-sourcing model is taking over.

The evolution reminds us of how the market for ‘driving’ and ‘drivers’ changed over the last century. People first drove their own cars around, then many hired personal drivers, then came along cars-for-hire services (cabs / limos) with ‘professional’ drivers that you didn’t personally know, and now to Uber/Lyft where you basically jump into some complete stranger’s car. Soon, we’ll jump into self-drivers cars without a second thought.

As we see, each new wave doesn't necessarily replace the last -- it's additive. Provided there is an economically superior ROI and value proposition, people also typically get over their fears of the unknown and will adopt something new and better. It just takes time.

All these vulnerabilities, rarely matter.

There is a serious misalignment of interests between Application Security vulnerability assessment vendors and their customers. Vendors are incentivized to report everything they possible can, even issues that rarely matter. On the other hand, customers just want the vulnerability reports that are likely to get them hacked. Every finding beyond that is a waste of time, money, and energy, which is precisely what’s happening every day. Let’s begin exploring this with some context:

Within any Application Security vulnerability statistics report published over the last 10 years, they’ll state that the vast majority of websites contain one or more serious issues — typically dozens. To be clear, we’re NOT talking about website infected with malvertizements or network based vulnerabilities that can trivially found via Shodan and the like. Those are separate problems. I’m talking exclusively about Web application vulnerabilities such as SQL Injection, Cross-Site Scripting, Cross-Site Request Forgery, and several dozen more classes. The data shows only half of those reported vulnerabilities ever get fixed and doing so take many months. Pair this with Netcraft’s data that states there’s over 1.7B sites on the Web. Simple multiplication tells us that’s A LOT of vulnerabilities in the ecosystem laying exposed. 

The most interesting and unexplored question to me these days is NOT the sheer size of the vulnerability problem, or why so many issue remain unresolved, but instead figuring out why all those ‘serious’ website vulnerabilities are NOT exploited. Don’t get me wrong, a lot of websites certainly do get exploited, perhaps on the order of millions per year, but it’s certainly not in the realm of tens or even hundreds of millions like the data suggests it could be. And the fact is, for some reason, the vast majority of plainly vulnerable websites with these exact issues remain unexploited for years upon years. 

Some possible theories as to why are:

1. These ‘vulnerabilities’ are not really vulnerabilities in the directly exploitable sense.
2. The vulnerabilities are too difficult for the majority of attackers to find and exploit.
3. The vulnerabilities are only exploitable by insiders.
4. There aren’t enough attackers to exploit all or even most of the vulnerabilities.
5. There are more attractive targets or exploit vectors for attackers to focus on.

Other plausible theories?

As someone who worked in the Application Security vulnerability assessment vendor for 15+ years, here is something to consider that speaks to theory #1 and #2 above. 

During the typical sales process, ‘free’ competitive bakeoffs with multiple vendors is standard practice. 9 out of 10 times, the vendor who produces the best results in terms of high-severity vulnerabilities with low false-positives will win the deal. As such, every vendor is heavily incentivized to identify as many vulnerabilities as they can to demonstrate their skill and overall value. Predictively then, every little issue will be reported, from the most basic information disclosure issues to the extremely esoteric and difficult to exploit. No vendor wants to be the one who missed or didn’t report something that another vendor did and risk losing a deal. More is always better. As further evidence, ask any customer about the size and fluff of their assessment reports.

Understanding this, the top vulnerability assessment vendors invest millions upon millions of dollars each year in R&D to improve their scanning technology and assessment methodology to uncover every possible issue. And it makes sense because this is primarily how vendors win deals and grow their business.

Before going further, let’s briefly discuss the reason why we do vulnerability assessments in the first place. When it comes to Dynamic Application Security Testing (DAST), specifically testing in production, the whole point is to find and fix vulnerabilities BEFORE an attacker will find and exploit them. It’s just that simple. And technically, it just takes the exploitation of one vulnerability for the attacker to succeed.

Here’s the thing: if attackers really aren’t finding, exploiting, or even caring about these vulnerabilities as we can infer from the supplied data — the value in discovering them in the first place becomes questionable. The application security industry industry is heavily incentivized to find vulnerabilities that for one reason or another have little chance of actual exploitation. If that’s the case, then all those vulnerabilities that DAST is finding rarely matter much and we’re collectively wasting precious time and resources focusing on them. 

Let’s tackle Static Application Security Testing (SAST) next. 

The primary purpose of SAST is to find vulnerabilities during the software development process BEFORE they land in production where they’ll eventually be found by DAST and/or exploited by attackers. With this in mind, we must then ask what the overlap is between vulnerabilities found by SAST and DAST. If you ask someone who is an expert in both SAST and DAST, specifically those with experience in this area of vulnerability correlation, they’ll tell you the overlap is around 5-15%. Let’s state that more clearly, somewhere between 5-15% of the vulnerabilities reported by SAST are found by DAST. And let’s remember, from an I-dont-want-to-be-hacked perspective, DAST or attacker-found vulnerabilities are really the only vulnerabilities that matter. Conceptually, SAST helps find them those issues earlier. But, does it really? I challenge anyone, particularly the vendors, to show actual broad field evidence.

Anyway, what then are all those OTHER vulnerabilities that SAST is finding, which DAST / attackers are not?  Obviously, it’ll be some combination of theories #1 - #3 above. They’re not really vulnerabilities, they’re too difficult to remotely find/exploit, or attackers don’t care about them. In either case, what’s the real value for the other 85-95% of vulnerabilities reported by SAST? A: Not much. If you want to know why so many reported 'vulnerabilities' aren’t fixed, this is your long-winded answer. 

This is also why cyber-insurance firms feel comfortable writing policies all day long, even if they know full well their clients are technically riddled with vulnerabilities, because statistically they know those issues are unlikely to be exploited or lead to claims. That last part is key — claims. Exploitation of a vulnerability does not automatically result in a ‘breach,’ which does not necessarily equate to a ‘material business loss,’ and loss is the only thing the business or their insurance carrier truly cares about. Many breaches do not result is losses. This is an crucial point that many InfoSec pros are unable to distinguish between — breach and loss. They are NOT the same thing.

So far we’ve discussed the misalignment of interests between Application Security vulnerability assessment vendors and their customers. The net-result of which is that that we’re wasting huge amounts of time, money, and energy finding and fixing vulnerabilities that rarely matter. If so, the first thing we need to do is come up with a better way to prioritize and justify remediation, or not, of the vulnerabilities we already know exist and should care about. Secondly, we must more efficiently invest our resources in the application security testing process. 

We’ll begin with the simplest risk formula: probability (of breach) x loss (expected) = risk.

Let’s make up some completely bogus numbers to fill in the variables. In a given website we know there’s a vanilla SQL Injection vulnerability in a non-authenticated portion of the application, which has a 50% likelihood of being exploited over a year period. If exploitation results in a material breach, the expected loss is $1,000,000 for incident handling and clean up. Applying our formula:

$1,000,000 (expected loss) x 0.5 (probability of breach) = $500,000 (risk)

In which case, in can be argued that if the SQL injection vulnerability in question costs less than $500,000 to fix, then that’s the reasonable choice. And, the sooner the better. If remediation costs more than $500,000, and I can’t imagine why, then leave it as is. The lesson is that the less a vulnerability costs to fix the more sense it makes to do so. Next, let’s change the variables to the other extreme. We’ll cut the expected loss figure in half and reduce the likelihood of breach to 1% over a year.

$500,000 (expected loss) x 0.01 (probability of breach) = $5,000 (risk)

Now, if vulnerability remediation of the SQL Injection vulnerability costs less than $5,000, it makes sense to fix it. If more, or far more, then one could argue it makes business sense not to. This is the kind of decision that makes the vast majority of information security professionals extremely uncomfortable and instead why they like to ask the business to, “accept the risk.” This way their hands are clean, don’t have to expose their inability to do risk management, and can safely pull an, “I told you so,” should an incident occur. Stating plainly, if your position is recommending that the business should fix each and every vulnerability immediately regardless of the cost, then you’re really not on the side of the business and you will continue being ignored.

What’s needed to enable better decision-making, specifically how to decide what known vulnerabilities to fix or not to fix, is a purpose-built risk matrix specifically for application security. A matrix that takes each vulnerability class, assigns a likelihood of actual exploitation using whatever available data, and containing an expected loss range. Where things will get far more complicated is that the matrix should take into account the authentication status of the vulnerability, any mitigating controls, the industry, resident data volume and type, insider vs external threat actor, a few other things to improve accuracy. 

While never perfect, as risk modeling never is, I’m certain we could begin with something incredibly simple that would far outperform our the way we currently do things — HIGH, MEDIUM, LOW (BLEH!). When it comes to vulnerability remediation, how exactly is a business supposed to make good informed decisions about remediation using traffic light signals? As we’ve seen, and as all previous data indicates, they don’t. Everyone just guesses and 50% of issues go unfixed.

InfoSec's version of the traffic light: This light is green, because in most places where we put this light it makes sense to be green, but we're not taking into account anything about the current street’s situation, location or traffic patterns. Should you trust that light has your best interest at heart?  No.  Should you obey it anyway?  Yes. Because once you install something like that you end up having to follow it, no matter how stupid it is.

Assuming for a moment the aforementioned matrix is created, all of a sudden it fuels the solution to the lack of efficiency in the application security testing process. Since we’ll know exactly what types of vulnerabilities we care about in terms of actual business risk and financial loss, investment can be prioritized to only look for those and ignore all the other worthless junk. Those bulky vulnerability assessment reports would likely dramatically decrease in size and increase in value.

If we really want to push forward our collective understanding of application security and increase the value of our work, we need to completely change the way we think. We need to connect pools of data. Yes, we need to know what vulnerabilities websites currently have — that matter. We need to know what vulnerabilities various application security testing methodologies actually test for. Then we need to overlap this data set with what vulnerabilities attackers predominately find and exploit. And finally, within that data set, which exploited vulnerabilities lead to the largest dollar losses.

If we can successfully do that, we’ll increase the remediation rates of the truly important vulnerabilities, decrease breaches AND losses, and more efficiently invest our vulnerability assessment dollars. Or, we can leave the status quo for the next 10 years and have the same conversations in 2028. We have work to do and a choice to make.