Monday, January 12, 2009

Product reviews are dead, not so much pen-testing

Have to hand it to Brian Chess for stirring up a hornets nest about the death of pen-testing in 2009. First, don’t worry. Pen-testing is remarkably resilient, capable of adapting, and quick to find greener pastures. This is what I believe Brian was trying to get at. In its current form of late stage one-offs, he’s saying pen-testing is dead, and I’m not inclined to disagree. On the whole though no one in their right mind will predict that the pen-testing market will end, especially since annual revenue is increasing year over year. I’ll tell you what is dead though, something that could have significant impact, product reviews.

With the decline of traditional media in print and online channels, cutbacks have taken a heavy toll on technically oriented product reviews. To generate revenue, in depth product reviews been replaced by paid-for 5-star advertising focusing largely on marketing talking points supported by precious little need to know information. Head-to-head shootouts, forget about it. This is troubling because product reviews are an invaluable resources for helping prospective buyers make educated decisions without spending inordinate amount of time comparing a long list of products themselves.

As an end-user this means you now must rely more on customer referrals, analyst reports, and general online chatter. Ideally you’d want to focus of on taking advice from organization of similar size and marketing vertical solving a similar problem.


Jordan said...

When Network Computing was rolled into InformationWeek and renamed IWeek Labs, the plan was to keep doing the same level of testing, but it really is a natural reaction to the shrinking budgets to see less testing going on.

Network Computing just didn't have the same profitability as other magazines in terms of dollars spent per eyeball fed. Of course, the argument could be made that those eyeballs were more likely to be in decision makers and worth extra testing budget (and hopefully likewise advertising budget).

It is sad to see though, to be sure.

Of course, my own lack of testing comes primarily from laziness.

I've been enjoying taking a nice break from writing altogether. You should see the email pile that's built up in my magazine inbox. I try to keep it somewhat tamed, but that doesn't last very long.

Jeremiah Grossman said...

Heya Jordan, you know... I think your review series was the only one for all 2008 in webappsec. Am I wrong? Where are all the others!?

Oh well, I guess the industry will just have to settle for awards. :)

Anonymous said...

I wholeheartedly agree that venues for product review are drying up, and as frustrating as that is for a consumer looking for advice, it is at least as troublesome to small, established software companies (like ours) trying to get noticed.

We have some killer security and performance tools that sometimes get lost in a larger marketplace lacking clear signposts. We're awaiting the moment a well-known figure picks up the mantle and fills the void with honest, objective reviews of new tools (hint, hint)...

Really enjoy your blog, thanks for the great info.

Jeremiah Grossman said...

I appreciate the vote of confidence, and as much as I'd like to review software tools, I am more than biased in that regard. Still, there really should be an outlet. Talk about an unfilled niche.