Home �   ITIL�  Index

There is No Evidence for ITIL

Feb 20, 2007

The IT Skeptic

I expect this last statement leaves this research with zero credibility with anyone who understands ITIL and ITSM.

No cost/benefit analysis. Not a single valid objective metric. Sure if you throw enough government money at anything and launch an aggressive enough PR campaign you can make the users happier. That proves nothing. And the fact that calls to the service desk went down over an initial nine-month period would to me be a cause for concern not celebration.

But BE WARNED: this paper will be quoted all over the place as evidence of the effectiveness of ITIL.

Pink Elephant have finally extended the number of anecdotal stories beyond the tired old Proctor and Gamble, Caterpillar and the internationally famous Ontario Justice Enterprise.

They now have a few more unsubstantiated statistics, this time for PEMCO, Zurich Life, Hospital Corporation of America, Nationwide Insurance, Ontario Ministry of Transportation (what is it with Ontario?), and Capital One. I’ve been in vendorland and I have generated this kind of case study. These amount to no more than selective quotes from managers justifying their decision after the fact.

HP is one vendor putting numbers where their hype is, though this pertains to a service desk product not ITIL:

"IDC found that, for the companies surveyed, IT productivity increased by an average of 14% … contributing an average cost savings over three years of almost $4.2 million annually. When normalized for company size, these savings amounted to $17,235 per 100 users … Based on these savings, the three-year hard ROI … averaged 130%, giving an average payback period of 13.5 months."

University researchers are you listening? Those are results. That’s what we want to see. Unfortunately, paid analysts doing surveys for a vendor are about the least useful sort of research. It is not subject to the same transparency, peer review or statistical rigour as academic research, and the results tend to be selectively quoted by the vendor.

But since we’ve started, here is another analyst survey: “Did you make a business case before decision? (Based on 62 European firms): No - 68%”

Fully two thirds had no business case! But wait. There’s more:

“Did you observe the expected ROI? (Based on 20 European firms). No - 50%; Don’t know - 30%; Yes - 20%.”

If less than a third built a business case, one would guess the ones that did build one represented a sample biased towards those who had a good case, and yet only one fifth of them achieved the expected ROI.

This is best business practice? I think I need to go lie down.

Before I do though, here is more. If you are willing to make major decisions based on informal research by vendors (as everyone adopting ITIL does), ponder this:

In a survey carried out by Noel Bruton of 400 sites, about half of the 125 organizations which were found to have adopted ITIL made no measured improvement in terms of their service performance or the rate at which they were able to close helpdesk calls.