ITIL Foundation Exams Don't Test Understanding

What is the ITIL v3 Foundation certification for? To assure us that the candidate knows the fundamentals of IT Service Management (ITSM)? Or at least that they understand the fundamental principles of ITIL (a narrower field)? Or does it tell us that they can memorise a hundred bullet points and decode arcane logic puzzles? This last one is closest and this damages ITIL.

I recently sat the ITIL v3 Foundation exam. Despite expounding about ITIL for years and building a pretty good following on my websitehttp://www.itskeptic.org/, I haven’t certified in ITIL since my v2 Foundation exam so many years ago. I argued that I would certify when the lack of it prevented me getting work. Well, it is 2009 and times are tough and off I went to Prometric to sit v3 Foundation. (I can’t imagine any recession persuading me to invest what ITIL v3 Expert demands, but that is another article).

Before sitting the exam, I ran through a number of practice exams available on the Web (personally I liked ITIL Campus and Taruu). I’ve been critical of the concept of multi-choice testing all along, and we’ve pulled apart a few examples on my blog, but as I was doing question after question on the practice exams (and the final real exam) it struck me how bad most of them were.

The Foundation exams should test an understanding of the fundamentals of ITSM, and a grasp of the key principles of ITIL. If someone has passed Foundation that should tell us they understand IT is all about delivery of services; that a catalogue defines what we do in users’ terms; that modifications to everything that impacts a service should go through Change; that an Incident is distinct from a Problem; that if what our suppliers promise doesn’t line up with we promise the business, we are screwed, etc., etc.

Who gives a rodent’s posterior exactly how many steps there are in the ITIL CSI process? (Especially since that process is different from every best practice CSI process already in existence―that too is another article), or what the “seven Rs” of Change are for? More precisely, why do candidates need to know these things by heart? These are not things that trainees should have absorbed as learned principles, they are things they can look up if they need to. Many of the questions test whether the candidate has memorised the sacred books rather than whether they have a useful knowledge of ITSM/ITIL.

In my satirical book Introduction to Real ITSM, I made fun of these exams with a little exam of my own. I’ll reproduce a couple of the Real ITSM questions as we go along to illustrate my points. We can start with:

1) On page 49 of Real ITSM what is the first word on line 10 (counting page headers as lines as well) a) the b) and c) This d) By

Testing Multiple Choice Skills

The second group of ITIL Foundation questions that really tick me off are the ones that test the candidate’s logic skills.

The satirical Real ITSM question that spoofed the logic tests was:

2) Which of the following is not the opposite of not being unresponding to a user’s failure to call the Service Desk?

a) not calling the user

b) not failing to call the user

c) not calling not the user

d) not the opposite of not calling someone other than the user

If that seems over the top, consider this question (reportedly from the official “mock” exams but we can’t say for sure as the mock exams to the best of my knowledge are not published – True!):

Which of the following are not included in Access Management?

1) Verifying the identity of users requesting access to services

2) Setting the rights or privileges of systems to allow access to authorised users

3) Defining security policies for system access

4) Monitoring the availability of systems that users should have access to

a) 2 and 4 only b) 1 and 4 only c) 1 and 3 only d) 2 and 3 only

Reportedly one of the live exam questions looks something like this:

Order the following CSI implementation steps in the correct order of the PDCA (Plan-Do-Check-Act) model:

1) Assign roles and responsibilities to work on CSI activities

2) Measure and review CSI plan objectives are being met

3) Identify CSI requirements, objectives and scope

4) Implement CSI enhancement

a) 1-3-2-4 b) 2-3-4-1 c) 3-1-2-4 d) 3-4-2-1