Training Evaluation: The Phillips ROI Model
Natali_ Mis/Shutterstock.com

Learn Why You Should Include The Phillips ROI Model In Your Training Evaluation

A persistent challenge of any Learning and Development professional is to demonstrate the effectiveness and overall worth of their training programs. Gone are the days where it’s sufficient to say that people “liked” a program, so therefore it must have been effective. More than ever, executives across all industries demand hard metrics to justify the investment into time-consuming and often costly training programs. The best-known model for analyzing the results of a training program is the Kirkpatrick model. Developed by Donald Kirkpatrick in the 1950s, this model takes into account the engagement of the learners as well as the longitudinal impact on job performance. For this reason, it has been the go-to evaluation model since its advent almost 70 years ago. This model begins with satisfaction and if planned and executed upon, an L&D professional can ultimately measure its impact at the organizational level.

The Kirkpatrick model consists of 4 phases starting with reaction and ending with business impact.

 1. Reaction

The initial phase of the Kirkpatrick model is standard for most training programs. It is a reaction assessment where the L&D professional gauges overall learner sentiment and satisfaction of the program. In addition, it also queries if the learner thought the program was useful to their job. It is typically referred to as a “smile sheet” evaluation. Examples for assessing level 1 include post-training questionnaires to assess satisfaction and participant interviews where feasible.

2. Learning

Evaluation at this stage is meant to determine if the learners were able to learn the knowledge or skills that the training program set out to accomplish. Compared to level 1, the second level of the model involves more planning and preparation. A designer should consider how this is going to be accomplished during the design phase of program development. Methods for evaluating level 2 include using a control group to compare differences, observations from peers/instructors, as well as pre- and post exams.

3. Behavior

In most cases, we should strive for training programs to, at a minimum, reach this stage of evaluation. This is the stage where we can truly evaluate if the training provided a behavioral change back at the learners' job site. Evaluation at this stage presents inherent challenges as it can only take place  (on average) no sooner than one month post training. Measuring behavior change must be planned for in advance of the training and should ultimately be the “why” for the program. Methods for collecting level 3 evaluation include post-training interviews with the learner’s manager and on-the-job observations. More than any level of evaluation, level 3 demands complete coordination and sponsorship of stakeholders and direct managers.

4. Business Impact

Level 4 examines the overall organizational change that can be attributed to the training program. As opposed to level 3, which only takes into account the individual learner, level 4 looks at learners as a whole and evaluates how the training impacted business key performance metrics. A level 4 evaluation assesses program effectiveness based on quality, cost and time. It is often considered the pinnacle of training evaluation.

The Phillips ROI Model

In the early 2000s, Jack Philips came out with an accessory to the Kirkpatrick model to include a Return On Investment. This level, commonly referred to as level 5 evaluation, compares the program's overall cost to its monetary benefits. This is presented as a cost/benefit ratio. A key component to evaluating at this level is isolating the training benefits and eliminating any non-training factors that may have contributed to the organizational impact. When applying evaluation at this level, a learning professional should eliminate such things as competitive environment and seasonal effects that temporarily contribute to business improvements.

Calculating The Return On Investment

ROI = (Total Program Benefits – Total Program Costs ROI/ Total Program Costs) x 100%

Here’s an example:

A safety training program delivered to 50 people resulted in a decrease of 20 accidents over the course of the year. This directly increases profits of the organization of $100,000 per year. The total cost of the program was $50,000.

The benefit-cost ratio (BCR) is: $100,000/$50,000 = 2. In other words, for every $1.00 spent on the training program, $2 is returned.

The ROI of the program is: ($100,000 -$50,000/$90,000) x 100% -----> (50,000/90,000) x 100% = 55%

This means that every dollar spent on the training program is returned and an additional $0.55 is returned as profit.

The ROI model is not meant for every training program. It is sufficient for basic check-the-box training to only reach level 1. However, if you want to show how your programs are truly helping the company save money and tangibly increase profits, the Philips model is the solution. There is no question that it takes practice and the biggest challenge is to eliminate non-training factors. However, continued focused attempts to show ROI will only improve over time.

Close