Level 4 Training Evaluation

Level 4 Training Evaluation

Level 4 Training Evaluation

Level 4 Training Evaluation - The key to measuring training value

Deceptively impossible

Achieving "real" results is the ultimate objective of any company's training initiatives. Organizations typically won't invest in a training program unless management is convinced that the proposed training will lead to specific benefits. The challenge that most corporate trainers face however is: How does one measure "real" organizational impact as a result of training? And the answer is: Through a Kirkpatrick Level 4 Evaluation. And when Level 4 (Results) can be substantiated, proponents of the training program can unequivocally proclaim success!Kirkpatrick’s four levels of evaluation strive to offer a framework for the meaningful evaluation of learning in an organization. Envisioned by Donald L. Kirkpatrick in 1959 (and further refined in 1998 in the publication titled "Evaluating Training Programs: The Four Levels."), the four levels of the Kirkpatrick model include:

These levels comprise of progressively difficult metrics against which success is evaluated, with Level 4: Results being the most difficult of them all. It is at this level (4) that evaluators measure the final impact that training has had on the organization.Because of its perceived difficulty, many training-centric initiatives don't strive to complete a Level 4 evaluation. In reality however, doing a Level 4 evaluation isn't that difficult, especially given that the impact of the training initiative will have been deconstructed significantly as the team progresses through the previous 3 levels. Level 4 evaluations will have significant number data points to work with as a result.Starting at the endThe best way to achieve success is to envision what success looks like. The same holds true with training programs. And when a company's trainers start at Level 4, which initially defines what training results should look like once the program has been delivered, closing the loop (once training has been delivered) on the review process with Level 4 evaluation becomes much easier.For instance, some Level 4 results that a company may look forward to achieving might be:

Having such Level 4 outcomes defined in advance then gives training evaluators a solid basis for working backward to quantify what Level 3 metrics are required to be achieved in order to hit the Level 4 evaluation criteria. This process continues through Level 2 and Level 1. In every case however, it is imperative to ensure that the proposed metrics are observable and measurable. For instance, a "poor" set of Level 4 evaluation metric might be:

By starting at the end, and defining clear and measurable Level 4 evaluation criteria, training proponents can clearly demonstrate to company management and training sponsors the precise value that a training program can bring to the organization. And it is that kind of precision that senior management like to see before approving a program.

Making it happen

A Level 4 evaluation of training results requires a certain framework in order to be credible to company management. Merely publishing test scores of the trainees won't cut it. Management needs to see demonstrable change before they acknowledge that a training exercise was a success. Here are some broad guidelines to consider when setting up a Level 4 evaluation:

Using these broad principles, trainers can put together a very effective framework to measure Level 4 evaluation outcomes.

Fingers on the pulse

Sports people need to continually practice their sport and be monitored by their coach, or they risk growing "stale". Similarly, trainees from a learning program must continually demonstrate their training has taken roots by exhibiting sustained pre-determined behavioral changes in the workforce or community.  Level 4 evaluations must therefore be designed to monitor those behavioral changes over the long-term. Some ways to accomplish this include:

Using these tools is a way of keeping a finger on the pulse of the trainees in order to determine whether, over a longer term, the training they received is being practiced as envisioned.

Delivering Proof and Managing Expectations

Financial professionals often tend to ask for proof of success in terms of Return on Equity (ROE), while learning professionals focus on Return on Expectations (ROE). While the Financial ROE might be straightforward to calculate (e.g. "Eliminated $50,000 worth of scrap each shift due to training"), it may be more challenging to quantify a Return on Expectation.Therefore, to deliver proof that expectations of the training program have been met, it is imperative that designers and evaluators of training programs know, in as much granularity as possible, precisely what those expectations are.  To accomplish that objective, learning professionals must:

It is extremely difficult for a training program in and of itself to meet the expectations set out by stakeholders. Many outside (of the training environment) factors also play a role in determining whether training was successful. Therefore, learning professionals should manage those expectations by encouraging stakeholders to exert influence over other favorable factors (outside of the training environment) which will impact how a trainee delivers to those expectations. It is that kind of partnership between all stakeholders that will ultimately ensure that Returns on Expectations are met.

The Final Test

The best way to determine whether Financial ROE or "expectation" ROE has been met is to ask those impacted by the training - including the trainees themselves. To that goal, learning professionals, in consultation with other key stakeholders, should ask appropriate questions and analyze the feedback received. Questions to ask include:

While such questions might be used as a way to receive significant insight into the question: "Did training deliver the results it was supposed to?", if structured strategically, they can also shed light on other elements of the training program, such as the quality of inclusiveness (Stakeholder engagement, Partnerships with other groups etc.) during the entire training initiative.

Exit mobile version