The Kirkpatrick Model Of Training Evaluation And Learning Analytics

The Kirkpatrick Model Of Training Evaluation And Learning Analytics
Peshkova/Shutterstock.com
Summary: A thorough evaluation of your training program is a pre-requisite for getting started with learning analytics. Mastering the Kirkpatrick Model can be your best plan for it. Read 5 simple steps to get started.

Implementing Learning Analytics With The Kirkpatrick Model

Corporate training isn’t just about curiosity and learning, it needs to be utilitarian and impactful. With businesses pouring millions of dollars every year into training, it’s no wonder they need real and quantified business results and Return On Investment (ROI). This is where learning analytics enters the L&D arena.

eBook Release: Leveraging Learning Analytics To Maximize Training Effectiveness - Practical Insights And Ideas
eBook Release
Leveraging Learning Analytics To Maximize Training Effectiveness - Practical Insights And Ideas
Practical ideas for L&D practitioners to get started with learning analytics.

When it comes to implementing learning analytics, the Kirkpatrick Model gives a simple yet powerful way to measure the effectiveness of training. Ever since its inception, it has been widely used by L&D professionals. Here is an overview of the 4 levels of the model.

Understanding The 4 Levels Of The Kirkpatrick Model For Training Evaluation

1. Reaction

It is the assessment of the initial reaction of learners to the course; reactions to its relevance, training methods used, and delivery. It is NOT a measurement of learning.

2. Learning

This level measures whether learners acquired the knowledge and skills taught in the course. Methods for assessing learning may be knowledge or performance-based.

3. Behavior

It measures the extent to which the acquired knowledge and skills are transferred to the job.

4. Result

The value of learning depends on the quantifiable impact it has on the organization. Results measure this impact.

Pushing A Boulder Uphill: The Challenge For Evaluation

Ask any L&D professional about training evaluation and they will most certainly talk about having the Kirkpatrick Model in place. But when you look carefully, typically only the first 2 levels are implemented. Even with its ubiquitous presence for many years now, why do most organizations still struggle with mastering a comprehensive evaluation program? These are a few of the most significant challenges.

There is a lot of “friction” when it comes to implementing the evaluation. Stakeholders are often more concerned about finishing up a training program and getting employees back on the job as soon as possible (and less concerned about detailed evaluation). Moreover, generally there is a level of trust with L&D to be effective and many stakeholders only want to know if employees passed or failed.

In general, Levels 1 and 2 are integrated with the training and under the direct control of the L&D function. Levels 3 and 4 call for a much higher level of involvement by stakeholders and the organization, thus making the process more challenging. Even with Levels 1 and 2, the appropriateness of the evaluation/assessment questions and extent to which the information gathered is used varies widely.

Many Level 2 evaluations “drift away” from the learning objectives and do not appropriately measure the achievement of those objectives. For example, using a multiple-choice question to measure if a participant can log in to a software program. Often, these evaluations are based on a passing threshold, and more meaningful and useful learning analytics are NOT enabled.

There often is a lack of available evaluation expertise (such as evaluation strategy, design, methodology, analysis, and reporting) plus limited resources, time and budget.

Getting Ahead Of The Odds: The Solution For Evaluation

There are 5 simple steps to make things clear.

1. Have The Talk: The First Step To Effective Evaluation

The first step to building successful training evaluations and implementing learning analytics is to have a comprehensive and clear conversation with all stakeholders. Educate them on how implementing learning analytics properly can enhance the impact of training, which in turn can translate to an increase in ROI. The goal is to collaborate and establish a strategy that outlines what learning analytics will be implemented.

2. Your Data Is Only As Good As The Questions You Ask!

After convincing all stakeholders about the value of learning analytics, devise a plan on how to go about implementing it. It is necessary to have all stakeholders participate and agree on:

  • How each level of the Kirkpatrick Model is going to be addressed; specifically, what questions should be asked and what decisions need to be made
  • How the evaluation strategy and learning analytics should unfold
  • From where the information will be gathered (sources)
  • How the information will be gathered (methods), including how the desired learning analytics will be enabled
  • Requirements for analyzing data
  • Requirements for how the findings and recommendations will be reported

3. Align Your Evaluation Instruments With The Model

Each level of the Kirkpatrick Model evaluates a particular stage in the learners’ journey. Each level has its own objective, role in the evaluation process, and needs proper instruments in place to be successful. Create templates for each instrument that can be adapted to each training solution.

Level/Role Reaction Learning Behavior Results
Feedback about learners’ experience Assess the difference between pre and post-training knowledge and skills Information on "success" on the job Most persuasive information for management
Sources/ Methods
  • Feedback surveys
  • Smiley sheets
  • Point rating scales
  • Pre and post
  • assessments
  • Focus groups
  • Mentoring
  • Feedback from supervisors
  • Peer comments
Review finances before and after training

4. ‘Capture, Analyze, Report’ To Leverage The Benefits Of Your LMS

Learning Management Systems (LMS) are meant to be the data tracking and reporting hub for eLearning courses. While most LMS do provide these facilities, managers do not really pay attention to most data other than completion, qualifying passing rates.

For example, if you have a training solution that aims to teach a 10-step procedure and your only focus is knowing if and when learners have completed the course, it doesn’t do much to help determine the quality and effectiveness of training. You need to leverage maximum benefits out of your LMS and enable deeper learning analytics.

You can adopt a Learning Record System (LRS) within your LMS which enables even nuanced tracking and recording of learning activities—online and offline. Having an xAPI-enabled LMS takes assessing learning experiences a step further.

5. Analytics Serve No Purpose Without Implementation

Reports are colorful, but only useless pie charts and graphs if you just set them aside after a glance. Implementing ways to collect evaluation metrics doesn’t make sense if they are not going to be put to use. You can update stakeholders about the reports and come up with strategies to implement changes based on them. It’s really about having good data to inform decisions. You should also periodically update your evaluation strategy and continuously improve your evaluation process.

The most important part of evaluation or analytics implementation is having resources, time and budget in place. To be honest, you will probably meet with a lot of challenges at every phase of the process of evaluation and implementation. But rest assured, successfully implementing learning analytics will transform your employee training program for good. So, why not walk the extra length if it ends up being the key to improving your business impact. And for insights on the practical aspects of learning analytics, including a collaborative 5-step process for implementation, register for this live webinar. Discover practical ideas to get started with learning analytics by downloading the eBook Leveraging Learning Analytics To Maximize Training Effectiveness - Practical Insights And Ideas.