Mobile Training And How You Can Measure Its Effectiveness
By providing your employees with mobile training, you are delivering eLearning to your employees seamlessly using the devices they are most comfortable with. Simply having mobile training, however, is not enough to keep your company at the forefront of training and development. It’s important to understand how effective your training is, and where to make tweaks to areas that need improvement. In this article, we are going to discuss ways that you can analyze your mobile training’s effectiveness using the Kirkpatrick’s Evaluation Model. The Kirkpatrick Evaluation Model was developed in the 1950s by University of Wisconsin Professor Donald Kirkpatrick. It contains 4 levels that help to measure the effectiveness of training. These 4 levels include reaction, learning, behavior, and results. Below are the 4 levels and some ideas you can use to measure the effectiveness of your training.
Level 1: Reaction
The first level, reaction, measures how learners reacted to the training. To measure this level:
Gather Learner Perception
Poll your audience about the training to gain an understanding of how well it was received. To what degree of effectiveness do they rate the training? Ask questions about their experience, such as “what did you like about your experience?”. Do you see a positive connection with your employees surrounding training, or do you think that they view training as just another obligation? Your employees are busy, and will not appreciate taking time for something they don’t find value in.
Level 2: Learning
The second level, learning, measures the knowledge and skills your audience has learned. You can measure this level by using these strategies:
Assess Online Forum Contribution
One way to assess your mobile audience is through the use of online forums. While you miss the opportunity to gauge learner understanding that comes with engaging students in a physical classroom, you can foster an online forum as a place for your learners to engage with each other and capture learner understanding. They can post to topics that are pre-determined by the course, or create threads about their own ideas or questions. Not only does this give you a glimpse into the learners' level of understanding surrounding the training topic, but the learners benefit from the social learning aspects of the exercise. Learning is reinforced when your audience engages with others to exchange ideas and answer questions their peers have.
Analyze Learning Outcomes
The goals you have for your mobile training that seek to produce a change in behavior amongst your organization are referred to as learning outcomes. These outcomes may vary amongst your course catalog and could be anything from improved customer service procedures to an increase in productivity. Analyzing learning outcomes is a good way to measure how effective your training is. Whether your intended learner outcome was to enhance productivity, improve customer satisfaction, or something else, your employees should be exhibiting behavior that meets this goal after they have completed training. For example, are you training your staff on safety procedures? Get a gauge for how effective it is by analyzing your employees’ behavior after they have taken the safety course. Has there been a change in the frequency or severity of safety incidents that you covered in your training? If there have been fewer, you can chalk up the improvement at least in part to your safety course. If so, you are off to a good start. In around 30 days, check in with this group to be sure that these changes are still taking place. Since studies show that we forget the majority of new information that we learn within 30 days, confirm that it stands the test of time.
Level 3: Behavior
The third level, behavior, measures how the training has influenced learners’ work performance. Some ways you can measure this level:
Measure Key Performance Indicators
Key Performance Indicators (KPI) are a set of metrics that a company uses to analyze its performance as a company over time. These are self-determined by an organization and typically set surrounding goals related to operations, finances, and performance against other businesses. It can be beneficial to compare KPIs figures pre-training and post-training to see where training may have impacted surprising areas. For example, a decreased employee turnover rate may be a KPI for your organization. People who leave an organization often cite a lack of training as one of the reasons for their departure. Your statistics may show that your turnover rate post-training has improved, and you could logically conclude that your mobile training is making your employees feel better prepared and confident in their roles.
Analyze User Statistics
Your LMS can give you a window into your training via user statistics. Has your audience accessed a certain portion of your training more than the rest? This can help guide you regarding the type of content your audience is seeking out. Among this most used content, is there a particular media type, such as videos, that are most utilized? This helps to inform your decisions as you build additional training content.
Level 4: Results
The fourth level, which is behavior, focuses on the results of training. A way to analyze this level:
Examine Return On Investment
An important consideration of any company expenditure is a Return On Investment (ROI). To measure your training ROI, calculate the cost of the course, and compare it against business impact. Did the training help you to see gains that offset the cost of the course? If so, you earned a positive ROI.
Conclusion
It’s important to understand the benefits that training brings to your organization. The strategies we discussed can help you to paint an accurate picture of how your Learning and Development programs are impacting your company. While it may be expensive to implement all levels of the Kirkpatrick model, you can implement a combination of levels based on your organization’s needs. For example, you could utilize the more extensive level 4 evaluation for programs that cost $50,000 or more, while utilizing levels 1-3 for programs that are less expensive. Measuring mobile training is only an important endeavor if you plan to act on the results you encounter. This should inform your decisions moving forward as you grow your training program. If the analysis finds components of your mobile training ineffective, you will need to carefully examine why it is missing the mark. Perhaps it needs supplemental information, opportunities to practice implementing the training, or that it needs to be redesigned entirely.
Download the eBook, Developing A Mobile Learning Solution That Embraces Anytime Anywhere Learning, to discover how to measure the effectiveness of mobile training and much more.
