Why You Need Learning Metrics That Matter
You’ve set up your Learning Management System (LMS) and have begun implementing courses. Congratulations! But don’t rest yet, because your job isn’t done until you’ve closed the loop. Your organization’s executives have invested into your Learning Management System, and they’ll want to know that the cost was worth it. Even if your Learning Management System was free, there were internal costs. You’ll also be asking for additional eLearning budget allocations in the future, so you’ll need to justify your past and current initiatives by showing a return on investment (ROI).
But where do you start? Many Instructional Designers report on activity measures—number of students, completion rate, etc. While these metrics can be used to show successful course adoption, ultimately you’ll need more than that to show ROI that impacts the bottom line. Quality and impact matter more than the number of students or quantity of training sessions. You’ll need to provide performance measures as well as activity measures.
Activity Measures Vs. Performance Measures
Activity measures are related to learners’ participation in a module. They provide statistical information about the use of the course and its success, in terms of participation. LMS administrators and course instructors need a meaningful analysis of module activities. These metrics help to evaluate the success of implementation and participation, and to measure the effectiveness and impact of learning programs.
Examples of activity measures include:
- Average number of students per learning module.
- Facility usage.
- Course completion rates.
- Course registration numbers.
- Revenue generated per course in a given quarter.
- Quality of responses provided by the instructor in a course.
Performance measures, on the other hand, tell you how well you have trained from a business-case perspective. Examples of performance metrics include:
- Job impact.
- Time to job impact.
- Change in strategic results isolated to the training.
- Instructor performance.
- Impact on business goals.
- Turnover and retention trends.
- Correlation to change in profit.
Reporting Training Activity And Performance Measures
Activity measures are generally easier to collect than performance measures, because they are primarily data associated with the training registration process. Because training needs to be coordinated and scheduled, the information is readily available within the Learning Management System. Most Learning Management Systems can easily export this data, although it’s debatable how well they report the information for useful analysis.
Most organizations do a decent job of reporting the activity measures that are easy to collect, such as number of courses run, or number of people trained. But performance measures are more difficult to obtain.
Some feel that certain performance measures are nearly impossible to obtain—although satisfaction-related measures such as instructor performance or courseware quality can easily be collected through end-of-course evaluations. It’s also fairly simple to obtain knowledge transfer measures by collecting pre- and post-test scores—but the testing exercise can be a significant drain on a company’s resources.
Return on investment is the brass ring of testing measures. Although ROI is a powerful metric, for many training groups it’s nothing more than a distant dream. The reasons abound, but often it’s because learning data and performance data often reside in different systems, making this type of comparison challenging. Other times, learning goals aren’t well-defined, or they aren’t aligned closely with the company’s business goals. And many organizations simply don’t track the right data, so the metrics to prove ROI aren’t being captured.
Nonetheless, performance measures are critical. If you don’t know which programs have the greatest impact on job performance and business goals, your measurement system has some significant shortcomings!
Performance Metrics Needed
Businesses are constantly looking for better ways to run lean. As a result, training departments can’t rely purely on activity measures to justify their budgets. Executives aren’t interested in the number of participants or how many modules were run in a quarter—they need to know what kind of impact training has on the company’s bottom line.
Your company’s CEO is constantly trying to control spending and costs, and your Chief Learning Officer is asking for a budget increase for the next fiscal year. If your budget is to be approved, you’ll need to show some evidence to not only justify the increase, but possibly to show the ROI on the current program in order to avoid budget cuts. Tools that help your company perform better, retain employees, and improve the bottom line are solid investments that prove their value.
If you’re reporting that 70% of the company received training in the prior year in a total of 250 courses, and everyone you talked to enjoyed the training—you can expect to lose your budget request.
Your CEO won’t be compelled to approve a training budget based on activity metrics alone, no matter how quantitative the data is. While activity metrics are valuable, they don’t reflect the impact on your company’s business goals. Your data needs to answer questions like these:
- Did the training help increase revenue?
- Did it help decrease costs?
- What kind of impact did training have on the average employee’s job performance, relative to their salary?
Implementing Performance Measurement
Most organizations that follow the Kirkpatrick model can’t evaluate their program beyond its first two levels.
The Learning Management Systems that many organizations use make lower-level evaluations easy, but they don’t provide any mechanism for higher-level evaluation.
Most Learning Management Systems will automatically track and report information required for Level One and Level Two analyses. They include assessment tools that can capture each learner’s reaction to the course, and templates that can create reports. For online and blended learning, the Level One assessment (the learner’s reaction) can be completely integral to the course. Likewise, training programs can inexpensively and easily administer pre- and post-tests that evaluate learning results (Level Two).
When it comes to evaluating learners’ ability to apply the training (Level Three) or judging whether the targeted outcomes are achieved (Level Four), these tools are silent.
More Powerful Performance Reporting
To successfully manage performance metrics and evaluate ROI of the training, you’ll need to have the right technical infrastructure and a good model of learning improvement. You’ll also need some expertise in:
- Data management and warehousing.
- A variety of systems and databases.
- Analytics.
- Web-based application development.
But Zoola™ Analytics closes the gap between your training and its impact on individual performance and organizational performance. Zoola makes it faster to perform the analysis and build reports. You can dig deeper into learner data, generate full insights into learning programs, and generate configurable, exportable reports quickly.
For Moodle and Totara LMS administrators, Zoola eliminates hours of effort to report on eLearning usage and outcomes, such as course activity, progress, effectiveness, and more.
Ready to go deeper in your LMS mastery? Check out our upcoming webinars!
