Are You Measuring These Essential Outcomes?
Data is the world’s most valuable resource. With the right data, companies and teams can make informed decisions to improve their performance, instead of taking guesses in the dark and hoping it works out. Without it, you may never know if your hard work was wasted on designing a course that missed the mark, frustrated learners, or bored them to tears.
"L&D plays a crucial role in maintaining employee engagement and growth in an organization…When using data to drive education, we can focus on what is required rather than what we think is needed," shares Vanessa Olsen, Learning and Development (L&D) officer for Campbelltown City Council, with over 20 years of experience [1]. So, what learning metrics should trainers and learning designers be collecting data for? Here are our top three recommendations.
1. Learner Engagement
Engagement is the art of catching and holding the attention of the learner. An engaged learner actively participates in the learning process, building meaningful connections with the information they encounter. Engagement is the critical first step toward learners remembering and implementing what they’ve learned.
However, engagement can be challenging to measure. It’s difficult to climb inside a person’s brain and know exactly what they’re focused on! Therefore, engagement is often measured through quantifiable proxy behaviors such as:
- Time spent on a page
- Clicks on page elements
- Time spent watching a video
- Time spent listening to audio
- Assessment results
However, interpreting the data produced by these measurements can be a challenge. After all, the time spent on a page could equally be caused by confusion with the content, attention split between the course and something else, or even walking away to grab a snack. As a result, this quantitative data is often matched with qualitative measures. The most popular of these is the humble survey, delivered after the learner has finished a course or training program.
Surveys can be a highly valuable tool for gathering feedback, Olsen confirms. "We ask for formal written feedback at the end of training sessions to capture the initial opinions of participants. We often identify new training opportunities not included in our original training plan by including questions regarding training needs in staff surveys." However, surveys present their own challenges. "One of the biggest challenges is obtaining honest feedback from staff. When there is a lack of transparency and trust in a workplace, employees will be less likely to provide the information we seek for fear of retribution. Poorly designed surveys can also lead to feedback that is not meaningful," Olsen adds. The best strategy is a combination of quantitative and qualitative data, with a healthy dose of experimenting and testing.
2. Knowledge Retention
Knowledge retention is perhaps the most simple of these three metrics. At its most basic, it refers to how much information learners retained after completing the training. Retention is important because the ability of a learner to remember what they have learned affects their ability to implement what they have learned.
Retention can also be one of the easiest metrics to measure. Almost all training courses conclude with an assessment asking learners to recall key facts from the learning content. However, there is a lot more to retention than just immediate recall. Numerous studies have shown that our ability to recall newly learned information begins to decline rapidly almost immediately. The AGES model is a popular framework for implementing research-backed methodologies for improving learning retention. One of the elements of the model, spacing, describes how asking learners to recall the information successive times in the period after the initial learning can enormously boost long-term retention.
The other aspect to consider is what learners can recall. Recalling minor and unrelated details is not a success if learners cannot recall the most important facts and lessons. To ensure learners are leaving with the right information, learning designers should use learning outcomes to frame their courses and shape their assessments [2].
3. Behavioral Change
Last but not least is the golden unicorn: behavioral change. Ultimately, behavioral change is the primary goal of all training, to cause learners to act in a specific way in specific situations. For example:
- Managing risks and prioritizing safety after safety training.
- Navigating online environments carefully after cybersecurity training.
- Communicating effectively and clearly after soft skills training.
However, it is perhaps the most challenging metric to measure as it takes place after the training experience (if at all). It is all too easy to implement a new training course and assume that if learners are passing the assessment, then they are implementing what they have learned and changing their behavior. Unfortunately, this is hardly guaranteed. Training courses can easily be undermined by managers with their own idea of how things should be done. Learners can also decide that the content is irrelevant, unrealistic, or simply untrue, and therefore reject the learning altogether.
Some behaviors can be directly measured through metrics such as a change in incident or accident reports before and after training is implemented. "Metrics such as staff turnover, WH&S psychological harm claims, and staff satisfaction survey results allow the L&D team to assess what training is required, prioritize training according to need and determine if current training meets the learning objectives," Olsen recommends. Other behaviors can require Learning and Development teams to be more creative to determine if they have changed as a result of training. For example, L&D teams can ask for observations from other team members. "After conducting leadership training, we can look at staff survey results to determine if a behavioral change has occurred," says Olsen.
Conclusion
Collecting data on the right metrics is the first step to making informed decisions and improvements to your learning courses and programs. Collecting and combining qualitative and quantitative data in an environment where employees feel safe to express themselves is often the best approach to getting the most accurate picture of how your training is performing.
However, data should always be gathered and examined carefully to ensure that the right conclusions are being drawn, as it’s all too easy to make a mistake. Above all, always try new things and experiment!
References:
[2] 5 Tips For Designing Learning Assessments
Originally published at www.howtoo.co.
