VR Equals To New Possibilities For Learning Analytics
Adaptive learning. Continuous improvement. Evidence-based learning efficacy. The rise of learning analytics is driving change in Learning and Development (L&D). If this area of focus gets you excited, you’re not alone, and VR training analytics is about to open up a whole new world for you.
“Learning analytics” jumped from third to first position in Donald Taylor’s annual L&D Global Sentiment Survey, which asks, “What will be hot in workplace learning in 2020?”[1] Yet despite the drive to collect and use data to adapt, improve, and assess the value of learning, it has been challenging for L&D to accomplish. An annual report by Watershed suggests that while 90% of L&D professionals surveyed want to measure the business impact of learning programs, 53% of L&D departments have either no or basic measurement [2].
Compared to eLearning, VR brings unparalleled access to user data because of the innate tracking capabilities inside the headset, which can then be exported to an analytics tool or dashboard. Analysis of VR data can demonstrate training effectiveness and show areas of improvement, both on an individual level and aggregated across entire learning audiences.
Combined with a strategic approach to learning evaluation outside the headset, we have a powerful formula for tying data to performance outcomes.
The Many Ways To Use VR Training Data
An important point to make is that VR user data is not only useful for analyzing after the fact. With sophisticated VR programming, we can collect data during the learning experience and adapt the experience in real time. New developments in VR include tracking body movements, facial expressions, voice recognition (speech) and vocal tone, and even biofeedback, such as heart rate.
What Can Be Tracked In VR?
Progress, Completions, And Replays
Measure and report on how many tries it took to finish a VR learning experience, and if/how many times the learner replayed the experience. This, combined with other data, can indicate whether you have areas for improvement in your learning experience.
Decision-Making
Continuous monitoring of data means you can gather data on key decision points, which can include anything from multiple-choice assessments to choices in a branching scenario. Analyzing this data can help you determine if you’ve made the questions or scenario choices too challenging or not challenging enough. It could also help you analyze patterns in an individual's thought process or decision-making. With a branching scenario in which the paths are based on emotional response (empathetic, severe, professional), the data will show which path an employee tends to take.
Voice Recognition And Analysis
Track the verbal interactions learners have with avatars and interactive components within the VR experience to gauge the emotion and energy level demonstrated by a learner’s verbal response. Pace can be determined by the words spoken per minute. Data can reveal the tone and overall sentiment of the speaker. You can even track how many “um's” or “likes” a person exhibits during verbal communication.
Body Position And Movement
Evaluate this data to assess if the proper physical techniques being instructed are being achieved—ideal for safety training. For soft skills training, this data can be monitored to evaluate the learner’s communication, interactions, and behavior exhibited through body language in the program. Monitor how and where learners move around a virtual environment to evaluate if you’ve created an intuitive path for them.
Retinal Eye Tracking
Data showing where learners are looking in the VR experience (and at whom they’re looking) can be visualized in a “heat map” for any point in the program. Evaluate this data to ensure key content is successfully being seen at the right moment in the learning journey. This data could also be used as key metrics for diversity and inclusion training, potentially indicating areas of unconscious bias.
Biometric Feedback
An exciting new development in VR, biometric feedback, may include monitoring a learner’s heart rate, energy expenditure, and even mapping brain activity (which is still in the early stage of development). This can provide insight into stress levels that learners feel at specific moments in the program. You can measure learners’ heart rates to determine if they’re exerting a specific amount of energy at a particular time.
Tracking Of Multimodal Data
You needn’t be limited by only VR learning data. Not only can your analytics tool monitor each VR program, but you can also aggregate data from other learning modalities, such as Augmented Reality and Mixed Reality. This gives L&D the flexibility to implement different learning modalities (for optimal learning potential) while monitoring learning activities from a single, customizable dashboard.
LMS Vs. VR Analytics Dashboard
Achieving the best integration with the LMS is a continuing journey in VR for learning. As self-contained, stand-alone applications, VR is typically launched via the VR hardware and software, rather than an LMS. However, if we want to go deeper and review and analyze more of the user data, then we need another tool: Enter the VR analytics dashboard.
The VR analytics dashboard can provide L&D professionals with these features:
- Track users, completion of experiences, and assessments
- Track the amount of time spent on a given situation
- Count repetitions and/or interactions in the experience
- Capture specific data for each unique user
- Support a dashboard view with a familiar LMS look, feel, and functionality
- Be customizable to provide insight into the most important data: you can add a data tracking element to just about anything in the Virtual Reality experience
Tying It All Together And Measuring Impact
To create the most effective evaluation strategy, you’ll want to combine data captured from VR learning with data gathered after the learning experience. This evaluation strategy should be tailored to your learning objectives and audience as well as business constraints.
If you’re new to training evaluation, good places to start are the Kirkpatrick Model of training evaluation and Will Thalheimer’s Learning-Transfer Evaluation Model [3] [4].
A comprehensive evaluation strategy might include surveys, observation, and data analysis to address the following areas (informed by the two models mentioned above):
- Engagement
How do learners rate the level of engagement? This can be combined with user data captured inside the headset on progress and replays, and, in the near future, biometric feedback as well. - Confidence
How confident did they feel about performing the task(s) on the job before the training, and how do they feel after the training? - Relevance
How relevant did they find the training? How often do they predict they will use what they learned on the job? - Motivation
How motivated are they to apply what they learned back on the job? - Knowledge
This refers specifically to the recall of facts and information, and it is typically measured via multiple-choice assessments. This could be combined with user data on decision-making, such as with branching scenarios captured inside the headset. - Competence/Transfer
Back on the job, is the learner making decisions and performing tasks by applying what they learned—in other words, displaying behavior change? This can be measured via manager observation and/or more sophisticated assessments (possibly in VR!). - Results
What is the effect of the transfer on the business or organization? This typically requires more in-depth data analysis on Key Performance Indicators before and after the learning intervention. Given the variables involved in these metrics (Is the increase in sales due to learning? New products? Discounts?), understanding impact at this level is challenging—the holy grail of learning analytics!
Learning Analytics And The Future of L&D
Data collected both inside and outside the headset create a comprehensive VR training analytics strategy, beyond what can be done in other delivery modalities. As we move forward through precautions around physical distancing, analytics is more important than ever for capturing every interaction and reaction your learners have. Armed with the granular data to track learning, instructors can support learners with better and more personalized feedback—without having to be in the same room.
Interested in discovering more about learning analytics in VR training and seeing examples of analytics in action? Download Your Guide To VR Training Programs: Virtual Reality For Our New Reality. In this comprehensive eBook, you’ll learn the steps to successfully building a VR training program within your organization and read the case studies that prove its effectiveness. Join the webinar VR Training Program Case Study to learn all you need to know through a success story!
References:
[1] THE L&D GLOBAL SENTIMENT SURVEY 2020 - FIRST THOUGHTS
[2] Measuring the Business Impact of Learning: 2020 Survey Results
[4] The Learning-Transfer Evaluation Model (LTEM)
