Measuring The AI Virtual Assistant ROI For Your Organization's L&D

Measuring The ROI Of An AI Virtual Assistant For Your Organization's L&D
Jacob Lund/
Summary: While we recognize the need for training, we find it difficult to measure its impact on efficiency and performance. Then, there are hard financial questions: what should we spend and what return will we see on our investment?

How To Measure The AI Virtual Assistant ROI For Your Organization

Ways Of Measuring ROI

There have been many attempts to quantify the impact of training on employees and validate the important role played by training. Perhaps, the most comprehensive approach is the Kirkpatrick/Phillips evaluation model.

eBook Release: Chatbots And Workplace Performance
eBook Release
Chatbots And Workplace Performance
Discover how chatbots, combined with Artificial Intelligence and better data tools, can create a more productive and efficient workplace.

As the table below shows, Kirkpatrick and Phillips identify 5 different levels of effectiveness ranging from ‘satisfaction’ with training (those ‘happy sheets’ handed in after a training session) to a profit-and-loss calculation of its financial benefit.

Fig. 1- Kirkpatrick/Phillips Evaluation Model

Fig. 1- Kirkpatrick/Phillips Evaluation Model

While the Kirkpatrick/Phillips model indicates how we can measure the success of training programs, it’s not prescriptive. It doesn’t tell you what to do to improve training and reinforce learning. It simply focuses on where the gaps may be.

Moreover, the model’s static. It applies to each discrete piece of training. In the modern working environment, where instant access to information is becoming ever more critical, the notion of training as an episodic event no longer holds. The model shows us where to focus, but we need something more dynamic and responsive to help today’s L&D.

The Limits Of Training

The Kirkpatrick/Phillips model presumes a traditional approach to training in organizations. Simply put, that means you have the training and assessment and then you’re deemed competent enough to perform the task. If the task or process changes, there’ll be more training in the new task or process—and so on.

There are problems with this approach. It presumes training and work are distinct, discrete areas of activity. It tends to involve training that’s away from the desk: taking employees out of work to attend classes or making them do eLearning courses at their desks. And, most important of all, it ignores the uncomfortable fact, first described by Ebbinghaus in the nineteenth century, that we forget much of what we’ve learned very shortly after we’ve learned it.

The Modern Learner

The other striking aspect ignored by traditional training is the needs and behavior of the learner. Modern learners have access to vast sources of information immediately, round the clock. For anyone with a mobile, smart device the answer (or many of them) is just a couple of taps away. Ease of access and constant connectivity are changing learner behavior.

The way people access information socially is becoming the way people expect to access information at work. If you want to know something today what do you do? Wait for L&D to run a training class or eLearning module or go to Google, YouTube, Pinterest, or wiki sites? And how can L&D compete with Google?

With such resources at their fingertips, modern learners feel in control of their learning. This is a powerful motivator. Handled correctly, it can make training more effective because learners invest in it, and they identify their training needs and how best to meet them.

The Advantages Of AI Virtual Assistants

Training is often removed from the environment in which it’s to be applied. It’s confined to the classroom, trapped in the LMS, or accessible only when timetabled. Imagine, though, that training in your organization was as readily available and as digestible as the information you access on Google, YouTube, or Wikipedia.

This is where AI Virtual Assistants can come in. We already use them in the house (Apple’s Siri or Amazon’s Alexa are common examples), but they’re increasingly being utilized in the workplace too. Properly integrated and embedded in the workflow, VAs can make a real difference to the way training works and bring an obvious, quantifiable ROI for your investment in L&D.

A VA can be given access to your organization's training resources and information on work practices. It can be placed on work management tools like Slack and Trello. Like the apps we use at home, VAs can provide instant responses offering a wealth of information. But, critically, it’s not just any information: it’s targeted, contextualized, and validated by L&D so that it’s the right information in the right place at the right time.

VAs In The Workflow

Let’s look at a couple of examples of where VAs can extend training into the workspace.

Firstly, a Virtual Assistant can make a difference to the way you support and train new employees in the onboarding process. In the past, this meant days, even weeks of training in the basics of their job and an introduction to HR procedures.

An AI Virtual Assistant can guide new employees through all the processes and information they need at their own pace when they need it. A VA offers instant responses to questions ranging from how to order office supplies, to holiday entitlements, to health and safety compliance and so on. Instead of being overwhelmed with too much information at one go, a new employee can access information just when needed.

Similarly, the VA can act as a personal assistant as you work. If you encounter a technical problem or need to know a procedure, you can just consult your Virtual Assistant. The VA can act as a personal organizer giving you reminders and access to resources for daily and weekly tasks. VAs can also make recommendations based on your requests (think of the similar services offered by Amazon or Netflix based on your browsing habits). If a piece of training is relevant, your VA can point you in that direction.

VAs help to manage learning, but also empower learners to take control and responsibility for their own training. Having a VA always on means there’s no reason to forget. They break the costly cycle of learning and forgetting that undermines all formal training. By being there in the workflow acting as a mentor, VAs make learning a continuous process, rather than an episodic event. AI Virtual Assistants offer just-in-time learning at the point of need.

Replace Trainers With VAs?

Despite the popular narrative of Artificial Intelligence replacing human beings in the workforce, VAs aren’t about to replace trainers. Rather they’re going to redefine the role of L&D and give it greater, rather than less prominence within an organization.

For VAs to be effective they need to have information in a format that’s accessible and usable by them. This means L&D producing bite-sized forms of learning and making content resources available to intelligent searching. They need to liberate the content from the classroom or the LMS.

You also need to train the VA, just as you would a real employee. Over time, however, a VA is capable of training itself by recognizing patterns of inquiry and adapting to the requests it receives.

A New Role For L&D

Integrating a Virtual Assistant in the workflow makes training more accessible, but it isn’t the complete answer. You need to move L&D into the workflow too and remove the barrier that often lies between training and the workplace. With L&D within the workflow trainers and learning designers can see more clearly where their intervention can make a difference.

The VA, in turn, isn’t just providing vital information to the employee, it’s also about delivering useful, quantifiable, hard data on learners’ requirements back to the learning designer. The data gives learning designers clearer insight into how training is applied. With this feedback, a learning designer can produce what learners are really asking for, instead of designing according to some notional idea of what learners ought to learn. With this approach, learners are at the center and learning designers move closer to where actual learning takes place.

This shift towards training in the workflow allows a virtuous circle to be established whereby learners learn when they need to, and learning is reinforced. The circle is completed as the learning designers learn from the learner.

VAs Offer Real ROI

Instructional Design models like ADDIE (Analysis, Design, Development, Implementation, and Evaluation) emphasize the need for constant evaluation, but that final element is often taken for granted or ignored as too hard to measure. With a VA in the workflow, you can directly measure the impact and effectiveness of L&D based on the evidence of employees’ on-the-job needs. Also, VAs offer the opportunity to re-use and re-purpose pre-existing content in a way that gives it more impact.

Both these outcomes are direct evidence of ROI. Evaluation becomes more than responses recorded on ‘happy sheets’. Learners are listened to and L&D responds effectively.

So, if you have doubts about whether L&D is making the right impact in your organization, add an AI Virtual Assistant to your team and reap the benefits.


  1. The Kirkpatrick/Phillips Model for Evaluating - Human Resource Development and Training
  2. Training Evaluation: Jack Phillips and ROI
  3. 5 Concrete Ways To Measure The ROI Of Training Programs
  4. A step by step approach to determine your return on training investment 
  5. Assessing the ROI of training
  6. 10 uses for Chatbots in learning (with examples)
  7. Five Steps: Changing Paradigms from Training to Performance
  8. Take your Team from Training to Performance Support
  9. Learning Trends In 2016 – Learning Paradigm Shift
  10. New Learning Paradigms
  11. Paradigm Shift in E-Learning