How To Create An eLearning Evaluation Plan
A few weeks ago, I wrote an article on how to Define A Workplace eLearning Strategy That Works, and a reader was kind enough to point out that the section on evaluation needed to be more elaborate. I agreed. In fact, I had planned for a separate post on evaluation, but his comment made me think deeper about when it should be planned for, and at what levels. Thanks Dave.
I believe evaluation needs to be planned for (note that I’m referring to ‘planning’ and not to the actual evaluation process) at two stages: First is at the time of defining your strategy for eLearning implementation at the workplace, and then before getting into the design stage of a particular learning solution.
In this connection, I’d like to clarify a few terminologies that I use frequently throughout this article:
It refers to one or a combination of the following, created for a specific set of outcomes:
- eLearning Course
- Synchronous Training (Online or Face-to-face)
- Mobile Learning
- Gamified Learning
- Discussion Communities
- Performance Support System, etc.
The combined force of all of the above solutions that are planned to be implemented within a said time period.
The first step in the process of planning for evaluation is to define success. Which of the following criteria do we mean when we think about success? (Note that I’m using the word ‘course’ below only for ease of reference. I am not in any way implying that a course is the only solution.)
- Participation (example: “72% of the learner population logged in and viewed the courses”).
- Completion (example: “80 out of every 100 people who started a course completed it”).
- Engagement (examples: “the courses received an average learner satisfaction rating of 90%” or “we received an average of 50 likes and 12 comments per course”).
- Learning (example: “90% of the learners who took the courses passed the end-of-course assessment”).
- Application (examples: “60% of those who went through the courses demonstrated improved skills / on-the-job performance” or, at a more specific course level “60% of those who went through the course on plant safety actually followed safety procedures while in the factory premises”).
- Results (example: “Safety incidents came down 8% in the three months following the rollout of the course”).
For a learning initiative to be meaningful, my guess is that you want to measure #6: Results. I’m not saying that the other criteria are not important. They are important, but worthless on their own.
You want people to participate in a course (i.e., view the course) so that they can complete it and be engaged with the content. You want them to complete the course and be engaged in the process so they learn and absorb the content well. You want them to learn well, so that they can take their newly-acquired skills and apply them on their job. You want them to apply these new skills on the job so that the quality of the job will improve, and business will see a merit in having put together the course in the first place.
The criteria are all interlinked, each one leading to another. The primary reason you are planning for eLearning Implementation at the workplace is so that business can improve in some manner or other.
A useful model that comes to mind is Kirkpatrick’s Four Levels of Evaluation. Now I know there’s plenty of great debate out there as to the efficacy and applicability of the model, but what I’m talking about here is my adaptation of the model, which can be applied in useful and practical ways to determine the impact of a learning initiative.
Taking the above criteria into account, here is what the evaluation plan for the complete initiative could look like. At this stage, we don’t know the details, the goals and the outcomes, of each solution, but we have a fair idea of how we want to go about the evaluation exercise.
Evaluation Plan Template For the Full Initiative
This should ideally be part of the learning strategy definition.
Evaluation Plan Template For Each Solution
Now, it’s time to get more specific. Since we have already laid out a high-level evaluation plan at the strategy stage, it should be easy to fill out the details for individual solutions. Note that this solution-specific plan ideally needs to be drawn out at the design stage itself, and not at the end of the development process, as an afterthought.
For ease of understanding, I’m taking an example of a specific solution. I’ve also provided the details of the design, so you can relate to how the individual criteria can be evaluated.
Example: A solution for encouraging people to report any code of conduct violations; the solution comprises:
- An eLearning course clarifying the specific details of the code of conduct, complete with detailed scenarios on how to recognize / report violations.
- 6 scenario-based quizzes, sent over a period of 6 weeks (one each week) after learners complete the course (via their mobile devices).
- Posters placed at strategic locations; around work desks and in common areas where employees congregate.
- Key information related to the code of conduct available on learners’ mobile devices.
Once the evaluation plan is drawn, we have a blueprint for understanding how successful our initiative is. However, execution and follow through are another matter altogether. Maybe this can form the topic for another post?
So, have I missed anything? Please share your thoughts in the Comments section below.
Learnnovators will shortly be releasing an eBook titled "The Ultimate Guide To Successful eLearning Implementation At The Workplace". Meanwhile, do get in touch at [email protected] if you would like to discuss any of the above pointers, or if you need assistance with implementing eLearning at your workplace.