eLearning 101 Part 1: Introduction to eLearning
- Getting Started
- Approaches to Learning
eLearning 101 Part 2: Planning eLearning
- Storyboard
- Script writing
- Interactivity
eLearning 101 Part 3: Developmental Roles
- Mandatory Roles
- Optional Roles
- Communications
eLearning 101 Part 4: Development Process
- Recording
- Implementing
- Publishing
eLearning 101 Part 5: Evaluating
Evaluation of your course is essential but often under used. Evaluators should assess student performance, determine program and cost effectiveness, monitor quality to include technology and support services, evaluate course design and instruction, and ascertain employer and student satisfaction. Higher-level evaluations cost more but also tell more in the long run.
Things to consider before you get started on evaluating a course: What is the purpose of your eLearning courses? Do you want the learner to retain and apply information to the job? What is your return of investment? Is what the learner applying relevant and effective? Aim for credibility not just accuracy, document business results, and gather data from those who should know! Also take into consideration that evaluating the course does not just happen after the fact. Evaluation should take place before, during and after the development of an online course.
Donald Kirkpatrick’s model of evaluating has 4 levels of evaluation and is a solid base to begin:
- Reaction: Did learners like the training? Did they complete it?
- Learning: How much is job performance improved, what could learners apply?
- Performing: What skills and knowledge did learners acquire?
- Results: How well did the organization meet its business goals?
Level 1: REACTION
The reaction of the learners assists with recognizing and defusing potential frustration of learners and identifying unmotivated learners. You can track these reactions by using a Learning Management System (LMS) or special tracking software.
Using the Internet such as a LMS assists with tracking:
- Frequency of accessing the course
- Number of pages accessed
- Assignments submitted
- Successful completion
- Participation in online discussions/posts
- Rate of progress through course
- Online evaluation/rating of the course
- You can let the learners vote on course design
- Set up a course discussion thread
- Use chat to focus discussion group
Level 2: LEARNING
What specific facts and skills has the participant learned that can be applied to the job?
- Knowledge of presented material
- Demonstration of skills
- Speed of work
- Accuracy of work
- Fluency of work
- Confidence of knowledge
- Application of knowledge
How do you rate that concepts have been learned and are being applied at work?
- Have learners evaluate their peers
- Observe reactions to collaborative discussions
- Have learners summarize discussion
- Challenge learners to post original questions for others
Level 3: PERFORMING
Can learners apply their skills to the job? Measuring application on the job can be done by:
- Consulting job records before and after learning
- Observing on-the-job behavior
- Monitoring fulfillment of participants action plans and performance contracts and measure their performance against an action plan
- Evaluate on-the job performance and individual action items
- Test with a control group, one that has the training and the other without it
Level 4: RESULTS
Did the training reach its original business and organizational goals? What is the rate of return of money invested in learning? The organization has to determine what is the single most important measure of success. When that is determined then the training can be applied to toward that end.
- Ask the participant to estimate
- Ask the learner to estimate how much time and money they have saved by taking the online course
- Ask for testimonials
- Ask supervisors to estimate
- What is the percentage of change of the participant after the training
- What is the value of that change?
- How confident are you in the estimate?
- Monitor business metrics
- Financial
- Intellectual capital
- Customers
- Operations
- Reputation
- Consider all benefits
- Hard benefits: direct costs, time, production, quality
- Soft benefits: work performance, work environment, career advancement
- Fuzzy benefits: satisfaction, initiative, leadership
- Non-monitory: customer satisfaction, morale, prestige, community relations, political
- Consider Lost Opportunity Costs: for example: the time it takes to go to an instructor-led training vs. time it takes for online course.
Benefits of online learning includes:
- Overall lowered cost (cuts travel expenses, reduces the time it takes to train people and eliminates or reduces need for classroom training).
- Enhanced business responsiveness (e-learning can reach an unlimited number of people simultaneously)
- Consistent messages (everyone gets the same content, presented the same way)
- Timely content (e-learning can be updated instantaneously, making the information more accurate and useful for a longer period of time)
- 24/7 learning (people can access e-learning anytime, anywhere)
References
- Integrated Learning Solutions Blog: Evaluating eLearning in a Crunch
- Learning Solutions Magazine: article by Jane Bozarth: How to Evaluate eLearning
- Fastrak Consulting: Clive Shepard: Evaluating Online Learning
Contributing Author Rob Porter with Training Objectives