What Is The True Power Of Feedback In eLearning?
“Feedback is one of the most powerful influences on learning and achievement, but this impact can be either positive or negative.” [1]
A large meta-study (500 meta-analyses, involving 450,000 effect sizes from 180,000 studies, representing approximately 20 to 30 million students) found that the impact of feedback on learning is in the top 5-10 factors, but its power depends on the type of feedback. The feedback that enables learners to improve is the most effective, while purely extrinsic motivators (awards, praises, trophies) may have a negative effect!
Giving and receiving feedback is a much wider and more complex topic. This article focuses on the power of feedback as it relates to eLearning and ways you can make it most impactful.
What Is Feedback?
While the focus of the study mentioned above was classroom feedback, the definition, or meaning, of feedback is universal:
“In this review, feedback is conceptualized as information provided by an agent (e.g., teacher, peer, book, parent, self, experience) regarding aspects of one’s performance or understanding.” [1]
What Is The Goal Of Feedback?
How would you define the goal of feedback? Is it praise? Encouragement? Motivation? Error correction?
When we hear the word, feedback, we often think about two types: positive and negative. Or, if you’re in the corporate world: pluses and deltas. We also like to use phrases such as constructive, to replace negativity in the tone. In eLearning, feedback often comes from the “system”, rather than a real person. Designers may use avatars to make it feel more human but overall, it is a message you receive on the screen without the ability to ask for clarification. Therefore, we must be very clear of what we’re trying to achieve with this message.
In their publication authors, John Hattie and Helen Timperley, argue that feedback should provide answers to the following three questions [1]:
“Where am I going? (What are the goals?), How am I going? (What progress is being made toward the goal?), and Where to next? (What activities need to be undertaken to make better progress?)”
These questions address long-term vision (“Where am I going?”), current status (“How am I going?”), and immediate actions (“Where to next?”). If this was a journey, the vision would provide you motivation to keep going towards the destination, the current status would offer you error corrections if necessary based on where you’ve been, and the immediate actions would power you with just enough fuel and turn by turn directions to make it to the next landmark.
Motivation Is Key
Motivation is one of the key elements of feedback. This is where gamification efforts often fail, assuming that people need extrinsic motivation such as points, badges, leaderboards, cash, etc. Purely extrinsic motivation often backfires in the long-run:
“Tangible rewards significantly undermined intrinsic motivation, particularly for interesting tasks (–0.68) compared with uninteresting tasks (0.18)” [1]
If a child loves reading books, a tangible reward to read books may undermine their intrinsic motivation. Numerous other studies showed the same results:
“Our results indicate that offering money or cash-equivalent incentives (such as tickets to an event) may have a negative effect on […] blood donor contribution.” [2]
It does not mean you can’t use points or scores in your eLearning, but you need to consider a balance between intrinsic and extrinsic motivators in your design.
What If They Skip The Feedback?
Do you ever get this argument from stakeholders or SMEs:
“Feedback should not contain any new information that is important because people don’t pay attention to it. They just close it as fast as they can.”
People don’t pay attention to feedback, therefore, we shouldn’t put any important information in them. How do you argue against this opinion? Here’s one way to start the conversation:
“If people don’t pay attention to feedback, you’re right; we shouldn’t try to use it to present something important they might miss. However, let me ask you this: why do you think people don’t pay attention to feedback?”
And this conversation often leads to interesting findings. For example, it might turn out that we taught people not to pay attention to feedback by doing some of these three major mistakes:
- Providing the same feedback, no matter what mistakes or errors people made.
- Using generic feedback text such as “Great job!” or “This is incorrect!”
- Designing interactions that were not engaging in the first place (and so, why would the feedback be any different?)
Changing the reputation of feedback starts with a mindset change from designers. Feedback should be part of the overall instructional strategy from the beginning with the end in mind. How we present the importance of feedback to users may determine its effectiveness. Another study on the impact of feedback showed some surprising side effect [3]:
“We expected learners to improve after they received the first feedback. However, we found that learners expecting feedback used better strategies right from the start.”
This study suggests that just by letting people know that they’re receiving valuable feedback on what they’re doing encouraged them to use better strategies for learning! Would that work in eLearning as well? What if we rebranded feedback as formative guidance: Show me what you can do, and I’ll help you get better at it. How powerful would that be? Now, we just need buy-in!
How Do You Get People’s Buy-In?
If we’re talking about courses, it’s slide 1. Slide 1 is probably the LATEST place to get buy-in. You can lose your audience with the first couple of slides between listing bullet points of meaningless (for them) learning objectives, locked and force audio, introduction how to navigate the page, and trying to understand how to pass this course as fast they can.
While David Boud and Elizabeth Molloy mainly address feedback in higher education in Feedback in Higher and Professional Education: Understanding it and doing it well, the book has many valuable lessons for workplace learning [4]:
“[…] when students buy into their learning objectives, they display more positive attitudes toward learning, more effort and perseverance, and greater engagement in their schooling (Pintrich & Schunk, 2002).”
This statement speaks even louder when we look at adult learning. Adults may not care about your learning objectives as much as their immediate application to work. Therefore, to engage and motivate adults to even pay attention to feedback, we need to start with performance objectives. "How to double your sales?" sounds more intriguing than “by the end of this course learners will be able to list and explain the four steps of our SALES methodology”. If you’re interested in why learning objectives do not need to be displayed on slide 1 for adult workplace learners, start with Will Thalheimer’s work [5].
Quick Challenge
Further, some research suggests that when feedback is delivered as formative guidance rather than summative evaluation, it can help students develop a learning orientation, in which they view improving their own competence as the goal of learning[…] (Shute, 2008)[4].
If the goal of your feedback is then formative guidance to improve competence, how would you approach this challenge?
Imagine you’re teaching a class of 6th graders. They submit their masterpieces for you to review. Which of the following method you would choose to provide feedback:
- Score
- Specific written messages
- Both score and specific written messages
The exact same study showed interesting results. The worst improvement was observed when teachers just gave a grade score. The best improvement happened when teachers gave a specific written message as feedback. However, when they combined these messages with a score, the score canceled out the effectiveness of the messages. A reason for that could be that high scores convinced students that they don’t need to read the messages, while low scores made students not want to read them [4].
In Practice: Effective Feedback Components Using Motivation And Error Correction
One of the challenges we face with eLearning is that there’s no human to provide feedback. There are no personalized messages written on assignments. How do we put all that we talked about feedback into practice then? How do we support both motivation and error correction?
In order to support both motivation and error correction, let’s break down a multiple-choice interaction based on result and intent. For simplicity, the result is correct or incorrect. The intent, on the other hand, represents the mental model of the participant used to make the decision. In other words, WHY they chose an answer. Is it because they applied the correct mental model? Because they guessed (no mental model to apply)? Or because they applied an incorrect mental model (but accidentally got the answer right)?
| Motivation (What’s the value for me?) | Error correction (How can I do better?) | |
| Correct (Knowledge: correct why, correct what) | Yasss! I'm smart! | I knew it. What else should I know? |
| Correct (Guess: unknown why correct what) | I might not know the answer to everything at work, but I can make an educated guess. | Okay, so that’s why!!! |
| Correct (Error: incorrect why, correct what) | I’m smart. Got it right! | Oops, I didn’t think that’s why. |
| Incorrect (Guess: unknown why, incorrect what) | Well, I didn’t know. I know now. | But why? |
| Incorrect (Mistake: correct why, incorrect what) | I was thinking right! I just made a mistake. | Oh, okay. Now I get it. |
| Incorrect (Knowledge: incorrect why, incorrect what) | I always thought… Hmm... Now I know. | But why is it wrong? |
How does this table help? You can use it to personalize feedback. For example, if you knew that someone answered a question correctly because they applied the right mental model, your feedback would acknowledge the achievement, reinforce the mental model, and add an additional piece of information (which could be a more elaborate level or explanation why other solutions are not correct).
Unfortunately, most of the time, we don’t know why someone chose an answer in a course, therefore we don’t know whether the concept or mental model was correct or not. While not perfect, there’s a trick you can apply to learn more about why an answer was chosen. The method is called confidence-based assessment, in which after answering a question, participants also indicate their confidence level in their answer. Using that information, you can then design more personalized feedback. For example, if the user was incorrect but their confidence was high, it might indicate that they had an incorrect mental model applied. On the other hand, if a user was correct but their confidence level was low, they might have guessed the answer.
Different Types Of Feedback
“Feedback is one of the most powerful influences on learning and achievement, but this impact can be either positive or negative.” [1]
Let’s say you do have the buy-in from participants, they’re engaged in the activities in the course. What kind of feedback is most useful? John Hattie and Helen Timperley propose four levels of feedback [1]:
“The model discriminates between four levels of feedback: the task, the processing, the regulatory, and the self levels.”
Providing a different type of feedback on each level has its own purpose:
- Task-level feedback is most effective when it aids in building cues and information regarding erroneous hypothesis and ideas and then leads to the development of more effective and efficient strategies for processing and understanding the material.
- The processing-level feedback assists in building better or more effective searching and strategizing.
- The regulatory-level feedback builds more confidence and engagement in further investigation of the subject.
- On the other hand, feedback at the self or personal level (usually praise) is rarely effective. Therefore, you should avoid feedback such as “Great job!” and “You’re awesome!” without providing any additional, effective elements on the three other levels.
When Should We Give Feedback?
Generally, learning designers use two types of timing for feedback: immediate and delayed. Immediate feedback is presented right after a decision or action. Delayed feedback is presented later in the course.
Immediate Or Delayed Feedback?
“The optimal timing of feedback seems to depend on the nature of the learning task. When students are acquiring new, complex knowledge or skills, real-time checks for understanding and tips can prevent them from developing misconceptions or incorrect practices. But when they are extending and applying knowledge (for example, writing an essay or solving a complex theorem), delaying feedback somewhat can enable them to self-correct, develop perseverance, and take responsibility for their own learning objectives.” [5]
If you want to study well-timed, motivational, immediate constructive feedback, play some good games! Well-designed games are engaging because they provide the perfect combination of all we talked about so far: meaningful context, engaging challenges, frequent and specific feedback, raising stakes, and driving competency:
“Let's face it—video games will likely always have more entertainment value than a biology class. But borrowing some of the principles of these games—in particular, the relevance, specificity, and timeliness of the feedback they provide—could go a long way toward powering up classroom environments, making them more engaging and rewarding for students.” [6]
Final Thoughts: Know Your Audience
One of the biggest challenges learning designers may face is not knowing the participant’s prior knowledge. Assuming a level of knowledge and relying on feedback for error correction can backfire in many ways. If the participant does not have adequate prior knowledge to build on or tries to rely on poorly understood concepts, more elaboration through instructions, worked problems are more efficient than feedback alone:
“With inefficient learners, it is better for a teacher to provide elaborations through instruction than to provide feedback on poorly understood concepts.” [6]
On the other hand, designing an eLearning course to the “lowest denominator” may lead to boredom and apathy as more advanced participants won’t feel challenged.
You may want to use a pre-assessment to determine prior knowledge or provide an option of diving into a challenge first and then adjust content accordingly. Either way, always keep feedback on top of your mind when designing learning. Done right, it can be a powerful influence on learning:
“Feedback is one of the most powerful influences on learning and achievement, but this impact can be either positive or negative.”
References:
[1] The Power of Feedback by John Hattie, Helen Timperley (http://rer.sagepub.com/content/77/1/81)
[2] Blood donor incentives: A step forward or backward (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2847338/)
[3] A surprising effect of feedback on learning (https://www.researchgate.net/publication/222391762_A_surprising_effect_of_feedback_on_learning)
[4] Feedback in Higher and Professional Education: Understanding it and doing it well by David Boud (Editor), Elizabeth Molloy (Editor)
[5] Work-Learning Research (https://www.worklearning.com/)
[6] Research Says / Good Feedback Is Targeted, Specific, Timely (http://www.ascd.org/publications/educational-leadership/sept12/vol70/num01/Good-Feedback-Is-Targeted,-Specific,-Timely.aspx)
