Is There A Rationale Behind The 80%?
In all my 20+ years of designing eLearning courses, I have always set the passing score at 80%. Most instructional designers at the customer end too insist on this passing percentage. 80% has become some kind of default industry standard. But what is the rationale for this figure? Is there any logic behind it? Or is it something that learning specialists have arbitrarily chosen with the rest of us following suit? Let’s see if this magic number has any mooring in the learning theory. And if not, how should we go about setting the final passing score?
When Do You Frame The Final Quiz?
The logical sequence of designing an eLearning course starts with identifying a business challenge that training can address to improve performance, followed by breaking down the training need(s) into measurable learning objectives (LOs) that are performance-based.
Once the performance-based learning objectives are set, the next step should be to translate them into their corresponding evaluation or test questions that would measure the achievement of the learning objectives. Most IDs skip this step and jump into content creation and come back to formulating a final quiz at the end. This sequence is not advisable as it may sidetrack the designer to come up with quiz questions that are not properly aligned to the LOs, resulting in the quiz not measuring the effectiveness of the course properly.
Therefore, the right sequence would be to start with LOs, set the final quiz questions, and then pick content. This will ensure perfect alignment between the LOs and Final Quiz, besides ensuring that the most appropriate content is chosen to address the LOs.
Are Some LOs More Equal Than Others?
Bloom’s taxonomy of classification of human cognition into different levels includes the most basic cognition level to the highest: knowledge, comprehension, application, analysis, synthesis, and evaluation. Recently, these nouns have been revised to verbs: remembering, understanding, applying, analyzing, evaluating, and creating.
As we all know, Bloom’s taxonomy gives us the framework for setting learning objectives at the appropriate level of cognition. It also shows that the learning objectives are arranged from enabling objectives leading to terminal objectives; that not all LOs can be seen as equal, some are enabling; and, some are terminal. Small, enabling objectives cannot be assigned the same score as a terminal objective as the latter is instrumental in delivering the desired performance.
Scoring Based On Levels Of Cognition
LOs can be classified based on Bloom’s taxonomy for their enabling and terminal nature and the rating of their corresponding performances based on the parameters of frequency, difficulty in learning, criticality, and associated risk. Once this is done, the corresponding quiz questions can be assigned scores depending on the nature of the antecedent LO.
Now, this kind of classification and assigning scores for each test question based on multiple parameters can be tricky and time-consuming. One easy and practical way of doing this is to categorize the test questions into three levels—basic, intermediate, and advanced—and assign the same score to questions in each level. The score becomes higher as the level advances.
For example, for a 12-question final quiz, questions 1-4 (basic) can be assigned 1 point each, questions 5-7 (intermediate) can be assigned 2 points each, and questions 8-12 (advanced) can be assigned 4 points each, with the total final quiz score adding up to 30 points.
Authoring tools such as Articulate Storyline and Lectora Inspire can provide this output with a little tweaking.
Are All Performances Critical?
Some tasks are difficult to perform, some are rarely performed (and some regularly), some are crucial for performance, and some not so much. Some performances are risky in the sense that if not done properly, they may result in damage or loss to the performer and/or organization. Therefore, it stands to reason that all tasks are not equal and so cannot be assigned the same score. Some questions warrant a higher score based on the frequency, ease of learning, criticality, and associated risk of the corresponding tasks/performances.
You would require learners to score 100% on all questions that address critical (like operating a mission-critical machine) and high-risk (like handling hazardous chemicals) tasks as even a 1% chance of failure would not be acceptable in those situations. When it comes to those that are frequently performed (like daily routine tasks), or easy, or less important and carry no risk, you can set a more lenient passing score, say 50 or 60%.
Setting The Passing Score
So, how do we bring all these aspects together to arrive at a passing score that is based on the types of performance? Once the objectives and the corresponding quiz questions are framed, we can assign points to each LO based on its fundamental nature, enabling or terminal. Then, identify the corresponding performance task of each LO and assign it a weight, based on parameters like criticality, risk, and difficulty of learning. By multiplying the points with performance weights, we can arrive at a minimum acceptable score, which is the passing score.
- Question 1, Basic (Enabling) Level, 1 Point, Performance Weight 0.3, Passing Score 0.3.
- Question 2, Basic (Enabling) Level, 1 Point, Performance Weight 0.3, Passing Score 0.3.
- Question 3, Basic (Enabling) Level, 1 Point, Performance Weight 0.3, Passing Score 0.3.
- Question 4, Basic (Enabling) Level, 1 Point, Performance Weight 0.3, Passing Score 0.3.
- Question 5, Intermediate (Enabling) Level, 2 Points, Performance Weight 0.5, Passing Score 1.0.
- Question 6, Intermediate (Enabling) Level, 2 Points, Performance Weight 0.5, Passing Score 1.0.
- Question 7, Intermediate (Enabling) Level, 2 Points, Performance Weight 0.5, Passing Score 1.0.
- Question 8, Intermediate (Enabling) Level, 2 Points, Performance Weight 0.5, Passing Score 1.0.
- Question 9, Advanced (terminal) Level, 4 Points, Performance Weight 0.8, Passing Score 3.2.
- Question 10, Advanced (terminal) Level, 4 Points, Performance Weight 0.8, Passing Score 3.2.
- Question 11, Advanced (terminal) Level, 4 Points, Performance Weight 1.0, Passing Score 4.0.
- Question 12, Advanced (terminal) Level, 4 Points, Performance Weight 1.0, Passing Score 4.0.
Total Points 30, Passing Score 19.6/30 (65%).
In this manner, you can arrive at a more logically derived final score that is grounded solidly in learning theory and terminal performance rather than going with an arbitrary, out-of-the-hat number like 80%, which assigns the same weightage to all questions, and then sets a passing score without any real basis.
