How To Write Multiple-Choice Questions Based On The Revised Bloom's Taxonomy

Creating Multiple-Choice Questions Using Revised Bloom's Taxonomy

Since formal education was introduced to the world hundreds of years ago, the testing process has been in a state of constant evolution. As eLearning developers on the forefront of a new digital education age, we are charged with finding the ideal way to insure that our learners have retained the information we have provided for them. One of the most effective methods of doing so is by offering multiple choice assessments and exams, which allow us to determine if our teaching methods or eLearning course design is doing its job (which is to provide the best possible eLearning experience). However, the question remains...what is the best technique to utilize when writing a multiple choice question? Bloom's Taxonomy may, very well, be the answer to that all-important eLearning question.

What Is The Revised Bloom's Taxonomy?

The revised Bloom's Taxonomy is based upon the cognitive objectives model that was developed in the 1950's by Benjamin Bloom. According to Bloom, there are six levels of cognitive behavior that can explain thinking skills and abilities used in the classroom (and in real life, for that matter):

• Creating
  Encouraging an individual to look at things differently or to generate new concepts or ideas. In Bloom's original taxonomy model this was known as "synthesis". It requires that the student use designing or organizational skills.
• Evaluating
  This requires that learners have a reason behind the course of action that they took and involves experimentation and hypothesis. In this process, the student is asked to critique or summarize information.
• Analyzing
  In this process, learners will have to break down the data that was provided in order to fully grasp the content (as it is now in more manageable parts). This usually requires learners to use comparative and/or deconstruction skills.
• Applying
  This asks learners to use information that they already have gained, in order to solve a problem that may be similar in nature. This involves implementation of prior knowledge and skills.
• Understanding
  Requires that the learners explain the situation or process in order to show that they have understood the materials. This usually involves summarizing, paraphrasing or detailed descriptions.
• Remembering
  The learner's ability to retain and recall information. This usually comes in the form of recognition, retrieving, or listing.

According to the model, the first two categories are typically not ideal for multiple-choice tests, as they don't generally allow for predetermined answers (and are classified as "divergent thinking"). However, the last four cognitive abilities allow for more predictable, concrete answers (also known as "convergent thinking"). As such, they are perfect for eLearning tests that are designed around the multiple-choice structure. It's important to mention that you can also transform a higher level thinking "divergent" question into a "convergent" thinking multiple-choice question if you ensure that it has a concrete answer. For example, you could present the learner with a question that outlines the protocol involved in manufacturing a turbine engine. Then, ask them to describe the next step in the process that you have purposefully left out. This would force the learner to evaluate the process to see which step is missing, which would enable you to determine if they have a firm grasp of the process' summary, rules, and concepts.

Why Use Bloom's Taxonomy For Multiple-Choice Questions?

Tests designed without a structured framework tend to lean on whatever is the easiest for learners to write. That simply means testing their ability to recall. Bloom’s Taxonomy gives instruction designers a proper roadmap to evaluate the effectiveness of their work. Its six levels of thinking can each be mapped to different questions, allowing designers to create multiple-choice questions that can go beyond just checking learner memory. That’s how they can gain a real measure of what someone can do with the material they’ve learned.

Improves Learning Outcomes

Tying questions to the different cognitive levels is an excellent way to gauge where a learner is struggling to grasp a concept. Someone who finds a Remember-level question difficult is struggling in a different way than someone who misses an Apply-level question, even though both show up as an incorrect answer in the test. That distinction gives course designers something concrete to leverage instead of guessing which modules need another look.

Measures Higher-Order Thinking

Recall questions are the low-hanging fruit. They’re easier to write and grade, which is why so many tests are built on them. The problem here is that job performance doesn’t rely on memorization alone. Rather, it rarely does. Bloom’s Taxonomy pushes designers to create questions that go up towards Apply, Analyze, and Evaluate, where learners have to genuinely reason through situations instead of just looking for familiar words, phrases, or concepts. This also helps institutions, teams, and course providers assess whether the intended learner is truly understanding the concept or just recognizing the vocabulary around it.

Creates Balanced Assessments

Any test made entirely of Remember-based questions is easy to pass and barely shows whether the learner can apply their acquired knowledge. Go the other direction, where you overload a test with Evaluate- or Create-based questions, and learners will likely get overwhelmed if they don’t have the right foundation yet. Bloom’s Taxonomy lets you mix up the questions purposefully, keeping the test fair to broader learner audiences while also leaving room for the stronger learners to shine.

5 Tips To Write A Multiple-Choice Test Based On The Revised Bloom's Taxonomy

One of the primary benefits associated with creating tests based upon this model is that the tests will not be unnecessarily difficult for the learner and are more effective in assessing the learner's knowledge of the subject matter. Not to mention that they are easier to correct and to modify. Here are 5 tips that you can use to write multiple-choice eLearning tests:

1. Always Use Plausible Incorrect Answers In The Questions

One of the biggest mistakes that eLearning test creators make is not making the incorrect answers convincing enough. You have to make them plausible so that you are able to test their ability to remember the information and apply it to the problem. This is the only true way to gauge if a learner fully understands the concept.

2. Integrate Charts Into The Exam

Include charts or graphs in your test, which will force the learner to use their analyzing skills. By having them interpret the data, they will be tested on whether or not they have really absorbed the information.

3. Transform The Verb

If you want to include a divergent thinking question on your test, you can generally accomplish this by turning it into a noun. For example, if you are trying to test your learner's ability to describe a scientific process, have them choose the best description for it. This can be used to test both their creating and evaluating skills.

4. Create Examples Or Stories To Test Their Understanding Abilities

Write out detailed stories or examples that the learner must read before answering corresponding multiple-choice questions. This will not only test their understanding ability, but their analyzing skills as well. You can also make the learner tap into their remembering or applying abilities if you create a story or example that asks them to draw upon knowledge they've already acquired.

5. Use Multilevel Thinking

These are the questions that include wording such as "the most appropriate" or "most important". Such questions serve to test the learners' judgment skills or understanding of an in-depth subject. For example, you could ask a learner a question about identifying a particular mental illness by first giving them a detailed explanation of a patient who exhibits a set of observed symptoms, then ask them to apply a particular psychological theory to come up with a diagnosis.

Examples Of Multiple-Choice Questions For Each Bloom's Level

The Taxonomy gets a lot easier to apply when you’ve seen how it works in real life. Here are some sample questions tied to each level, aimed at addressing the kind of thinking it calls for.

Remember

Remember-level questions focus on recall, which means they revolve around asking facts, terms, and sequences, presented in the same way they were taught. For example, asking, "Which of the following definitions talks about onboarding in the context of employee training?” with four brief descriptions. The wrong ones need to appear believable and differ slightly from the correct answer.

Understand

Understand-level questions ask learners to either choose an explanation that captures a concept correctly or showcase it in their own words. An example: “Which argument below best summarizes the purpose of analyzing learner needs before designing a course?” This is where the test goes beyond simple definitions and checks the learner’s ability to comprehend or interpret an idea.

Apply

Apply-level questions place learners in a scenario and ask them to use a specific process or rule to work through it. Here’s an example: “The sales team submits a quarterly expense report missing a few key receipts. Based on company policy, what should the team manager’s next step be?” Such questions directly show whether training translates into real-world thinking rather than just recall.

Analyze

Analyze-level questions ask learners to go through a scenario and take it apart, compare the available options, or identify a pattern that isn’t very obvious. An example would be presenting two customer feedback conversations and asking, “Which sales response better follows our company’s de-escalation protocols, and why?” Pairing analysis-level questions with case studies or charts also works very well.

Evaluate

Evaluate-level questions force learners to interpret and judge a concept: how effective, good, or valid it is against the criteria they have been given. For instance, providing imaginary budget-constraint scenarios and asking, “Based on the budget constraints mentioned below, which vendor proposal aligns the most with our development project’s requirements?” A well-written question at this level will ask the learner to compare various factors simultaneously rather than choosing whichever option seems best on the surface.

Create

This is the toughest level to frame multiple-choice questions for, since creating something new means there have to be open-ended responses. However, you can still frame questions that ask learners to select the ideal approach or plan from a set of options. Here’s an example: “Which of the following outlines the best approach to create a training module on 1:1 feedback for new managers?” It’s not the same as building from scratch, but such questions still suit the multiple-choice nature of the test while gauging the learner’s creative thinking.

Best Practices For Writing Effective Assessments

If you’re wondering how to make test questions using Bloom’s Taxonomy, here’s something to remember: choosing the right level is only one side of the coin. These four best practices outline what truly goes into creating assessments that can measure learning effectiveness fairly.

Align Questions With Learning Objectives

Every multiple-choice question should trace back to a genuine learning objective the course was designed to measure. For example, if an objective was based on the Apply level, the test question must also ask a learner to apply the skill they acquire during the course, not just recall a fact or statement tied to it.

Use Real-World Scenarios

A vague question that simply talks about the hypothetical isn’t enough. It rarely tells you how much someone has learned and how well they can really perform at work. Including real-world scenarios, such as situations they’re likely to face or have faced before, gives learners a reason to apply what they’ve learned.

Review Questions For Bias

Bias often comes into test questions without you realizing it: phrasing that suits certain audience groups, a scenario that stronger learners can interpret better, or certain cultural references others may not recognize. Hence, before rolling out an assessment, an external member must analyze the questions for fairness and clarity.

Pilot Test Your Assessment

Running the test on a small pilot group before the entire learner audience gets it helps you catch misses or errors beforehand. You can spot questions that are too easy or confusing, or those that might address concepts yet to be covered in the course.

Conclusion

Remember that many multiple-choice tests are works-in-progress. You must be willing to reassess them every now and then to ensure that they are still effective in determining a learner's knowledge retention and understanding to the subject matter. If need be, rework faulty questions that may be too easy or too difficult, and don't be averse to changing your instruction if you determine that that may be the root of the problem.

Bloom's Taxonomy isn't the only Instructional Design Model to consider. Find out everything about ID models in our ultimate guide Successful Instructional Design Models: Top ID Skills And Strategies For Newcomers. It's perfect for both beginners and professionals in the field who want to stay up-to-date on the latest ID skills and strategies. You'll find out what makes an ID model successful, steps in their design process, and other tips to help you choose the right one and apply it effectively. Also, you'll discover top ID skills and knowledge to make you a master. Secure your copy today and leverage this resource to design like a pro.

References:

• Aiken, Lewis R., (1982). Writing multiple-choice items to measure higher-order educational objectives. Educational and Psychological Measurement, 1982, Vol. 42, pp. 803-806.
• Bloom, B. S., Englehart, M. D., Furst, E. J., Hill, W. H., & Krathwohl, D. R. (1956). Taxonomy of educational objectives. The classification of educational goals: Handbook I. Cognitive domain.New York: David McKay.
• Bloom's Taxonomy, Retrieved from Wikipedia 01/22/2014, http://en.wikipedia.org/wiki/Bloom' s_Taxonomy
• Revised Bloom's Taxonomy, Retrieved from UTAR 01/22/2014, http://www.utar.edu.my/fegt/file/Revised_Blooms_Info.pdf
• Teaching with the Revised Bloom's Taxonomy, Retrieved from Northern Illinois University 01/22/2014, http://www.niu.edu/facdev/programs/handouts/blooms.shtml