The Importance Of Learning Evaluation And Training Needs Analysis To Measure L&D Success
This article is part of a series focused on the Seven Elements of Needs Analysis. The Elements are: Curriculum Mapping, Know/Do/Believe, Defining the Audience, Designing a Learning Brand, Defining the Return on Investment, Building the Team, and Scoping the Project.
What you are about to read is a fable. The company, AshCom, is fictional, but the learning challenges faced by Kathryn, AshCom’s CLO, and her team are real and commonly shared by learning teams in corporations, non-profits, associations, and education. It is our hope that you will be able to connect with the characters, their challenges, and the solutions they discover. Building and following a Needs Analysis system is vital to the success of the learning opportunities we create.
Training Needs Analysis And ROI: The Last Minute Meeting
Mondays were hectic. While Kathryn prepared for a day of back-to-back meetings, a text notification interrupted her thoughts. It was Amy.
We need to talk. Do you have an hour today to discuss the acquisition?
This was not how Amy normally did things. Kathryn’s day was already scheduled out, but she didn’t want to make Amy wait. Something important must be on her mind.
Kathryn texted her back.
I can squeeze in a working lunch. I’ll have salads delivered. Sound good?
Amy responded right away.
That would be great. See you then.
Amy was not technically an employee of AshCom, but she was a trusted member of the L&D team. As the Chief Learning Officer, Kathryn brought her on as an ongoing consultant committed to 15-20 hours per week on a two-year contract that would expire in three months’ time. Kathryn had every intention of renewing it.
Amy had other clients. She worked with some of the largest companies in Minneapolis and across the country. She also had five years of experience as a learning leader and instructional designer at a pharmaceutical company in Kalamazoo, Michigan, and the next 18 years she consulted on learning, training, and development projects in various industries. Amy knew the trends and had multiple touchpoints for benchmarking what other companies were doing. In a pinch, she would also jump into instructional design work.
Amy arrived promptly at 11:45 a.m. with a burgundy folder in her hand. She sat down and pulled out a paper copy of the 2020 Business Chief EU article, “Why do up to 90% of Mergers and Acquisitions Fail?”
Game-Changing Insights
The article was short, but Amy had taken the time to run a yellow highlighter over the opening paragraphs.
The first simply stated that:
According to collated research and a recent Harvard Business Review report, the failure rate for mergers and acquisitions (M&A) sits between 70 percent and 90 percent.
The second read:
Mergers and acquisitions should never be taken lightly. Not only are you asking two companies to integrate under one corporate mission, but you are bringing together large groups of people with their own personalities, ambitions, behavioral traits, and ways of working.
Amy gave a little background. She had seen some acquisitions that worked, but she had also seen more failures that were miserable for all involved. So many systems needed to come together well for it to be successful, and often, learning teams were essential to the outcome.
As Amy spoke, Kathryn could feel her anxiety level elevate. When she thought about her learning team and the progress they were making, she was confident. When she considered the big picture and what was a stake, which usually happened at night, she was restless and somewhat unsettled.
Part of Amy’s value, beyond her expertise, was that she truly cared for Kathryn, the learning team, and the learners at AshCom (including those who would be joining them from Globex). Amy felt responsible as part of the team, something she communicated by spending the weekend reading articles on mergers and acquisitions.
“That,” said Amy, “is what prompted the early morning text. Sorry if I disturbed your normal morning routine, but I needed to talk with you.” Kathryn was accurate in her assessment; this was important to Amy and that made it important to her.
“Okay,” Kathryn said with a smile, “you have me all amped up while I’m trying to eat my lunch. I’ve worked with you for almost two years, and I believe I’ve come to know your patterns. You aren’t the kind of person who brings up a potential problem unless you’ve also got thoughts on possible solutions. So, calm my stomach down and tell me what’s on your mind.”
Amy appreciated Kathryn’s directness and trust. “What I’m about to tell you isn’t just a problem when two companies merge or one acquires another,” Amy said. “I’ve seen this in almost every company I’ve consulted in with a few exceptions.”
Measuring ROI
Feeling a little emboldened, Amy walked to Kathryn’s whiteboard and wrote “ROI: Return on Investment.”
Still standing while Kathryn finished her salad, Amy said, “Demonstrating a solid return on investment might be one of the most important tasks of a corporate learning team. But it is often given somewhere between little and no attention, at least as learning needs are being analyzed and systems are being renewed or built. It isn’t easy to do at the start, and so it is often ignored at the beginning of a big initiative. This can be a costly mistake, as I’ve seen in other companies. AshCom’s acquisition of Globex raises the stakes even more.”
Kathryn responded, “Amy, I’ve really come to appreciate your expertise and care for our company. I also respect your ability to speak the unvarnished truth with just the right amount of graciousness. So, I have to ask, looking back at the projects that have involved you, what grade would you give us for clarifying, measuring, and reporting our ROI to the owners of the company?”
“B-,” said Amy, like she had anticipated the question and had already given it some thought. “AshCom’s learning has done enough in ROI to be successful and justify to the owners the budget that we get each year. But with the acquisition, the budget is going to go up along with the expectations. I like where we are headed. As a matter of fact, I can’t think of many learning leaders who have done a better job of getting their head around the learning needs of the company than you these last weeks.”
“But…,” Amy hesitated.
Kathryn finished, “but we are not where we need to be on ROI. Or maybe to put more positively, we are not where we could be.”
“Yep,” said Amy. “That’s how I see it, and I think now is the time to be asking the ROI question rather than waiting for someone else to ask and not being able to answer.”
“I’ll assume you’ve done this before at the pharma company when you were on that team and with many of the clients you’ve had over the years. If you could put this in one question—and I’m going to ask you to do just that on the whiteboard—what would that question be?”
Defining Success
This time, Amy didn’t hesitate. With confidence, she wrote, “What does success look like?” She underlined the word “success” somewhat unnecessarily.
Amy explained, “This same question applies on multiple levels. For each module or course we develop, we need a clear answer. Same with entire series of courses. What I am asking has to do with the entire learning program at AshCom.”
“We are not starting with nothing. We have the Curriculum Map which helped us connect our learning plan with AshCom’s business objectives. Also, we have the three buckets of KNOW/DO/BELIEVE. We are gaining a solid understanding of our learners, although we still have work to do with the Globex people who will be joining us. The planning so far will help us prevent problems, solve problems, and create opportunities.”
Amy continued, “My question is this: How we will measure those things? Or more specifically, by what metrics will we be measuring? We should also be thinking about when we do our measurements. How we measure will matter too. We will need to do that for the learning we create, but we also need a system that puts those individual course metrics into a larger metric so that we can demonstrate and quantify what we did and what difference it made in achieving AshCom’s goals. How was everyone involved successful?”
Kathryn had finished her lunch. With only 10 minutes remaining in the hour, she asked Amy to map out at least her big ideas on tracking ROI. Amy walked Kathryn through what she called the 5 Levels of Evaluation, based on Kirkpatrick’s 4 Levels which considered both quantitative and qualitative metrics to give a fuller picture of the impact on learners and, ultimately, AshCom.
Amy knew time was up and that Kathryn needed to move on to her next meeting. “I really appreciate you taking the time to spend an hour with me and hear me out. Later today, I will send a chart that will give you a visual of the 5 Levels. It will get a lot clearer for you and for the team if you choose to share it with them.”
“Trust me,” replied Kathryn, “this will be the topic of our next team meeting in two days. I’m glad you spent the time thinking and reading. I also appreciate your willingness to push me and be honest with me. I look forward to seeing the visual.”
When Amy returned to her cubicle, she forwarded the following resources to Kathryn...
Needs Analysis: Designing The ROI - What Will Be Accomplished, And How Will You Know?
Conclusion
To read the rest of the chapters in this series on Needs Analysis and to see Kathryn and her team take on the other Essentials, please download the eBook How To Change The Way You Approach Needs Analysis: A Story Around The 7 Essential Elements. You can also join the webinar to nail your training needs analysis and develop a culture of continuous improvement.
References
Why do up to 90% of Mergers and Acquisitions Fail?
