How To Evaluate The ROI Of Your Learning Programs
This article is part of a series on tracking the right metrics using the right methodology to visually display and quantify the investment in learning and prove ROI. What you are about to read is a fable. The company, AshCom, is fictional, but the learning challenges faced by Kathryn, AshCom’s CLO, and her team are real and commonly shared by learning teams in large organizations. It is our hope that you will be able to connect with the characters, their challenges, and the solutions they discover. We also invite you to read the first eBook in the series.
The Learning Circle
AshCom’s main lobby door felt heavy. A heavy that Amy, a frequent visitor, didn’t remember before today. Her stomach was a tight, hard ball. Her mind jumped through the different scenarios of her upcoming meeting. She steeled herself for whatever she was about to learn.
Amy was both inside and outside the learning circle at AshCom. She had been active on their learning team for several years and was familiar with each member. Her relationship with Kathryn, AshCom’s Chief Learning Officer, blossomed from professional to friendship. But Amy remained a consultant to Kathryn and was not an employee. She continued to serve learning teams in a variety of other companies and non-profit organizations. Her client base was mostly made up of large organizations.
Kathryn appreciated Amy’s professionalism and experience, which was why she hired her as a consultant years before. One benefit Kathryn had not fully realized when Amy came aboard was her knowledge of the trends among learning and development leaders. Amy provided a more global perspective. Her ability to benchmark what Kathryn and AshCom were doing compared to what other organizations were doing made Amy invaluable to Kathryn.
The Unvarnished Truth
In addition to having the perspective of an outsider, Amy could hold things in confidence. What Kathryn might not be willing to share with the team who worked for her, she could share with Amy with no fear that their conversations were anything but private. Sometimes Kathryn simply needed to bounce an idea off someone without causing any drama or starting the rumor mill rolling. Amy was that person for Kathryn.
Amy was brutally honest with Kathryn. She called it her “unvarnished truth.” Amy understood that Kathryn was strong enough to listen to what she might not want to hear. Kathryn was grateful to be given Amy’s perspective without wondering if she was hearing everything she needed to hear.
When Kathryn emailed Amy that she needed an hour, “the sooner, the better,” Amy knew something was wrong. She was around AshCom enough to have heard the rumors. AshCom was struggling financially, having lost money in the previous quarter for the first time in its history. From the sound of things, the losses were continuing.
Amy cleared her schedule for the following morning, suspecting the conversation might be longer than 60 minutes. She was wise to do so.
“Thanks for making time for me so quickly,” said Kathryn when Amy entered her office. “Please close the door.”
Amy studied Kathryn’s face for a moment. She had always known her as a strong individual. Unflappable even. Kathryn was strategic and rational. She didn’t much care for overly emotional situations. Clearly, those tendencies were failing her this morning. It was obvious she had not slept well the previous night.
As Amy sat at the small table in the office, Kathryn asked, “So what have you heard? How much do you know?”
“Those are two different things,” replied Amy. “I’ve heard a lot, but I’m not sure how much I know because I don’t know if what I’ve heard is actually true.”
“Fair enough,” said Kathryn. “What have you heard?”
A Financial Microscope
“It seems AshCom had a terrible financial last quarter,” replied Amy. “I’ve heard that the losses have continued into the past two months and that this quarter is not likely to show any improvement. I’ve heard that every department is under a financial microscope.”
“That’s all true,” said Kathryn.
“I’ve also heard about something Kurtis your CFO is calling ‘Defend the Spend,’ which I assume is the name chosen for the microscope,” said Amy.
“I told Kurtis that that name might not have been the best choice. He was proud of himself for making it rhyme, which I suppose counts for something,” said Kathryn smiling weakly.
“And we both know what that means for your learning team,” said Amy. “I’ve seen it every time a company struggles. The first job cuts usually happen on the learning team.”
“I’ve seen that too,” said Kathryn. “Some part of me understands that approach. We have built a lot of learning assets for this company over the years, and the benefit of those learning experiences will carry through for a long time. If I had Kurtis’ job, I suppose I might do the same. I likely would not cut jobs from the production or sales team. If they stop, the impact is large and immediate.”
“You are worried that you will lose some members of your team?” asked Amy, more as a statement than a question.
“That’s the weird part,” replied Kathryn. “I met with Kurtis yesterday. He told me that there would be no budget cuts or personnel layoffs on the learning team.”
“Hmmm...,” said Amy. “Between the rumors, what usually happens in these situations, and the look on your face, I thought we would be having a very different conversation. I’ll tell you what I thought when I received your email. I assumed we were meeting for you to tell me that I was done as a consultant and that you would be letting go of several people on your learning team. You and I have become friends, so I thought you wanted to do this in person.”
Kathryn’s face showed some surprise. “I never thought of it like that, and I’m sorry if that was the impression I gave you.”
“No worries,” said Amy. “But if this isn’t THAT conversation, what are we talking about? What did Kurtis ask of you in the meeting?”
Creating New Learning Experiences
“That is where this gets a little muddled,” said Kathryn. “He wants my team to create an entirely new set of learning experiences around preventative maintenance. It turns out that this is where we are losing a lot of money.”
“Machines not being properly maintained,” said Amy. “I’ve seen that before too. Failing to meet production goals. Too much unscheduled downtime for machines. Missing spare parts. Scrap rates that are too high. I can help with that.”
“You certainly understand the topic,” said Kathryn. “But that isn’t the part of the conversation with Kurtis that is giving me heartburn. He wants us to use this topic to build a system that measures our return on investment, starting with preventative maintenance, that will eventually cover every learning experience we offer.”
“Defend the Spend,” said Amy. “Now I’m seeing it. I’m sure your team can create a great learning experience around preventative maintenance. It is the ROI that’s causing your anxiety. And that’s where you need my help. And why you asked me to close your office door.”
“I don’t want you to think that Kurtis was in any way threatening,” said Kathryn. “But it was clear that we need to figure out ROI for everything. At least he didn’t try to make us create a system for every learning asset. He’s asking us to start small and figure out something that can scale.”
“You and I have talked about this before,” said Amy. “Last time we spoke about ROI, I told you that figuring this out is one of the most important things a successful learning team can do.”
“I remember,” said Kathryn. “But you also told me it usually receives somewhere between little and no attention and that this can be a costly mistake.”
“I also asked you to grade yourself on how well you were doing,” said Amy. “I remember the grade you gave yourself. Do you remember?”
Making The Grade
Kathryn smiled warmly for the first time in the meeting. “I gave us a B-. Now that I think of it, it was actually you that gave us a B-. I have a feeling you were being abundantly gracious.”
“You know better than I do,” said Amy, now laughing.
“Seriously though,” said Kathryn. “I can’t say we are any better than a B-. We may have gotten worse. We’ve had a lot of projects thrown at us that were mission-critical, and we haven’t taken the time to actually work our way through how to do this properly. Kurtis pointed out that we have had a huge increase in our learning over the last couple of years—a bigger increase than any other department by percentage, he pointed out.”
“That can be a mixed blessing,” said Amy. “More budget brings a much bigger spotlight. And now the company is financially struggling, so the spotlight is even brighter.
Amy paused a moment, choosing her next words carefully. “I have been around L&D for more than 20 years, as you know. I remember back when the markets crashed in 2009. A lot of my friends and colleagues in learning lost their jobs. You have a great team. I would really hate to see that happen here.”
“I own this, don’t I?” asked Kathryn, not really expecting an answer. Her stress levels were clearly rising.
“You do own this, and I don’t think your team will be all that helpful here,” said Amy. “But I want to offer some hope. I’ve worked through this with other clients, and I can help you. We need to take this one step at a time. For now, don’t worry about making an entire system that will measure the ROI for everything that you do. Kurtis was wise for asking you to focus exclusively on the results of the preventative maintenance learning you are about to build. Let your team work on the learning content. They are talented people, so trust them. You and I will work on the return-on-investment portion. I’m confident we can get to where you need to be.”
“Thank you,” said Kathryn. “I’m grateful for your insight and understanding. For your friendship, too. Can we meet again tomorrow? I really have to get to another meeting.”
“My schedule is full for tomorrow,” said Amy. “How about Friday morning? We can start at 8 a.m. here and work through 10 a.m.”
“Wonderful,” said Kathryn. “I’ll make sure we have something good to eat, and I’ll clear my schedule. I’m looking forward to it.”
Conclusion
Download the eBook The Learning Scorecard: How To Measure L&D Impact And Prove ROI to delve into the data and discover which key metrics your L&D team should consider. You can also join the webinar to discover a completely new approach to measuring ROI.
Dear Reader, if you would like to see a demo of MindSpring’s Learning Scorecard, please click here to schedule a time convenient for you and the learning experts at MindSpring will be happy to walk you through it.
