Redefining the ROI of Corporate Learning

In an era of economic uncertainty and belt-tightening, organizations need to be able to show the impact of the work they’re doing in all areas. Corporate learning is no exception. 

Classically, showing impact meant “calculating return on investment,” or ROI. This is nothing new to learning leaders, as ROI has largely been treated as synonymous with Kirkpatrick’s 4th and final level of training evaluation. And of the four levels in that evaluation scheme, it’s the only one that speaks to the interests of organizational stakeholders. 

But evaluating such returns is easier said than done. Individual industries, organizations and teams have their own specific benchmarks and metrics such as efficiency metrics, revenue quotas, market share and so on. Plus, some types of effects may be easier to calculate than others, either by their nature or because the data is more readily available (for example, measuring how cost-effective marketing campaigns are before and after training, and examining trends in course completions). But for as many ways as there are to define what a “return” on learning is—usually lashed closely to a training budget—there are many ways to run different numbers to reach conclusions. 

Despite the challenge, measuring ROI matters. It’s crucial to driving and determining the success of learning and development (L&D) programs, which includes all programs that employees may engage with, not limited only to learning content developed in-house. Organizations need to push past the “feel goods” and figure out bottom-line impact. But how are modern-day organizations approaching this task? 

We wanted to find out. So in 2023, D2L and Training Industry, Inc., conducted a survey of 329 learning leaders and asked how their organizations measure the impact of the learning programs they deliver to employees. In this report, we’ll share insights from this survey and discuss: 

• how organizations rate their learning programs’ effectiveness 
• the ways in which they define and measure success 
• what the future looks like for assessing the impact of corporate learning initiatives 

The Rubik’s Cube of ROI

Measuring the effectiveness of and demonstrating clear ROI are common challenges for most organizations, but it isn’t enough to state that as an accepted fact. There are many different types of challenges that organizations face when it comes to learning strategy. In that light, the question we wanted to ask was how common of a challenge is measuring and demonstrating ROI? 

As seen below in Figure 1, almost half of organizations noted that demonstrating the impact of learning programs was one of their biggest challenges—second only to learner engagement. Historically, the most common L&D-related challenges faced by organizations involved familiar aspects of keeping training consistent across the enterprise and working with limited resources. What Figure 1 suggests is that in the wake of the COVID-19 pandemic and all the reshuffling of strategic priorities that occurred, what’s top-of-mind for learning leaders extends beyond mere consistency or resourcing concerns. Whether an L&D initiative can be pulled off has been replaced for many by questions such as: How effective is the training we’re delivering to employees? Is it having a tangible impact? 

What our data showed is that ROI measurement isn’t a higher-order challenge only affecting those organizations that have solved most of their L&D issues. It’s also not something typically faced by those just beginning to grapple with rolling out programs across the enterprise. The challenge of demonstrating impact is something that touches many organizations regardless of other circumstances. As shown in Figure 2, roughly a third of the learning leaders who completed our survey indicated they faced three challenges from the list presented in Figure 1. 

Digging into our data further, we wanted to know if there was a pattern in the number of overall challenges in an organization and whether ROI was included among them. What we found was that when an organization only identified one or two challenges, ROI was the culprit in 11% of those instances. On the other end of the spectrum, when a company identified between three and six challenges, ROI was included in 66% of those cases. Also worthy of note is that there were no differences in ROI being a challenge based on the size of the organization—45% of smaller organizations identified this as a challenge compared to 48% of larger organizations. In other words, ROI is an equal opportunity struggle. 

But what is it that organizations are ultimately trying to get out of their learning programs? Figure 3 shows the factors that drive investments in corporate learning, as endorsed by learning leaders in our survey. What our data shows is that there are impact endpoints, whether they’re about culture, compliance or technology investments, that underpin the resources organizations devote to employee learning. While these endpoints might seek different information, they’re all arguably some form of ROI—confirmation that the enterprise is getting what it expects from what it’s putting in. 

That brings us to another crossroads: What is ROI, exactly? Well, we asked. Several learning leaders represented ROI as a simple formula: 

(net benefits – total training cost) / total training cost 

While this is close, it exposes how some in L&D might miss factors across their learning programs, such as ongoing versus one-time costs. The equation for ROI suggested by D2L is as follows: 

((annual program benefit – annual ongoing program costs) / one-time setup costs) x 100 = ROI 

This equation is intended to be inclusive, as it’s applied to measuring the effectiveness of programs as a whole across the enterprise, and not just the ones where the learning programs are developed in-house. Learning ROI applies to everything—whether a program is homemade from scratch by the L&D team, bought off-the-shelf, custom built by a third-party provider or some combination of each. 

But the halfway-there formula above wasn’t the only way our survey respondents characterized how ROI is treated at their organizations. Here are some of the other working definitions of ROI that learning leaders provided when prompted: 

Learning leaders have different thoughts about what constitutes ROI, which makes it unsurprising that they would frame it as an active challenge (Figure 1). Tools are available to gauge the impact of L&D—such as D2L’s ROI calculator—but are organizations capturing their outcomes? In the next section, we’ll explore what organizations are doing when it comes to learning metrics. 

A Reality Check on Learning Metrics

Most organizations are seeking better metrics as part of their future strategy. But let’s step back for a second, because “better” is relative to what organizations are currently doing. So, what are they doing? 

Figure 4 provides a breakdown of the many different types of learning outcomes and metrics that can be collected in the course of training evaluation and provides a snapshot of what’s currently being used at organizations. The figure has been organized to roughly correspond with familiar evaluation models such as Kirkpatrick. While it’s noteworthy that over 50% of organizations are capturing some form of level 1 and level 2 metrics—those that measure learner reactions and substantiate knowledge or skill acquisition—collecting metrics is hardly a universal constant from organization to organization. Only 33% of the learning leaders in our sample indicated that the metrics they collect include ROI or some form of assessing the impact of learning programs in a financial sense. 

Again, only 33% formally assess ROI. Of course, some of the endorsement of gauging the impact of employee learning on business goals or outcomes might fall into the same territory or be treated as the primary marker of L&D impact—but that’s not quite synonymous with ROI. Based on Figure 4, one might think the majority of organizations are struggling with their learning programs. In the next section, we’ll discuss the extent to which that’s a reality. 

Minding the Gaps

Despite the gap between the depth of metrics that get collected—ROI in particular—most organizations still say their learning programs are successful. 

Learning leaders at 61% of organizations indicated their learning programs are typically effective (Figure 5). On the other end of the spectrum, only 9% gave their programs poor marks. The two highest percentages in Figure 5 form a middle ground where—at least some of the time—training works, but it might be plagued by issues or face challenges with accountability or impact. 

Where might some of this variability come from? Where are the majority of organizations faring better? To dive deeper into this issue, we also collected effectiveness ratings parsed by delivery modality (Figure 6). Organizations are slightly better at some types of delivery than others, and while the margins may not seem large, they’re meaningful at both ends of the spectrum.  

Let’s take the example of eLearning, which 61% of organizations are effective at delivering. While this seems to be tracking with the overall effectiveness ratings from Figure 5, the disparity lies at the bottom of the spectrum where 13% of organizations say they’re not effective. This 4% disparity compared to the lower end of Figure 5 is one of several areas where organizations can be disjointed across learning initiatives or delivery modalities. Perhaps an organization can provide a pretty decent in-person classroom experience but falls short when it comes to eLearning, and this is what we’re seeing manifested in the numbers. 

Strengths and weaknesses within an organization’s corporate learning orbit are bound to happen. A logical question would be whether the organizations these data represent are new to implementing their own training. This would mean less time to realize a learning strategy, align resources (both internal and external) and execute on that strategy. In part, this could explain the gap between the proportion of organizations with effective learning programs (Figure 5) and how many of those programs are likely to include metrics centered on business outcomes and financials (Figure 4). However, this data is coming from organizations that are relatively mature (Figure 7). 

So, what are we to make of all this? Let’s look at the results through a slightly different lens. If 33% of organizations measure ROI (Figure 4), but only 26% say their learning programs are always effective (Figure 5), we can begin to parse the story in a different way. We know in general that we’re dealing with mature L&D programs, they vary in effectiveness across delivery modalities and most of them are collecting an average of four or five evaluation metrics. It can’t be that the 26% of organizations whose learning programs are always effective overlaps perfectly with the 33% that measure ROI, because Figure 8 shows that is patently not the case. 

From what we can see in Figure 8, certainly there is a pattern of organizations that do collect ROI tending to have much more effective corporate learning programs. While this doesn’t doom those that don’t calculate that metric, it certainly doesn’t guarantee success. Recall that in Figure 1, we showed that 46% of organizations tagged ROI as one of their foremost challenges when it comes to corporate learning. Of those organizations, 40% currently do not measure ROI (at least not yet). But that also means that among those companies collecting ROI, it’s not automatically going smoothly. Does that suggest their training is not effective? Not necessarily—but it implies that even when that’s the case, learning leaders might not have a clear picture of the impact their learning programs are ultimately making. 

How Organizations Can Measure ROI

If an organization doesn’t have a means to assess ROI, what is a viable starting point? Below, we’ve suggested best practices that are based on tackling the biggest challenges learning leaders have specifically with ROI. If learning organizations can wrestle with these practices, they’ll likely leapfrog over the competition in terms of understanding how to optimize corporate learning programs. 

Consistently Measure Program Benefits

Getting a general feel for what benefits come out of learning programs isn’t hard—again, research showed the average organization measures four or five different types of outcomes. But when it comes to demonstrating ROI, it takes committing to the data collection. Investments in employee learning aren’t made once and never thought of again, and ROI can be similarly dynamic. In other words, snapshots are useful, but trends are actionable. Even if the benefits that can be measured consistently don’t include everything, the best place to start is with what can be repeatedly assessed. 

Understand Variable Costs

There’s good reason that variable costs are one of the inputs in D2L’s ROI calculator. Fixed costs are easy to nail down, it’s the variable costs of learning that can upend ROI if they’re not managed. Put simply, variable costs are costs that increase (usually) or decrease depending on the volume of learning activities changes. Examples of these are things like materials, direct and sourced labor, or transaction fees. If a cost depends on how much or how little is going on in L&D, it needs to be monitored closely to realistically appraise ROI. 

Calculate Productivity and Efficiency Gains

The first step to having valuable ROI insights to share with stakeholders is trying to crunch the numbers. Sounds almost too simple, right? But if the actual calculation steps of ROI are never actually completed, an organization will never have a sense of where it stands—even if confidence is lacking in the metrics being shared with stakeholders. Having a baseline—even an imperfect one—allows organizations to react to it, either by making changes in L&D processes or tweaking the inputs to the ROI calculation. Until there’s something to react to, organizations are in the dark about ROI, even if the qualitative information about program benefits exists. Don’t wait until the right formula is presented or until a deadline comes up on the calendar—try as soon as possible. 

Recognize Disruption Costs

Despite careful planning, cautious integrations, diligent pilot programs, and best-in-class L&D providers and tools, L&D exists in the ebb and flow of the rest of the organization. To that end, employee learning is subject to the forces and pressures that govern day-to-day and quarter-to-quarter business operations.  

Disruption can come from all sorts of angles, such as losing experienced L&D staff, changes in the competitive market, switching providers or trusted partners that provide learning content or services, and more. The point of getting a handle on disruption costs isn’t to provide fodder for excuses, but to permit some degree of “look ahead” at how a disruption will affect L&D operations and what changes can be made to support ROI through the rough patch. 

Identify Behaviors That Can Be Monetized

The problem with many so-called metrics is that there’s no straightforward way to monetize the resulting data. Organizations seeking clarity on learning program ROI need to focus on the things that are tied to L&D costs and their subsequent revenues or savings. While standard learning metrics can provide valuable insights into employee perceptions of training, these same metrics often lack a clear connection to the organization’s bottom line.  

What outcomes should organizations turn to? Some examples are improved sales figures, increased customer satisfaction ratings, reduced error rates or similar endpoints that may or may not already be key performance indicators for the organization. What’s crucial is to have timepoints, so that learning program changes can be tied to changes in these outcomes. 

Conclusion 

In summary, 46% of organizations say ROI is a challenge, yet only 33% of organizations say they measure the financial outcomes of corporate learning. But 61% of organizations say that their learning programs are effective. 

What it boils down to is how effectiveness is defined for training strategy. Organizations want learning experiences that employees will react positively to and that they can learn new knowledge and skills from to improve their job performance—but the context matters. Regardless of the type of organization, number of employees or industry, corporate learning programs need to make an impact on the organization to justify their expense. No training is ever truly free—think about salary hours, opportunity cost, resources and systems. Most of us would agree that a fantastic restaurant with no idea of its profits or operating costs sounds like an irresponsible business plan. Why would an L&D function run under different rules? While that may sound a bit pessimistic, it’s possible to both have your cake and eat it too: Learning programs can provide a positive ROI and drive employee engagement and commitment. 

It’s not a secret in L&D that effective employee learning can impact desirable business outcomes such as employee engagement and retention, help attract new talent, nurture career growth and pathways, and help organizations avoid replacement costs for critical roles and address skill gaps. Does demonstrating ROI automatically mean the needle moves on these outcomes? Not necessarily. Sometimes the effect may be indirect or elusive. For example, ROI might be easy to calculate for programs that are delivered with the aid of a trusted partner or provider, yet difficult to assess when a program was created by an internal L&D team, or vice versa. But without a sense of what the organization is getting for its investments across all learning programs, there’s no way to gauge whether these programs are comparably strong or just masquerading as such based on opinions and anecdotes. 

Right now, measuring success might mean leaning into available qualitative feedback until ROI metrics can be brought up to speed. The value of qualitative information about learning isn’t going to disappear, and it’s still an important piece of the narrative about the overall value of corporate learning. The gaps organizations want and need to fill are in building out the quantitative evidence to show that corporate learning programs are cost-effective and deliver a positive ROI.  

Ready to maximize the return on your learning investment?

Amplify your workforce’s onboarding speed, job satisfaction, retention and internal mobility with D2L for Business.

LEARN MORE

Written by: