Since 2015, global spending on learning and development (L&D) has consistently exceeded $350 billion annually, yet despite this substantial investment, many  organizations don’t track long-term training outcomes, and a significant portion of L&D efforts are guided by outdated evaluation models.

For decades, the training industry has relied on oversimplified, traditional frameworks to evaluate training outcomes. However, as business needs continue to become more complex, these traditional models alone are no longer sufficient. Enter the International Organization for Standardization (ISO) and their new guidelines for training and development metrics, which offer a more comprehensive and modern approach to training evaluation.

This article explores the limitations of traditional models, introduces the ISO standards and provides practical insights for adopting these new guidelines to enhance training measurement.

The Limitations of Traditional Measurement Models

Traditional models, with their check boxes and levels, have long been the go-to for training evaluation, despite the fact that they were developed before the arrival of personal computers, the internet, eLearning platforms, mobile learning and advanced analytics tools. As a result, they often provide a very simple, broad framework for assessing the effectiveness of training programs.

Tom Whelan, Ph.D., director of research at Training Industry, explains, “One of the limitations is that those models were created in a world of scarcity, where the amount of information you had on training was very low and you had to find it yourself. Whereas now, there’s so much information that we’ve gone from ‘Do I have anything I can use?’ to ‘How do I separate signal from noise to actually demonstrate something of value?’” Modern training requires modern measurement methods that can adequately capture the complexity of today’s learning experiences and diverse learner needs.

Parker Donnafield, CPTM, a learning consultant at Canvas Credit Union, echoes this sentiment, saying that, with their prescribed levels and categories, older models don’t offer the flexibility needed in today’s learning environment. “Each level is meant to show a different success story, but people can get stuck in those ‘requirements.’ Analytics used to be very rigid, but now it’s more flexible and you can make a story out of the data more easily,” he notes.

This rigidity often results in an incomplete understanding of training effectiveness, preventing organizations from making meaningful improvements. As a result, there has been a growing need for more adaptable measurement methods that better reflect the flexibility and diversity of today’s training and development efforts.

The Emergence of ISO Standards

Recognizing this need for a more nuanced approach, the International Organization for Standardization (ISO) introduced the ISO 30437 standards for L&D metrics. These standards provide a much-needed framework for learning professionals to approach training measurement in a more comprehensive way. This framework does away with levels in favor of three considerations for developing a measurement and analytics strategy:

1. Who needs the metrics?

2. Why is training being measured?

3. What is being measured?

These standards can be thought of as a “5-4-3 framework,” representing five types of end users, four reasons for measuring training and three types of metrics.

The 5-4-3 Framework of ISO 30437

Let’s take a closer look at each of these categories and how they influence learning measurement and reporting.

5 Types of End Users

Understanding who will be utilizing your data will help tailor your measurement strategy to meet the specific needs and expectations of various stakeholders. This understanding ensures you collect relevant data that provides the most useful information for each group’s decision-making needs and demonstrates training’s value across different levels of the organization. Depending on the user, there are different reasons to measure training activities and different metrics of importance between stakeholders.

4 Reasons for Measuring Training

Identifying the reasons for measuring training in the first place will clarify the purpose for collecting the measurement data. This helps select appropriate metrics and reporting formats that align with organizational goals.

The primary reasons for evaluating training:

1. Inform: Provide straightforward answers about learning, such as usage trends or number of offerings.
2. Monitor: Contextualize metrics against historical benchmarks (e.g., changes in engagement metrics over time).
3. Evaluate: Classic program evaluation, encompassing efficiency, effectiveness and target outcomes.
4. Manage: Use a suite of metrics to drive ongoing improvements in training.

3 Types of Metrics

Categorizing metrics into efficiency, effectiveness and outcomes ensures a balanced approach to data collection and analysis. This view allows organizations to assess not only the quantity and quality of training, but also its impact on overarching business goals. All three factors are essential to impactful training programs that bring the most value to the business.

Examples of specific metrics for each category:

1. Quantity (Efficiency): Metrics such as the number of courses, learners, costs and utilization rates.
2. Quality (Effectiveness): Metrics comparing Kirkpatrick’s levels and Phillips Level 5 ROI, including learner reactions, amount learned and application rates.
3. Goals for Change (Outcomes): Metrics tied to organizational impact, like increased sales or reduced cybersecurity incidents.

The 5-4-3 framework shifts the focus from a level-based approach to one that considers the specific needs and contexts of different stakeholders.

Barriers to Adopting New Measurement Methods and How to Overcome Them

1. Resistance to Change

Despite the clear benefits of the ISO standards, many organizations are reluctant to move beyond traditional methods. Chris Massaro, CPTM, an HR manager and instructor for Training Industry’s Learning Analytics Workshop, explains, “Any conversation you have about organizational change, resistance is always going to be there.” This resistance is often rooted in the familiarity and perceived simplicity of existing models.

To combat this, Massaro suggests shifting the mindset: “You have to move from this idea of ‘I’m measuring because we have to’ to ‘I’m measuring this because I want high impacts to be able to make changes, show value and show ROI.’” Without utilizing a robust evaluation and analytics process, he says, you won’t be able show how training brings value to the business.

2. Reluctance to Put in the Time

Another barrier can be that people are overwhelmed by having to collect new data for a new system. Massaro advises tapping into existing data to help build your case. “There’s already data out there in your learning management system, your human capital system, your performance improvement system. You don’t need to start from scratch,” he says. By leveraging existing data, organizations can make a more compelling case for transitioning to comprehensive frameworks like the ISO standards.

3. Stakeholder Buy-In

One of the keys to successful adoption of new measurement frameworks is stakeholder engagement. Massaro advises, “Engage them early on in the evaluation process — because if you have them in the beginning, they’re going to see a connection not just to measurement, but to the training and the entire process. Then you’re going to have absolute buy-in and a change champion.”

Donnafield also highlights the importance of proactive engagement: “Work with your stakeholders, not at them. Give them a voice in the process.” Find out what outcomes they would like to see and show them how you plan to get there. When trainers partner with stakeholders, he says, they set the precedent that learning is important across the organization.

Final Thoughts

The time has come for the training industry to move beyond outdated models. The ISO 30437 standards offer a modern and comprehensive framework for training measurement, addressing the complexities of today’s business environment. By adopting the 5-4-3 framework and engaging stakeholders throughout the evaluation process, organizations can gain deeper insights and drive meaningful improvements in their training programs.

While the transition to comprehensive measurement frameworks may require effort and investment, by making that change, learning leaders can ensure their training programs are not only effective but also aligned with the strategic goals of their organizations.

To learn more about the ISO standards for training evaluation and how to adopt them in your organization, check out our e-book, “Learning Measurement for Modern Business: Making Sense of ISO 30437.”

And don’t miss our webinar on Sept. 26, “The 800lb Gorilla of Learning Measurement: Know Your Stakeholders,” where Dr. Tom Whelan will discuss how to map the entire training stakeholder universe, from top-level management to the employees themselves, and everyone in between.