Credit: Omar:. Lopez-Rincon on Unsplash You’re likely familiar with some version of this narrative: companies adopt AI, performance gains eventually plateau, and leaders are left wondering about the ROI from their investments. Experts disagree on how to remedy this gap between adoption and results. Some call for better governance, others for better deployment.
Or my personal favorite, better enablement. Each one of these remedies depends upon organizations being aligned on what success looks like and being able to measure it. Take learning and development (L&D) professionals. They may be supporting your AI adoption plans while facing the same organizational issues.
That’s right, the people most prepared to build skills are also struggling to measure the ROI of AI. We surveyed more than 800 workplace learning professionals across more than 25 countries for our annual AI in Learning & Development Report. Here’s what we can learn from them about getting from adoption to results. Without agreed-upon outcomes, teams default to speed How L&D teams use AI reflects what their organization rewards and measures.
If employees are working at organizations that track AI usage and reward more tokens being used, then teams’ AI use will reflect that. This type of reward only becomes an issue when it isn’t paired with clear measurement outcomes. Organizations need to be able to say, we want you to use more AI, in order to accomplish X. Otherwise, they’re left with productivity gains that amount to little performance improvement.
In our report, 75% of respondents named faster content creation as one of the clearest impacts they’ve seen from AI adoption. This looks like reducing the time it takes to produce new training and increasing the output of training. Meanwhile, only 11% named improved on-the-job performance as an impact. Most workplace training is designed to improve on-the-job performance.
Yet only 35% of L&D teams are consistently measuring that (compared to 72% who consistently measure participation and completions). This suggests that leaders are either rewarding and measuring L&D performance against those metrics, or that no one has agreed on what success should look like. Our survey responses point to the latter. More than one in five name a lack of agreed-upon success metrics as their biggest barrier to proving impact.
Teams are told to use AI, then hit roadblocks Rewards aren’t the only organizational constraint L&D teams face when they adopt AI. What an organization permits, how it governs AI, matters just as much. That starts with which AI solutions teams can access. 73% of L&D teams that started from a leadership mandate to use AI still need departmental or cross-functional permission to pilot a new AI tool. Some teams are limited to company-approved AI tools and can’t test beyond that.
Teams also need the right data to evaluate AI’s impact. A majority of L&D teams are confident that AI is leading to improvements, but struggle to provide evidence of those gains. Asked for the biggest barrier to proving impact, 53% pointed to an organizational constraint, including the lack of agreed-upon success metrics and access to data or measurement tools. That might look like data needed to measure impact living in disconnected systems, such as a Learning Management System, an HR Information System, a survey tool, and an AI video platform.
If L&D teams are using AI to churn out training faster, but aren’t evaluating whether that training is leading to behavioral change, then they’re just using AI as a content factory. Until leaders agree on outcomes beyond speed, more content is all those teams will have to show for AI adoption. Build the structure AI needs to deliver ROI The good news is that about a third of L&D teams report performance gains from AI, like faster time to proficiency or improved on-the-job performance. Several traits set these teams apart, like being able to freely pilot or build AI solutions, or using AI to evaluate their work.
Those factors often depend on organizational factors beyond L&D’s control. Based on our data, there are three structural changes leaders can make to their AI strategy to deliver ROI. First, align on what AI is supposed to accomplish, then on how its usage is rewarded and measured. Without a defined outcome, teams produce what’s easiest to show, which in L&D is faster content.
For L&D teams, that means working with leaders to identify the performance changes they want to see, and how to measure them, before designing any new training. And yes, that includes AI enablement. Second, invest in data integration before scaling new AI solutions or automating more workflows. Make sure teams across functions have access to the data they need to measure AI’s impact.
Third, partner with L&D on your AI strategy. If the goal of AI is to automate manual work, then the goal should be to free up time for more strategic capability building, like how to use AI to change how work gets done. L&D is well suited to lead that capability building because they can identify where training can be part of the solution and where it can’t. Only then can training help close the gap between AI adoption and ROI.
Expertise from selected TNW Council members, admitted through an application-and-review process to a fee-based program. Opinions expressed by the authors are their own. Get the TNW newsletter ✕ Close Get the TNW newsletter Get the most important tech news in your inbox each week.
