Why it’s time to embrace FinOps for AI

By Hemang Davé and Reinier Aerdts

One of the inherent challenges of innovation is squaring benefits with the bottom line. This certainly has been the case so far for AI and generative AI. 

Although these transformative technologies are changing the way, speed and efficiency with which organizations across sectors operate, research from Kyndryl shows that 36% of leaders view ROI as a barrier to AI adoption. Of the organizations that have implemented AI, only 42% have seen a positive return on their investments.

The implications are clear: AI-related expenditures must create greater business value to justify the technology’s hard and soft costs. FinOps — a discipline that combines technology, best practices and reimagined cross-functional teams — can play a key role as organizations strive to balance the promise of AI with fiscal responsibility.

Why businesses should apply FinOps to AI workloads

Most (75%) of Forbes’ Global 2000 enterprises now use FinOps to rein in cloud and hybrid cloud spending.¹ Companies can apply the same methodologies to AI and generative AI workloads to:

• Reduce operating expenses
  FinOps provides a structured approach to managing expenses caused by variability in data processing, model training and inference workloads. Teams can also use forecasting tools tailored for AI workloads to project future costs and plan budgets more effectively.
  
  
• Improve resource allocation
  FinOps can help improve resource management and boost ROI by optimizing AI workloads. For example, FinOps teams can monitor usage patterns of graphics processing units (GPUs) and implement spot instances or reserved instances to maximize performance.
  
  
• Align spending with business value
  FinOps tracks the performance of AI models to determine the business value of AI-related spending. The process involves establishing clear frameworks for measuring AI’s impact on cost efficiency, resilience, user experience, productivity, sustainability and business growth.
  
  
• Increase financial accountability
  FinOps promotes greater accountability and helps all stakeholders understand the financial ramifications of their decisions. Finance, engineering and business teams work together to establish governance frameworks, conduct regular cost reviews, optimize resource allocation, and manage expenses.
  

The savings potential of FinOps for AI in context

Research from McKinsey shows that FinOps can help enterprises lower their cloud and hybrid cloud costs by 20-30% each year.² Meanwhile, companies like LifeLabs have cut their cloud spend by 35-40% annually. Applying the same percentage of savings, FinOps could potentially help a company save US$2 million to US$4 million in operating costs on a US$10 million AI investment. 

At the granular level, consider the expense a company incurs each time an AI model processes an input and generates an output.³ This measurement — known as cost per inference — allows FinOps teams to track the operational efficiency of applications like chatbots, recommendation engines and image recognition systems. 

So, for example, if the total inference cost for a company’s human resources chatbot is US$10,000 and the system processes 150,000 inference requests, the cost per inference is US$0.06 per request. Engineers can use this data to identify cost spikes caused by inefficient code or infrastructure and distribute resources more strategically.

How to implement FinOps with AI workloads

Since traditional FinOps wasn’t explicitly designed for AI and generative AI, applying the methodologies to workloads can present unique challenges. These steps will help CTO, CIO, CFO, line of business and DevOps teams address common concerns and lay a foundation for success: 

1. Provide ongoing FinOps training
   During the early stages of FinOps deployment, offer team members education and resources on FinOps principles⁴ and specialized training in data analysis to control costs and optimize resource usage. Ongoing education should prioritize adapting to evolving AI technologies and pricing models to drive greater business value. 
   
   
2. Assess current AI workloads
   Before designing a FinOps strategy, review current AI workloads to determine baseline functionality and consumption rates, including where data is located, how tokens are stored and generated, and what factors drive costs. This evaluation provides essential information for enhancing performance and optimizing costs, such as running AI next to data or retraining large language models. 
   
   
3. Create a governance framework
   Develop a FinOps framework for compliance, performance measurement and cost containment to align financial management with organizational objectives.⁵ Policies and governance structures should outline protocols for managing AI expenses, conducting periodic (daily, weekly, monthly) performance and usage reviews, and adjusting resource usage to maximize efficiency.
    
4. Establish monitoring and reporting processes
   Continuous monitoring provides real-time visibility into AI operations, enabling FinOps teams to track, analyze and adjust resource usage dynamically. Use cloud-native tools or an open integration platform to gather insights, display data and generate reports. Tagging mechanisms can help teams categorize expenses by project, department or application and identify trends and anomalies. 
   
   
5. Foster cross-functional collaboration
   FinOps is rooted in collaboration, so employees from IT, operations, finance and business units must work across departments to apply the methodologies to AI workloads.⁶ Setting up clear and dedicated communication channels and defining roles and responsibilities is critical. You’ll also want to host regular workshops and meetups to build trust, share knowledge and encourage cross-functional team building.
   
   
6. Optimize resource usage and iterate
   After establishing observability into resource costs and consumption, monitor usage patterns to uncover trends and areas for improvement, such as over-provisioned resources or underutilized instances. You can then implement cost-saving measures like rightsizing and using machine learning models to predict future resource requirements while iterating and optimizing as needed.
   

Striking a balance with FinOps and AI

Even as AI and generative AI usher in an era of innovation, many organizations are searching for greater returns on their AI investments. By combining FinOps principles with a mindset of continual improvement and collaboration, companies may be able to draw a clearer line between spending and value as they gain greater financial and operational control of their AI initiatives.

Hemang Davé and Reinier Aerdts are Client Technical Leaders for the Technology, Media and Telecommunications market at Kyndryl, a member of the FinOps Foundation and FinOps Foundation Governing Board.

1 FinOps breaks out of the cloud, CIO, March 2025
2 The FinOps way: How to avoid the pitfalls to realizing cloud’s value, McKinsey, January 2023
3 FinOps for AI overview, FinOps Foundation, April 2025
4 FinOps principles, FinOps Foundation, April 2025
5 FinOps policy and governance, FinOps Foundation, April 2025
6 FinOps workload optimization, FinOps Foundation, April 2025