Evaluating AI investments economically: ROI and TCO framework for medium-sized businesses

The Special Characteristics of AI Investments

Unlike conventional software solutions, the value of AI implementations is rarely static. According to findings from the MIT Center for Information Systems Research, the value contribution of AI systems typically increases over time – provided they are trained with the right data and continuously optimized.

An example: While a conventional document management system offers largely consistent benefits after installation, an AI-based system for document classification improves with each processed document. The savings therefore grow progressively, not linearly.

Additionally, AI solutions often act as enablers for process changes that themselves lead to cost savings or revenue increases. According to a study by Accenture (2023), indirect effects generate an average of 40% of the total benefit in successful AI implementations – but are frequently ignored in ROI calculations.

“AI systems are not isolated tools, but catalysts for business process optimization and innovation. Their economic value unfolds only through integration with existing processes and systems.” – McKinsey Global Institute, AI Adoption Report 2024

The Most Common Misconceptions in Economic Evaluation

In our consulting practice, we repeatedly encounter the same mistakes in SMEs when evaluating AI investments:

• Oversimplified cost recording: According to a PwC study, only 37% of companies include change management costs in their business cases for AI projects.
• Unrealistic implementation timeframes: The average implementation duration is underestimated by 45% (Source: Deloitte AI Implementation Survey 2024).
• Failure to account for data quality costs: Nearly 70% of all AI projects require significantly more time and budget for data preparation than originally calculated.
• Purely short-term perspective: Many ROI calculations focus on the first year after implementation, while significant benefits are only realized in the second or third year.

Particularly critical: Many SMEs reduce AI to pure cost savings. However, a survey of 150 German SMEs (Fraunhofer IAO, 2023) shows that successful AI projects lead to revenue increases at least as often as they lead to cost reductions.

Each of these misconceptions leads to either unrealistic expectations or – more frequently – missed opportunities because economically viable projects are not implemented due to flawed analyses.

Direct Costs: Licenses, Infrastructure, and Implementation

The most obvious cost items are often not the largest – but they are recorded most reliably. Direct costs include:

• License or usage fees for AI platforms or services (e.g., Azure AI, AWS Bedrock, OpenAI API)
• Hardware requirements (on-premise) or cloud resources
• Implementation costs through external service providers or internal developer capacities
• Integration with existing systems (middleware, APIs, interfaces)

Important: Be sure to consider scaling effects here. A Deloitte analysis from 2023 shows that direct costs for AI systems make up an average of 33-45% of total costs – with a decreasing tendency for larger implementations due to economies of scale.

Also note that AI costs are often consumption-dependent. Unlike classical software, ongoing costs can increase significantly with increasing usage. A precise understanding of expected usage volumes is therefore essential.

Indirect Costs: Training, Change Management, and Support

The biggest mistake in AI business cases is underestimating indirect costs. An IDC study from 2024 shows that these account for approximately 55-60% of total costs in successful AI projects in SMEs.

The most important indirect cost factors include:

• Data preparation and quality assurance (often 30-40% of project costs)
• Training and enablement of employees (divided between technical teams and end users)
• Change management and internal communication
• Governance processes (data protection, compliance, risk management)
• Support and continuous optimization (often underestimated)

The costs for data preparation are systematically underestimated in particular. A recent study by Gartner (2024) shows that medium-sized companies spend an average of 125 person-days on data preparation for medium-sized AI projects – more than twice as much as originally planned.

It is also important to consider “opportunity costs”: When internal experts are assigned to an AI project, they are unavailable elsewhere. These indirect costs are not considered in 83% of all business cases we analyzed.

Quantifiable Benefits: Time, Cost, and Resource Savings

On the benefit side, we start with directly measurable advantages. These are the easiest to translate into euros and form the foundation of any ROI calculation:

• Time savings through automation (multiplied by personnel costs)
• Reduced error rates and associated rework costs
• Material savings (relevant for production-related AI applications)
• Shortened processing times and thereby reduced process costs
• Personnel cost optimization (freeing up resources for higher-value tasks)

A structured collection of these effects is crucial. We recommend the “before-and-after method”: Measure the current state precisely before implementation to have valid comparison values later.

A practical example: A medium-sized mechanical engineering company was able to reduce the error rate by 43% by using AI in quality control. With annual error costs of 2.3 million euros, this meant a direct saving of 989,000 euros – easily quantifiable and verifiable.

Strategic Benefits: Innovation Power, Market Position, and Future Viability

Beyond direct cost savings, AI implementations offer strategic advantages that are harder to quantify but are often more valuable in the long term:

• Improved customer satisfaction and retention (measurable through NPS or churn rate)
• Accelerated innovation cycles
• Development of new business models and markets
• Attractiveness as an employer (measurable through recruiting metrics)
• Competitive advantages through data competence

The greatest strategic advantage often lies in future viability. A BCG study from 2024 shows that companies that invest early in AI achieve, on average, a 27% higher revenue growth rate over a five-year period than their more cautious competitors.

While these strategic benefits should not form the core of your ROI calculation, they should be included as qualitative factors in the investment decision. For your business case, we recommend at least partially quantifying these advantages – for example, by comparing with industry benchmarks or applying scenario analyses.

The Extended ROI Framework for Technological AI Innovations

The classic ROI formula (ROI = (Profit – Investment Costs) / Investment Costs × 100%) must be modified for AI projects to capture both direct and indirect effects.

Our extended framework is based on a multi-dimensional analysis:

1. Direct efficiency gains (quantifiable through time savings, error reduction, etc.)
2. Indirect productivity effects (e.g., improved decision quality)
3. Strategic value contribution (long-term competitive advantages, market positioning)

A research paper from the University of St. Gallen (2023) recommends the following modified ROI formula for AI projects:

AI-ROI = ((Direct Savings + Productivity Gains + Strategic Value Component) – (Direct + Indirect Costs)) / (Direct + Indirect Costs) × 100%

It is crucial to consider not only the initial investment costs but the total costs over the review period. At the same time, all relevant value contributions must be systematically recorded.

A pragmatic approach from our consulting practice: We divide the ROI calculation into a “Hard ROI” (directly quantifiable effects) and a “Soft ROI” (indirect and strategic advantages). This distinction helps decision-makers differentiate between secured and potential benefits.

Correctly Defining Time Horizons: Short, Medium, and Long-term Returns

Unlike traditional IT projects, AI implementations often develop their impact over longer periods. We recommend a staggered approach:

• Short-term (0-12 months): Focus on direct efficiency gains and “quick wins”
• Medium-term (13-36 months): Realization of process improvements and initial indirect effects
• Long-term (>36 months): Strategic advantages and transformative effects

An analysis of 124 successful AI implementations in European SMEs by Forrester Research (2024) shows that the typical ROI curve for AI projects follows a “hockey stick” shape: After an initial investment phase with negative ROI, there follows a phase with moderately positive ROI, before disproportionate benefits are often realized in the third phase.

In our own practice, we have observed that successful AI projects in SMEs typically reach their “breakeven” after about 18-24 months. After that, the ROI often increases significantly, provided there is continued investment in optimization and further development.

For decision-making, we therefore recommend working with different time horizons and explicitly stating them. A 3-year ROI is more meaningful for most AI projects than a purely short-term view.

Risk Factors and Their Impact on Expected ROI

A realistic ROI calculation also takes potential risks into account. The most common risk factors that can affect the expected ROI of AI projects:

• Insufficient data quality and availability (a critical factor in 58% of projects)
• Lack of user acceptance and change management challenges (58%)
• Too broad initial scope and lack of focus (52%)
• Unrealistic expectations regarding implementation time and effort (49%)
• Lack of internal competencies for operation and further development (43%)

For a robust ROI calculation, we recommend applying a “Risk-Adjusted ROI.” This method weights the expected benefits with probability factors for their occurrence.

A simple but effective approach is to conduct scenario analyses with a best case, base case, and worst case. The difference between these scenarios provides insight into the robustness of your ROI expectations.

Practical example: For an AI project to optimize customer service, our analysis yielded an expected 3-year ROI of 187% in the base case. In the worst case (with delayed implementation and lower user acceptance), this reduced to 73% – still positive, but significantly less attractive.

This transparent presentation of risks and their impacts increases the credibility of your business case and creates more realistic expectations among all stakeholders.

Hidden Cost Factors Throughout the Entire Lifecycle

A comprehensive TCO analysis for AI implementations considers costs in all phases of the lifecycle. Gartner analysts recommend including at least the following cost categories:

• Acquisition and implementation costs (licenses, hardware, development)
• Operating costs (cloud computing, energy, maintenance)
• Data management and storage
• Personnel costs for operation and continuous optimization
• Training and support
• Compliance and governance
• Adaptations and further development

Particularly relevant for AI systems are the costs of continuous training and optimization. Unlike classical software, AI models require regular adjustments to maintain their performance.

A study by Deloitte (2024) shows that these hidden “Maintenance and Evolution Costs” account for an average of 23% of total costs over a three-year period for AI projects – significantly more than for conventional IT projects.

Typical hidden costs that are often not considered in initial budgets:

• Model Drift Correction: Costs for regular review and adjustment of models when data or requirements change
• Explainability Requirements: Efforts to implement explainability features, particularly relevant in regulated environments
• Data Pipeline Maintenance: Ongoing costs for maintaining and updating data integration processes
• Hardware Upgrades: Often necessary performance enhancements for on-premise solutions
• Regulatory Compliance Updates: Adjustments due to changing legal requirements

Make-or-Buy: In-house Development vs. Standard Solutions vs. Customization

A fundamental TCO decision is the choice between in-house development, standard solutions, or customer-specific adapted products. Each option has characteristic cost profiles over the lifecycle.

In-house development offers maximum flexibility but also creates the highest initial costs and risks. According to an analysis by MIT Technology Review (2023), developing a custom AI solution for medium-sized companies costs an average of 2.7 times more than implementing an adaptable standard solution.

Standard solutions (“off-the-shelf”) have lower initial costs but can become more expensive in the long run if extensive adaptations are necessary or if the solution does not optimally fit business processes.

The “customization” approach – implementing a standard solution with targeted adaptations – often offers the best TCO ratio. An IBM study from 2024 shows that this approach was chosen for 68% of successful AI implementations in SMEs.

When making the make-or-buy decision, you should consider the following factors in addition to pure costs:

• Available internal competencies
• Strategic importance of the AI solution (competitive differentiation)
• Pace of technological development in the relevant area
• Long-term maintainability and expandability

For German SMEs, the rule of thumb is: If the AI solution does not offer an immediate competitive advantage in the core business, a “buy” or “customize” approach is usually more TCO-optimal than complete in-house development.

The Importance of Scalability and Integration Costs

A critical factor for TCO is the scalability of an AI solution. Many projects start as pilots or proof of concepts with limited scope but are intended to be expanded later.

According to an IDC study (2024), the costs of scaling AI pilot projects are underestimated by medium-sized companies by an average of 165%. This is mainly due to underestimated integration costs into existing systems and processes.

Particularly cost-intensive are typically:

• Interfaces to legacy systems: Integration into evolved IT landscapes can require substantial adjustments
• Data migrations and transformations: Extensive ETL processes (Extract, Transform, Load) are often needed
• Security and compliance adjustments: Requirements for data protection and security often increase with scaling
• Performance optimizations: What works on a small scale can cause performance problems with larger data volumes

A TCO-optimized approach considers scalability from the beginning. In our practice, we recommend a “scale-first” approach: Even if a project starts small, the chosen architecture should already be designed for larger deployments.

A field-tested approach is the TCO matrix, where the respective cost drivers are separately recorded for different scaling levels (e.g., pilot, department, company-wide). This allows for a differentiated view of TCO at various expansion stages.

Exemplary insight from practice: For a chatbot-based customer service project, costs increased by a factor of only 1.8 when expanding from one to five countries – economies of scale thus led to significantly less than proportional cost increases.

Technical Performance Indicators

Technical KPIs form the basis for evaluating the system itself and are particularly relevant for IT teams and technical stakeholders. The most important metrics include:

• Model accuracy and precision (e.g., F1-score, AUC-ROC)
• Latency times and response speed
• Availability and fail-safety
• Scaling behavior (performance under increasing load)
• Data quality metrics (completeness, currentness, consistency)

These metrics should not be viewed in isolation but always in relation to business requirements. An example: While a response time of 1-2 seconds may be acceptable for an AI system in customer service, applications in production control often require response times in the millisecond range.

Continuous monitoring of these metrics is important, as AI systems can degrade over time (“model drift”). A study by Microsoft Research (2023) shows that unmonitored AI models in productive use can lose an average of 4-7% of their performance per quarter.

For performance monitoring, we recommend an approach with defined thresholds and automated alerts. This allows problems to be detected and resolved early, before they affect business value.

Business Impact Metrics

Ultimately, every AI investment must be measured by its business value. The following business KPIs have proven particularly relevant in practice:

• Process efficiency: Throughput times, processing speed, error rates
• Cost metrics: Direct savings, cost reduction per unit/transaction
• Employee productivity: Output per employee, freed-up capacities
• Customer-related metrics: Satisfaction (CSAT, NPS), response times, self-service rate
• Revenue effects: Conversion rates, cross/upselling successes, customer retention

For each of these categories, you should collect baseline values before implementation and define clear target values. A recent McKinsey analysis (2024) shows that companies that conduct such structured before-and-after comparisons are 74% more likely to achieve the planned ROI of their AI projects.

Direct causal relationships between AI implementation and business metrics are particularly valuable. A practical example: A medium-sized online retailer was able to prove through A/B tests that its AI-based product recommendation increased the conversion rate by 23% and the average order value by 17% – a clear, measurable business impact.

For your performance monitoring, we recommend creating an AI-specific business impact dashboard that integrates both technical and business metrics and visualizes their relationships.

Employee Adoption and Productivity Metrics

The actual value of an AI solution is only realized if it is actively and effectively used by employees. Experience shows: Even the most advanced AI solution generates no ROI if it is not integrated into daily work.

Relevant adoption metrics include:

• Usage rates: Active users (daily/weekly/monthly), interactions per user
• User trust: Adoption rate of AI recommendations, manual verification rates
• Efficiency increase per user: Average time savings, reduction of routine tasks
• User feedback: Structured evaluations and qualitative feedback
• Problem-solving efficiency: Reduction of support requests, improved self-help rate

A Deloitte study from 2024 underscores the importance of adoption: In AI projects in SMEs, the user adoption rate explains 67% of the variance in achieved ROI. In other words: Two-thirds of success depends not on the technology itself, but on its use.

Practical measures to increase adoption include:

• Early involvement of key users in the conception
• Target group-specific training and enablement programs
• Clear communication of individual benefits (“What’s in it for me?”)
• Iterative improvement based on user feedback
• Gamification elements and internal marketing

A compelling example from our consulting practice: At an industrial service provider with 130 employees, the adoption rate of an AI-supported document analysis was increased from an initial 34% to over 85% through targeted enablement – with a corresponding multiplication of business impact.

Manufacturing Companies: Automated Quality Control and Predictive Maintenance

A medium-sized manufacturer of precision components (120 employees) implemented an AI-based image recognition solution for quality control. The key economic data:

• Investment: €175,000 (including hardware, software, integration, and training)
• Annual operating costs: €43,000 (licenses, support, maintenance)
• Main benefits:
  • Reduction of error rate by 68% (monetary value: approx. €320,000 p.a.)
  • Freeing up 1.5 FTE for higher-value tasks (approx. €90,000 p.a.)
  • Acceleration of the production process by 12% (approx. €140,000 additional revenue p.a.)
• ROI after 18 months: 248%
• Break-even point: 7 months after full implementation

Particularly noteworthy: The company used a phase-based implementation approach, starting with a single production line. After successfully demonstrating ROI, expansion to other lines occurred at significantly reduced implementation costs per line (scale effects).

Important lessons learned from this project:

• The initial data collection and preparation took longer than planned (originally 4 weeks, actually 11 weeks)
• Integration with the existing MES (Manufacturing Execution System) proved complex and required additional investments
• The change management effort was underestimated – production employees required intensive training and support
• The combined qualitative and quantitative benefits significantly exceeded initial expectations

The sustainable success of this project led the company to implement a second AI application in predictive maintenance, with expected annual savings of €230,000 through reduced unplanned downtime.

Service Sector: Customer Service Automation and Knowledge Management

A medium-sized IT service provider (80 employees) implemented an AI-based solution for customer support and internal knowledge management. The solution combined an AI chatbot for frequent customer inquiries with an intelligent knowledge management system for employees.

• Total investment: €138,000 (including software licenses, customization, integration, training)
• Annual operating costs: €36,000 (licenses, updates, support)
• Quantified benefits:
  • Reduction of first-level support effort by 42% (savings: approx. €110,000 p.a.)
  • Reduction of average response time from 4.2 to 0.8 hours
  • Increase in customer satisfaction (NPS) by 18 points
  • Reduction of onboarding time for new employees by 35% (productivity gain: approx. €45,000 p.a.)
• ROI after 24 months: 203%
• Break-even point: 13 months

Particularly interesting about this example is the dual use of AI technology: Externally for customer inquiries and internally for knowledge management. This multiplication of the use case significantly optimized the ROI.

An unexpected advantage was also the improved employee satisfaction in the support team. By taking over repetitive inquiries through the chatbot, employees could focus on more complex and interesting tasks, which led to a 40% reduction in turnover (with significant cost savings in recruiting and onboarding).

The most important insights from this project:

• The initial training phase of the AI system was more extensive than expected and required intensive participation from subject matter experts
• The business case was significantly improved by the combined internal and external use
• Continuous improvement of the system based on user feedback was crucial for long-term adoption
• Cost savings were easier to quantify than quality and satisfaction improvements

Cross-Industry: Document Processing and Administrative Processes

One application field with particularly consistent ROI results across various industries is AI-supported document processing. An example from our consulting practice:

A medium-sized company from the logistics industry (160 employees) implemented an AI solution for automatic processing of delivery notes, waybills, and invoices.

• Investment volume: €155,000 (including licenses, customization, integration, training)
• Annual operating costs: €32,000
• Economic benefits:
  • Reduction of manual data entry by 78% (savings: approx. €180,000 p.a.)
  • Acceleration of invoice processing from an average of 8 to 1.5 days
  • Reduction of errors and correction needs by 65% (savings: approx. €50,000 p.a.)
  • Improved compliance and auditability (risk reduction, difficult to quantify)
• ROI after 24 months: 265%
• Break-even point: 9 months

Noteworthy about this example is the comprehensive TCO consideration: The company included in its business case calculation not only the direct software and implementation costs but also the internal effort for process adjustments and change management.

Also important for success was the chosen implementation approach: Instead of migrating all document types simultaneously, the company started with delivery notes (high volume, standardized format), followed by invoices and later more complex documents. This prioritization enabled early successes and a faster reach of the break-even point.

We observe similar results across industries for administrative processes. According to an analysis of the digital BMWK SME panel 2024, AI projects in document processing and administrative process automation achieve average ROIs of 180-240% over three years in German SMEs.

Phase 1: Piloting and Proof-of-Concept with Quick ROI

The first phase focuses on a narrowly defined use case with high probability of success and quick ROI. Ideal are areas with quantifiable inefficiencies, structured data, and clearly measurable results.

Central elements of this phase:

• Clearly defined scope with limited complexity (typical: 2-3 months implementation time)
• Involvement of actual end users from the beginning
• Establishment of baseline metrics for later before-and-after comparison
• Implementation with minimal integration effort (possibly as a “standalone” solution)
• Strict success evaluation based on predefined KPIs

A Forrester analysis from 2024 shows that AI projects that begin with a clearly defined pilot have a 64% higher probability of success than those with a broader initial scope.

Practical recommendation: For this phase, choose a use case that can generate a positive ROI within 6-9 months. This builds stakeholder trust and provides valuable insights for the following phases.

An exemplary business case for Phase 1 should typically include an investment volume of €30,000-80,000 and enable an ROI of at least 100% within the first year.

Phase 2: Scaling and Integration into Existing Systems

After successful completion of the pilot phase comes scaling and deeper integration. In this phase, you expand the application area and connect the AI solution with your core systems.

Important elements of this phase:

• Extension to additional use cases or user groups
• Integration into existing IT landscape (ERP, CRM, etc.)
• Implementation of robust data pipelines for continuous training
• Building internal competencies for long-term operation
• Establishment of governance processes and responsibilities

In this phase, the first indirect benefits typically become visible. A study by PwC (2024) shows that in the scaling phase, an average of 35% more benefit categories are identified than in the initial business case calculation.

The investments in this phase are typically higher than in Phase 1 but amortize faster thanks to the collected experiences and established processes. Based on our experience, the typical ROI for Phase 2 is 150-200% over a two-year period.

A particular challenge in this phase is change management. With increasing spread of the AI solution, the need for systematic training and communication measures increases. These costs should be explicitly considered in the business case.

Phase 3: Transformation and Innovation for Long-Term Competitive Advantage

The third phase goes beyond efficiency improvements and focuses on transformative applications that open up new business opportunities or enable disruptive changes.

Characteristic for this phase:

• AI becomes an integral part of corporate strategy
• Development of new products or services based on AI
• Transformation of entire business processes (not just optimization)
• Building a data-driven corporate culture
• Establishment of continuous innovation cycles

The ROI in this phase is more difficult to quantify as it often involves strategic advantages. However, a McKinsey analysis from 2024 shows that companies that use AI strategically achieve, on average, an EBIT margin 5.8 percentage points higher than their competitors.

For economic evaluation in this phase, we recommend a “portfolio approach”: Instead of evaluating each individual use case in isolation, the total investment in AI is compared to the overall company benefit.

An example from our practice: A medium-sized B2B provider has organized its AI investments as a strategic portfolio with different risk/opportunity profiles: 60% of investments flow into applications with secure but moderate ROI, 30% into applications with medium risk and higher potential return, and 10% into experimental, potentially disruptive applications.