A SAP AI project that succeeds in isolation is a proof of concept. A SAP AI project that scales across the enterprise is a Centre of Excellence. The difference between the two is not technology — it is organisational design.
Companies that deploy one successful SAP AI use case and then wonder why the next three projects stall, the fourth gets cancelled, and the fifth has to reinvent the wheel are missing a CoE. This guide gives you the exact structure, roles, governance cadence, toolchain, and 6-month launch roadmap used by the SAP AI programmes that consistently scale.
The multiplier effect: Enterprises with a formal AI CoE deploy new SAP AI use cases 3.2× faster, at 40% lower cost, with 28% higher first-year adoption than those without one. The CoE pays for itself on the second use case it delivers.
Step 1: Choose Your CoE Operating Model
Before hiring a single person, decide which operating model fits your organisation. Each has fundamentally different implications for headcount, reporting lines, and decision-making authority.
Centralised CoE
One central team owns all AI development, deployment, and governance. Business units submit requests; the CoE delivers. Common in highly regulated industries.
Federated CoE
Each business unit has its own AI team, loosely coordinated by a central standards body. Maximum speed but minimum consistency — common in large multinationals.
Hybrid (Hub & Spoke)
A central hub sets standards, owns the platform (SAP BTP), and delivers complex use cases. Embedded "spoke" champions in each business unit drive adoption and surface requirements.
Step 2: The Core Team — Roles & Hiring Guide
The minimum viable SAP AI CoE for a mid-market enterprise needs these 7 roles. Not all need to be full-time from day one — the phasing section shows how to build incrementally.
AI CoE Lead
Owns the AI strategy, budget, roadmap, and board-level reporting. Sets priorities across use cases. The single accountable leader for AI business outcomes.
SAP BTP Architect
Owns the technical architecture on SAP BTP — integration patterns, security model, AI Core configuration, and the API layer between AI and SAP backends.
AI / ML Engineer
Builds, trains, and monitors AI models. Handles prompt engineering for LLM use cases, fine-tuning, RAG pipelines, and model performance optimisation post-go-live.
SAP Data Engineer
Owns the data pipeline from SAP (S/4HANA, ECC) into the AI layer. Responsible for data quality remediation, SAP HANA Cloud, and the vector store for RAG use cases.
Business Analyst (Process)
Bridges between AI capability and business process. Documents current-state workflows, defines exception logic, writes user acceptance test scripts, and owns adoption metrics.
AI Governance & Risk
Owns EU AI Act compliance, model risk management, audit trail governance, and the ethical AI framework. Interfaces with Legal, Compliance, and the DPO.
Change Management Lead
Drives adoption across business units. Runs the champions network, designs role-specific training, monitors adoption KPIs, and surfaces resistance before it kills go-lives.
Hiring order matters: Hire the AI CoE Lead first — they hire everyone else. Hire the BTP Architect second (they unblock technical progress). Hire ML Engineers third. Add the Change Management Lead before your first go-live, not after. Skipping change management is the most expensive false economy in an AI CoE build.
Step 3: The Governance Model — Who Decides What
Governance is not bureaucracy — it is the mechanism that stops every use case from becoming a political fight. Clear decision rights prevent the two most common CoE failures: the project that launches without IT sign-off, and the project that IT kills without business input.
| Forum | Attendees | Decisions Made | Cadence |
|---|---|---|---|
| AI Steering Committee | CIO, CFO/COO, AI CoE Lead, CISO | Strategic priorities, budget allocation, portfolio approval, risk escalations, board reporting | Quarterly |
| CoE Operating Board | AI CoE Lead, BTP Architect, BAs, Process Owners | Use-case backlog prioritisation, go/no-go decisions, resource allocation, milestone reviews | Monthly |
| Technical Review | BTP Architect, ML Engineers, Data Engineer, IT Security | Architecture decisions, vendor selection, security sign-off, model accuracy thresholds | Bi-weekly |
| Project Stand-up | Delivery team (implementation-specific) | Daily blockers, task progress, escalations within implementation sprints | Daily (in-flight) |
| Adoption Check-in | Change Management Lead, Business Champions, Process Owners | Adoption rate review, resistance identification, training gaps, feedback loop | Weekly (first 90 days) |
| Model Health Review | ML Engineers, Data Engineer, Governance & Risk | Accuracy drift alerts, retraining decisions, EU AI Act audit evidence collection | Monthly |
Step 4: The AI CoE Toolchain
Standardise your toolchain early — it is significantly harder to migrate between platforms mid-programme than to start on the right stack. These are the tools used by the most efficient SAP AI CoEs in 2026.
AI Platform
LLM & AI Models
Data & Storage
Monitoring & Governance
Step 5: The 6-Month CoE Launch Roadmap
Foundation: Team & Platform
- Hire AI CoE Lead & BTP Architect
- Stand up SAP BTP tenant (AI Core, Integration Suite)
- Draft AI governance charter & decision-rights RACI
- Convene first Steering Committee meeting
- Shortlist Wave 1 use cases (score & prioritise)
- Conduct SAP data readiness assessment
First Delivery: Wave 1 Use Case in Shadow Mode
- Hire ML Engineers & Data Engineer
- Wave 1 use case: data extraction & model training
- Shadow mode enabled — accuracy tracking begins
- EU AI Act risk classification documented
- Publish AI standards & development playbook v1
- Launch champions network in Finance & Procurement
First Go-Live & Wave 2 Pipeline
- Wave 1 production go-live
- First ROI measurement report to Steering Committee
- Hire Change Management Lead & BAs
- Wave 2 use-case scoped & approved
- Reusable component library documented (accelerators)
- CoE operating model presented to board for funding renewal
Step 6: CoE-Level KPIs to Report to the Board
A CoE that cannot prove its business impact will lose funding within 18 months. Report these five metrics to the Steering Committee every quarter — and publish them to the wider organisation.
| KPI | Target (Year 1) | Target (Year 2) | Reporting Cadence |
|---|---|---|---|
| Use cases in production | 2–3 | 8–12 | Quarterly |
| Total annualised ROI delivered | 3–5× CoE cost | 8–12× CoE cost | Quarterly |
| Avg. use-case delivery time (pilot→prod) | ≤90 days | ≤60 days | Quarterly |
| Portfolio AI accuracy (weighted avg.) | ≥92% | ≥95% | Monthly |
| User adoption rate (live tools) | ≥70% | ≥85% | Monthly |
What Does a SAP AI CoE Cost?
Budget transparency prevents the most common CoE failure mode: being funded for the platform but not the people. Both are essential. Here are realistic cost ranges for a mid-market enterprise.
| Cost Category | Year 1 | Year 2+ |
|---|---|---|
| People (7 FTE fully-loaded, blended rate €100k) | €700,000 | €750,000 |
| SAP BTP platform & AI Core licences | €120,000 | €140,000 |
| LLM API access (Claude, GPT-4o via Azure) | €40,000 | €60,000 |
| Tooling (monitoring, DevOps, BI dashboard) | €30,000 | €25,000 |
| Training & certifications (SAP BTP, AI Core) | €25,000 | €15,000 |
| SI / implementation partner (Wave 1 support) | €100,000 | €50,000 |
| Total Annual CoE Investment | €1,015,000 | €1,040,000 |
With 3–5× ROI in Year 1 as the target, the CoE should deliver €3–5M in measurable business value to fully justify its cost. Use-cases with the fastest payback (invoice processing, GR/IR) fund the CoE's continued existence — sequence them first.
5 AI CoE Anti-Patterns That Guarantee Failure
The "Build It and They'll Come" CoE
Team built, platform deployed, governance documented — then waiting for business units to submit requests. Without proactive engagement, the queue stays empty.
Platform Without People
Fully funded SAP BTP deployment with two engineers and no change management, no business analyst, and no data engineer. The platform sits idle.
Governance Theatre
Governance documents exist but decisions are still made informally. The Steering Committee meeting becomes a status update with no actual decisions.
The Invisible CoE
The CoE team works hard but nobody in the business knows what they've delivered. No internal comms, no success stories, no visible KPI dashboard.
Perfection Before Production
Spending 6 months refining the governance framework, the toolchain architecture, and the operating model before delivering a single use case to production.
Building Your SAP AI CoE? We've Done This 30+ Times.
SAVI AI can accelerate your CoE from zero to first production use case in 90 days — providing the BTP architecture, AI model library, and change management playbook so you don't start from scratch.
Book a Free CoE Design Session