Every SAP Business AI capability — SAP Joule, autonomous enterprise agents, Joule Studio custom agents, and SAVI AI's agentic modules — runs on a common foundation: SAP AI Core and SAP Generative AI Hub on the SAP Business Technology Platform (BTP). Understanding this foundation is no longer optional for SAP IT leaders and architects. It determines which AI models you can use, how your SAP data grounds those models, what governance controls apply, and how you extend AI into custom workflows.
This guide explains the SAP BTP AI stack from first principles — what each layer does, which LLMs are supported in 2026, how grounding and orchestration work, what the Orchestration Service is, and how SAVI AI's agentic platform sits on top of this foundation to deliver autonomous enterprise outcomes.
What Is SAP AI Core? The Engine Under Every SAP AI Scenario
SAP AI Core is the runtime and lifecycle management service for AI models and workflows on SAP BTP. Every AI scenario in the SAP ecosystem — whether it's a Joule assistant, a predictive analytics model in SAP Analytics Cloud, or a custom LLM application built with Joule Studio — executes through SAP AI Core.
Think of SAP AI Core as the enterprise-grade middleware between your AI models and your SAP business data. It handles:
- Model lifecycle management: Training, versioning, deployment, monitoring, and retirement of AI models — with full audit trail and rollback capability for production-grade governance
- Inference execution: Running LLM inference requests against any supported model — whether that's Claude (Anthropic), GPT-4o (Azure OpenAI), Gemini (Google), Llama 3 (Meta), or Mistral — through a unified API endpoint
- Resource orchestration: Automatically scaling compute resources for AI workloads on SAP BTP infrastructure — GPU instances for training, CPU instances for lightweight inference
- Security and compliance controls: Data residency enforcement, PII scrubbing before model calls, encryption of prompts and completions, and access control via SAP BTP role collections
- Consumption tracking: Token-level usage tracking per AI scenario, cost allocation by business unit, and budget controls to prevent runaway inference costs
Data privacy guarantee: SAP AI Core routes your prompts and data to LLM providers through agreements that prohibit training on customer data. Your SAP business data — customer records, financial transactions, employee data — is never used to train the underlying foundation models. This is a contractual guarantee from SAP's agreements with model providers, not just a policy statement.
SAP Generative AI Hub: Your Enterprise LLM Marketplace
SAP Generative AI Hub sits on top of SAP AI Core and provides a curated catalogue of large language models accessible through a unified API — the SAP AI Core Orchestration Service. Rather than each development team negotiating their own Azure OpenAI or Anthropic API agreements, enterprise architects use SAP Generative AI Hub as the single, governed access point for all LLM consumption across the SAP landscape.
LLMs Available in SAP Generative AI Hub (2026)
The SAP BTP AI Stack: Four Layers Explained
To understand where your AI investment sits and how value flows from model to business outcome, it helps to see the full stack as four distinct layers:
Layer 4 — Business Applications (SAP Joule, SAVI AI Agents, Custom Apps)
End-user AI experiences: Joule assistant in S/4HANA, Joule Studio custom agents, SAVI AI autonomous business process agents, and custom SAP BTP applications. This is what users and CXOs interact with daily.
Layer 3 — Orchestration Service (Prompt Management, Grounding, Tool Use)
The SAP AI Core Orchestration Service manages prompt templates, RAG grounding against SAP data (HANA vector store, SAP Knowledge Graph), content filtering, data masking, and multi-step agent orchestration across models and tools.
Layer 2 — SAP Generative AI Hub (Model Catalogue & Access)
Curated catalogue of 30+ LLMs from Anthropic, Azure OpenAI, Google, Meta, NVIDIA, and SAP — accessible via unified API with consumption tracking, cost controls, and data privacy guarantees per model provider.
Layer 1 — SAP AI Core (Runtime, Lifecycle, Security)
The foundational runtime: model deployment, inference execution, resource scaling, audit trails, data residency enforcement, role-based access control, and consumption metering. Everything runs through AI Core.
The Orchestration Service: Where Enterprise AI Gets Its Power
The most underappreciated component of the SAP BTP AI stack is the Orchestration Service — introduced in 2025 and now the backbone of all multi-step AI scenarios in SAP. It's what turns a raw LLM API call into a governed, grounded, enterprise-grade AI workflow.
RAG Grounding with SAP HANA Vector Store
Retrieval-Augmented Generation (RAG) is how you make an LLM answer questions about your SAP data — not just what it was trained on. The Orchestration Service integrates with SAP HANA Cloud's vector store capability, allowing you to:
- Store embeddings of your SAP documents — vendor contracts, product specs, HR policies, financial procedures — in the HANA vector store
- At inference time, the Orchestration Service retrieves the most relevant documents from the vector store and injects them into the LLM's context window as grounding material
- The LLM answers based on your actual SAP documents, not its training data — eliminating hallucinations and ensuring responses are grounded in your business context
- All retrieval and grounding happens within the SAP security perimeter — documents never leave your BTP tenant to an external RAG service
Content Filtering & Data Masking
Before any prompt reaches an LLM, the Orchestration Service applies configurable content filters (blocking harmful content categories) and data masking (replacing PII — employee names, customer data, financial figures — with anonymised tokens before the prompt leaves your SAP system). The LLM response is then de-anonymised before returning to the application. This is the mechanism that makes GDPR-compliant LLM usage in SAP possible.
Prompt Management & Versioning
The Orchestration Service includes a centralised prompt registry where enterprise AI developers store, version, test, and deploy prompt templates. When a Joule agent or SAVI AI module is updated, the underlying prompt template is versioned and auditable — enabling governance teams to review exactly what instructions the AI is operating under at any point in time.
SAP Joule Studio: Building Custom AI Agents on BTP AI Core
SAP Joule Studio Agent Builder — reached GA in 2026 — allows enterprise developers and power users to build custom AI agents on top of the SAP BTP AI foundation without writing model code. Agents built in Joule Studio automatically inherit all AI Core capabilities: grounding, content filtering, data masking, audit trails, and multi-model support.
From a technical architecture perspective, Joule Studio agents are composed of:
- Agent instructions: Prompt templates stored in the Orchestration Service's prompt registry — defining what the agent does, what SAP data it can access, and what tools it can call
- Joule Skills: 2,400+ pre-built SAP task execution capabilities — each Skill is a validated BTP function that reads or writes SAP data via standard APIs — that agents can invoke as tools
- MCP (Model Context Protocol) tools: External tool integrations that agents can call — third-party APIs, data sources, or enterprise systems outside SAP — via the standardised MCP interface that SAP adopted in 2025
- Memory and context: Short-term session memory (within a conversation) and long-term memory (persisted in SAP HANA) that agents use to maintain context across multiple interactions
SAVI AI on SAP BTP: SAVI AI's agentic platform runs natively on SAP BTP, using SAP AI Core as the inference runtime and the Orchestration Service for grounding, data masking, and prompt management. Our Finance, Procurement, Supply Chain, and Manufacturing agents are fully aligned with the Joule Studio agent architecture — meaning they inherit SAP's enterprise governance controls automatically, with no custom security configuration required.
SAP BTP AI Core vs DIY Cloud AI: Why the Platform Matters
| Capability | DIY (Azure OpenAI / AWS Bedrock direct) | SAP BTP AI Core + Gen AI Hub |
|---|---|---|
| SAP data grounding | Custom RAG pipeline required; complex SAP API integration | Native HANA vector store integration; SAP Knowledge Graph built in |
| Data residency & GDPR | Developer responsibility; requires custom PII scrubbing and logging | Built-in data masking, content filtering, and residency controls via Orchestration Service |
| SAP authorization model | Separate IAM setup; no native integration with SAP role-based access | BTP role collections control AI access — same system as all SAP authorizations |
| Multi-model flexibility | Separate SDK/API per provider; vendor lock-in risk | 30+ models via single SAP AI Core API; switch models without code changes |
| Cost governance | Token costs visible in cloud provider console only; no SAP-side budgeting | Token consumption tracked per BTP service, cost center, and AI scenario in SAP |
| Audit & compliance | Custom audit logging required; no built-in AI action audit trail | Full audit trail of all AI model calls, prompts, and responses within SAP AI Core |
| Joule & agent integration | Not possible — Joule and Joule Studio only work through AI Core | Full access to 2,400+ Joule Skills, MCP tools, and Joule Studio agent builder |
Frequently Asked Questions: SAP BTP Generative AI Hub & AI Core
Build Enterprise AI on SAP BTP with SAVI AI
SAVI AI's agentic platform runs natively on SAP BTP AI Core — giving you the governance, grounding, and security of the SAP AI Foundation with the autonomous process automation capabilities of SAVI AI agents. Book a BTP AI architecture session and see how to activate enterprise-grade AI across your SAP landscape.
