LLM Providers

Purple8 Hyper Graph integrates with nine LLM providers for JourneyAIAdvisor, SchemaDetector, RAG generation, and embedding generation. All providers implement the same LLMProvider interface — you can swap them with a one-line config change or at runtime via PUT /rag/config.

RAG Studio integration

Since v0.27.2, the RAG Studio ships a 39-model LLM registry and a 21-model embedding registry. You can browse and switch models in the Configure tab or via GET /rag/models. The provider/model listed here are used for both the genai subsystem (Journey, extraction, NL-to-query) and the RAG pipeline generation step.

Quick reference

Provider	Class	Install extra	Best for
OpenAI	`OpenAIProvider`	`pip install purple8-hyper-graph[openai]`	Production, highest quality
Anthropic	`AnthropicProvider`	`pip install purple8-hyper-graph[anthropic]`	Long-context, safety-focused
Google Gemini	`GeminiProvider`	`pip install purple8-hyper-graph[google]`	Multimodal, Google Cloud
Vertex AI	`VertexAIProvider`	`pip install purple8-hyper-graph[google]`	Google Cloud with VPC-SC / CMEK
Mistral	`MistralProvider`	`pip install purple8-hyper-graph[mistral]`	Cost-effective, European data residency
Cohere	`CohereProvider`	`pip install purple8-hyper-graph[cohere]`	RAG-optimised reranking
AWS Bedrock	`BedrockProvider`	`pip install purple8-hyper-graph[bedrock]`	AWS-native, IAM auth
Azure OpenAI	`AzureOpenAIProvider`	`pip install purple8-hyper-graph[openai]`	Enterprise Azure, VNET, managed identity
Ollama / Self-Hosted	`OllamaProvider`	Ollama server running locally	Offline, private, no API cost

OpenAI

python

from purple8_graph.genai import OpenAIProvider

provider = OpenAIProvider(
    api_key="sk-...",               # or set OPENAI_API_KEY env var
    model="gpt-5.4-mini",          # default: gpt-5.4-mini
    embedding_model="text-embedding-3-small",  # default
    temperature=0.1,
    max_tokens=1024,
)

bash

OPENAI_API_KEY=sk-...

Supported models: gpt-5.4, gpt-5.4-mini, gpt-5.4-nano, o3, o4-mini
Embedding models: text-embedding-3-small (1536-dim), text-embedding-3-large (3072-dim), text-embedding-ada-002 (1536-dim, legacy)

Model selection

gpt-5.4-mini (400k context, 128k output) is the recommended default for RAG — it balances quality, speed, and cost. Use gpt-5.4 (1M context) for complex multi-hop reasoning. Use o3 or o4-mini for math/science/coding tasks that benefit from extended thinking.

Anthropic

python

from purple8_graph.genai import AnthropicProvider

provider = AnthropicProvider(
    api_key="sk-ant-...",           # or set ANTHROPIC_API_KEY env var
    model="claude-sonnet-4-20250514",  # default
    max_tokens=1024,
)

bash

ANTHROPIC_API_KEY=sk-ant-...

Supported models: claude-sonnet-4-20250514, claude-opus-4-20250514, claude-haiku-4-20250514, claude-3.5-haiku-20241022

Long context

All Claude 4 models support a 200K token context window. Claude Sonnet 4 (64k max output) is the best default — strong tool use and coding. Claude Opus 4 (32k output) is the most intelligent for sustained autonomous workloads. Claude Haiku 4 (8k output) is the fastest for real-time classification and routing.

Google Gemini

python

from purple8_graph.genai import GeminiProvider

provider = GeminiProvider(
    api_key="AIza...",              # or set GOOGLE_API_KEY env var
    model="gemini-2.5-flash",      # default
    embedding_model="gemini-embedding-001",
)

bash

GOOGLE_API_KEY=AIza...

Supported models: gemini-3.1-pro-preview, gemini-3-flash, gemini-2.5-flash, gemini-2.5-pro, gemini-2.5-flash-lite
Embedding models: gemini-embedding-001 (768-dim), text-embedding-004 (768-dim)

1M context window

All Gemini models support a 1M token context window. gemini-2.5-flash is the best price-performance starting point. gemini-3.1-pro-preview is the most advanced for deep reasoning and agentic coding.

Vertex AI

python

from purple8_graph.genai import VertexAIProvider

provider = VertexAIProvider(
    project="my-gcp-project",
    location="us-central1",
    model="gemini-2.5-flash",
)

bash

GOOGLE_APPLICATION_CREDENTIALS=/path/to/service-account.json
VERTEX_PROJECT=my-gcp-project
VERTEX_LOCATION=us-central1

Supported models: Same as Google Gemini, deployed through Vertex AI with VPC-SC, CMEK, and enterprise audit logging.

Mistral

python

from purple8_graph.genai import MistralProvider

provider = MistralProvider(
    api_key="...",                  # or set MISTRAL_API_KEY env var
    model="mistral-large-latest",  # default
    embedding_model="mistral-embed",
)

bash

MISTRAL_API_KEY=...

Supported models: mistral-large-latest (Large 3), mistral-medium-latest (Medium 3.1), mistral-small-latest (Small 4), magistral-medium-latest (reasoning), magistral-small-latest (fast reasoning), codestral-latest (256k context, code)
Embedding models: mistral-embed (1024-dim)

European data residency

Mistral is headquartered in Paris and offers EU data processing. If GDPR compliance requires European data residency, Mistral is the right choice. mistral-small-latest (Small 4) is a hybrid model that unifies instruct, reasoning, and coding — excellent value.

Cohere

python

from purple8_graph.genai import CohereProvider

provider = CohereProvider(
    api_key="...",                  # or set COHERE_API_KEY env var
    model="command-a-03-2025",     # default
    embedding_model="embed-v4.0",
    rerank_model="rerank-v4-pro",  # optional, used in hybrid search
)

bash

COHERE_API_KEY=...

Supported models: command-a-03-2025 (256k context), command-a-reasoning-08-2025 (reasoning, 23 languages), command-r7b-12-2024 (7B, fast)
Embedding models: embed-v4.0 (1536-dim, 128k context, text+image+PDF), embed-english-v3.0 (1024-dim), embed-multilingual-v3.0 (1024-dim), embed-english-light-v3.0 (384-dim), embed-multilingual-light-v3.0 (384-dim)
Reranking: Cohere Rerank v4 Pro and v4 Fast are available as reranker options in PUT /rag/config.

Ollama / Self-Hosted (OpenAI-compatible)

python

from purple8_graph.genai import OllamaProvider

provider = OllamaProvider(
    base_url="http://localhost:11434",   # default
    model="llama3.3",
    embedding_model="nomic-embed-text",  # or bge-large, mxbai-embed-large
)

bash

OLLAMA_BASE_URL=http://localhost:11434
OLLAMA_MODEL=llama3.3

Prerequisites: Install Ollama and pull your model:

bash

ollama pull llama3.3
ollama pull nomic-embed-text

Recommended self-hosted models:

Model	Parameters	Best for
`llama-4-maverick-17b-128e`	17B active (128 MoE)	Latest Llama — 1M context, multimodal, open-weight
`llama-4-scout-17b-16e`	17B active (16 MoE)	Efficient Llama 4 — 512k context, open-weight
`llama-3.3-70b`	70B	Strong general-purpose, open-weight
`llama-3.1-8b`	8B	Lightweight, edge / budget deployments
`mistral-nemo-12b`	12B	Multilingual, Apache 2.0 license
`qwen3-next-80b-a3b`	80B	Multilingual + strong reasoning
`deepseek-r1`	671B (MoE)	Math, code, logic reasoning

Ollama runs fully offline — no API keys, no outbound network calls, no data leaves your machine. Ideal for air-gapped deployments and regulated environments.

Any OpenAI-compatible API server (vLLM, TGI, llama.cpp) works with OllamaProvider — just set the base_url to your server's endpoint.

AWS Bedrock

python

from purple8_graph.genai import BedrockProvider

provider = BedrockProvider(
    region="us-east-1",            # or set AWS_REGION env var
    model="amazon.nova-pro-v1:0",  # default
)

bash

AWS_REGION=us-east-1
AWS_ACCESS_KEY_ID=AKIA...       # or use IAM role / instance profile
AWS_SECRET_ACCESS_KEY=...

Supported models:

Amazon Nova: amazon.nova-pro-v1:0 (300k context, multimodal), amazon.nova-lite-v1:0 (budget multimodal), amazon.nova-micro-v1:0 (128k, text-only, lowest cost)
Cross-provider via Bedrock: anthropic.claude-sonnet-4-20250514-v1:0, anthropic.claude-3-5-haiku-20241022-v1:0

Embedding models: amazon.titan-embed-text-v2:0 (1024-dim, 8192 tokens)

No API key management

AWS Bedrock uses IAM authentication — no separate API keys to manage. Ideal for teams already on AWS who want to access Claude, Nova, and Titan through a single billing and governance plane.

Azure OpenAI

python

from purple8_graph.genai import AzureOpenAIProvider

provider = AzureOpenAIProvider(
    api_key="...",                         # or use managed identity
    endpoint="https://my-resource.openai.azure.com/",
    deployment="gpt-5-4-mini",             # your deployment name
    api_version="2024-12-01-preview",
)

bash

AZURE_OPENAI_API_KEY=...
AZURE_OPENAI_ENDPOINT=https://my-resource.openai.azure.com/
AZURE_OPENAI_DEPLOYMENT=gpt-5-4-mini

Supported models: Same as OpenAI (gpt-5.4, gpt-5.4-mini), deployed through Azure with VNET integration, managed identity, content filtering, and enterprise audit logging.

Enterprise governance

Azure OpenAI adds VNET private endpoints, content filtering policies, and Azure Monitor diagnostics on top of the OpenAI model capabilities. Use this provider when your organisation mandates Azure-managed AI deployments.

Multi-provider fallback

MultiProvider tries providers in order, falling back on rate-limit or error:

python

from purple8_graph.genai import MultiProvider, OpenAIProvider, AnthropicProvider

provider = MultiProvider([
    OpenAIProvider(api_key="sk-..."),
    AnthropicProvider(api_key="sk-ant-..."),
])
# Falls back to Anthropic if OpenAI returns a 429 or 5xx

RAG Studio runtime model swap

Since v0.27.2, you can switch the RAG generation model at runtime without redeploying:

bash

# Switch from OpenAI to Anthropic
curl -X PUT /rag/config \
  -H "Authorization: Bearer $TOKEN" \
  -d '{"provider": "anthropic", "model": "claude-sonnet-4-20250514"}'

# Switch to a self-hosted model
curl -X PUT /rag/config \
  -H "Authorization: Bearer $TOKEN" \
  -d '{"provider": "self-hosted", "model": "llama-3.3-70b"}'

The model change takes effect immediately for the tenant — no server restart needed. Browse all available models via GET /rag/models.

Using a provider for embeddings only

Any provider can be used purely for embedding generation, without LLM calls:

python

from purple8_graph import GraphEngine
from purple8_graph.genai import OpenAIProvider

engine = GraphEngine("./data")
provider = OpenAIProvider(api_key="sk-...", embedding_model="text-embedding-3-small")

# Add a node with an auto-generated embedding
node_id = engine.add_node(
    "Document",
    {"title": "Introduction to graph databases", "body": "..."},
)
engine.add_embedding(
    node_id,
    provider.embed("Introduction to graph databases"),
)

Choosing a provider

Scenario	Recommendation
Production RAG, best quality-to-cost ratio	OpenAI `gpt-5.4-mini`
Maximum intelligence, complex multi-hop reasoning	OpenAI `gpt-5.4` or Anthropic `claude-opus-4`
Very long audit trails (>50 stages)	Anthropic Claude Sonnet 4 (200K context, 64k output)
Google Cloud deployment	Gemini 2.5 Flash (or Vertex AI for VPC-SC)
European data residency required	Mistral `mistral-small-latest` (Small 4)
Best hybrid search reranking	Cohere `command-a` + `rerank-v4-pro`
AWS-native, IAM-managed	AWS Bedrock `amazon.nova-pro-v1:0`
Azure enterprise governance	Azure OpenAI `gpt-5.4-mini`
Air-gapped / regulated / no internet	Ollama + `llama3.3`
Cost-optimised, high volume	Gemini 2.5 Flash Lite or Mistral Small 4
Math / science / coding reasoning	OpenAI `o3` or DeepSeek R1 (self-hosted)

LLM Providers ​

Quick reference ​

OpenAI ​

Anthropic ​

Google Gemini ​

Vertex AI ​

Mistral ​

Cohere ​

Ollama / Self-Hosted (OpenAI-compatible) ​

AWS Bedrock ​

Azure OpenAI ​

Multi-provider fallback ​

RAG Studio runtime model swap ​

Using a provider for embeddings only ​

Choosing a provider ​

LLM Providers

Quick reference

OpenAI

Anthropic

Google Gemini

Vertex AI

Mistral

Cohere

Ollama / Self-Hosted (OpenAI-compatible)

AWS Bedrock

Azure OpenAI

Multi-provider fallback

RAG Studio runtime model swap

Using a provider for embeddings only

Choosing a provider