Purple8 Hyper Graph — KMS & Envelope Encryption Guide

This guide covers Purple8 Hyper Graph (v0.72.0) KMS envelope encryption — provider configuration, key rotation, multi-tenant isolation, encrypted backup verification, and the admin API.

---

Table of Contents

1. Concept Overview
2. Wire Format
3. Quick Start — Local Provider (Zero Dependencies)
4. Provider Setup
   • Local (file-backed)
   • HashiCorp Vault
   • AWS KMS
   • GCP Cloud KMS
   • Azure Key Vault
5. Docker & Compose Configuration
6. Key Rotation
7. Frontend — Key Management Page
8. Admin REST API Reference
9. Upgrading an Existing Database
10. Multi-Tenant Behaviour
11. Security Considerations
12. Troubleshooting

---

1. Concept Overview

Purple8 Hyper Graph uses envelope encryption — the same pattern used by AWS S3, GCP Cloud Storage, and HashiCorp Vault.

┌────────────────────────────────────────────────────────────────┐
│  Your data  →  encrypted with DEK  →  stored in Cortex storage   │
│                                                                  │
│  DEK (Data Encryption Key) — random 256-bit AES key             │
│    └─ encrypted with KEK  →  stored alongside the blob          │
│                                                                  │
│  KEK (Key Encryption Key) — managed by your KMS provider        │
│    └─ lives outside the database (Vault / AWS / GCP / Azure)    │
└────────────────────────────────────────────────────────────────┘

Why envelope encryption?

Concern	Plain encryption	Envelope encryption
Rotate key without re-encrypting all data	❌ Must re-encrypt everything	✅ Rotate KEK; old DEKs re-wrapped on demand
Performance	Slow (every op calls KMS)	Fast (DEK cached per tenant in-process)
Per-tenant isolation	Difficult	✅ Each tenant gets its own DEK
KMS call rate	N per op	1 per tenant per server lifetime

Component map

Component	File	Role
KMSProvider ABC	src/purple8_graph/kms.py	Common interface for all backends
LocalKMSProvider	kms.py	AES-GCM, file-backed keystore, no external dependencies
VaultKMSProvider	kms.py	HashiCorp Vault Transit secrets engine
AWSKMSProvider	kms.py	AWS KMS symmetric keys
GCPKMSProvider	kms.py	GCP Cloud KMS CryptoKeys
AzureKMSProvider	kms.py	Azure Key Vault RSA/EC keys
KMSEnvelopeCodec	kms.py	Generates DEK, encrypts/decrypts blobs, caches DEKs
BrickCoreStorage	core/brick_storage.py	Calls _encode()/_decode() on every Purple8 Cortex storage write/read
GraphEngine	core/engine.py	Passes kms_codec + kms_tenant_id down to BrickCoreStorage
TenantManager	tenancy.py	Passes kms_codec + tenant ID to each GraphEngine
/admin/kms/*	secure_server.py	REST endpoints for key management
KeyManagement.tsx	frontend/src/pages/	UI for key status, rotation, enable/disable

---

2. Wire Format

Every encrypted blob stored in Cortex storage has this structure:

┌──────────┬─────────┬──────────────┬────────────┬────────────┬──────────────────┐
│  4 bytes │ 1 byte  │   2 bytes    │  N bytes   │  2 bytes   │  M bytes         │
│  "P8KM"  │ version │ key_id_len   │  key_id    │ dek_len    │  encrypted_dek   │
│ (magic)  │  = 0x01 │              │  (UTF-8)   │            │  (provider-enc.) │
└──────────┴─────────┴──────────────┴────────────┴────────────┴──────────────────┘
│  rest of blob: AES-256-GCM ciphertext (12-byte nonce ‖ tag ‖ ciphertext)       │
└─────────────────────────────────────────────────────────────────────────────────┘

Backward compatibility: _decode() checks for the P8KM magic bytes before attempting decryption. Blobs without the magic are returned as-is, so you can open an existing un-encrypted database with KMS enabled and read all legacy data without a migration step.

---

3. Quick Start — Local Provider (Zero Dependencies)

The local provider uses AES-256-GCM with a file-backed keystore. No cloud account or sidecar needed.

Via Python API

from purple8_graph.kms import LocalKMSProvider, KMSEnvelopeCodec
from purple8_graph import GraphEngine

# 1. Create / open the keystore
provider = LocalKMSProvider(
    keystore_path="./kms-keys",   # directory — created automatically
    root_password="strong-secret-change-me",
)

# 2. Create a key (first time only; subsequent runs load existing keys)
key_id = provider.create_key(alias="my-graph-key", description="Primary graph key")
# key_id looks like: "local-a3f9c2b1d044"

# 3. Wrap in a codec
codec = KMSEnvelopeCodec(provider=provider, default_key_id=key_id)

# 4. Pass to GraphEngine
engine = GraphEngine(
    "./my_graph",
    kms_codec=codec,
    kms_tenant_id="default",   # or tenant-specific ID in multi-tenant setups
)

# All reads/writes are now transparently encrypted
engine.add_node("n1", labels=["User"], properties={"name": "Alice"})
node = engine.get_node("n1")  # decrypts automatically
print(node.properties["name"])  # → Alice

Via environment variables (server)

KMS_PROVIDER=local
KMS_KEY_ID=local-a3f9c2b1d044        # from provider.create_key() above
KMS_LOCAL_KEYSTORE=/data/kms          # must be a persistent volume mount

---

4. Provider Setup

4.1 Local (file-backed)

Install: No extra dependencies — included in purple8-hyper-graph[kms].

pip install purple8-hyper-graph[kms]

Keystore structure (inside KMS_LOCAL_KEYSTORE):

/data/kms/
  keys.json          # encrypted KEKs (chmod 600)

The keystore is encrypted with a root key derived from root_password via PBKDF2-HMAC-SHA256 (600 000 iterations). Never lose this password — there is no recovery without it.

Environment variables:

KMS_PROVIDER=local
KMS_KEY_ID=local-<hex>              # key_id returned by create_key()
KMS_LOCAL_KEYSTORE=/data/kms        # path to the keystore directory

---

4.2 HashiCorp Vault

Install:

pip install purple8-hyper-graph[kms-full]   # or: pip install hvac

Vault setup (once):

# Enable Transit secrets engine
vault secrets enable transit

# Create a named key
vault write -f transit/keys/purple8-hyper-graph

# Create a policy
vault policy write purple8-hyper-graph - <<EOF
path "transit/encrypt/purple8-hyper-graph"   { capabilities = ["update"] }
path "transit/decrypt/purple8-hyper-graph"   { capabilities = ["update"] }
path "transit/datakey/plaintext/purple8-hyper-graph" { capabilities = ["update"] }
path "transit/rotate/purple8-hyper-graph"    { capabilities = ["update"] }
path "transit/keys/purple8-hyper-graph"      { capabilities = ["read"] }
EOF

# Create a token (or use AppRole / Kubernetes auth)
vault token create -policy=purple8-hyper-graph -period=24h

Environment variables:

KMS_PROVIDER=vault
KMS_KEY_ID=purple8-hyper-graph            # Vault key name (not the token)
VAULT_ADDR=http://vault.internal:8200
VAULT_TOKEN=hvs.xxxxxxxxxxxxx       # or use VAULT_ROLE_ID + VAULT_SECRET_ID
VAULT_MOUNT=transit                 # default

Dev sidecar (local testing):

# Already included in docker-compose.dev.yml
docker compose -f docker-compose.dev.yml up
# Vault UI: http://localhost:8200/ui  token: dev-root-token

---

4.3 AWS KMS

Install:

pip install purple8-hyper-graph[kms-full]   # or: pip install boto3

AWS setup (once):

# Create a symmetric KMS key
aws kms create-key \
  --description "Purple8 Hyper Graph encryption key" \
  --key-usage ENCRYPT_DECRYPT \
  --key-spec SYMMETRIC_DEFAULT \
  --query 'KeyMetadata.KeyId' --output text
# → arn:aws:kms:us-east-1:123456789012:key/abc-def-...

# Create an alias
aws kms create-alias \
  --alias-name alias/purple8-hyper-graph \
  --target-key-id <key-id>

# IAM policy needed by the API container's role/user:
# kms:GenerateDataKey, kms:Decrypt, kms:DescribeKey, kms:ListKeys

Environment variables:

KMS_PROVIDER=aws
KMS_KEY_ID=arn:aws:kms:us-east-1:123456789012:key/abc-def-...
                                    # or just: alias/purple8-hyper-graph
AWS_DEFAULT_REGION=us-east-1
# If not using IAM role / instance profile:
AWS_ACCESS_KEY_ID=AKIA...
AWS_SECRET_ACCESS_KEY=...

---

4.4 GCP Cloud KMS

Install:

pip install purple8-hyper-graph[kms-full]   # or: pip install google-cloud-kms

GCP setup (once):

# Create key ring
gcloud kms keyrings create purple8-hyper-graph \
  --location global

# Create symmetric encryption key
gcloud kms keys create primary \
  --location global \
  --keyring purple8-hyper-graph \
  --purpose encryption

# Grant the service account access
gcloud kms keys add-iam-policy-binding primary \
  --location global \
  --keyring purple8-hyper-graph \
  --member serviceAccount:purple8@my-project.iam.gserviceaccount.com \
  --role roles/cloudkms.cryptoKeyEncrypterDecrypter

Environment variables:

KMS_PROVIDER=gcp
KMS_KEY_ID=projects/my-project/locations/global/keyRings/purple8-hyper-graph/cryptoKeys/primary
GCP_PROJECT=my-project
GCP_KMS_LOCATION=global
GCP_KMS_KEYRING=purple8-hyper-graph
GOOGLE_APPLICATION_CREDENTIALS=/run/secrets/gcp-sa.json

---

4.5 Azure Key Vault

Install:

pip install purple8-hyper-graph[kms-full]   # or: pip install azure-keyvault-keys azure-identity

Azure setup (once):

# Create Key Vault
az keyvault create \
  --name purple8-kv \
  --resource-group my-rg \
  --location eastus \
  --sku standard

# Create RSA key
az keyvault key create \
  --vault-name purple8-kv \
  --name purple8-hyper-graph-key \
  --kty RSA \
  --size 2048

# Assign Key Vault Crypto User role to service principal
az role assignment create \
  --role "Key Vault Crypto User" \
  --assignee <sp-object-id> \
  --scope /subscriptions/.../resourceGroups/my-rg/providers/Microsoft.KeyVault/vaults/purple8-kv

Environment variables:

KMS_PROVIDER=azure
KMS_KEY_ID=purple8-hyper-graph-key        # key name in the vault
AZURE_VAULT_URL=https://purple8-kv.vault.azure.net/
AZURE_CLIENT_ID=<service-principal-app-id>
AZURE_CLIENT_SECRET=<service-principal-secret>
AZURE_TENANT_ID=<azure-ad-tenant-id>

---

5. Docker & Compose Configuration

Production (docker-compose.yml)

KMS env vars are already present with empty defaults (disabled by default):

environment:
  - KMS_PROVIDER=${KMS_PROVIDER:-}          # set to local/vault/aws/gcp/azure
  - KMS_KEY_ID=${KMS_KEY_ID:-}
  - KMS_LOCAL_KEYSTORE=${KMS_LOCAL_KEYSTORE:-/data/kms}
  - VAULT_ADDR=${VAULT_ADDR:-}
  - VAULT_TOKEN=${VAULT_TOKEN:-}
  - VAULT_MOUNT=${VAULT_MOUNT:-transit}
  # AWS / GCP / Azure vars also present — see docker-compose.yml

Add to your .env:

KMS_PROVIDER=vault
KMS_KEY_ID=purple8-hyper-graph
VAULT_ADDR=https://vault.internal:8200
VAULT_TOKEN=hvs.your-token

Dev (docker-compose.dev.yml)

The dev file sets KMS_PROVIDER=local by default and includes a Vault sidecar:

# Default: local KMS (no Vault needed)
KMS_PROVIDER: "${KMS_PROVIDER:-local}"
KMS_KEY_ID:   "${KMS_KEY_ID:-purple8-dev-key}"

# Override to use Vault sidecar:
#   KMS_PROVIDER=vault
#   VAULT_TOKEN=dev-root-token

The purple8-hyper-graph-vault-dev sidecar starts automatically. It is in-memory only — data is lost on restart. This is intentional for development.

---

6. Key Rotation

Rotation replaces the KEK material while preserving the ability to decrypt any existing DEK (old KEK versions are archived). No data re-encryption is needed.

Via the Frontend (recommended)

1. Open the Purple8 Hyper Graph UI → Key Management (sidebar, shield icon)
2. Find the key to rotate in the table
3. Click Rotate → confirm
4. The UI shows a success toast and refreshes the key metadata

Via REST API

TOKEN="<admin-bearer-token>"

# Rotate key (archives old KEK, generates new one, invalidates all DEK caches)
curl -X POST http://localhost:8000/admin/kms/keys/local-a3f9c2b1d044/rotate \
  -H "Authorization: Bearer $TOKEN"

# Response:
# {"success": true, "key_id": "local-a3f9c2b1d044", "rotated_at": "2026-03-05T..."}

What happens during rotation?

Before rotation:                    After rotation:
┌────────────────────┐              ┌────────────────────┐
│ Active KEK (v3)    │              │ Active KEK (v4) ← NEW
│ Previous KEK (v2)  │    rotate    │ Previous KEK (v3)  │
│ Previous KEK (v1)  │  ─────────▶  │ Previous KEK (v2)  │
└────────────────────┘              └────────────────────┘

Existing DEKs encrypted with v1/v2/v3 → still decryptable
New DEKs encrypted with v4
Last 3 versions retained (configurable)
All in-process DEK caches invalidated → next read fetches new DEK

Rotation schedule recommendations

Environment	Suggested frequency
Development	On demand
Production	Quarterly (at minimum)
PCI-DSS / HIPAA	Annually (mandatory), quarterly (recommended)
Post-compromise	Immediately

---

7. Frontend — Key Management Page

Navigate to /keys in the Purple8 Hyper Graph UI (shield icon in the left sidebar).

Status Banner

Shows whether KMS is enabled, which provider is active, and the active key ID. If KMS is disabled, a warning banner is shown with setup instructions.

Keys Table

Column	Description
Key ID	Unique identifier (e.g. local-a3f9c2b1d044)
Alias	Human-readable name
State	enabled (green) / disabled (yellow) / pending_deletion (red)
Created	ISO timestamp
Last Rotated	ISO timestamp or Never
Actions	Rotate / Disable / Enable buttons

Actions

Action	Result
Rotate	Archives old KEK, generates new KEK, invalidates all DEK caches
Disable	Blocks new encryptions with this key; existing data remains readable
Enable	Re-enables a disabled key

Keys in pending_deletion state cannot be re-enabled (cloud provider constraint).

---

8. Admin REST API Reference

All endpoints require Authorization: Bearer <token> with the ADMIN_SETTINGS permission.

GET /admin/kms/status

{
  "enabled": true,
  "provider": "vault",
  "key_id": "purple8-hyper-graph"
}

GET /admin/kms/keys

Returns an array of KeyMetadata objects:

[
  {
    "key_id": "local-a3f9c2b1d044",
    "alias": "my-graph-key",
    "provider": "local",
    "state": "enabled",
    "created_at": 1741130400.0,
    "rotated_at": 1741216800.0,
    "description": "Primary graph key",
    "tags": {}
  }
]

GET /admin/kms/keys/{key_id}

Returns a single KeyMetadata object (same schema as above).

POST /admin/kms/keys/{key_id}/rotate

{ "success": true, "key_id": "local-a3f9c2b1d044", "rotated_at": "2026-03-05T12:00:00Z" }

Also invalidates all in-process DEK caches for all tenants.

POST /admin/kms/keys/{key_id}/disable

{ "success": true, "key_id": "local-a3f9c2b1d044" }

POST /admin/kms/keys/{key_id}/enable

{ "success": true, "key_id": "local-a3f9c2b1d044" }

Error responses:

Status	Meaning
503 Service Unavailable	KMS not configured (KMS_PROVIDER not set)
404 Not Found	Key ID does not exist
403 Forbidden	Token lacks ADMIN_SETTINGS permission

---

9. Upgrading an Existing Database

You can enable KMS on an existing un-encrypted Purple8 Hyper Graph database without downtime or migration.

How it works

BrickCoreStorage checks for the 4-byte P8KM magic header before decrypting:

• Old blob (no magic) → returned as-is (plain JSON)
• New blob (has magic) → decrypted with the DEK

This means all existing data is instantly readable even after enabling KMS. New writes will be encrypted; old data remains un-encrypted until it is overwritten.

Step-by-step

# 1. Stop the server (recommended but not required for in-place migration)
docker compose down

# 2. Configure KMS env vars (see provider sections above)
# Add to .env:
KMS_PROVIDER=local
KMS_KEY_ID=local-a3f9c2b1d044
KMS_LOCAL_KEYSTORE=/data/kms

# 3. Start the server — existing data is immediately readable
docker compose up -d

Full re-encryption & Six-Sigma Storage Integrity Controls

To encrypt all existing blobs proactively, iterate and rewrite each node and edge via the SDK or admin API.

Purple8 Hyper Graph enforces Six-Sigma operational controls across encrypted storage and backups:

• Streaming gzip/JSONL backups with P8KM magic verification: Backups stream directly as compressed JSONL archives without buffering the full graph in memory. During backup and restore, the engine validates the 4-byte P8KM envelope header on encrypted payloads and enforces the P8G_BACKUP_MAX_ARCHIVE_MB archive size guard to prevent decompression bombs or unbounded disk consumption.
• Online storage bit-rot scrubber (POST /admin/storage/scrub): Operators can trigger an online integrity scrub at any time via POST /admin/storage/scrub (or engine.scrub_integrity() in Python) to verify BrickCoreStorage / BrickWAL checksums and P8KM AES-256-GCM authentication tags without taking the database offline.

---

10. Multi-Tenant Behaviour

Each tenant gets its own DEK:

Tenant A ──▶ DEK_A ──▶ encrypted with KEK ──▶ stored in graph
Tenant B ──▶ DEK_B ──▶ encrypted with KEK ──▶ stored in graph

• DEKs are cached in-process per tenant (no KMS call on every read/write)
• POST /admin/kms/keys/{id}/rotate invalidates all tenant DEK caches, so each tenant fetches a new DEK on its next operation
• Tenants cannot access each other's DEKs

---

11. Security Considerations

Do

• ✅ Store KMS_LOCAL_KEYSTORE on a persistent, backed-up volume (not a tmpfs)
• ✅ Use Vault / AWS / GCP / Azure in production (local provider is for dev/staging only)
• ✅ Rotate keys at least quarterly in production
• ✅ Restrict ADMIN_SETTINGS permission to infrastructure/ops roles only
• ✅ Back up the keystore or record the KMS key ARN — without it, encrypted data is unrecoverable
• ✅ Use IAM roles / workload identity instead of static credentials for cloud providers

Don't

• ❌ Use the local provider in production with sensitive data
• ❌ Store VAULT_TOKEN / AWS_SECRET_ACCESS_KEY in source control
• ❌ Disable a key while data encrypted by it has not been re-encrypted
• ❌ Delete KMS keys without verifying all DEKs have been re-wrapped

Threat model

Threat	Mitigation
Cortex storage volume compromised	Data encrypted with per-tenant AES-256-GCM DEK
Container image/layer exfiltrated	DEKs in memory only; keystore separate from data volume
DEK cache exfiltrated	DEKs are ephemeral; rotate KEK to invalidate
KMS provider unavailable	Server continues to serve cached DEKs; new tenant sessions queue until KMS recovers

---

12. Troubleshooting

GET /admin/kms/status returns 503

KMS is not configured. Set KMS_PROVIDER environment variable and restart.

ValueError: Failed to decrypt with key local-xxx

Possible causes:

1. Wrong keystore — KMS_LOCAL_KEYSTORE points to a different directory than the one used to create the key
2. Wrong password — root_password used to open the LocalKMSProvider does not match the one used to create the key
3. Corrupt keystore — restore from backup

IO error: lock hold by current process

You are trying to open the same BrickCoreStorage (Cortex) directory twice in the same process. Ensure only one GraphEngine / BrickCoreStorage instance is open per database path at a time. Always call .close() before re-opening.

PermissionError: Key xxx is not enabled

The key has been disabled. Re-enable it via:

curl -X POST http://localhost:8000/admin/kms/keys/<key_id>/enable \
  -H "Authorization: Bearer $ADMIN_TOKEN"

Vault 403 permission denied

The Vault token does not have the purple8-hyper-graph policy. Verify:

vault token capabilities <your-token> transit/encrypt/purple8-hyper-graph

Should return [update]. If not, recreate the token with the correct policy.

AWS AccessDeniedException

The IAM role/user is missing one of: kms:GenerateDataKey, kms:Decrypt, kms:DescribeKey. Update the IAM policy.

Performance: high latency on first request per tenant

This is expected — the first request per tenant triggers a GenerateDataKey call to the KMS provider. Subsequent requests use the in-process DEK cache. Warm up the cache at startup if this is a concern.

---

See Also

• src/purple8_graph/kms.py — Full provider implementation
• tests/test_kms.py — 30 test cases covering all providers, rotation, and upgrade path

---