Purple8 Graph

Appearance

How Purple8 Scales ​

This page answers the genuine question directly: can this system hold up at the scale of real-world healthcare networks, manufacturing plants, insurance operations, and government agencies — or does it fall apart?

Short answer: it scales to large enterprise production workloads today. It is not yet a hyperscaler. Here is exactly what it handles, where the limits sit, and what is on the roadmap.

What a single node handles today ​

A single Purple8 node on commodity server hardware (m6i.2xlarge, 8 vCPU, 32 GB RAM, NVMe SSD):

WorkloadNumber
Nodes in the graph~500M (with int8 vector quantization)
EdgesMultiple billions (edge storage is lightweight — ~50 bytes/edge)
Graph traversal queries (3-hop MATCH)4,120 QPS at P99 5.9 ms
Hybrid search (BM25 + vector + graph)1,847 QPS at P99 8.7 ms
Journey Engine transitions/sec~12,000/sec
Ingestion (pre-extracted entities)~48,000 nodes+edges/sec
Memory for 1M nodes, 768-dim vectors0.9 GB (binary quantization)

For context: a mid-size hospital network has roughly 5–10 million patient-journey events per year. A national insurer handles 2–4 million claims per year. A large manufacturer runs 500K–2M production orders per year. A single Purple8 node absorbs all of these with headroom.

Start on a single machine ​

The default GraphEngine runs in-process. No Docker required, no separate server to manage. Install, import, query.

python
from purple8_graph import GraphEngine

engine = GraphEngine("./data")

engine.add_node("Document", {
    "title": "Q4 Report",
    "body": "Revenue grew 34% year-on-year...",
})

results = engine.query("""
    CALL db.vector.search('Document', $vec, 5)
    YIELD node, score
    RETURN node.title, score
""", vec=my_embedding)

This is production-ready for most enterprise workloads. Many teams run Purple8 this way for months or years before they need to scale out. The embedded storage handles hundreds of millions of nodes and billions of edges on a single NVMe drive.

Scale to a multi-shard cluster ​

When your dataset grows beyond a single machine — or you need higher write throughput — promote to ShardedGraphEngine. The API is identical.

python
from purple8_graph.distributed import ShardedGraphEngine, HashPartitioner

engine = ShardedGraphEngine(
    shard_configs=[
        {"path": "./data/shard-0"},
        {"path": "./data/shard-1"},
        {"path": "./data/shard-2"},
    ],
    partitioner=HashPartitioner(),
)

# Same Cypher — the engine federates across shards automatically
results = engine.query("""
    MATCH (i:JourneyInstance {entity_id: $id})-[t:ADVANCED_TO]->()
    RETURN t.from_stage, t.to_stage, t.actor, t.timestamp
    ORDER BY t.timestamp
""", id="P-44821")

Queries are automatically federated: Purple8 issues them in parallel across all shards and merges the results. Write throughput scales linearly with shard count — a 3-shard cluster delivers roughly 3× the journey transition throughput of a single node (~36,000/sec). You don't rewrite a single line of application code.

Add fault tolerance with Raft ​

For production systems that cannot afford data loss or downtime, wrap shards in Raft consensus groups. Each shard gets 3 or 5 replicas. The cluster survives the loss of any minority of nodes. Leader election completes within 300 ms.

python
from purple8_graph.distributed import RaftShardedEngine, HashPartitioner

engine = RaftShardedEngine(
    shard_groups=[
        {
            "replicas": [
                {"host": "node-0a.internal", "port": 9010, "path": "/data/s0"},
                {"host": "node-0b.internal", "port": 9010, "path": "/data/s0"},
                {"host": "node-0c.internal", "port": 9010, "path": "/data/s0"},
            ]
        },
        {
            "replicas": [
                {"host": "node-1a.internal", "port": 9010, "path": "/data/s1"},
                {"host": "node-1b.internal", "port": 9010, "path": "/data/s1"},
                {"host": "node-1c.internal", "port": 9010, "path": "/data/s1"},
            ]
        },
    ],
    partitioner=HashPartitioner(),
    election_timeout_ms=300,
)

Writes require a quorum commit (majority of replicas acknowledge before returning success). Reads from the leader are always consistent. Use read_consistency="eventual" for high-volume read paths that can tolerate replica lag.

Spread across data centres ​

Assign replicas to different availability zones or regions using the zone tag. The Raft leader-election algorithm is zone-aware — it avoids electing a leader in a zone that already holds a majority of replicas, so a full AZ outage doesn't take down your cluster.

python
engine = RaftShardedEngine(
    shard_groups=[
        {
            "replicas": [
                {"host": "us-east-1a.db.internal", "port": 9010, "path": "/data/s0", "zone": "us-east-1a"},
                {"host": "us-east-1b.db.internal", "port": 9010, "path": "/data/s0", "zone": "us-east-1b"},
                {"host": "eu-west-1a.db.internal", "port": 9010, "path": "/data/s0", "zone": "eu-west-1a"},
            ]
        },
    ],
    partitioner=HashPartitioner(),
    election_timeout_ms=500,   # slightly higher for cross-region latency
)

Scaling path at a glance ​

StageConfigurationNodesJourney transitions/secFault tolerant?
DevelopmentGraphEngine1~12,000WAL durability
Large production (single machine)GraphEngine + DiskANN1~12,000WAL durability
Scale-outShardedGraphEngine (3 shards)3~36,000No (single replica per shard)
High availabilityRaftShardedEngine (3×3)9~36,000Yes — minority failure tolerance
Multi-regionRaftShardedEngine + zones9+~36,000+Yes — full AZ failure tolerance

What stays identical at every stage ​

This is the point that matters most for long-lived production systems:

  • Cypher queries — not a single line changes between laptop and 9-node cluster
  • Python SDK — engine.query(...), engine.add_node(...), journey_engine.advance(...)
  • REST and GraphQL endpoints — same paths, same auth, same payloads
  • KMS encryption — config carries through to every shard and replica
  • Journey Engine — SLA monitoring, HITL queues, CDC events, AI advisor — all continue firing across the cluster unchanged
  • MCP server — Claude and Cursor query the cluster the same way they query a single node

No migration scripts. No re-indexing. No schema changes. Add nodes to your cluster config and you're done.

Where the real limits are — honestly ​

Purple8 is production-grade for large enterprise workloads. It is not (yet) a hyperscaler. Here is where the real boundaries sit:

Cross-shard graph traversal ​

When a multi-hop query crosses shard boundaries — node A is on shard 0, its neighbour is on shard 1 — the FederatedQueryEngine resolves this with a second scatter-gather pass. The latency cost is real: a 3-hop query that crosses shards twice may take 15–25 ms instead of 2–5 ms.

Mitigation: Use LabelPartitioner to co-locate related node types on the same shard. For healthcare, Patient, JourneyInstance, and all their journey edges live on the same shard — cross-shard hops are rare in practice.

python
partitioner=LabelPartitioner({
    "Patient":         0,
    "JourneyInstance": 0,
    "ClinicalEvent":   0,
    "Document":        1,
    "Staff":           1,
    "Organisation":    2,
})

Deep traversals on very large graphs & Built-in DoS Protection ​

Unbounded multi-hop queries (-[*]-> with no depth limit) on graphs with hundreds of millions of nodes and high average degrees can easily behave like a Denial of Service (DoS) attack, causing explosive memory usage and CPU saturation as the query fan-out explodes.

Mitigation & Hard Limits: Purple8 now natively protects clusters against graph-expansion DoS attacks. By default, MAX_TRAVERSE_DEPTH = 10 acts as a hard ceiling on all queries and internal operations. Any query attempting a BFS traversal past 10 hops returns the truncated valid 10-hop graph bounds instead of freezing the cluster.

If you specifically require deeper analytics, always bound traversals cautiously with LIMIT. You can also adjust P8G_MAX_TRAVERSE_DEPTH cautiously via environment configurations, but the 10-hop cap natively isolates tenant queries and guarantees system resilience.

Global aggregations at massive scale ​

COUNT(*) or SUM() across a billion edges is a full-table scan. On a multi-shard cluster this is parallelised, but it still takes seconds, not milliseconds, at extreme scale.

Mitigation: Materialise frequently-needed aggregations as graph nodes using CDC. When an SLA_BREACHED edge is written, a background handler increments a counter node rather than re-scanning the full edge set at query time.

Single-cluster upper bound ​

The current architecture scales to roughly 50 shards before federation overhead outweighs parallelism gains. In practice, 50 shards with 5 Raft replicas each — 250 nodes — handles several billion nodes and trillions of edges. Very few organisations on earth have workloads that exceed this today.

What's on the roadmap ​

The current architecture handles every healthcare, manufacturing, insurance, and government workload described in the use cases. What is in active development for the next scale tier:

CapabilityStatus
Raft-native shard rebalancing (zero-downtime shard split)In development
Distributed HNSW index shardingIn development
Streaming ingestion from Kafka / Kinesis nativelyPlanned
Read replica auto-scaling (serverless replica pool)Planned
Cross-shard traversal cost-based optimiserPlanned

The core design principle — that the graph, vector index, journey engine, and CDC bus are one system — does not change as these capabilities land. The complexity that a distributed system usually pushes onto application code stays inside Purple8.

The honest comparison ​

Most databases that claim to handle this class of workload require you to:

  1. Run a separate vector database
  2. Run a separate graph database
  3. Build ETL pipelines between them
  4. Run a separate workflow orchestration layer (Temporal, Airflow)
  5. Build a separate audit log
  6. Manage the operational complexity of 4+ distributed systems

The scaling challenge for those architectures is not just data volume — it is the compounding operational overhead of keeping multiple distributed systems in sync, monitored, and healthy. Purple8 replaces all of them with one system that you scale once, operate once, and monitor in one place.

Further reading ​

Purple8 Graph is proprietary software. All rights reserved.

© 2026 Purple8