Unified Data Store
One engine. Four databases eliminated. Zero sync bugs.
Purple8 Hyper Graph is a Hyper Graph DB — a new category of database that unifies four traditionally separate systems into a single embedded engine. It is not a graph database with plugins. It is not a vector database with a graph layer. It is a purpose-built engine where graph, vector, document, and full-text are equal citizens sharing one storage layer, one write path, and one query engine.
| Capability | What it replaces | Purple8 module |
|---|---|---|
| Graph Database | Neo4j, FalkorDB, Kùzu | core.engine — property graph with nodes, edges, labels, adjacency indexes |
| Vector Database | Pinecone, Weaviate, Qdrant, Milvus | core.vector — native HNSW index with multi-vector slots, int8/binary quantization, ANN pre-filtering |
| Document Store | MongoDB, Postgres JSONB, DynamoDB | core.models.Node.properties — schemaless dict[str, Any] on every node, arbitrary JSON depth |
| Full-Text Search | Elasticsearch, OpenSearch, Typesense | fulltext — BM25 inverted index with configurable tokenisation, stopwords, and hybrid text+vector search |
All four live in the same BrickCoreStorage (Cortex) instance, encrypted at rest with the same KMS envelope, backed up in one call, and queryable through a single REST API and Cypher endpoint. Durability is provided by a Write-Ahead Log (WAL) with fdatasync — there is no RocksDB and no Raft consensus layer.
Why this matters
The infrastructure sprawl problem
A typical RAG or AI-powered application today looks like this:
App → Postgres (documents)
→ Pinecone (vectors)
→ Elasticsearch (keyword search)
→ Neo4j (relationships)
→ sync layer (to keep them consistent)That is four connection strings, four auth configurations, four backup strategies, four failure modes, and a custom synchronisation layer that you wrote and now have to maintain forever. Every new developer on the team has to understand all five systems. Every deployment is a multi-container orchestration. Every data migration touches four schemas.
The Purple8 answer: a Hyper Graph DB
App → Purple8 Hyper Graph (everything)One pip install. One storage directory. One backup command. One encryption key. One auth layer. One container. Zero synchronisation bugs.
We call this a Hyper Graph DB because it goes beyond what any single database category offers. It is not "a graph database with vector search bolted on." It is a new kind of engine where all four modes — graph relationships, vector embeddings, document properties, and keyword search — are first-class citizens with independent storage paths, shared atomicity, and a single query surface.
How the four engines are isolated
A natural concern: "If everything runs in one process, won't a heavy document write stall my vector search?" The answer is no — because the four storage modes use physically separate data structures that don't compete for the same resources:
Storage-level isolation
┌──────────────────────────────────────────────────────────┐
│ Purple8 Hyper Graph │
│ │
│ ┌──────────────────────┐ ┌──────────────────────┐ │
│ │ RocksDB │ │ HNSW Index │ │
│ │ (graph + documents) │ │ (vectors — hnswlib) │ │
│ │ │ │ │ │
│ │ n: nodes + props │ │ Separate memory │ │
│ │ e: edges │ │ Separate files │ │
│ │ o: outgoing adj │ │ (hnsw.bin) │ │
│ │ i: incoming adj │ │ │ │
│ │ l: label index │ │ No RocksDB writes │ │
│ │ p: property index │ │ during search │ │
│ │ m: metadata │ │ │ │
│ └──────────────────────┘ └──────────────────────┘ │
│ │
│ ┌──────────────────────┐ ┌──────────────────────┐ │
│ │ Full-Text Index │ │ LRU Caches │ │
│ │ (BM25 — in-memory │ │ │ │
│ │ inverted index) │ │ Traversal cache │ │
│ │ │ │ Vector cache │ │
│ │ No RocksDB writes │ │ Label allow-set │ │
│ │ during search │ │ cache │ │
│ └──────────────────────┘ └──────────────────────┘ │
└──────────────────────────────────────────────────────────┘| Component | Storage medium | Contention with graph writes? |
|---|---|---|
| Graph topology (nodes, edges, adjacency) | RocksDB key prefixes (n:, e:, o:, i:) | Write lock serialises mutations only |
| Document properties | Stored inside the node record (same RocksDB key) | Same write as node — no extra I/O |
| Vector index (HNSW) | Separate hnswlib data structure + hnsw.bin file | No RocksDB I/O — reads are pure in-memory ANN |
| Full-text index (BM25) | Separate in-memory inverted index | No RocksDB I/O — reads are pure in-memory BM25 scoring |
| Property indexes | RocksDB p: prefix keys | Read-only lookups — O(k) not O(n) |
Write path isolation
All writes go through a single _write_lock (reentrant threading.RLock). This guarantees atomicity across all four modes — when you call add_node(), the graph topology, document properties, vector embedding, and full-text tokens are all written in one atomic operation. There is no eventual consistency. There is no sync layer.
But critically, the write lock only blocks other writes. Read operations — vector_search(), full_text_search(), traverse(), get_node() — are lock-free and run concurrently without blocking.
# All four storage modes updated atomically in one call:
fte.add_node("doc-1", labels=["Contract"], properties={
"title": "Master Services Agreement",
"body": "This agreement is entered into by...",
"clauses": [{"id": "2.3", "text": "Limitation of liability..."}]
}, embedding=contract_embedding)
# Under the hood:
# 1. RocksDB write: node record + properties + adjacency indexes ← graph + document
# 2. hnswlib insert: add vector to HNSW graph ← vector (no RocksDB I/O)
# 3. BM25 index: tokenise + update inverted index ← full-text (no RocksDB I/O)
# All three happen inside the same write lock — atomically.What this means in practice
- A bulk document ingest (10,000 nodes with properties) does not degrade vector search latency — because vector search reads from a separate in-memory HNSW structure, not RocksDB.
- A heavy vector search workload does not slow down graph traversals — because traversals use RocksDB prefix-seek reads while vector search uses hnswlib.
- A full-text search does not touch RocksDB at all — it scans the in-memory inverted index.
- Property index lookups are O(k) reads from RocksDB's
p:prefix — they don't compete with graph topology reads onn:ore:prefixes.
Document ingestion pipeline
Purple8 includes a full document ingestion pipeline that flows cleanly into the unified store:
┌──────────┐ ┌───────────┐ ┌──────────┐ ┌───────────┐
│ Source │ → │ Extract │ → │ Compare │ → │ Publish │
│ (upload / │ │ (DocIntel │ │ (diff vs │ │ (commit │
│ URL / │ │ parses + │ │ existing│ │ to graph │
│ paste) │ │ chunks) │ │ graph) │ │ store) │
└──────────┘ └───────────┘ └──────────┘ └───────────┘How it works
- Source — Upload a file (PDF, DOCX, IFC, DXF, images with OCR, and 50+ other formats), paste a URL, or paste raw text.
- Extract — Purple8 DocIntel parses the document, detects entities and relationships, and chunks the content using one of 6 strategies (fixed, recursive, semantic, sentence window, parent-child, or custom).
- Compare — The extracted entities are diffed against the existing graph. You see what's new, what's updated, and what conflicts before anything is committed.
- Publish — Commit to the graph with one click. Entities become nodes, relationships become edges, document text becomes properties, and embeddings are indexed — all in one atomic operation.
What the ingestion pipeline stores
When a document is ingested, Purple8 creates one node per entity with:
- Labels —
["Document"],["Person"],["Company"], etc. - Properties — the full extracted content as schemaless JSON (title, body, metadata, source info, parsed fields)
- Embedding — vector embedding for semantic search
- Edges — typed relationships linking entities to each other
All four storage modes are populated in a single atomic write. There is no two-phase commit, no message queue, no background sync job.
Index management
Purple8 provides editable, user-controlled indexes — much like creating indexes in MongoDB or DynamoDB, but covering all four storage modes from one interface.
Property indexes (document-style queries)
Build secondary indexes on any node property for O(k) lookups instead of O(n) full scans:
# Create indexes — like MongoDB's createIndex() or DynamoDB's GSI
engine.build_property_index("Invoice", "vendor")
engine.build_property_index("Invoice", "status")
engine.build_property_index("Contract", "effective_date")
# Fast equality lookup — O(k), not O(n)
invoices = engine.find_nodes_by_property("Invoice", "vendor", "Acme Corp")
# Multi-filter query with automatic index selection
# Purple8 picks the most selective indexed property as the primary key,
# then post-filters the rest in memory — just like a query planner.
results = engine.find_nodes(
"Invoice",
filters={"vendor": "Acme Corp", "status": "pending", "currency": "USD"},
limit=50,
)
# List all indexes
indexes = engine.list_property_indexes()
# [("Contract", "effective_date"), ("Invoice", "status"), ("Invoice", "vendor")]Vector index configuration
The HNSW vector index is tunable at both build time and query time:
# Construction-time: P8G_HNSW_EF_CONSTRUCTION=200 (default)
# Higher = better recall, slower build
engine = GraphEngine("./data", embedding_dimension=1536)
# Query-time: ef_search controls accuracy/speed tradeoff
engine._vector_index.ef_search = 100 # higher = better recall, slower search
# Quantization: 4× RAM savings (int8) or 32× (binary)
# Set via env var: P8G_VECTOR_QUANTIZATION=int8
# Backend: swap HNSW for DiskANN (larger-than-RAM indexes)
# Set via env var: P8G_VECTOR_BACKEND=diskann
# Auto-compaction: tombstoned vectors trigger background rebuild
# after P8G_VECTOR_COMPACT_THRESHOLD deletions (default: 500)
# or 10% of index size — whichever is smallerFull-text index configuration
from purple8_graph.fulltext import FullTextEngine
fte = FullTextEngine(
engine,
index_fields=["title", "body", "tags", "clause_text"], # choose what to index
k1=1.5, # BM25 term frequency saturation
b=0.75, # BM25 document length normalisation
)
# The full-text index auto-updates on add_node / update_node / delete_node —
# no manual reindexing required.REST API for index management
All indexes are manageable via the REST API — no code changes needed:
# Build a property index
POST /api/indexes
{"label": "Invoice", "property": "vendor"}
# List all indexes
GET /api/indexes
# Configure RAG retrieval settings (chunking, embedding, reranking)
PUT /rag/config
{
"chunking_strategy": "semantic",
"embedding_model": "text-embedding-3-small",
"retrieval_k": 15,
"min_similarity": 0.7,
"reranker": "cohere",
"hybrid_weight": 0.5
}Querying: the MongoDB / DynamoDB experience
Purple8's document store provides querying capabilities comparable to MongoDB or DynamoDB — but with graph traversals, vector search, and full-text search available in the same query:
Simple document queries (MongoDB-style)
# Find by property (like MongoDB's find())
invoices = engine.find_nodes("Invoice", filters={
"vendor": "Acme Corp",
"status": "pending",
})
# Find by property with limit (like DynamoDB's query with Limit)
recent = engine.find_nodes("Contract", filters={
"effective_date": "2026-04-01",
}, limit=10)
# Get a single document by ID (like MongoDB's findOne / DynamoDB's GetItem)
doc = engine.get_node("invoice-42")
print(doc.properties["line_items"][0]["sku"]) # "WDG-001"
# Update properties (like MongoDB's updateOne / DynamoDB's UpdateItem)
engine.update_node("invoice-42", properties={
"status": "approved",
"approved_by": "alice@acme.com",
"approved_at": "2026-04-07T10:30:00Z",
})Full-text search (Elasticsearch-style)
# Keyword search with BM25 scoring
hits = fte.full_text_search("limitation liability indemnification", top_k=20)
# Hybrid: BM25 + vector similarity (like Elasticsearch kNN + text)
hits = fte.hybrid_text_vector_search(
query="limitation of liability",
embedding=query_embedding,
top_k=10,
text_weight=0.3,
vector_weight=0.7,
)Graph traversals (Neo4j-style)
# Follow relationships
results = engine.traverse("contract-101", edge_types=["SIGNED_BY"], max_depth=1)
# Cypher queries — combine everything
results = engine.execute_cypher("""
CALL db.vector.search('Contract', $vec, 10) YIELD node, score
WHERE score > 0.8
MATCH (node)-[:SIGNED_BY]->(person:Person)
MATCH (person)-[:WORKS_AT]->(company:Company)
WHERE company.industry = 'Financial Services'
RETURN node.title, person.name, company.name, score
ORDER BY score DESC
""", {"vec": query_embedding})What you cannot do with separate databases
None of these queries are possible when your data lives in four different systems:
# "Find contracts similar to this one, then follow the supersession chain,
# then return all parties involved in the latest version"
chain = engine.hybrid_search(
query_embedding, top_k=5,
expand_edges=["SUPERSEDES"], max_depth=3,
)
# With separate databases, this requires:
# 1. Pinecone query → get similar doc IDs
# 2. Neo4j query → traverse SUPERSEDES edges
# 3. Neo4j query → get party nodes
# 4. Postgres query → get document properties
# 5. App-level join code to stitch results together
# 6. Hope nothing changed between steps 1 and 5Impact on developers
Before Purple8 (4 systems)
# Connection setup
pg = psycopg2.connect("postgres://...")
pinecone.init(api_key="...", environment="...")
es = Elasticsearch(["https://..."], api_key="...")
neo4j_driver = GraphDatabase.driver("bolt://...", auth=("...", "..."))
# Ingest a document — 4 systems, 4 transactions, no atomicity
pg.execute("INSERT INTO documents ...") # 1. store doc
pinecone.upsert(vectors=[(id, embedding)]) # 2. index vector
es.index(index="docs", body=doc_text) # 3. index text
neo4j_driver.session().run("CREATE (d:Doc)…") # 4. create graph node
# What if step 3 fails? Now your data is inconsistent.
# You need: retry logic, dead letter queues, reconciliation jobs.With Purple8 (1 system)
from purple8_graph import GraphEngine
from purple8_graph.fulltext import FullTextEngine
engine = GraphEngine("./data")
fte = FullTextEngine(engine)
# One call. Atomic. Done.
fte.add_node("doc-1", labels=["Document"],
properties={"title": "MSA", "body": "..."},
embedding=vec)Developer experience wins
| Concern | 4 systems | Purple8 |
|---|---|---|
| Lines of setup code | 20+ (4 clients, 4 configs) | 3 |
| Dependencies in requirements.txt | 8+ (psycopg2, pinecone-client, elasticsearch, neo4j, ...) | 1 (purple8-hyper-graph) |
| Error handling | Compensating transactions, DLQs, reconciliation | Single try/except |
| Schema migrations | 4 migration scripts per change | None — schemaless by default |
| Local development | Docker Compose with 4+ services | pip install purple8-hyper-graph |
| Integration tests | Mock or spin up 4 services | One in-memory engine instance |
| New developer onboarding | Learn 4 query languages + sync layer | Learn one SDK |
| Time to first working prototype | Days (infra setup) | Minutes |
Impact on non-developer users
Purple8's web UI (RAG Studio) makes the unified store accessible to non-technical users — no code required:
Document ingestion
The 4-step Ingest wizard (Source → Extract → Compare → Publish) lets business users upload documents (PDF, DOCX, spreadsheets, even CAD files) and see the extracted entities and relationships before committing them to the graph. No CLI, no API calls.
Search and query
RAG Studio's Query tab lets users ask questions in natural language. Under the hood, Purple8 searches the vector index, the full-text index, and the graph simultaneously — but the user just types a question and gets an answer with source citations.
Configuration
The Configure tab exposes all RAG settings (chunking strategy, embedding model, retrieval parameters, reranker) through a graphical interface. Changes take effect immediately — no redeploy needed.
What this means for teams
- Data analysts can upload CSVs and immediately query relationships
- Legal teams can search contracts by meaning, not just keywords
- Compliance officers can trace audit trails through the graph without writing Cypher
- Product managers can prototype RAG features without waiting for engineering
Performance characteristics
Read performance (no contention)
| Operation | Latency | How |
|---|---|---|
get_node(id) | ~0.1 ms | RocksDB point read |
vector_search(vec, top_k=10) | ~1–3 ms | In-memory HNSW — no RocksDB I/O |
full_text_search(query, top_k=10) | ~0.5–2 ms | In-memory BM25 — no RocksDB I/O |
find_nodes_by_property(label, prop, val) | ~0.2 ms | RocksDB p: prefix seek |
traverse(node, depth=2) | ~2–10 ms | RocksDB o:/i: prefix seeks |
hybrid_search(vec, top_k=10, expand=2) | ~3–12 ms | HNSW + RocksDB traversal |
Write performance
| Operation | Throughput | How |
|---|---|---|
add_node (with embedding) | >100k nodes/min at 1M scale | Single write lock, RocksDB batch write + HNSW insert |
add_edge | >200k edges/min | RocksDB prefix-key writes |
| Bulk ingest (10k docs) | ~100 seconds | DocIntel parallel extraction + serial commit |
Why document processing doesn't impact search
The critical insight: write operations hold the lock briefly (microseconds for RocksDB + HNSW insert), then release it. Read operations never acquire the lock. So even during a 100k-document bulk ingest:
- Vector search latency stays at ~1–3 ms (reads don't block on writes)
- Full-text search latency stays at ~0.5–2 ms (separate in-memory index)
- Graph traversal latency stays at ~2–10 ms (RocksDB concurrent reads are lock-free)
The only potential contention is write-write: if two threads try to write simultaneously, one waits for the other. This is intentional — it's what gives you atomicity. For write-heavy workloads, Purple8 offers horizontal sharding with per-shard write locks, so writes to different shards proceed in parallel.
De-risking: "All eggs in one basket?"
The most common objection to a unified store is: "Aren't I putting all my eggs in one basket?" Here is how Purple8 de-risks this:
1. Raft consensus replication
Purple8's Raft implementation replicates all four storage modes across multiple nodes. Every write — graph mutations, document updates, vector inserts, full-text index changes — is committed to the Raft log and applied to all replicas after majority quorum.
Client write → Leader (RaftNode)
│
├──append_entry──► Follower 1 ✓
├──append_entry──► Follower 2 ✓
│ ← majority ACK ───────────
▼
Apply to ShardedGraphEngine (all 4 storage modes)- Leader election: If the leader goes down, a follower is elected within 150–300 ms.
- Linearisability: Committed entries are applied synchronously in tick order — no stale reads.
- WAL persistence: The Raft log is backed by a
WALWriterwith fsync — log entries survive process restarts.
2. Backup and restore
One backup captures everything — graph, documents, vectors, indexes, metadata:
from purple8_graph.production import BackupManager, BackupConfig
mgr = BackupManager(BackupConfig(
backup_dir=Path("./backups"),
compress=True,
checksum=True, # SHA-256 integrity verification
max_backups=10, # auto-rotate old backups
))
# One call backs up all four storage modes
manifest = await mgr.create_backup(engine, description="Pre-migration snapshot")
print(f"Backed up {manifest.node_count} nodes, {manifest.edge_count} edges")
print(f"Size: {manifest.size_bytes / 1024 / 1024:.1f} MB (compressed)")
# Restore — verified with SHA-256 checksum
await mgr.restore_backup(manifest.backup_id, engine, verify_checksum=True)Compare this with 4-system backup:
| Concern | 4 systems | Purple8 |
|---|---|---|
| Backup commands | pg_dump + Pinecone export + ES snapshot + Neo4j dump | mgr.create_backup() |
| Restore commands | 4 restore scripts, ordered correctly | mgr.restore_backup() |
| Point-in-time consistency | Very hard — 4 systems at different times | Guaranteed — one atomic snapshot |
| Integrity verification | 4 separate checksums | One SHA-256 |
| Storage overhead | 4× metadata | One compressed archive |
3. Horizontal sharding
For deployments that need write scale beyond a single node, Purple8 supports horizontal sharding with per-shard isolation:
from purple8_graph import GraphEngine
from purple8_graph.sharding import ShardedGraphEngine, HashPartitioner
# Each shard is an independent GraphEngine with its own RocksDB, HNSW, and full-text index
shards = [GraphEngine(f"./shard_{i}") for i in range(4)]
engine = ShardedGraphEngine(shards, HashPartitioner(num_shards=4))
# Writes route to the correct shard via consistent hashing
# Reads scatter-gather across all shards
# Per-shard write locks — writes to different shards proceed in parallel4. Data portability
Purple8 is not a lock-in:
- Export: Backup data is plain JSON (or compressed JSON) — you can read it with any tool.
- Cypher: Standard Cypher queries work — migrating to/from Neo4j is straightforward.
- REST API: Standard HTTP endpoints — any client can integrate.
- Open-core: The engine is open-source. You can inspect every line of storage code.
Impact on time to market
Traditional architecture (weeks)
Week 1: Set up Postgres — schema design, migrations, connection pooling
Week 1: Set up Pinecone — create index, configure dimensions, API key management
Week 2: Set up Elasticsearch — cluster, mappings, analyzers, index templates
Week 2: Set up Neo4j — Docker, Cypher schema, driver config
Week 3: Build sync layer — message queues, change data capture, reconciliation
Week 3: Write integration tests — mock all 4 services, test failure scenarios
Week 4: Deploy — multi-container orchestration, health checks, monitoring
Week 4: Fix sync bugs — the first batch that slipped throughPurple8 architecture (hours)
Hour 1: pip install purple8-hyper-graph
Hour 1: engine = GraphEngine("./data")
Hour 2: Build ingestion pipeline using Purple8 DocIntel
Hour 3: Add vector search + graph traversal
Hour 3: Deploy — one container, one volume, one health checkWhy it is faster
| Phase | Traditional | Purple8 | Savings |
|---|---|---|---|
| Infrastructure setup | 2 weeks | 1 hour | ~99% |
| Data model design | 4 schemas × 1 day | 0 — schemaless | ~100% |
| Sync layer development | 1 week | 0 — atomic writes | ~100% |
| Integration testing | 1 week | 1 day (one engine mock) | ~80% |
| Deployment and ops | Ongoing | One container | ~90% |
| Total to MVP | 4–6 weeks | 1–2 days | ~95% |
The four engines in detail
1. Graph Store
The property graph is the backbone. Every entity is a Node with labels and typed relationships (Edges):
from purple8_graph import GraphEngine
engine = GraphEngine("./data")
# Create nodes with labels
engine.add_node("doc-1", labels=["Document", "Contract"],
properties={"title": "NDA v3", "status": "active"})
engine.add_node("person-1", labels=["Person"],
properties={"name": "Alice Chen", "role": "Legal"})
# Create a typed relationship
engine.add_edge("doc-1", "person-1", edge_type="AUTHORED_BY",
properties={"date": "2026-03-15"})
# Traverse: who authored this document?
results = engine.traverse("doc-1", edge_types=["AUTHORED_BY"], max_depth=1)The graph layer uses RocksDB key prefixes with prefix-seek indexes:
n:— node records (properties stored inline)e:— edge recordso:source:type— outgoing adjacency indexi:target:type— incoming adjacency indexl:label— label-to-node indexp:label|prop|value— secondary property index
All reads are O(log N + k) via RocksDB prefix seeks — not O(N) full scans.
2. Document Store
Every node's properties field is a schemaless JSON document — dict[str, Any] with arbitrary nesting:
engine.add_node("invoice-42", labels=["Invoice"], properties={
"vendor": "Acme Corp",
"amount": 15400.00,
"currency": "USD",
"line_items": [
{"sku": "WDG-001", "qty": 100, "unit_price": 154.00},
{"sku": "WDG-002", "qty": 50, "unit_price": 0.00},
],
"metadata": {
"source": "SAP",
"imported_at": "2026-04-07T10:30:00Z",
"raw_text": "Full OCR text of the scanned invoice..."
},
"tags": ["procurement", "q2-2026", "approved"]
})| Type | Example |
|---|---|
| Strings | "title": "Annual Report" |
| Numbers | "amount": 15400.00 |
| Booleans | "approved": true |
| Nested objects | "address": {"city": "SF", "zip": "94105"} |
| Arrays | "tags": ["urgent", "legal"] |
| Mixed arrays | "data": [1, "two", {"x": 3}] |
| Null | "deleted_at": null |
No depth limit. No schema constraint (unless you opt in via SchemaValidator). All property data is serialised with orjson (4–10× faster than stdlib json) and stored in RocksDB with optional AES-256-GCM envelope encryption.
3. Vector Store
Every node can carry one or more embedding vectors, indexed in an HNSW graph:
# Single embedding (default slot)
engine.add_node("doc-1", labels=["Document"],
properties={"title": "Climate Report"},
embedding=embedding_vec)
# Multi-vector: separate embeddings for title vs body
engine.add_node("doc-2", labels=["Document"],
properties={"title": "Q2 Earnings", "body": "Revenue grew..."},
embeddings={"title": title_vec, "body": body_vec})
# Semantic search
results = engine.vector_search(query_vec, top_k=10, labels=["Document"])
# Hybrid: vector + graph traversal
results = engine.hybrid_search(query_vec, top_k=10,
expand_edges=["CITES"], max_depth=2)Vector features:
- Multi-vector slots per node
- ANN pre-filtering by label (cached label allow-set — O(1) after first call)
- Quantization:
int8(4× RAM savings) orbinary(32×) - DiskANN backend for larger-than-RAM indexes
- Auto-compaction of tombstoned vectors
4. Full-Text Search
Built-in BM25 inverted index — no Elasticsearch required:
from purple8_graph.fulltext import FullTextEngine
fte = FullTextEngine(engine, index_fields=["title", "body", "tags"])
# Auto-indexed on write
fte.add_node("doc-1", labels=["Document"],
properties={"title": "Climate Change Report",
"body": "Global temperatures rose by 1.2C..."})
# BM25 keyword search
results = fte.full_text_search("climate temperature", top_k=10)
# Hybrid: BM25 + vector similarity
results = fte.hybrid_text_vector_search(
query="climate report",
embedding=query_vec,
top_k=10,
text_weight=0.3,
vector_weight=0.7,
)Full-text features:
- Configurable tokenisation and stopword lists
- Per-field indexing (choose which properties to index)
- BM25 with tunable
k1andbparameters - Automatic index updates on add/update/delete
- Hybrid text + vector fusion with configurable weights
Summary: before and after
| Concern | Without Purple8 | With Purple8 |
|---|---|---|
| Document storage | Postgres / MongoDB | node.properties |
| Vector search | Pinecone / Weaviate | engine.vector_search() |
| Keyword search | Elasticsearch | fte.full_text_search() |
| Relationships | Neo4j / manual joins | engine.add_edge() / engine.traverse() |
| Property indexes | 4 separate index configs | engine.build_property_index() |
| Encryption at rest | 4 per-system configs | One KMS envelope |
| Backup | 4 separate backup jobs | One mgr.create_backup() call |
| Replication | 4 replication topologies | One Raft log |
| Auth / tenancy | 4 auth configs | One multi-tenant auth layer |
| Deployment | 4+ containers | 1 container |
| Consistency | App-level sync code | Single-writer, atomic writes |
| Time to MVP | 4–6 weeks | 1–2 days |
| Dependencies | 8+ packages | 1 package |
| New developer ramp-up | Learn 4 systems + sync | Learn 1 SDK |
One import. One storage directory. One backup. One encryption key. Zero synchronisation bugs.