Hybrid Search

The core query pattern in Purple8 — vector similarity, BM25 full-text, and graph traversal unified in a single Cypher statement.

Why one query matters

In most systems, hybrid search is two queries:

1. Vector search → top-k node IDs
2. Graph query → pass those IDs in as a filter

The latency budget is doubled. The failure domain is doubled. And the boundary between "find by meaning" and "follow connections" lives in your application code, not the query engine.

Purple8 erases that boundary. CALL db.vector.search is a first-class Cypher procedure — its output feeds directly into the next MATCH clause in the same query planner, same process, same round-trip.

Benchmarked results (v0.72.0):

• Query P99 @ 100K: 0.33 ms | HTTP P95 (1024d): 23 ms
• VectorDBBench (1M × 768d): 916 QPS, 0.946 recall@10
• Batch ingest: 40,551 docs/s | 8.84M MS MARCO docs in ~9 GB peak RSS
• Hybrid passage retrieval: 3.53 ms p50 at 100,000 documents (MS MARCO, all-MiniLM-L6-v2)
• Sustained soak: 1.18M requests across 500 concurrent users with 0% errors

The three search modalities

1. Vector search (semantic)

CALL db.vector.search('Document', $queryVec, 10) YIELD node, score
WHERE score > 0.80
RETURN node.title, score
ORDER BY score DESC

• Searches the HNSW index on the Document label
• Returns up to 10 results with cosine similarity scores (0–1)
• WHERE score > 0.80 filters post-search

2. Full-text search (BM25)

results = engine.fulltext_search("transformer attention", labels=["Document"], top_k=10)

For weighted keyword/vector fusion, use the FullTextEngine wrapper:

from purple8_graph.fulltext import FullTextEngine

search = FullTextEngine(engine)
search.index_existing_nodes()
results = search.hybrid_text_vector_search(
    query="transformer attention",
    embedding=query_vec,
    labels=["Document"],
    text_weight=0.3,
    vector_weight=0.7,   # 70% semantic, 30% keyword
    top_k=10,
)

Weights must be finite and non-negative, with at least one positive weight. A zero weight disables that modality entirely. Equal scores within a modality (including a single hit) normalise to 1; ties in the combined score use node ID ordering for deterministic results. Retrieval failures propagate rather than silently returning partial results.

Cold-segment publication and demand-load share a lock: readers cannot load the binary index and mappings halfway through an update. A published mutation also invalidates cached copies so later loads cannot retain a superseded generation.

3. Graph traversal (structural)

MATCH (p:Person {name: 'Alice Chen'})<-[:AUTHORED_BY]-(doc:Document)
MATCH (doc)-[:BELONGS_TO]->(topic:Topic)
RETURN doc.title, topic.name

Combining all three

-- Vector seeds → filter → follow relationships → return enriched results
CALL db.vector.search('Document', $queryVec, 20) YIELD node AS doc, score
WHERE score > 0.75
  AND doc.year >= 2020
MATCH (doc)-[:AUTHORED_BY]->(author:Person)
MATCH (author)-[:WORKS_AT]->(org:Organization)
OPTIONAL MATCH (doc)-[:CITES]->(cited:Document)
RETURN
    doc.title        AS title,
    doc.year         AS year,
    author.name      AS author,
    org.name         AS org,
    count(cited)     AS citation_count,
    score
ORDER BY score DESC, citation_count DESC
LIMIT 10

Filtered vector search

Labels, properties, and shared-tenant prefixes constrain eligible candidates inside the ANN search, before top-k selection. They are not implemented by fetching a fixed number of global results and discarding ineligible hits. HNSW remains approximate; pre-filtering is not a guarantee of exact nearest-neighbour recall.

results = engine.vector_search(
    query_embedding=query_vec,
    labels=["Document"],
    k=10,
    filters={"region": {"$eq": "APAC"}},
)

Supported filter operators: $eq, $ne, $lt, $lte, $gt, $gte, $in, $nin, $exists

Unknown operators are rejected. $in and $nin require a list, and $exists requires a boolean, even when the index is empty.

Named embedding slots

A node can carry multiple embeddings — for example, a document with both a title_embedding and a body_embedding:

engine.add_node(
    "doc1",
    labels=["Document"],
    properties={"title": "Attention is all you need"},
    embeddings={"title": title_vec, "body": body_vec},
)

# Search against the title slot
engine.vector_search_slot(query_vec, slot="title", k=10)

# Search against the body slot
engine.vector_search_slot(query_vec, slot="body", k=10)

Named-slot writes and queries use that slot's dimension, not the default index dimension. The first index creation establishes the dimension for that name; later writes must match it. Creation, update, and bulk writes validate every vector before mutation, including finite float32 values and consistency across the bulk request. Different named slots may have different dimensions. embeddings["default"] aliases the primary embedding; conflicting values are rejected rather than indexed independently.

Shared-tenant engines scope named-slot candidates before selection. Shared tenants use common slot indexes and therefore share the dimension contract for a given slot name.

Graph expansion constraints

engine.hybrid_search(..., edge_types=["A", "B"]) accepts either relationship type at each hop; other types are excluded during expansion, so disallowed edges cannot act as bridges. Shared-tenant traversal cannot cross a different tenant's nodes.

engine.traverse(..., max_nodes=N) caps discovered nodes, including the seed. Adjacency expansion now uses bounded native pages and lazy edge decoding, rather than first copying the node's entire adjacency list. Traversal has a cooperative 30-second default deadline, configurable by administrators through P8G_QUERY_TIMEOUT_SECONDS. This requires the matching rebuilt native extension. engine.graph_guided_search(..., max_candidates=N) applies this cap during candidate traversal, including widening passes. These are node-count budgets, not execution deadlines or bounds on the memory used to read one node's adjacency.

Vector index backend

By default Purple8 uses hnswlib (C++) for the per-segment HNSW index. An alternative pure-Rust backend is available for off-GIL vector operations:

export P8G_VECTOR_BACKEND=rust   # default: hnswlib

Same public API — zero code changes. The choice is global per-process; segment save/load formats do not mix, so don't switch backends on an existing index directory.

Quantized vector indexes

Reduce memory footprint at scale with int8 or binary quantization:

export P8G_VECTOR_QUANTIZATION=int8   # or: binary

• int8: calibrated scale, pre-screen then full-precision rerank — best accuracy/memory trade-off
• binary: Hamming pre-screen then full-precision — maximum compression, modest accuracy drop

Benchmark results

All runs on all-MiniLM-L6-v2 (384-dim) embeddings.

MS MARCO — Passage Retrieval (MRR@10)

Scale	Purple8 MRR	BM25 MRR	Purple8 p50	vs BM25
2,500 passages	0.9658	0.8233	0.93 ms	—
25,000 passages	0.8810	0.7117	1.99 ms	—
100,000 passages	0.7511	0.6025	3.53 ms	182×

Entity disambiguation (Suite A)

Task	Purple8	Neo4j + Vector	Kùzu + HNSW	FalkorDB
Disambiguate author by graph context	✅ 100%	⚠️ 50%	⚠️ 50%	⚠️ 50%
Multi-hop topic cluster	✅ 67%	✅ 67%	✅ 67%	✅ 67%
Knowledge synthesis	✅ 100%	✅ 100%	✅ 100%	✅ 100%
Temporal chain reconstruction	✅ 100%	✅ 100%	✅ 100%	✅ 100%
Mean accuracy	🥇 91.7%	79.2%	79.2%	79.2%

Full benchmark methodology →