Sentinel — Human-Gated Proactive Intelligence

Sentinel watches Purple8 and the data inside it. It surfaces findings: something changed, or something could be better. It never changes production behaviour by itself. Every proposal waits in the inbox until a person approves it.

detector / evolver ──► Finding (open) ──► human approves ──► applier runs ──► applied
                                     └──► human/agent rejects ──► rejected

UI: Dashboard → Sentinel (inbox, model behaviour panel, evolution experiments). API prefix: /sentinel.

---

1. Findings and the human gate

A finding contains:

Field	Meaning
source	Where it came from: drift, models, soc.threshold, soc.emerging, evolution.internal, evolution.external
kind, title, summary	What happened, in plain language
severity, score	Triage from the decision gate (low → critical)
evidence	Numbers behind the claim (baseline vs current, diffs, sample sizes) and recommendations
proposal	{action, params, rationale}: what will run if you approve
fingerprint	Dedup key. A repeat detection refreshes the open finding instead of creating a new one

Endpoint	Who
GET /sentinel/findings?status=&source=&min_severity=	node:read
GET /sentinel/findings/{id}, GET /sentinel/summary	node:read
POST /sentinel/findings/{id}/approve	Humans only (JWT session; API keys get 403). Needs soc:configure for soc.* findings, admin:settings otherwise
POST /sentinel/findings/{id}/reject	Humans or agents with the same permission

source filtering is by prefix: source=evolution matches evolution.internal and evolution.external.

2. Data drift

• POST /sentinel/drift/baseline records a reference. The body is {kind, name, values|vectors|score}, where kind is distribution, embeddings or quality.
• POST /sentinel/drift/check compares current data against the baseline and opens a finding when the change is significant. Approving the finding (sentinel.rebaseline) accepts the new state as the baseline.

3. Model behaviour

LLM calls are sampled in-process: error rate, refusals, latency and fallbacks per model.

• GET /sentinel/models/telemetry?window_s=3600 shows live numbers.
• POST /sentinel/models/baseline with {model} freezes today's behaviour as "normal".
• POST /sentinel/models/check opens findings for models that degraded. Approving one excludes the model from routing.
• POST /sentinel/models/{model}/include puts an excluded model back.

Limit: telemetry is in memory and per process. It covers calls made through the built-in LLM client.

4. SOC tuning

• POST /sentinel/soc/evolve-thresholds evolves detector thresholds against labelled windows. It proposes the change only if recall improves without more false positives.
• POST /sentinel/soc/discover-threats finds recurring anomaly patterns that are absent from benign traffic and proposes them as new rules.

Both need soc:configure. See SOC Agent.

---

5. Capability evolution — one engine, many applications

Purple8 has a single, general-purpose evolutionary engine (core/evolution.py). Sentinel applies it in two families:

	Internal intelligence	External intelligence
Who benefits	Purple8 itself	Your application
What evolves	Routing policy, RAG retrieval, SOC thresholds	Any parameters you declare
Fitness	Recorded outcomes, eval sets	Scores your app reports
Result	Finding → applied to Purple8 config	Finding → promoted production config you read back

Guarantees common to both:

• Never worse than today. The current config is always in the starting population.
• Minimal diff. After the search, every change that doesn't improve the score is reverted.
• Bounded cost. Evaluations are cached and capped (max_evaluations).
• Human gate. The search only proposes. Nothing is applied until someone approves the finding.

5.1 Internal: model routing policy

POST /sentinel/evolution/routing-policy        (admin:settings)
{
  "outcomes": [
    {"profile": {"task_type": "classify", "input_tokens": 400}, "tier": "small", "success": true},
    {"profile": {"task_type": "summarise", "input_tokens": 12000}, "tier": "small", "success": false}
  ],
  "cost_weight": 0.3, "generations": 25, "min_gain": 0.02
}

Each outcome is a task profile, the tier that handled it, and whether it succeeded. Outcomes are scored on the assumption that a stronger tier would also have succeeded and a weaker one would also have failed. Cases the data cannot answer count as uncertain (0.5). Fitness is quality − cost_weight × cost, with a small penalty for drifting from the current policy. Approving the finding runs models.set_policy for your tenant. You need at least 10 outcomes.

5.2 Internal: RAG retrieval

POST /sentinel/evolution/rag-retrieval         (admin:settings)
{"eval_set": [{"question": "...", "expected_answer": "..."}], "profile_name": "support-docs"}

This evolves fusion_alpha, max_hops and seed_k, starting from the corpus' topology-aware recommendation. The score is token-F1 between the retrieved context and the expected answers (retrieval only, no generation). Approving the finding saves a collection profile. You can then use it with collection_profile: "support-docs" in rag.hybrid_query. It needs a tenant LLM configuration for embeddings.

/rag/tune-collection is a quick fixed grid that saves immediately. Use this endpoint when you want a wider search behind review.

5.3 External: your own experiments

Use an experiment to tune anything your application controls from real outcomes. Examples: prompt temperature, ranking weights, discount levels, retry counts, batch sizes.

POST /sentinel/evolution/experiments           (node:update)
{
  "name": "checkout-ranker",
  "genes": [
    {"name": "recency_weight", "gene_type": "float", "lo": 0, "hi": 1, "default": 0.3},
    {"name": "top_k", "gene_type": "int", "lo": 3, "hi": 20, "default": 10}
  ],
  "maximize": true, "population_size": 8, "min_trials": 20
}

Gene types: float, int, bool (up to 32 genes). The default values are today's production config.

The loop your app runs:

s = GET  /sentinel/evolution/experiments/checkout-ranker/suggest   # {genome_id, config}
# ... use s["config"] for this request / user / batch ...
POST /sentinel/evolution/experiments/checkout-ranker/report  {"genome_id": s["genome_id"], "score": 0.42}

• Each candidate is tried at least min_trials times. Then the population breeds a new generation automatically. Raise min_trials when outcomes are noisy.
• GET .../{name} shows status: generation, every candidate's mean score and trial count, history.
• POST .../{name}/propose with {min_gain: 0.05} opens an evolution.external finding when the best candidate beats production by min_gain (relative).
• After approval, GET .../{name}/production returns the promoted config. Serve this to traffic that is not part of the experiment.
• DELETE .../{name} removes the experiment (admin:settings).

Experiments are stored as EvolutionExperiment graph nodes, isolated per tenant. Each one has a fixed-size footprint: one population and bounded history.

Use "maximize": false for costs and latencies.

5.4 From an AI agent (MCP)

All Sentinel and Evolution workflows are exposed as dot-namespaced MCP tools (part of Purple8's 125 MCP tools across 16 namespaces), so an AI agent can inspect findings, check drift, and run experiments without custom application code:

Tool	Role	Endpoint
sentinel.summary	viewer	GET /sentinel/summary
sentinel.list_findings	viewer	GET /sentinel/findings
sentinel.get_finding	viewer	GET /sentinel/findings/{id}
sentinel.reject	editor	POST /sentinel/findings/{id}/reject
sentinel.drift_check	editor	POST /sentinel/drift/check
evolution.routing_policy	admin	POST /sentinel/evolution/routing-policy
evolution.rag_retrieval	admin	POST /sentinel/evolution/rag-retrieval
evolution.create_experiment	editor	POST /sentinel/evolution/experiments
evolution.list_experiments	viewer	GET /sentinel/evolution/experiments
evolution.suggest	viewer	GET .../{name}/suggest
evolution.report	editor	POST .../{name}/report
evolution.production	viewer	GET .../{name}/production
evolution.propose	editor	POST .../{name}/propose

Agents can propose and reject, but cannot approve. Approval is a human-only action (JWT session required) in the Sentinel inbox.

---

Permissions summary

Action	Permission
Read findings, telemetry, experiments; suggest	node:read
Create experiments, report outcomes, propose	node:update
Baselines, routing/RAG evolution, delete experiments	admin:settings
SOC tuning and approving soc.* findings	soc:configure
Approve any finding	A human session with the permission above