Core Concepts

The mental model behind Lunaris: you write Episodes, ingest turns them into stored primitives, recall composes a read plan over them — and every layer is bi-temporal, scope-partitioned, and committed in one atomic write. Read this once and the rest of the book is configuration detail.

The flow: Episode → ingest → storage → recall

          Episode
             |
             v
      +-------------+        (optional)         +----------------+
      |   ingest    | -- graph pipeline ON --> |  extract       |
      |  (chunk,    |                          |  entities +    |
      |   embed,    |                          |  relations +   |
      |   atomic    |                          |  facts         |
      |   write)    | <---- validator --------+----------------+
      +------+------+
             |                                            ^
             v                                            |
      +-------------+                             +--------------+
      |   storage   | <-- MVCC supersede -------> | forget       |
      | (KV/Vector/ |       (soft/hard)           | (GDPR/audit) |
      |   Graph/    |                             +--------------+
      |   Queue)    |
      +------+------+
             |
   recall    v        rerank
   builder -+-> [vector] ----\                       queue
            +-> [bm25 ]  ---- fuse_rrf --> rerank --> Hit[]
            +-> [graph] ----/                          |
                                                       v
                                               +-------+-------+
                                               | __lunaris_    |
                                               |  verify__     |  <-- verifier worker (default OFF)
                                               | __lunaris_    |
                                               |  consolidate__|  <-- consolidator worker (default OFF)
                                               | __lunaris_    |
                                               |  audit__      |  <-- audit sink
                                               +---------------+

• ingest chunks the episode body (markdown-aware, ~500-token target with ~100-token overlap), embeds each chunk in batches, and commits everything — episode KV row, per-chunk KV rows, per-chunk vector upserts, plus graph/fact rows if the graph pipeline is on — in a single StoragePort::atomic_write call. Then it publishes one message to __lunaris_consolidate__ (best-effort; the data is already durable).
• forget is the inverse: it also commits in one atomic_write (soft delete stamps the MVCC sys end; hard delete is a KvDelete fan-out behind a two-step confirmation token), and emits one audit event.
• recall is read-only: a RetrievalBuilder composes Vector, Keyword, Graph, and Tree (RAPTOR hierarchical) operators (plus fusion / rerank / fallback wrappers) into a plan, executes it once, and returns Vec<Hit>. The one-shot ScopedLunaris::recall(query) is the same builder with its default root left in place. No LLM is on this path — that’s the sub-25 ms moat.

The primitives

Six primitive types, each carrying a BiTemporal stamp and a Scope (crates/lunaris-core/src/primitives.rs):

Only Episode is written by your code; everything below it is derived. Recall queries one of four whitelisted index names — chunks, entities, facts, communities — e.g. Vector::new("chunks", 30).

Bi-temporal MVCC + the HLC

Every primitive carries a BiTemporal (crates/lunaris-core/src/bitemporal.rs):

use lunaris_core::Hlc;
async fn demo() -> Result<(), lunaris::LunarisError> {
pub struct BiTemporal {
    pub valid: (Hlc, Option<Hlc>),  // when the fact is true in the world
    pub sys:   (Hlc, Option<Hlc>),  // when the system observed/recorded it
}
Ok(())
}

Both are half-open [from, to) intervals; to = None means “still current”. valid is world time — “Alice loved chocolate from Jan to March”. sys is system time — “we recorded that on Feb 3rd, and superseded it on April 1st”. The two axes are independent, which is what lets you ask “what did the agent believe at time T?” as a query (.as_of(ts) on the retrieval builder, or read_as_of on the storage port) rather than reconstructing it from a log.

Updates are MVCC supersede, never in-place mutation: a soft delete or an arbitration outcome stamps the old row’s sys end and writes a new row — old as_of reads still resolve correctly. (Backends derive persisted bitemporal from the payload bytes, so a typed-only mutation would be silently lost; the supersede writers patch both.)

Which half is universal. The bi-temporal write model above holds on every backend. As-of reads do not: StoragePort::read_as_of can answer a historical pin only where the backend keeps a KV version chain (supports_historical_kv_reads() == true), and the two backends that did were deleted in 0.7.0. Moon stores Lunaris rows as plain hashes with no version chain, so since v0.6.2 it refuses a historical pin with StorageError::NotSupported (HTTP 501 not_supported) rather than answering with present-time data. Latest-state reads — what every recall, hydrate and forget actually issues — work everywhere, and Moon’s search/graph lanes remain temporal via FT.SEARCH AS_OF and GRAPH.QUERY VALID_AT. Closing the KV gap upstream is tracked against Moon’s TemporalKvIndex.

The clock is a Hybrid Logical Clock (crates/lunaris-core/src/hlc.rs): Hlc { wall_ms: u64, counter: u32, node_id: u16 }, totally ordered by (wall_ms, counter, node_id). When the wall clock doesn’t advance between two ticks, the counter increments — so no two stamps ever collide, even under burst writes. node_id is 0 in single-node v0.

Scope: the multi-agent partition key (RFC 0001)

Every Lunaris operation is partitioned by a Scope — a validated newtype (crates/lunaris-core/src/scope.rs), not a user_id string the caller could swap:

use lunaris::{EpisodeBuilder, Scope};
use lunaris::Lunaris;
async fn demo() -> Result<(), lunaris::LunarisError> {
let lunaris = Lunaris::open("moon://localhost:6380").await?;
let scope  = Scope::new("acme.agent-1")?;   // validates against [A-Za-z0-9_\-.]{1,128}
let scoped = lunaris.scoped(scope);          // ScopedLunaris — every op partitioned
scoped.ingest(EpisodeBuilder::new("user-msg", "Alice loves chocolate.")).await?;
Ok(())
}

Scope::new rejects the empty string, anything over 128 bytes, and any character outside [A-Za-z0-9_\-.]. : is not in the alphabet — by design, so the KV key format can’t byte-alias across scopes (see below). Scope does not derive Deserialize transparently — the hand-rolled impl re-runs the validating constructor, so the wire can’t smuggle an invalid or unintended scope past the type. (Scope::dev() exists but is #[doc(hidden)] — a migration crutch for code paths that haven’t threaded a real scope through yet.)

On the HTTP surface, the tenant claim resolved for the bearer token is the only source of truth for the partition scope — route handlers ignore any scope field on the request body. On Moon, the scope is baked into the key, the FT index name and the graph name (lunaris:{scope}:{kind}:{ulid}, lunaris_{scope}_{kind}_idx), so a cross-scope read has nothing to address rather than being filtered after the fact. Same ULID under two different scopes = two distinct rows, no leak.

Through 0.6.2 a second boundary existed: Postgres row-level security (USING + WITH CHECK, under a NOSUPERUSER NOBYPASSRLS role). That backend was removed in 0.7.0, so today the isolation is structural in the Moon keyspace rather than row-filtered in a database. Details: Multi-Agent & Scope.

The keyspace

KV keys are minted exclusively by lunaris_core::keyspace — episode_key, chunk_key, entity_key, relation_key, fact_key, community_key — in the canonical format:

lunaris:{scope}:{kind}:{ulid}

e.g. lunaris:acme.agent-1:chunk:01J.... Backend crates re-export these helpers; minting a Lunaris KV key from a local helper is a bug. Because : is forbidden in Scope, the {scope} segment can never absorb the :{kind} delimiter — scope isolation is closed at the type level, not by operator discipline.

The single atomic_write invariant (INGEST-04)

This is the correctness moat. One ingest call = exactly one StoragePort::atomic_write — never one commit per chunk. The pipeline builds the full Vec<WriteOp> (episode KV + per-chunk KV + per-chunk vector upsert, plus graph/fact rows when the graph pipeline is on) and hands it to the backend in a single call. Either all of it lands or none of it does — across the vector index, the KV store, the BM25 index, the audit log, and the queue. Fan-out architectures (write to the vector DB, then the store, then the graph) can’t make that guarantee; Lunaris can because it owns the substrate.

It’s grep-pinned and CI gates on it on every push: after stripping comment lines, exactly one real storage.atomic_write call site must remain in crates/lunaris-ingest/src/pipeline.rs (grep -v '^\s*//' crates/lunaris-ingest/src/pipeline.rs | grep -c 'storage\.atomic_write' == 1) — see Ingesting Observations for the exact gate and the graph-on counterpart. Any new ingest fan-out extends the single WriteOp vector — it does not add a second atomic_write. forget holds the same property: one atomic_write per call (zero for a dry-run preview).

Opt-in pipelines: graph, verify, consolidate

Three background pipelines, all default OFF — your dev box doesn’t download a Gemma extractor until you ask:

Each handle is obtained from the Lunaris value after open. The consolidator handle additionally exposes .enable_for_scope(prefix) (a source-prefix filter) for per-scope rollout; the graph and verify handles are .enable() / .disable() only in v0.2.x. Enabling a pipeline with no real backend wired just runs the shipped Noop* impl — no crash, but no useful work either. Details: The Graph Pipeline and Consolidation & Verification.

The two backends

Lunaris::open(url) dispatches on the URL scheme:

Moon is the only backend. Start it with --shards 1 — an ingest is one MULTI/EXEC transaction and a sharded Moon rejects it. Embedding-dimension details and the STORE-07 as-of gap: The Storage Backend.

Next

• The Retrieval DSL — every operator and combinator.
• Ingesting Observations — the chunker, the embedder driver, the graph branch.
• Durability & Recovery — how the bi-temporal store survives a crash.
• Configuration Reference — every feature flag and LUNARIS_* variable.

Where this chapter disagrees with the Rust source, the source wins — the path:line references above point back into the crates.