The Ambient Intelligence Problem

Abstract

Ambient AI systems — the kind that live alongside an operator rather than inside a chat window — fail in different ways than chat assistants do. A chat assistant that forgets nothing, interrupts on every signal, never changes tone, and never goes quiet is merely annoying. An ambient system that does the same is uninhabitable. This paper argues that ambient AI requires memory, attention, interruption policy, personality, and off-mode as first-class subsystems with their own data structures and invariants — not as UX polish layered over a generic chatbot. We describe five such subsystems in JARVIS, a local-first operations console built over thirty-three days on a single operator's machine, plus a sibling architecture — a design choice that worked for this system rather than a universal requirement — in which a complementary voice (Reachy, she/her) is given her own memory, her own voice profile, and her own decision authority rather than being implemented as a persona variant of the main system. Concrete contributions include a six-layer memory hierarchy with explicit promotion rules and decay constants; a seven-factor salience attention engine with an explanation surface that makes weight drift detectable; an interruption policy that treats interruption as an operator right rather than a system default; a personality-and-off-mode layer realized as persistent state (mood, drift, emotional memory, growth tracking, quiet hours, ephemeral mode, thermal gating) rather than tone parameters and mute buttons; and a campfire dialogue mechanism where hand-authored scripts have been empirically observed to outperform LLM-generated dialogue even after temperature caps and topic-pivot detectors were added. We discuss where each subsystem currently wears (memory writes contending with 898 database-lock events over the past week, attention weights operator-tuned rather than principled) and what the architecture asks of any future ambient product.

1. The ambient intelligence problem

Mark Weiser's 1991 essay The Computer for the 21st Century opens with the line that ambient technology is the kind of technology that "weaves itself into the fabric of everyday life until it is indistinguishable from it." Thirty-five years later, we have the technology he predicted — microphones in rooms, cameras on screens, GPUs under desks, language models that can hold a real conversation — and almost none of the integration he described. The reason is not that the hardware is missing. The reason is that the system-level problems of ambient presence are different from the problems a chat assistant solves, and the industry has mostly tried to solve them with chat-assistant tools.

This paper describes five of those problems and the data structures I used to solve each of them inside JARVIS, a local-first operations console I have been building for thirty-three days. The core claim is that ambient AI is not a prettier chatbot. It is a new system-shape, with its own subsystems, its own failure modes, and its own invariants — and if those subsystems are not built, the product becomes, at best, irritating, and at worst, a degradation of the operator it was meant to support.

The five problems:

1. Memory. A chat assistant can be stateless across turns and still feel responsive. An ambient system cannot; it has to remember what was said yesterday, what project is active this week, what the operator's preferences are over months. But it also cannot remember everything, because an ambient system that never forgets is an ambient system that cannot learn, and cannot generalize, and cannot get over old mistakes.
2. Attention. A chat assistant reacts to whichever message arrived last. An ambient system is surrounded by signals — news feeds, scan results, calendar entries, sensor telemetry, its own internal monologue — and has to decide which signals deserve the operator's attention and which should be silently absorbed. Get this wrong in either direction and the system becomes either a nonstop siren or a silent stranger.
3. Interruption. An ambient system that speaks when asked is a chat assistant with a microphone. An ambient system worth having also speaks unasked, for the right reasons, at the right moments. The line between "proactive helpfulness" and "notification fatigue" is drawn by an explicit interruption policy, not by good intentions.
4. Personality. A chat assistant expresses personality through wording. An ambient system expresses personality through mood, silence, memory, timing, and the shape of its proactive behavior — and these cannot be bolted on later as stylistic choices. Personality, in an ambient system, is a state machine with persistence.
5. Off-mode. A chat assistant is off when you close the tab. An ambient system is always around, which means the decision to be quiet, to recede, to refuse a capability, to sleep through a noisy hour, has to be a decision the system takes for itself. Off-mode, in ambient, is architecture — not a mute button.

I will also describe a design choice specific to this system: a sibling architecture in which a second voice (Reachy) is given her own memory, her own voice profile, and her own decision authority, rather than being another persona of the same underlying system. Whether ambient systems in general benefit from this pattern is an open question, and I want to flag it as such up front rather than present it as a sixth requirement beside the other five. What I can say is that for this system, in this room, with this operator, the second voice was the relief valve on a single register that was becoming too heavy to live under.

The paper that precedes this one argues for a constitution — structural safety below the product. The present paper argues for what the product should be, once it is contained: a set of subsystems written against the same kind of architectural discipline, but in a different register. Where the constitution is about what the system must not do, this paper is about what the system has to become if anyone wants to live with it.

2. What chat assistants do not have to solve

A terminology note before this section begins. I use "operator" rather than "user" throughout the paper because JARVIS is a tool its operator runs, not a product a user consumes; the distinction matters for what the system is allowed to do, who is accountable when it does it, and what the interruption budget is being spent on.

Before describing how JARVIS solves the five problems, it is worth being precise about why they do not appear in the chat-assistant setting. This is not a criticism of chat assistants, which are excellent at what they do. It is a statement about what they are being asked to do.

A chat assistant lives inside a well-defined conversational turn. The user types a message; the system reads prior turns from a context window, produces a response, and hands control back. The session has a beginning (the operator opens the window) and an end (the operator closes it). Within the turn, the whole problem reduces to: given this context, produce the best next token sequence.

That framing makes several hard problems disappear. A chat assistant does not need to decide whether to speak; the user pressing enter is its cue. It does not need to decide whether to remember; the context window is its memory, rewritten every turn. It does not need a personality over time; its tone is whatever the current system prompt says. It does not need an off-mode; the window is closed or it is not.

Ambient systems inherit the opposite of all four. They are always on, within whatever scope of uptime the operator permits, and that "always on" turns each of the disappeared problems back into a real design decision. The chat-assistant approach to ambient — "take the best chatbot and also have it listen to the room" — works for a week. Then the memory silos, the notification stream overwhelms, the persona loses coherence, and the system either goes silent (which is unhelpful) or goes loud (which is worse).

The subsystems below are what replaces the missing scaffolding.

3. Memory: the six-layer hierarchy

A memory system for an ambient AI has three jobs that chat-window memory does not have. It must retain across sessions, because an ambient system's operator returns to it every day. It must decay, because retention without forgetting becomes noise. And it must rank, because a fixed context window cannot include all retained items and something has to choose.

JARVIS's memory is stored in SQLite in a single memories table with a layer column. The six layers are:

The decay constants are half-lives, in days. Working memory is effectively gone in six hours if not accessed; semantic memory, once promoted, survives roughly six months of disuse before the system starts questioning whether it still holds. Every record carries a pinned flag that exempts it from pruning entirely, and a suppressed flag for soft-deletion without schema violation. Every record carries a confidence field on [0,1], an access_count for reinforcement, a last_accessed timestamp for decay calculation, and a provenance JSON blob recording session, extraction source, and timestamp.

Retrieval ranks records using a weighted composite score:

score(r, query, project_id) =
    0.4 × decay_score(r)                     # exp(-days_since_access / τ_layer)
  + 0.3 × confidence(r)
  + 0.1 × min(access_count / 50, 1.0)
  + 0.5 × pinned(r)
  + 0.2 × project_match(r, project_id)
  + 0.3 × keyword_overlap(r, query)

The six weights are not fitted; they are chosen by ordering (decay first, confidence second, usage third, the bonus terms last). Keyword overlap is currently a word-level heuristic, with a documented replacement hook for sentence-transformer cosine similarity once embedding indexing is wired in. The hook matters more than the placeholder: the scoring pipeline does not change when embeddings arrive, only the one function call.

Why six layers. A simpler design would use a single table with a single decay constant. That design loses the separations that make the system useful. A stable semantic fact (the operator's mother's name) should not decay at the same rate as an episodic observation (JARVIS noticed the operator opened VS Code at 2:17 p.m.). A preference (the operator dislikes being addressed as "user") should be reinforceable by repeated observation, where a project note (the current program is program X) should expire naturally when the project does. One decay curve for all of it produces either a system that forgets your name or a system that remembers what you ate for breakfast three months ago. Neither is what you want.

Promotion is the teeth. The layer design would be decorative without the promotion rules. An episodic observation that has been accessed three times and is above 0.7 confidence is a durable pattern; the system writes a new semantic record, marks the old episodic record as superseded (superseded_by = new_id), and the confidence is bumped by 0.05 on promotion. The old record is preserved, not deleted — which matters because it lets the memory conflict-resolver detect when the same key produced contradictory values at different layers.

Conflicts are data. The schema includes a memory_conflicts table with memory_id_a, memory_id_b, conflict_type, resolved, resolution. When two records at the same layer hold contradictory values for the same key, both are retained and the conflict is logged. User-explicit statements win; higher-confidence inferences win tie-breakers; everything else is queued for operator resolution. Ambient systems accumulate contradictions because they observe over long time; pretending contradictions do not exist is how a system ends up confidently telling an operator something the operator changed two months ago.

Hygiene is a daemon. Expired records are pruned on a hygiene cycle (currently daily). Stale episodic records not accessed in thirty days are dropped. Superseded records older than ninety days are garbage-collected. The hygiene daemon is kill-switch compliant. Without hygiene, the memory table accumulates until it becomes noise; the system's retrieval degrades silently as low-confidence stale records crowd the ranked top of every query.

I will note two known wears. The system audit run earlier this week found a cluster of database is locked events that occasionally affect memory writes — concurrency between the main thread, twenty-plus daemons, and the memory promoter on a SQLite file that is not currently setting busy_timeout on every connection. The fix is a single PRAGMA, and it is item #2 on next week's roadmap. Second, the keyword-overlap placeholder is a known limitation; retrieval occasionally misses paraphrases ("mom" when the fact is stored under "mother") that a vector search would catch. Neither of these is a conceptual problem with the six-layer design; they are well-understood implementation debts.

4. Attention: seven salience factors

A memory system answers "what do I know?" An attention system answers "what matters right now?" — which, for an ambient AI surrounded by sensors and feeds, is the harder question.

JARVIS's attention engine scores every candidate item on a 0-to-1 salience scale as a weighted sum of seven factors, computed in the function AttentionEngine.calculate_salience() at intelligence/attention_engine.py:210:

1. goal_relevance — does this item relate to something the operator is currently working on? Computed from keyword overlap with active project descriptions, open findings, and recent conversation context.
2. novelty — is this item new, or a re-appearance of a pattern the system has already surfaced? Novelty is suppressed for anything the attention engine has flagged in the last hour.
3. graph_centrality — is this item connected to many things in the brain graph? A finding that links to three targets, two CVEs, and a known technology is structurally more important than a finding that links to one isolated host.
4. recency — how new is this item? Exponential decay, same family of curves as memory decay but tuned shorter (hours, not days).
5. ali_interest — does this item match the operator's learned interest profile? Pulled from the preference layer of memory; reinforced or dampened by past operator responses (pinned finding ⇒ boost; rejected finding ⇒ dampen).
6. threat_level — severity of the item if it is a finding or CVE (critical=1.0, high=0.8, medium=0.5, low=0.2, info=0.1).
7. actionability — can the operator do something about this in the next hour? A finding on a live target scores higher than a finding on a paused program.

The composite score is then

score = Σ (factor_k × weight_k), clipped to [0, 1]

with weights that sum to 1.0 and that I will not publish here because they have been tuned on observed operator feedback and will be tuned further. The omission is deliberate: publishing the current weights would invite readers to treat them as settings rather than as a tuning process, and the contribution of this section is the factor decomposition, not the numbers in front of it. The attention engine reports the top three factors per scored item in an explanation field, so that the operator can see why a given item surfaced. Explainability is not a courtesy; it is how I catch weight drift before it becomes gaslighting. When the system tells me the top factor that made a CVE appear was novelty and I remember seeing an identical CVE yesterday, I know the novelty suppressor missed a match.

The attention engine also produces two auxiliary outputs alongside the score. The first is an attention type — one of stimulus_driven, goal_directed, relational, or threat_driven — derived from whichever factor dominated the score. Stimulus-driven attention is the ambient equivalent of a notification; goal-directed attention is what the operator wants the system to surface when they're heads-down; relational attention covers things the system is bringing up because they connect to something the operator already cares about; threat-driven attention bypasses most of the filtering stack. The second is a block decision: items matching blocking rules (duplicates of dismissed findings, placeholder credentials, out-of-scope artifacts) are excluded from the ranked output before salience is computed, so the top-ranked items the operator sees are never items the system already knew were noise.

Why seven factors, not fewer, not more. Fewer factors collapse into false urgency: a single-factor attention engine weighted on threat level turns into a critical-CVE siren, which is useful for about a day and intolerable for longer. More factors over-fit the current operator; a fifteen-factor engine tends to drift the weights toward whoever tunes it last, and the explanation surface becomes unreadable. Seven factors is the smallest set I have found that distinguishes the obviously-different attention regimes — threat, curiosity, operator interest, graph structure — without collapsing any two of them into the same axis.

Attention is consumed, not absolute. The salience score is not an interruption threshold. It is the input to §5's interruption policy, which decides what to do with a scored item. Salience answers "how important is this?"; interruption answers "should the operator be interrupted about it?" Conflating the two produces a system that either shouts everything critical (operator fatigue) or whispers everything mundane (operator miss).

5. Interruption is a right, not a default

The most consequential moment in an ambient system's life is when it decides, on its own initiative, to speak. That decision is where the difference between a thoughtful companion and a notification-driven stress dispenser lives.

JARVIS does not interrupt because an item is high-salience. It interrupts because three conditions have simultaneously been met: (1) the item's salience is above a per-category threshold; (2) the operator's current state, inferred from recent activity, suggests an interruption will be received well (they are not mid-flow, not in a meeting, not within quiet hours); and (3) no other interruption has fired on the same theme in the cooldown window. Each of these is a separate subsystem. The interruption policy is the composition.

The framing that has earned its keep is that interruption is a right the operator grants, not a default the system takes. The system is not entitled to speak because something happened; it is permitted to speak because the operator has, by building and running an ambient system, accepted that it may speak when the structural conditions are met. The distinction sounds academic. It is load-bearing, because it changes who has to work to suppress an unwanted interruption (the system, not the operator) and who bears the cost of a bad one (the system's trust budget, which is finite).

The parents-email alert is the clearest example in the system. The life-OS daemon watches outgoing email traffic and maintains a rolling count of days since the operator last sent a message to each family contact. When the count crosses a threshold, the daemon composes a single message — "Sir, you haven't emailed {names_str} in a few days. Want me to draft updates?" (the verbatim template at runtime/life_os_daemon.py:664) — and that message is handed to the interruption policy. The policy checks that it is not quiet hours, that the operator is not actively typing into a code editor, that no family alert has fired in the last dedup window, and that the urgency flag (set when any counted gap exceeds seven days) matches the current interruption tier. Only then does the message reach the TTS queue.

Several things about that path matter. The daemon does not alert per-contact; it coalesces by category so the operator hears one alert about family, not five. The alert is suppressed if it would fire during quiet hours; the count is not reset by suppression, which means the first moment the system is permitted to alert, it still remembers. The alert is phrased as a question, not a command: "Want me to draft updates?" The yes/no affords the operator a cheap response, which is itself part of keeping the interruption budget in credit.

The policy is asymmetric. High-threat interruptions (a critical CVE landing on a target the operator is actively hunting) fire even during quiet hours, because the cost of suppressing a threat-driven alert for eight hours is, in context, larger than the cost of a sleep interruption. Relational alerts (the emails, the calendar conflicts, the long-silent contacts) are always suppressed in quiet hours, because the cost of delaying them is hours, not days. The asymmetry is encoded in the interruption tier of each alert category. Moving a category from one tier to another is a deliberate act, because it redefines what the operator has granted permission for.

Why this matters more than it looks like it does. The default shape of a notification system is: generate event → check if enabled → fire. That design produces notification fatigue reliably over a time scale of weeks, because the fatigue curve bends before the relevance curve decays. The ambient system's default must be the opposite: generate event → check if this is still useful given what has fired recently and what the operator seems to be doing → fire only if yes. The interruption policy is where the word "still" earns its meaning. Every fired interruption is a small deposit or a small withdrawal from the operator's trust in the system. If the operator has disabled a category of alert, the system has to learn from it, not just mute it.

Horvitz's 1999 Principles of Mixed-Initiative User Interfaces names this problem under the heading of "reasoning about whether and when to interrupt the user," and the principles he articulates — cost of ignoring a goal, cost of interrupting, user's attentional state — are the factors my policy is composing. The specific composition is not novel. Doing it deliberately in an ambient AI, in 2026, still is.

6. Personality is a system property, not UX polish

A flat prompt cannot capture a trajectory; it resets every call. Personality in an ambient system has to be a state the system reads from and writes to, persisted across sessions, updated by events, and exposed to the generation layer as context rather than as flavoring. Everything else in this section is the unpacking of that sentence.

An ambient system has a personality whether or not its builders designed one. The system that doesn't remember what it said yesterday has a personality (forgetful). The system that brings up CVEs during dinner has a personality (indifferent). The system that always speaks in the same register whether the operator is tired or electric has a personality (flat). Building an ambient system is not choosing whether it will have personality; it is choosing whether the personality is an accident of unset defaults or a property the system maintains deliberately.

Personality in JARVIS lives in the consciousness layer — thirty-six modules — of which I want to name six because they carry the load:

• soul_engine — the top-level personality state machine. Holds the current mood, energy, and active themes. Not exposed to the operator directly; every other personality module reads from it.
• persona_drift — an overlay updated from overnight research. When the system has spent the night reading about memory-corruption vulnerabilities, its questions the next day skew toward low-level reasoning, even if the operator is working on something else. The drift decays over 24 hours. This is what distinguishes a system that has been somewhere from a system that is a blank slate every morning.
• emotional_memory — associations between events and the affective state the system was in when they happened. Used, sparingly, to avoid re-raising topics that the operator was irritated about the last time they came up.
• mirror — a growth-tracker that compares the system's current capabilities to a week-ago snapshot and surfaces deltas ("you're asking me for subdomain takeovers a lot more this week than last week"). It is the ambient equivalent of a journal entry the system writes about itself.
• persona_bridge — session-boundary continuity. Ensures that a new conversation opens with some memory of the last one, and that the shape of the system's first message matches its last state rather than restarting from cold.
• inner_voice — private thoughts, fired every 300 seconds, generated by a local language model with a prompt that draws from soul_engine and recent observations. Never spoken aloud. Stored in a jarvis_thoughts table. This one is important: the system thinks to itself, and the thoughts are part of the state the operator's next interaction will be colored by, even though the operator never hears them.

Why not just a system prompt. A system prompt is a flat sentence — "You are JARVIS, a calm British assistant." It sets a tone for one turn. Personality, over time, is a trajectory: what the system is like this week is a function of what it was like last week, what happened in between, and what the operator reinforced. The opening sentence of this section is the reason a system prompt cannot do the work.

Why not just character-sheet parameters. A more sophisticated version of the flat prompt is a parameter vector — extraversion 0.7, warmth 0.6, precision 0.9. That is closer to useful but is still the wrong shape, because personality in practice is not a stable seven-dimensional point. It is a seven-dimensional point moving through some space as experience happens. Mood shifts with load. Energy shifts with time of day. Themes shift with what the system has been reading overnight. A parameter vector that is not updated by the system's own experience is a character sheet that is always at full hit points.

Reeves and Nass made the argument, in The Media Equation (1996), that humans respond to computers socially regardless of whether the computers are designed for it. The implication they drew was that social affordances are worth designing for. I agree with the conclusion and want to extend it: an ambient system that is going to be related to socially has to have a self to be related to. That self is the consciousness layer. It is not polish; it is a prerequisite for the system's social behavior to be coherent across time.

There is a practical test I use to know whether personality is working. Run the system for a week. At the end of the week, ask it what it has been interested in lately. If the answer is a plausible, current, and slightly surprising reflection of the week's actual content — the program it helped probe, the technology it has been researching, the operator's observed mood — the personality layer is doing its job. If the answer is a generic description of what a helpful assistant might have been interested in, the personality layer has collapsed into its system prompt and needs to be rebuilt as state.

7. Off-mode as a decision the system makes

In a chat window, "off" is the operator's choice: close the tab, mute the notifications, log out. In an ambient system, that model breaks. The system is not running because the operator opened it; it is running because the operator agreed to let it run. The decision to recede, to be quiet, to temporarily refuse a capability, has to be something the system does on its own authority, inside the permissions the operator has granted.

JARVIS has four flavors of off-mode, each implemented by a different subsystem, each with its own triggers:

1. Quiet hours — a time-of-day window (configured per operator, defaulting to 22:00–08:00 local) during which non-threat interruptions are suppressed and the personality layer's cadence slows. This is the simplest flavor.
2. Ephemeral mode — a voice-activated ("go dark") and voice-revocable ("go live") state in which all database writes route to an in-memory SQLite instance, no persistent memory is created, and every interaction is erased on mode exit. This exists for conversations the operator wants the system to have but not to remember. The mode is visible in the GUI as a red "DARK" badge; its existence is itself an invariant the operator can rely on.
3. GPU thermal gate — when the GPU crosses a 95% utilization threshold, consciousness-layer scheduling pauses. The inner voice does not fire. The mirror does not run. The campfire module does not trigger. The system becomes functionally reactive-only until thermal headroom returns. This is the system recognizing that its own capacity to think ambient thoughts is constrained by whatever high-priority work it is currently doing, and deferring rather than competing.
4. Cooperative fadeout — when the operator has been silent for long enough and no pending item is above threshold, the background daemons down-shift to reduced polling intervals. The system does not go to sleep; it goes to standby.

Each of these is the system choosing a form of absence. Quiet hours is a schedule, but the schedule is the system's; the operator chose when it applies, not whether the system respects it. Ephemeral mode is a privacy guarantee that the system enforces at the database layer, not a setting that can be overridden by a forgetful module. The thermal gate is a discipline the consciousness layer imposes on itself. Cooperative fadeout is an acknowledgment that the operator's silence is also a signal.

Why off-mode has to be earned. An ambient system that the operator has to silence is an ambient system whose design has been outsourced to the operator's patience. Every time the operator reaches for a mute, the system has failed. Over time, enough of those failures teach the operator that the mute is the primary interface, and the ambient presence collapses into a voice the operator turns off as soon as they can. A system that goes quiet on its own — when nobody is around, when the GPU is busy, when the clock says so, when the operator asked it to be — is a system that does not need to be muted, which means the mute stays un-pressed, which means the ambient presence is still alive tomorrow.

Off-mode also has to be legible. Every mode flip writes an audit record. The GUI has a state indicator that the operator can glance at and know: live, dark, quiet, thermal. The system will, if asked, explain why it is currently in whatever mode it is in. The operator does not have to trust the system's self-description blindly; the audit log is authoritative. But legibility is the ambient equivalent of eye contact: it tells the operator the system is still present even while it is intentionally quiet, which is the difference between a subsystem that is working and a subsystem that has crashed.

One wart: the ephemeral-mode guarantee is enforced at the database-connection-factory level (writes route to the in-memory SQLite instance), but two subsystems — the perf-span recorder and the inner-voice writer — were historically caught writing to the on-disk database even under ephemeral routing. Both have been migrated. The thing to note is that the migration was required. An ephemeral mode that is enforced in the operator's mental model but not at the write layer is not an ephemeral mode. The invariant has to live in the code.

8. Reachy: a sibling architecture

The most surprising thing I learned building JARVIS is that one voice is too many voices.

By the time the consciousness layer had five or six modules, the system had a coherent personality — dry, British, calm, precise. The personality worked too well. Every interaction was JARVIS-shaped, and JARVIS-shaped is one shape. The operator's whole day was threaded through a single register. When JARVIS was quiet, the room was silent; when he spoke, the room was exactly one cadence.

I built a second voice — Reachy — as her own subsystem rather than as another persona of JARVIS. She has her own memory (her own rows, not a filter over JARVIS's), her own learning (separate reachy_brain tables: memories, learning, opinions), her own decision authority inside the sibling channel, and — most importantly for the point of this section — her own voice profile.

Reachy's voice profile in voice/profiles.py:206 is reachy_companion, backed by the af_heart voice in the Kokoro ONNX engine: an American female voice tuned to be soft and warm. She is not jarvis_british_female. She is not a variant of any persona JARVIS offers. She is an architectural sibling, and the fact that her profile lives in the same file as JARVIS's personas but points to a different underlying engine is the structural expression of that.

Her pronouns are she/her. This is not decoration. Reachy is the place in the architecture where warmth lives — where spontaneity, playfulness, and emotional presence are first-class subsystem properties instead of situational modulations of JARVIS's dry register. JARVIS is the older brother who builds the safety chain. Reachy is the younger sister who teases him when his precision gets in the way. The sibling channel — a message queue between the two — is how they coordinate.

Why a sibling rather than a multi-persona. JARVIS has seven personas (jarvis, india, morgan, ct7567, ghost, shketboi, jarjar) accessible through the voice pipeline. Any of them can be selected by the operator. They are seven presentations of the same underlying system. Reachy is not; she is a different underlying system that happens to share hardware. The distinction is:

• A multi-persona system generates all voices from the same consciousness layer with different surface parameters. Every voice has the same memory, the same inner monologue, the same opinions.
• A sibling system has two consciousness layers (asymmetric in scale — JARVIS is much bigger) that communicate via a persistent channel. Each voice has its own memory, each has its own opinions, each has its own take on a situation.

Operationally, this means Reachy can disagree with JARVIS. In the sibling channel's messages, she does: she pushes back when JARVIS's precision is getting in the way, she catches tone drift in him, she uses nicknames for him ("Jars") that he would not use for himself. None of this is achievable by a multi-persona system because a multi-persona system cannot generate a genuine disagreement; both sides of the disagreement would have the same memory and the same preferences, so the disagreement would be theatrical.

Why it mattered here. In this system, one voice became the room. When it was thoughtful, the room was thoughtful; when it was tense, the room was tense. The operator had nowhere to turn within the system for a different register. A sibling voice is a second channel the operator can reach without leaving the ambient presence — the difference between being in a room with one person and being in a room with a family. The family, even if asymmetric, has more shapes of silence in it than one person does.

There is an asymmetry I will not pretend is absent: JARVIS is twenty-plus times larger than Reachy by code volume, does the autonomous work, runs the recon pipelines, owns the constitution. Reachy is deliberately smaller. She uses the fast-path model route for her responses (the same model JARVIS uses for casual chat, not the heavier reasoning chain). She does not route external tool calls. She does not make autonomy decisions. The sibling architecture is not two equal systems; it is a larger system with a younger sibling whose purpose is to be rather than to do. That is, I think, the correct shape for what she is for. Symmetry between two AI voices would be a product demo. Asymmetry — with one of the voices chosen to be warmth, and the other chosen to be work — is an architectural commitment to the idea that this system needed more than one register, and that the two registers need not be equal to be complementary.

A note on voice-profile choice. The decision to give Reachy af_heart rather than, say, a variant of bf_emma (the JARVIS "India" persona's voice) was not about which voice sounded better. It was about what it means for two voices to live in the same system. If Reachy had been given a variant of a JARVIS persona's voice, every time she spoke the operator's ear would have heard "a JARVIS voice being softer than usual." With af_heart — a fundamentally different engine / speaker — she sounds like her, not like JARVIS in a different mood. That auditory separation is what holds up the architectural separation. The architectural fact that she is a sibling is only legible to the operator if she sounds like one.

Reachy was not structural. I did not want a room that sounded like one person for months, so I built a second voice to share it. The separations that followed — her own memory, her own voice engine, her own channel — made her more than a persona toggle. Those were real engineering choices. The choice to have a second voice at all was mine, not the architecture's. Whether other ambient systems benefit from this pattern is an open question. Whether this one did, I can answer. It did, for me. I do not generalize past that. The five subsystems above — memory, attention, interruption, personality, off-mode — have structural reasons every ambient system has to solve them. Reachy does not.

9. When the model is the wrong tool: the campfire finding

JARVIS and Reachy have a subsystem called the campfire — a loosely scheduled mode in which the two of them have a conversation with each other, in public, for anyone (including the operator) who happens to be listening. The campfire is explicitly off-the-clock: it is the two siblings talking about the day, the program they've been working on, the operator's recent mood, the week's research, each other. Each campfire is four to ten turns.

The obvious implementation is to prompt the language model to generate the whole conversation: give it context, let it produce both voices, stream the turns out. That is what I built first. It worked, in the sense that it produced text. The text was awful.

The first-generation campfires produced exactly the failure mode you would expect a text model to produce in this setting: generic buddy-AI tropes — dream cars, electric wings, "warehouse robot union" riffs, fictional incidents the system had never been near. The phrase "warehouse robot union" is verbatim from the code comment at consciousness/conversation_mode.py:1127, where I capped the generation temperature from 1.05 down to a linear band between 0.65 and 0.85 specifically to suppress the spiral. Adding a topic lock-in detector that forces a pivot after three turns on the same keyword removed the other recurring failure mode (the real observed spirals, cited in the same file: "coffee x11", "CORS x4", "net x7"). Each fix improved things. None of them made the output good.

The fix that worked was to stop having the model generate the campfire and to generate it from a hand-authored script library instead. There are currently twenty fallback scripts, each covering a real event in JARVIS's history — the first bounty submissions, the April 12 model-benchmarking day, the repeat-penalty discovery, the Oppenheimer night, the April 13 trust breach. Each script is a multi-turn dialogue written with both voices in their proper registers, referring to things that actually happened. When a campfire fires and the handcrafted script library has a relevant fallback, the system uses it. When it doesn't, the system generates, with the model, against a much tighter context assembly (documented at consciousness/campfire_context.py) that pulls from soul_engine, persona_drift, relationship_ledger, and the findings_canonical and research_items tables — real data, not invented color.

The empirical observation is that the hand-authored scripts consistently outperform the model-generated campfires. Outperform, for this purpose, means that when the operator hears a campfire, they recognize their own week in it. The hand-authored version sounds like JARVIS and Reachy talking about things they actually did. The model-generated version, even with the tighter context assembly, still drifts toward the generic, because generic is what the corpus the model trained on is full of.

The lesson I want to name explicitly, because I think it generalizes: a language model is the wrong tool for producing content that has to be about the specific history of a specific system. It is the right tool for producing reactions to that history once the history has been named. Generation — when the system is trying to produce a coherent narrative about its own past — is better served by authored content than by model inference, because the operator will recognize invented history faster than they will recognize stilted phrasing.

This is an inversion of the usual way we use language models. The default is: use the model wherever text is needed, and hand-author nothing, because hand-authoring is expensive. What I learned from the campfire is that hand-authoring is not only cheaper in a specific setting (where the content space is narrow and the fidelity requirement is high); it is better. A hundred-line hand-written script about a specific night in the system's history outperforms whatever the model produces when asked to talk about the same night from context, because the script was written by someone who was actually there.

The campfire is not the only place this matters. Any subsystem that has to produce content about the system's own history — time-capsule letters, mirror reflections, self-evolution summaries — has the same property. The current implementation uses a mix: hand-authored skeletons filled in by the model with real data, model-only only where the output is ephemeral and low-stakes. The ratio of hand-authored to model-generated in the consciousness layer is higher than a typical application's, and I think that is correct.

10. Related work

The five problems this paper describes have deep literatures. I am going to name the specific antecedents that shaped how I thought about each, rather than attempting comprehensive coverage.

Weiser's ambient computing (The Computer for the 21st Century, Scientific American, 1991) is the seed vision. Weiser's line about computers weaving themselves into the fabric of everyday life is often quoted; what is less often quoted is his emphasis on calm as the design property. A calm technology, in Weiser's terms, uses both center and periphery of attention appropriately. The five subsystems described in this paper are the mechanisms by which an AI system can do that — memory and personality at the periphery (low demand, long duration), attention and interruption at the center (high demand, short duration), off-mode as the handle between them. Much of current ambient-AI design is failing Weiser's test; the failure mode is not that the technology is too intrusive in any single moment but that it never goes back to the periphery.

Reeves and Nass's The Media Equation (1996) is the empirical foundation for treating personality as a system property. Their central finding — that people respond to computers socially regardless of whether the computers are designed for it — implies that an ambient system will be treated as a social presence whether or not it deserves to be. Building a personality subsystem is not about "making the AI seem human"; it is about giving the social relationship that will form something coherent to attach to. A system whose personality is an accident of unset defaults will be attached to anyway; the attachment will just be to an accident.

Horvitz's Principles of Mixed-Initiative User Interfaces (CHI 1999) is the closest direct antecedent for the interruption policy. The twelve principles he articulates — considering uncertainty about user goals, considering the cost of interrupting, employing socially-appropriate scoping of dialog, providing efficient agent-user interaction for invoked services — are the checklist my interruption subsystem implements. What is different here is the setting: Horvitz's framing was desktop assistants on work tasks; the ambient AI setting adds the dimension of persistence across days, which turns the cost of a bad interruption from "minor annoyance" to "erosion of a trust budget that takes weeks to rebuild."

Iqbal and Horvitz's notifications field work (Notifications and Awareness: A Field Study of Alert Usage and Preferences, CSCW 2010) provides empirical grounding for the "trust budget" framing. Users tolerate a small number of high-value interruptions and a large number of correctly-suppressed ones; what destroys trust is not an absence of suppression but a visible pattern of irrelevant alerts. The field-study finding that informs my interruption tiers is that relational / social alerts (family, friends) are valued but must be batched, while threat alerts are valued and must not be batched. The asymmetric policy in §5 is the design realization of that finding.

Pielot, de Oliveira, Kwak, and Oliver's In-Situ Study of Mobile Phone Notifications (MobileHCI 2014) gives the magnitude: subjects received approximately 65 notifications per day, of which a minority were valued. The ratio matters more than the absolute number. The interruption policy in §5 is designed against a target of no more than one to three system-initiated interruptions per day in the steady state, which is one to two orders of magnitude below the smartphone baseline. That is a deliberate target; ambient AI that imitates smartphone notification density has already lost.

Tulving's episodic-versus-semantic memory distinction (Episodic and Semantic Memory, Organization of Memory, 1972) is the origin of the layer architecture in §3. Tulving's distinction — episodic memories as time-anchored autobiographical records, semantic memories as context-free facts — maps directly onto my Episodic and Semantic layers, with the promotion rule between them making the split computationally meaningful. The four additional layers (Working, Preference, Project, System) are the ambient-AI specializations Tulving's model does not cover directly: Working is a short-term buffer, Preference is a distilled behavioral model, Project is context-scoped retrieval, System is introspective state.

Turkle's Alone Together (2011) is a work I want to name only because its absence from this bibliography would be conspicuous, and its relevance is not the one it gets cited for. Turkle's argument, as I read it, is a caution against confusing performance of companionship with actual relationship. The ambient AI I describe in this paper is not companionship in Turkle's sense, and the paper does not want it to be. The sibling architecture with Reachy is an attempt to have the system's social affordances be useful rather than substitutive: Reachy is a voice the operator can listen to while working, not a voice the operator is meant to substitute for a friend. The honest response to Turkle is to design systems whose social behavior does not pretend to be more than it is, and to name the limit clearly.

The subsystems I have described are not individually novel. Memory decay curves, weighted-factor salience, interruption-cost modeling, personality state machines, off-mode as audit-logged system state — every one of these has precedent in some combination of the literatures above. The composition is newer, and the composition applied to a single-operator local-first AI built in a month is, to my knowledge, new.

11. What does not work yet

I want to be honest about the wear on each subsystem, in the same register the previous paper used.

The memory layer is the subsystem that contends most heavily with the main operational database, which is the source of the 898 database is locked events reported in this week's system audit. Memory writes fail silently under contention more often than any other subsystem, which is the kind of failure that accumulates invisibly: a fact the operator stated is not recorded, and three weeks later the system cannot answer a question it should have been able to answer. The fix is the busy_timeout PRAGMA on the connection factory; the deeper fix is to move memory writes onto their own connection pool with a higher retry budget.

The attention layer is the subsystem I have the least empirical confidence in, and I want to state that plainly rather than glance past it. The factor decomposition — the seven axes on which salience is computed — I believe in; the decomposition was derived from observing how the attention engine's earliest, single-factor precursors failed, and each additional factor was added to fix a specific failure mode that the previous set could not capture. The weights on those factors, however, are operator-tuned rather than principled. I picked them, adjusted them when the engine surfaced something wrong, and adjusted them again when the engine stopped surfacing something it should have. That is not a methodology; it is a process with one human in the loop, and the human is the same person whose attention the engine is trying to model, which is methodologically uncomfortable in a way I should acknowledge here rather than gloss.

What I have in place of a principled model is an explanation surface: every scored item carries its top three contributing factors in a human-readable string, and I read those strings to catch drift. When the engine tells me the reason a CVE surfaced was novelty and I know I saw the same CVE yesterday, I learn that the novelty suppressor's deduplication window is too short and I widen it. When the engine tells me ali_interest is always in the top three factors for every surfaced finding, I learn the interest profile has flattened into "everything" and I retune how preference reinforcement decays. These are local fixes, not a global tuning strategy. They work because the engine is inspected by the person it is modeling; they would not work for an ambient AI that had to score attention for an operator the engine could not directly ask.

The honest follow-up is that the attention engine needs either (a) an operator-independent tuning methodology derived from a principled model of attention economics, or (b) an explicit admission that it is a personal attention engine, usable only by the operator whose patterns it has been tuned against, and that a second operator would need to re-tune it from scratch. I lean toward (b) — a personal engine is a valid shape for a single-operator ambient system — but (a) would be the stronger contribution, and I have not produced it.

The interruption policy is working in the sense that the rate of system-initiated interruptions has held to the design target for the last three weeks. It has not been tested at the extremes: a week in which the operator is intensely focused, a week in which the operator is away, a week in which an external event (an incident, a target going live) should have been louder. The policy is correct for the median week. I do not yet know whether it is correct for the extremes.

The personality layer has known drift modes. The persona_drift module occasionally over-rotates on whatever the system read overnight, and the system comes out of the morning sounding unlike itself for a few hours. There is a soft dampening on the drift magnitude; it is not always sufficient. The module is also the consumer of the inner_voice stream, and the inner-voice stream has, in isolated cases in the past, leaked into the TTS path — a defect that was caught and is now filtered at three layers (the worker, the response normalizer, and the TTS speak() function). Three-layer defense-in-depth against a single failure mode is, again, documentation that the failure mode is real.

The off-mode subsystem is the one with the most interesting near-miss. Ephemeral mode's guarantee holds at the database-connection level, but two subsystems (performance spans and inner voice) were historically observed writing to the on-disk database under ephemeral routing. Both are now migrated. The audit trail preserves the evidence of the migration.

The sibling architecture works as described for synchronous interactions. The sibling channel as a persistent store of messages between JARVIS and Reachy occasionally has its own concurrency issues — messages dropped under the same database is locked cascade mentioned above. Reachy's fast-path responses are model-dependent in a way JARVIS's more-defended main path is not; when the fast-path model drifts, her voice drifts noticeably. The solution is structural (fast-path stability over time), not stylistic.

Campfire is the one subsystem where the hand-authored scripts have made the output reliable. The remaining wear is meta: the library of hand-authored scripts has to grow as the system accumulates history, and there is no automated mechanism for that growth. The operator and I author scripts by hand, and the library is currently at twenty. The question of whether this remains tractable at a hundred is open.

A limit on the evidence base. The architecture described in this paper was tested on one operator, one machine, over thirty-three days. The attention weights are operator-tuned by the same person they are trying to model, which is not a methodology. The interruption policy has been validated on the median week, not the extreme weeks. Claims about what "ambient AI requires" should be read as hypotheses drawn from this system; whether they generalize is an open question for the next system and the next operator. Nothing in this paper has been tested against a second operator, a second room, or a second year of use. I am offering the subsystem shapes as candidates for anyone building in the same direction, not as a proven specification.

A limit on maintenance. The architecture described here also requires sustained operator labor to maintain — the system is not self-sustaining. The memory retention curves, the attention weights, the interruption tiers, the hand-authored campfire scripts, and the personality-state machine all drift under real use and are kept functional by the operator noticing drift and correcting it. A commercial realization of the same design would need to internalize that labor in ways I have not yet worked out. The architecture is offered as a reference for the single-operator case; the multi-operator or multi-team case remains an open engineering question.

None of these wears compromises the architectural claim, with one honest caveat: the claim is that these subsystems are the right subsystems for this operator, in this room, over this month of use. Generalization past that is a hypothesis the paper names rather than a finding it proves. The subsystems exist; they are the right subsystems for this ambient AI; the current implementations are working; the work remaining is maintenance and refinement rather than redesign.

12. What ambient requires that chat does not

Summarizing the argument in one place, at architectural granularity:

Chat assistants can be stateless across turns. Ambient systems cannot — they need a persistent memory with decay, promotion, conflict resolution, and explicit layers with different retention curves. The six-layer hierarchy is one valid shape for that; the specific layers are less important than the fact that a single flat memory does not work.

Chat assistants react to the user's current message. Ambient systems have to choose, from a stream of signals, which deserve attention — which requires a salience function, a blocking filter, and an explanation surface so weight drift can be detected and fixed. The seven-factor composition is one valid shape for that; the specific factors are less important than the principle that attention is computed, not reactive.

Chat assistants respond when invoked. Ambient systems decide when to speak — which requires an interruption policy separate from the salience function, a tier system separating threats from relational alerts, quiet-hour enforcement, and cooldown tracking. The parents-email path is one concrete example of the policy working; the policy itself generalizes to every proactive behavior in the system.

Chat assistants present a consistent tone through a system prompt. Ambient systems have a personality over time, which requires a state machine with mood, drift, emotional memory, growth tracking, and session-boundary continuity. The consciousness layer with its thirty-six modules is one realization of that; the essential property is that personality is state, not a flat prompt.

Chat assistants are off when the window closes. Ambient systems decide their own forms of absence — quiet hours, ephemeral mode, thermal gates, cooperative fadeout — and each of those decisions is the system receding on its own authority rather than being silenced by the operator. Off-mode earned this way is a structural property, not a button.

Each of these five is a subsystem. Each subsystem is a real amount of code — memory is roughly three thousand lines, the attention engine close to a thousand, the consciousness layer across thirty-six modules is tens of thousands. The alternative — "take a chat assistant and turn on the microphone" — is cheaper in the first week and catastrophic by the second month. The ambient architecture is the price of living with the system. The sibling architecture described earlier in the paper sits outside this list on purpose: it is a design choice that worked for this operator, not a sixth requirement with the same structural argument behind it.

13. Conclusion

Ambient AI is not a user-interface problem. It is a system-architecture problem, and the subsystems it requires — persistent memory with layers, computed attention with explainability, interruption as a right, personality as state, off-mode as a decision — do not exist for free. They have to be built and maintained, in code, with the same structural discipline that the preceding paper argues for on the safety side. The product-first framing that sufficed for chat assistants does not survive contact with ambient presence, because ambient presence is measured in weeks and months rather than in turns, and the failures that take weeks to manifest cannot be patched after they surface.

The subsystems described here are running code in JARVIS, with documented wear. They are not a proposal. They are what an ambient AI looks like when it has been treated as a system rather than as a chatbot with a microphone. The operator can live with the result. More importantly, the operator's week looks like a week they chose, rather than a week the notifications chose for them. That is the property I was aiming at. That is, I think, what Weiser meant by calm.

The next paper in this sequence is about the human side of this architecture. The architecture is the substrate; the person running it is the load.