Auto-Ingest

Auto-ingest is a continuous learning signal that mines every successfully completed agent session for architectural patterns and decisions, then contributes them to the knowledge graph without any manual trigger. It is the primary mechanism for keeping the graph current as agents do real work.

Trigger

Auto-ingest fires inside a next/server's after() block at session terminal time, when the session reaches a terminal status and workResult === 'passed'. Sessions that fail, crash, or are manually stopped do not produce observations - only sessions where the work was accepted contribute to the corpus.

session terminal (workResult = 'passed')
  └─ after()
       └─ maybeFlushArchObservations({ sessionId, workResult, orgId })

The after() wrapper ensures the 200 ack is already on the wire before any DB writes happen. The function never throws - errors are caught and returned in the outcome object.

Extraction heuristics

Two signals are extracted from the session_activities stream:

Decisions

session_activities rows where activity_type = 'thought' are scanned for decision-marker phrases. When a match is found, a kind: 'decision' observation is emitted with the thought text (truncated to 280 characters).

Decision markers matched (case-insensitive):

decision:   |   i decided   |   we decided   |   chose to
design choice   |   architectural decision   |   rationale:

These markers are intentionally conservative. False-positive decision observations are noisier than false-negatives - over-emitting drowns the review dashboard.

Patterns (file-write clusters)

session_activities rows where activity_type = 'action' and tool_name is one of Edit, Write, MultiEdit, or NotebookEdit are aggregated by session. All touched file paths are collected into a single kind: 'pattern' observation:

Session <sessionId> touched files: src/lib/auth.ts, src/app/api/auth/route.ts

This coarse summary is by design. The arch-intel pipeline's clustering step lifts recurring path-clusters into architectural patterns - auto-ingest just feeds it a candidate stream.

Idempotency

Observations are idempotent on (org_id, project_id, agent_id, content_hash) using ON CONFLICT DO NOTHING. The content hash mixes sessionId + signalKey + text so:

Two distinct sessions with identical thought text produce different rows (intentional - each session is its own data point).
Double-invocation on the same session is a no-op (content hash is stable).

Confidence

Auto-ingested observations use weight: 0.3 (the AUTO_CONFIDENCE constant). This is deliberately below the 0.55 used by PR ingest, which benefits from human review having filtered the work. The lower weight keeps auto-ingest signals appropriately ranked below merge-confirmed signals in the arch-intel review dashboard.

Downstream: the graph-extraction cron

Auto-ingest writes raw rows to the observations table - it does not refine them into graph nodes itself. The refinement into typed graph nodes (pattern, convention, decision, deviation) runs on the platform's scheduled extraction driver:

/api/cron/graph-extraction runs every 15 minutes (Vercel Cron). It enumerates memory-enabled projects, filters each through isGraphEnabledForProject (per-project graph enablement - see Knowledge Graph enablement), and runs one bounded extraction pass per enabled (org, project) scope.
Each pass drains a bounded batch of not-yet-extracted observations through the LLM extractor and upserts the results into graph_nodes / graph_edges. Extraction is idempotent - re-running a pass over already-extracted observations never duplicates nodes.
Operators can drain a single scope on demand via POST /api/graph/extraction/run with { "orgId": "...", "projectId": "..." } (CRON_SECRET bearer auth).

See Extraction Pipeline for what each pass does.

Each fresh insert also emits emitMemoryObservation('created', obs) on the in-process memoryHookBus, but no pipeline subscriber is attached in the deployed platform - the event is a no-op today, kept as a seam for future in-process consumers. The cron driver is the authoritative path from observations to graph nodes. Duplicate observations (idempotency hits via ON CONFLICT DO NOTHING) do not emit a bus event.

API

import { maybeFlushArchObservations } from '@/lib/arch/auto-ingest'

const outcome = await maybeFlushArchObservations({
  sessionId: 'sess_...',
  workResult: 'passed',     // anything else short-circuits
  orgId: 'org_...',
  projectId: 'proj_...',    // optional - resolved from Redis if omitted
  agentId: null,            // optional - defaults to 'system/arch-auto-ingest'
})

// outcome:
// {
//   contributed: boolean,
//   insertedCount: number,
//   candidateCount: number,
//   reason?: string   // present when contributed = false
// }

The reason field on a contributed: false outcome is a diagnostic string, not an error. Common values:

`reason`	Meaning
`non-passed-work-result:<value>`	Session did not pass; expected
`no-session-activities`	No activities found for the session
`no-candidates-extracted`	Activities found but none matched decision/write heuristics
`session_activities-read-failed`	DB read failed; logged as warning

Pure extraction helper

The extraction logic is separated into a pure, synchronous helper that can be unit-tested without a DB:

import { extractObservationCandidates } from '@/lib/arch/auto-ingest'

const candidates = extractObservationCandidates({
  sessionId: 'sess_...',
  orgId: 'org_...',
  projectId: 'proj_...',
  agentId: 'system/arch-auto-ingest',
  activities: [...],   // typeof sessionActivities.$inferSelect[]
})
// returns SynthesizedObservation[]

PR Ingest - higher-confidence ingest from merged PRs
Extraction Pipeline - converts observations to graph nodes
Context Injection - graph triplets surfaced to agents
Graph Feedback - outcome signals fed back to node weights

On this page