Observation Store

The observation store is the persistence backbone for all agent memory on the Rensei platform. It exposes two complementary interfaces - IMemoryStore (retrieval path) and IObservationStore (export and retention path) - both backed by PostgreSQL with pgvector.

Architecture overview

The two interfaces serve different call sites:

The observations table

Observations are rows in a shared observations table scoped by (org_id, project_id, agent_id). Key columns:

Content-hash deduplication ensures the same observation text is never stored twice for the same org/project/agent triple. Writes that collide on content hash are silently discarded.

IMemoryStore - retrieval

The retrieval store is the MemoryStore type used by buildSessionMemoryBlock and the recall MCP tool.

interface MemoryStore {
  retrieve(scope: RetrievalScope, query: RetrievalQuery): Promise<ScoredObservation[]>
  store(scope: RetrievalScope, observation: ObservationInput): Promise<Observation>
  deleteById(scope: RetrievalScope, id: string): Promise<void>
}

interface RetrievalQuery {
  text: string
  embedding?: number[]   // float4[] - pgvector query vector
  limit?: number         // default 20
}

When pgvector embeddings are configured (EMBEDDING_PROVIDER env), retrieval uses cosine distance (<=>) against content_embedding. Without embeddings it falls back to ILIKE text matching. Both paths return the same ScoredObservation[] shape.

IObservationStore - export and retention

PgObservationStore (created by createPgObservationStore()) implements the export and retention contract. It returns null when the database is not configured so the export route degrades to a 501.

const store = createPgObservationStore()
if (!store) {
  return Response.json({ error: 'DB not configured' }, { status: 501 })
}

const observations = await store.list({
  orgId: 'org_abc',
  projectId: 'proj_xyz',            // optional
  agentId: 'agent_sdlc_dev',        // optional
  dateRange: { from, to },          // optional
})

ObservationRecord shape

interface ObservationRecord {
  id: string
  orgId: string
  projectId?: string
  agentId: string
  content: string
  contentHash: string
  classification: MemoryClassification   // derived from metadata.classification
  createdAt: Date
  updatedAt: Date
  softDeletedAt: null                    // always null until schema migration
  feedbackWeight: number
  provenance?: {
    sessionId?: string
    derivedFrom?: string[]
  }
}

Classification derivation

classification is read from metadata.classification when it is a valid MemoryClassification value (public | internal | confidential | restricted). Observations without an explicit classification default to 'internal'. See Memory Classification for how this field is populated at write time.

Retention state

Once the schema migration lands the inert guards will be replaced with real TTL queries keyed on soft_deleted_at and classification.

Scope filtering

list() accepts an optional scope that narrows results to a single org, project, agent, or date window. All scopes are AND-combined:

SELECT ... FROM observations
WHERE org_id = $1
  AND project_id = $2   -- when scope.projectId is set
  AND agent_id = $3     -- when scope.agentId is set
  AND created_at >= $4  -- when scope.dateRange.from is set
  AND created_at <= $5  -- when scope.dateRange.to is set
ORDER BY created_at DESC

Related pages

• Context Injection - how observations flow into session start prompts
• Feedback Retrieval - EMA weight updates that drive the weight column
• Memory Classification - BFSI tier tagging at write time
• Feedback Retention Audit - purge policies and audit trail
• Memory Export - operator-facing export UI