Feedback Retention Audit
Weights, retention, and audit.
Partial. The feedback weight (EMA updates), audit trail, and retention policy engine are fully implemented. Automated TTL purge (findExpired / softDelete) is currently safe-inert pending the soft_deleted_at schema column migration. Hard-delete and classification update on specific IDs work today.
This page covers three related systems that manage the lifecycle of memory observations: feedback weights (how useful an observation is estimated to be), retention policies (how long it is kept), and the audit events emitted by the implemented memory paths listed below. The audit trail is tamper-evident for a supplied retained segment; it is not proof that every possible memory operation was captured.
Feedback weights (EMA)
Each observation has a weight column (default 1.0) that is updated via an Exponential Moving Average after every session outcome. Higher weight = more likely to surface in retrieval; lower weight = pushed down or eventually retired.
Update formula
newWeight = previousWeight × (1 - alpha) + signal × alpha
where signal = 1.0 for 'accepted' outcomes
signal = 0.0 for 'rejected' or 'rework' outcomesalpha (the learning rate) differs by outcome direction:
| Outcome | Alpha |
|---|---|
accepted (positive) | alphaSuccess (default 0.1) |
rejected / rework (negative) | alphaFailure (default 0.15) |
Negative feedback uses a higher alpha to ensure harmful observations decay faster than helpful ones accumulate. The asymmetry is intentional.
Weight history
Every update writes a row to observation_weight_history:
SELECT * FROM observation_weight_history
WHERE observation_id = 'obs_abc'
ORDER BY recorded_at DESC;| Column | Description |
|---|---|
observation_id | UUID of the affected observation |
session_id | The session that triggered the update |
outcome | accepted, rejected, or rework |
previous_weight | Weight before the update |
new_weight | Weight after the EMA update |
alpha | The alpha used (may be doubled for misleading observations) |
org_id, project_id | Tenant scope |
recorded_at | Timestamp |
Aggressive downweight
When an observation is classified as misleading (≥ 3 failed sessions, 0 successful sessions), the diagnostics module applies a 2x alphaFailure update. See Memory Diagnostics for the detection and application logic.
Retention policies
Retention policies control how long observations are kept based on their BFSI classification tier.
Default retention schedule
| Classification | retentionDays | softDeleteGraceDays |
|---|---|---|
public | indefinite (null) | 0 |
internal | 365 days | 30 days |
confidential | 90 days | 14 days |
restricted | 30 days | 7 days |
After retentionDays the observation is soft-deleted (still in the store but invisible to queries). After an additional softDeleteGraceDays it is hard-deleted.
Org-level overrides
Orgs can configure custom retention windows that override the platform defaults:
import {
getRetentionPolicy,
type RetentionPolicy,
} from '@/lib/memory/retention'
const policy = getRetentionPolicy(
'org_abc',
'confidential',
orgOverrides, // RetentionPolicy[] from org config
)
// Returns org-specific override if found, else the platform defaultBFSI orgs must set restricted retention to 7 years (2555 days) to satisfy SR 11-7 requirements. This override must be applied before any restricted observations are written - retroactive policy changes do not backdate existing rows.
// BFSI SR 11-7 retention override for restricted observations
const bfsiRetentionOverrides: RetentionPolicy[] = [
{
classification: 'restricted',
retentionDays: 2555, // 7 years
softDeleteGraceDays: 30,
},
{
classification: 'confidential',
retentionDays: 365, // 1 year (tighter than default 90)
softDeleteGraceDays: 14,
},
]
// Apply at org-config write time, not at query time
const policy = getRetentionPolicy('org_bfsi', 'restricted', bfsiRetentionOverrides)
// policy.retentionDays === 2555isExpired
import { isExpired } from '@/lib/memory/retention'
const expired = isExpired(
{ createdAt: observation.createdAt, classification: 'internal' },
policy,
)
// true when now >= createdAt + retentionDaysReturns false when policy.retentionDays is null (indefinite retention).
Audit trail
The implemented memory paths below call the platform audit writer. Entries produced by the current append path are hash-linked and carry per-workspace Ed25519 signatures; retained history can include legacy unsigned entries. The observation's raw content is not included in these audit payloads; the content hash is used instead.
Audit event types
| Event type | Triggered by |
|---|---|
memory.store | Observation write (auto-capture or tool) |
memory.retrieve | Query returning results |
memory.delete | Explicit deletion |
memory.remember | af_memory_remember MCP tool |
memory.recall | af_memory_recall MCP tool |
memory.forget | af_memory_forget MCP tool |
memory.purge | Retention purge job |
memory.feedback | EMA weight update |
When these events reach the current append path, they use the same predecessor-hash linkage and Ed25519 signing as other current-path audit entries. audit_events is currently an ordinary, non-partitioned PostgreSQL table. Merkle-head maintenance and configured SIEM delivery are separate, non-blocking follow-up paths. Signature presence is not verification; see Audit Trail for the explicit checks and their completeness, retention, and anchoring limits.
Querying memory audit events
import { queryMemoryAuditEvents } from '@/lib/memory/audit'
const events = await queryMemoryAuditEvents('ws_rensei', {
eventType: 'memory.feedback',
limit: 50,
offset: 0,
})memory.purge event shape
interface MemoryPurgeInput {
workspaceId: string
actorId: string // system service account for cron-triggered purges
orgId: string
projectId?: string
observationIds: string[]
retentionPolicyId?: string
classification: MemoryClassification
dryRun: boolean // true for preview runs that don't delete
}memory.feedback event shape
interface MemoryFeedbackPayload {
sessionId: string
outcome: 'accepted' | 'rejected' | 'rework'
observationIds: string[]
weightDeltasById: Record<string, { previous: number; new: number }>
orgId?: string
projectId?: string
}Retention purge job
The automated purge (cron-triggered findExpired → softDelete → grace-period hardDelete path) is currently safe-inert. The findExpired method returns [] until the soft_deleted_at and classification columns are added to the observations table. Manual hardDelete by ID works today.
The full purge pipeline is:
store.findExpired(policies) - identify observations past their retention window with softDeletedAt IS NULL.
For each expired observation: optional Cedar policy check (cedarCheck). Deny → skip and emit a denial audit event.
store.softDelete(id, reason) - marks softDeletedAt = now(). Observation becomes invisible to queries.
store.listSoftDeleted() - find soft-deleted observations past their grace period.
store.hardDelete(id) - permanently removes the row. Emits memory.purge audit event with dryRun: false.
Related pages
- Memory Classification - the tier that drives retention schedules
- Memory Diagnostics - the aggressive downweight path for misleading observations
- Feedback Retrieval - where
observations.weightis consumed during retrieval - Observation Store - the
IObservationStoreimplementation - Audit Trail - the hash-chain mechanics underlying
memory.*events - Memory Export - export format that uses
IObservationStore.list()