Files
JC Beasley 0bd719ab91 Add cross-project pattern registry for retrieval-augmented generalization
- Create patterns/ directory with README, manifest, and 10 initial patterns
  covering Ollama JSON fallback, API escaping, deprecation, PTY auth,
  queue-poll, LLM-as-parser, credential rotation, reverse proxy binding,
  human approval gates, and transient retry.
- Wire pattern loading into architecture/pipeline.js based on task tags.
- Update architecture/orchestrator.js to load patterns and surface them in
  the system prompt.
- Update MEMORY.md, ARCHITECTURE.md, and CONTEXT.md to document the registry
  and record the decision.
2026-08-06 12:46:09 -07:00

7.5 KiB

OpenClaw Agent Architecture

Core Philosophy

LLM = Execution Engine (stateless)
Orchestrator = Brain (decides what LLM sees)
Database = Source of Truth
Router = Workflow Controller
Validator = Quality Gate


Three-Layer Context System

Layer 1: Behavior Rules (Always Injected)

These are non-negotiable and loaded every session. They live in:

  • SOUL.md - Core operating principles
  • AGENTS.md - OpenClaw operating instructions
  • IDENTITY.md - Role scope and responsibilities
  • workflow/*.md - Workflow-specific rules

Priority: ABSOLUTE — These override everything else.

Layer 2: Persistent Facts (Database)

User preferences, confirmed decisions, validated fixes. Stored in:

  • Structured Memory (memory/items/) - JSON with metadata
  • Project Memory (Projects/*/memory/) - Per-project facts
  • Cross-Project Patterns (patterns/) - Reusable technical patterns loaded by the preprocessing pipeline

Priority: HIGH — Loaded at session start, refreshed as needed.

Layer 3: Ephemeral Context

Current task, conversation history, tool outputs. This is:

  • Session transcript (limited history)
  • Current tool results
  • Active workflow state

Priority: LOW — Constantly changing, not relied upon for consistency.


Preprocessing Pipeline

Every request goes through this pipeline before reaching the LLM:

Step 1: LOAD RULES
├── Inject SOUL.md
├── Inject AGENT.md
├── Inject IDENTITY.md
└── Inject active workflow rules

Step 2: LOAD PREFERENCES
├── Query structured memory (high-importance items)
├── Load user preferences (format, communication style)
└── Load project-specific context

Step 3: LOAD RELEVANT MEMORY
├── Vector search for similar past tasks
├── Load project STATUS.md
├── Load recent DECISIONS.md entries
└── Load matching cross-project patterns from `patterns/` registry

Step 4: BUILD CONTEXT PACKET
├── Priority order: Rules → Prefs → Memory → Patterns → Task
└── Truncate to fit context window

Priority Enforcement

1. SYSTEM RULES (absolute)
   └── Safety, format, tool constraints

2. USER PREFERENCES
   └── Communication style, format preferences

3. TASK INSTRUCTIONS
   └── Current goal, acceptance criteria

4. CHAT INPUT
   └── User's current message

If lower priority conflicts with higher priority → Higher wins.


Workflow Router

Before responding, classify intent and route to fixed workflow:

Intent Workflow Fixed Template
coding workflows/coding.md Code structure, tests, docs
troubleshooting workflows/debug.md Diagnose → Fix → Verify
planning workflows/planning.md Breakdown → Dependencies → Timeline
deployment workflows/deploy.md Check → Backup → Execute → Verify
audit workflows/audit.md Inventory → Assess → Report

Each workflow has:

  • Fixed prompt template
  • Fixed output format
  • Fixed tool access rules

Validation Layer

After LLM responds, run validation:

validator.check(response, against={
    "rules_followed": bool,
    "format_compliance": bool,
    "workflow_adherence": bool,
    "safety_constraints": bool
})

if not validator.passed:
    response = regenerate_with_feedback(validator.errors)

Memory Write Policy

Only store:

  1. Repeated preferences (observed 2+ times)
  2. Confirmed fixes (user verified it worked)
  3. Validated decisions (documented in DECISIONS.md)
  4. Explicit "remember this" commands

Never store:

  1. Random conversation text
  2. Guesses or speculation
  3. Partial ideas
  4. Tool outputs (ephemeral)

Format Locking

Enforce structure at system level:

Code Changes

## Summary
[What changed]

## Files Modified
- file1.py: [change description]
- file2.py: [change description]

## Verification
- [ ] Tests pass
- [ ] Lint passes
- [ ] Manual verification complete

Debug Reports

## Problem
[Symptom]

## Cause
[Root cause]

## Fix
[What was changed]

## Validation
[How verified]

Cross-Project Pattern Registry

The patterns/ directory stores reusable technical lessons (workarounds, failure modes, design choices) that recur across projects. The preprocessing pipeline loads matching patterns into the context packet based on the task input.

How it works

  1. architecture/pipeline.js reads patterns/patterns.json.
  2. It matches task input against pattern tags and affected_projects.
  3. Matching pattern files are injected as context with priority 6 (above generic memory, below project status).
  4. Agents receive the pattern content and can apply the documented countermeasure.

Adding patterns

  1. Create patterns/<pattern-id>.md using the standard template.
  2. Add an entry to patterns/patterns.json.
  3. Test retrieval: node architecture/pipeline.js "your task description".

This enables generalization: an agent encountering a new task can recognize a known shape and reuse a proven fix.


Implementation Status

Component Status Location
Behavior Rules (Layer 1) Implemented SOUL.md, AGENTS.md, IDENTITY.md
Persistent Facts (Layer 2) Implemented memory/items/, Projects/*/memory/
Ephemeral Context (Layer 3) Built-in Session transcript
Preprocessing Pipeline Implemented architecture/pipeline.js
Workflow Router Implemented architecture/workflow-router.js
Validation Layer Implemented architecture/validator.js
Format Locking Implemented architecture/format-locker.js
Orchestrator Implemented architecture/orchestrator.js
Memory Write Policy Documented This file + MEMORY.md

Component Details

Workflow Router (architecture/workflow-router.js)

  • Intent Classification: Pattern-based matching with confidence scoring
  • 5 Workflows: coding, debug, deploy, audit, planning
  • System Prompt Building: Context-aware prompt construction
  • Usage: node workflow-router.js "your request here"

Validation Layer (architecture/validator.js)

  • Rule Checking: Prohibited phrases ("it should work", "probably")
  • Format Compliance: Section headers, tables, checkboxes
  • Workflow Adherence: Required sections per workflow type
  • Safety Constraints: Destructive commands, DB operations, permissions
  • Usage: node validator.js <workflow> <response-file>

Format Locker (architecture/format-locker.js)

  • Template Enforcement: Required sections per workflow
  • Auto-Fix: Adds missing sections automatically
  • Validation: Checks section content quality
  • Templates: Markdown with placeholders for each workflow
  • Usage: node format-locker.js <workflow> [response-file]

Orchestrator (architecture/orchestrator.js)

  • Integration: Pipeline → Router → Validator → Format Locker
  • Context Building: Rules + Preferences + Memory + Task
  • Full Pipeline: Single entry point for all requests
  • Usage: node orchestrator.js "your request" --verbose

Next Steps

  1. Integration Testing: Test full pipeline with real requests
  2. Workflow Templates: Refine format templates based on usage
  3. Memory Enforcement: Add write policy validation to memory system
  4. Performance: Optimize pipeline execution time
  5. Documentation: Update agent instruction files to use orchestrator