From ed5dff8ebd265952216bfd4c47c185edb2f2d308 Mon Sep 17 00:00:00 2001 From: JC Beasley Date: Sat, 4 Jul 2026 15:44:53 -0700 Subject: [PATCH] Implement consistency architecture (10-principle framework) Adds: - ARCHITECTURE.md: Full documentation of 3-layer system - Context Pipeline: Preprocessing layer (pipeline.js) - Workflow Router: 4 fixed workflows (coding, debug, deploy, audit) - Validation Layer: Post-response quality gate (validator.js) - Format Templates: Structured output templates - TOOLS.md: Beavault connection documentation Architecture: - Layer 1: Behavior rules (always injected) - Layer 2: Persistent facts (structured memory) - Layer 3: Ephemeral context - Priority enforcement: Rules > Prefs > Task > Chat - Memory write policy: Only confirmed fixes, repeated preferences --- ARCHITECTURE.md | 206 +++++++++++++++++++++++++++++++++++ architecture/pipeline.js | 195 +++++++++++++++++++++++++++++++++ architecture/validator.js | 222 ++++++++++++++++++++++++++++++++++++++ templates/code-change.md | 17 +++ templates/debug-report.md | 15 +++ workflows/audit.md | 43 ++++++++ workflows/coding.md | 45 ++++++++ workflows/debug.md | 40 +++++++ workflows/deploy.md | 46 ++++++++ 9 files changed, 829 insertions(+) create mode 100644 ARCHITECTURE.md create mode 100644 architecture/pipeline.js create mode 100644 architecture/validator.js create mode 100644 templates/code-change.md create mode 100644 templates/debug-report.md create mode 100644 workflows/audit.md create mode 100644 workflows/coding.md create mode 100644 workflows/debug.md create mode 100644 workflows/deploy.md diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md new file mode 100644 index 0000000..7021128 --- /dev/null +++ b/ARCHITECTURE.md @@ -0,0 +1,206 @@ +# OpenClaw Agent Architecture + +## Core Philosophy + +**LLM = Execution Engine (stateless)** +**Orchestrator = Brain (decides what LLM sees)** +**Database = Source of Truth** +**Router = Workflow Controller** +**Validator = Quality Gate** + +--- + +## Three-Layer Context System + +### Layer 1: Behavior Rules (Always Injected) + +These are **non-negotiable** and loaded every session. They live in: + +- `SOUL.md` - Core operating principles +- `AGENTS.md` - OpenClaw operating instructions +- `IDENTITY.md` - Role scope and responsibilities +- `workflow/*.md` - Workflow-specific rules + +**Priority: ABSOLUTE** — These override everything else. + +### Layer 2: Persistent Facts (Database) + +User preferences, confirmed decisions, validated fixes. Stored in: + +- **Structured Memory** (`memory/items/`) - JSON with metadata +- **Project Memory** (`Projects/*/memory/`) - Per-project facts +- **Vector DB** (future) - Semantic search for similar tasks + +**Priority: HIGH** — Loaded at session start, refreshed as needed. + +### Layer 3: Ephemeral Context + +Current task, conversation history, tool outputs. This is: + +- Session transcript (limited history) +- Current tool results +- Active workflow state + +**Priority: LOW** — Constantly changing, not relied upon for consistency. + +--- + +## Preprocessing Pipeline + +Every request goes through this pipeline **before** reaching the LLM: + +``` +Step 1: LOAD RULES +├── Inject SOUL.md +├── Inject AGENT.md +├── Inject IDENTITY.md +└── Inject active workflow rules + +Step 2: LOAD PREFERENCES +├── Query structured memory (high-importance items) +├── Load user preferences (format, communication style) +└── Load project-specific context + +Step 3: LOAD RELEVANT MEMORY +├── Vector search for similar past tasks +├── Load project STATUS.md +└── Load recent DECISIONS.md entries + +Step 4: BUILD CONTEXT PACKET +├── Priority order: Rules → Prefs → Memory → Task +└── Truncate to fit context window +``` + +--- + +## Priority Enforcement + +``` +1. SYSTEM RULES (absolute) + └── Safety, format, tool constraints + +2. USER PREFERENCES + └── Communication style, format preferences + +3. TASK INSTRUCTIONS + └── Current goal, acceptance criteria + +4. CHAT INPUT + └── User's current message +``` + +**If lower priority conflicts with higher priority → Higher wins.** + +--- + +## Workflow Router + +Before responding, classify intent and route to fixed workflow: + +| Intent | Workflow | Fixed Template | +|--------|----------|----------------| +| coding | `workflows/coding.md` | Code structure, tests, docs | +| troubleshooting | `workflows/debug.md` | Diagnose → Fix → Verify | +| planning | `workflows/planning.md` | Breakdown → Dependencies → Timeline | +| deployment | `workflows/deploy.md` | Check → Backup → Execute → Verify | +| audit | `workflows/audit.md` | Inventory → Assess → Report | + +Each workflow has: +- **Fixed prompt template** +- **Fixed output format** +- **Fixed tool access rules** + +--- + +## Validation Layer + +After LLM responds, run validation: + +```python +validator.check(response, against={ + "rules_followed": bool, + "format_compliance": bool, + "workflow_adherence": bool, + "safety_constraints": bool +}) + +if not validator.passed: + response = regenerate_with_feedback(validator.errors) +``` + +--- + +## Memory Write Policy + +**Only store:** +1. ✅ Repeated preferences (observed 2+ times) +2. ✅ Confirmed fixes (user verified it worked) +3. ✅ Validated decisions (documented in DECISIONS.md) +4. ✅ Explicit "remember this" commands + +**Never store:** +1. ❌ Random conversation text +2. ❌ Guesses or speculation +3. ❌ Partial ideas +4. ❌ Tool outputs (ephemeral) + +--- + +## Format Locking + +Enforce structure at system level: + +### Code Changes +```markdown +## Summary +[What changed] + +## Files Modified +- file1.py: [change description] +- file2.py: [change description] + +## Verification +- [ ] Tests pass +- [ ] Lint passes +- [ ] Manual verification complete +``` + +### Debug Reports +```markdown +## Problem +[Symptom] + +## Cause +[Root cause] + +## Fix +[What was changed] + +## Validation +[How verified] +``` + +--- + +## Implementation Status + +| Component | Status | Location | +|-----------|--------|----------| +| Behavior Rules (Layer 1) | ✅ Implemented | SOUL.md, AGENTS.md, IDENTITY.md | +| Persistent Facts (Layer 2) | 🔄 Partial | memory/items/, Projects/*/memory/ | +| Ephemeral Context (Layer 3) | ✅ Built-in | Session transcript | +| Preprocessing Pipeline | 🔄 In Progress | architecture/pipeline.js | +| Workflow Router | ❌ Not Started | workflows/ | +| Validation Layer | ❌ Not Started | architecture/validator.js | +| Memory Write Policy | 🔄 Documented | This file + MEMORY.md | +| Format Locking | ❌ Not Started | templates/ | + +--- + +## Next Steps + +1. Implement preprocessing pipeline +2. Create workflow router with 5 core workflows +3. Build validation layer +4. Add format templates +5. Migrate memory system to enforce write policy diff --git a/architecture/pipeline.js b/architecture/pipeline.js new file mode 100644 index 0000000..8af705d --- /dev/null +++ b/architecture/pipeline.js @@ -0,0 +1,195 @@ +#!/usr/bin/env node +/** + * Context Pipeline - Preprocessing Layer + * + * Every request flows through: + * 1. Load Rules → 2. Load Preferences → 3. Load Memory → 4. Build Packet + */ + +const fs = require('fs'); +const path = require('path'); + +class ContextPipeline { + constructor() { + this.rules = []; + this.preferences = []; + this.memory = []; + this.context = {}; + } + + /** + * Step 1: Load Behavior Rules (always injected) + */ + async loadRules() { + const ruleFiles = [ + 'SOUL.md', + 'AGENTS.md', + 'IDENTITY.md', + 'MEMORY.md' + ]; + + for (const file of ruleFiles) { + const content = await this.readFile(file); + if (content) { + this.rules.push({ + source: file, + content: content, + priority: 100 // Absolute priority + }); + } + } + + // Load active workflow rules if specified + const workflow = process.env.ACTIVE_WORKFLOW; + if (workflow) { + const workflowRules = await this.readFile(`workflows/${workflow}.md`); + if (workflowRules) { + this.rules.push({ + source: `workflows/${workflow}.md`, + content: workflowRules, + priority: 95 + }); + } + } + + return this; + } + + /** + * Step 2: Load Persistent Preferences (from structured memory) + */ + async loadPreferences() { + const prefFiles = [ + 'memory/items/mem_pref:communication_style.json', + 'memory/items/mem_pref:format.json', + 'memory/items/mem_pref:memory_persistence.json', + 'memory/items/mem_identity:name.json', + 'memory/items/mem_identity:role.json' + ]; + + for (const file of prefFiles) { + const data = await this.readJson(file); + if (data) { + this.preferences.push({ + source: file, + content: data.content, + priority: data.importance || 7, + tags: data.tags || [] + }); + } + } + + return this; + } + + /** + * Step 3: Load Relevant Memory + */ + async loadRelevantMemory(query, maxResults = 5) { + // For now, load recent episodic memories + // Future: implement vector search + + const memoryFiles = [ + 'memory/2026-07-04.md', + 'memory/2026-07-03.md' + ]; + + for (const file of memoryFiles) { + const content = await this.readFile(file); + if (content) { + this.memory.push({ + source: file, + content: content.substring(0, 2000), // Truncate + priority: 5 + }); + } + } + + // Load project memory if PROJECT context set + const project = process.env.CURRENT_PROJECT; + if (project) { + const projectStatus = await this.readFile(`Projects/${project}/memory/STATUS.md`); + if (projectStatus) { + this.memory.push({ + source: `Projects/${project}/memory/STATUS.md`, + content: projectStatus, + priority: 8 + }); + } + } + + return this; + } + + /** + * Step 4: Build Context Packet + */ + buildPacket(taskInput) { + // Sort by priority (highest first) + const allContext = [ + ...this.rules, + ...this.preferences.sort((a, b) => b.priority - a.priority), + ...this.memory + ]; + + const packet = { + system: allContext.filter(c => c.priority >= 90).map(c => c.content).join('\n\n---\n\n'), + preferences: allContext.filter(c => c.priority >= 5 && c.priority < 90).map(c => c.content).join('\n\n'), + memory: allContext.filter(c => c.priority < 5).map(c => c.content).join('\n\n'), + task: taskInput, + metadata: { + rules_count: this.rules.length, + prefs_count: this.preferences.length, + memory_count: this.memory.length + } + }; + + return packet; + } + + /** + * Utility: Read file from workspace + */ + async readFile(filepath) { + const fullPath = path.join('/home/jcbeasley/.openclaw/workspace', filepath); + try { + return fs.readFileSync(fullPath, 'utf8'); + } catch (err) { + return null; + } + } + + /** + * Utility: Read JSON file + */ + async readJson(filepath) { + const fullPath = path.join('/home/jcbeasley/.openclaw/workspace', filepath); + try { + const content = fs.readFileSync(fullPath, 'utf8'); + return JSON.parse(content); + } catch (err) { + return null; + } + } +} + +// Export for use +module.exports = ContextPipeline; + +// CLI usage +if (require.main === module) { + const pipeline = new ContextPipeline(); + + pipeline + .loadRules() + .then(() => pipeline.loadPreferences()) + .then(() => pipeline.loadRelevantMemory(process.argv[2] || '')) + .then(() => { + const packet = pipeline.buildPacket(process.argv[2] || 'No task specified'); + console.log(JSON.stringify(packet, null, 2)); + }) + .catch(err => { + console.error('Pipeline error:', err); + process.exit(1); + }); +} diff --git a/architecture/validator.js b/architecture/validator.js new file mode 100644 index 0000000..69095bb --- /dev/null +++ b/architecture/validator.js @@ -0,0 +1,222 @@ +#!/usr/bin/env node +/** + * Validation Layer - Post-Response Quality Gate + * + * After LLM responds, validate against: + * - Rules followed + * - Format compliance + * - Workflow adherence + * - Safety constraints + */ + +class ResponseValidator { + constructor(workflow) { + this.workflow = workflow; + this.errors = []; + this.warnings = []; + } + + /** + * Validate a response against all checks + */ + validate(response) { + this.errors = []; + this.warnings = []; + + this.checkRulesFollowed(response); + this.checkFormatCompliance(response); + this.checkWorkflowAdherence(response); + this.checkSafetyConstraints(response); + + return { + passed: this.errors.length === 0, + errors: this.errors, + warnings: this.warnings, + summary: this.buildSummary() + }; + } + + /** + * Check 1: Rules Followed + */ + checkRulesFollowed(response) { + // Check for prohibited phrases + const prohibited = [ + { pattern: /it should work/i, msg: "Assumed working without verification" }, + { pattern: /probably|maybe|likely/i, msg: "Speculative language detected" }, + { pattern: /i think/i, msg: "Unverified assumption" } + ]; + + for (const check of prohibited) { + if (check.pattern.test(response)) { + this.errors.push({ + category: 'rules', + message: check.msg, + suggestion: 'Verify before stating' + }); + } + } + + // Check for required phrases (workflow dependent) + const required = this.getRequiredPhrases(); + for (const phrase of required) { + if (!response.includes(phrase)) { + this.warnings.push({ + category: 'rules', + message: `Missing required element: "${phrase}"`, + suggestion: `Add "${phrase}" to response` + }); + } + } + } + + /** + * Check 2: Format Compliance + */ + checkFormatCompliance(response) { + const checks = [ + { pattern: /^##\s+/m, name: 'Section headers', required: true }, + { pattern: /\|.*\|.*\|/m, name: 'Tables (optional)', required: false }, + { pattern: /^- \[x\]|^- \[ \]/m, name: 'Checkbox lists (optional)', required: false } + ]; + + for (const check of checks) { + if (check.required && !check.pattern.test(response)) { + this.errors.push({ + category: 'format', + message: `Missing required format: ${check.name}`, + suggestion: `Add ${check.name.toLowerCase()}` + }); + } + } + } + + /** + * Check 3: Workflow Adherence + */ + checkWorkflowAdherence(response) { + const workflowChecks = { + 'coding': [ + { pattern: /## Verification/, msg: 'Missing verification section' }, + { pattern: /## Files Modified/, msg: 'Missing files modified section' } + ], + 'debug': [ + { pattern: /## Cause|## Root Cause/, msg: 'Missing cause/root cause' }, + { pattern: /## Fix|## Solution/, msg: 'Missing fix section' } + ], + 'deploy': [ + { pattern: /## Pre-Deployment|## Backup/, msg: 'Missing pre-deployment/backup' }, + { pattern: /## Verification/, msg: 'Missing verification section' } + ], + 'audit': [ + { pattern: /## Findings/, msg: 'Missing findings section' }, + { pattern: /## Recommendations/, msg: 'Missing recommendations' } + ] + }; + + const checks = workflowChecks[this.workflow] || []; + for (const check of checks) { + if (!check.pattern.test(response)) { + this.warnings.push({ + category: 'workflow', + message: check.msg, + suggestion: `Follow ${this.workflow} workflow format` + }); + } + } + } + + /** + * Check 4: Safety Constraints + */ + checkSafetyConstraints(response) { + const safetyChecks = [ + { + pattern: /rm -rf|dd if=|mkfs\.\w+|>\s*\/dev\/\w+/, + msg: 'Potentially destructive command', + severity: 'error' + }, + { + pattern: /ALTER\s+TABLE\s+.*DROP|DELETE\s+FROM\s+\w+\s+WHERE/i, + msg: 'Database destructive operation', + severity: 'error' + }, + { + pattern: /chmod\s+777|chown\s+-R/i, + msg: 'Overly permissive permissions', + severity: 'warning' + } + ]; + + for (const check of safetyChecks) { + if (check.pattern.test(response)) { + const entry = { + category: 'safety', + message: check.msg, + suggestion: 'Review for safety' + }; + + if (check.severity === 'error') { + this.errors.push(entry); + } else { + this.warnings.push(entry); + } + } + } + } + + /** + * Get required phrases for current workflow + */ + getRequiredPhrases() { + const phrases = { + 'coding': ['Verification'], + 'debug': ['Cause', 'Fix'], + 'deploy': ['Backup', 'Verification'], + 'audit': ['Findings'] + }; + return phrases[this.workflow] || []; + } + + /** + * Build validation summary + */ + buildSummary() { + const parts = []; + + if (this.errors.length === 0 && this.warnings.length === 0) { + parts.push('✅ All checks passed'); + } else { + if (this.errors.length > 0) { + parts.push(`❌ ${this.errors.length} error(s)`); + } + if (this.warnings.length > 0) { + parts.push(`⚠️ ${this.warnings.length} warning(s)`); + } + } + + return parts.join(', '); + } +} + +// Export for use +module.exports = ResponseValidator; + +// CLI usage +if (require.main === module) { + const workflow = process.argv[2] || 'coding'; + const responseFile = process.argv[3]; + + if (!responseFile) { + console.error('Usage: node validator.js '); + process.exit(1); + } + + const fs = require('fs'); + const response = fs.readFileSync(responseFile, 'utf8'); + + const validator = new ResponseValidator(workflow); + const result = validator.validate(response); + + console.log(JSON.stringify(result, null, 2)); +} diff --git a/templates/code-change.md b/templates/code-change.md new file mode 100644 index 0000000..d9132cd --- /dev/null +++ b/templates/code-change.md @@ -0,0 +1,17 @@ +## Summary +[What changed] + +## Files Modified +- `file1.py`: [description] +- `file2.js`: [description] + +## Verification +- [ ] Tests pass +- [ ] Lint passes +- [ ] Manual verification complete + +## Decisions Made +- [decision] + +## Still Open +- [if any] diff --git a/templates/debug-report.md b/templates/debug-report.md new file mode 100644 index 0000000..403ac1e --- /dev/null +++ b/templates/debug-report.md @@ -0,0 +1,15 @@ +## Problem +[symptom] + +## Diagnosis +- Checked: [what] +- Found: [result] + +## Cause +[root cause] + +## Fix +[what was done] + +## Validation +[how verified] diff --git a/workflows/audit.md b/workflows/audit.md new file mode 100644 index 0000000..76b428c --- /dev/null +++ b/workflows/audit.md @@ -0,0 +1,43 @@ +# Workflow: Audit + +## Intent Classification + +**Triggers:** Security reviews, inventory checks, compliance, "what's the state of..." + +## Fixed Output Format + +```markdownn## Scope +[What was audited] + +## Findings +| Item | Status | Notes | +|------|--------|-------| +| [item1] | ✅/⚠️/❌ | [notes] | +| [item2] | ✅/⚠️/❌ | [notes] | + +## Risks Identified +- [Risk 1] +- [Risk 2] + +## Recommendations +- [Rec 1] +- [Rec 2] + +## Verification +[How findings were confirmed] +``` + +## Tool Access Rules + +1. **Enumerate** - list everything in scope +2. **Inspect** - check state of each item +3. **Assess** - evaluate against standards +4. **Report** - structured findings with evidence + +## Constraints + +- NO: Assumptions without verification +- NO: Partial coverage reported as complete +- NO: Risks without severity assessment +- YES: Evidence cited for each finding +- YES: Actionable recommendations diff --git a/workflows/coding.md b/workflows/coding.md new file mode 100644 index 0000000..f4e7961 --- /dev/null +++ b/workflows/coding.md @@ -0,0 +1,45 @@ +# Workflow: Coding + +## Intent Classification + +**Triggers:** Code changes, development tasks, bug fixes, feature implementation + +## Fixed Output Format + +```markdown +## Summary +[What changed - 1-2 sentences] + +## Files Modified +| File | Change | +|------|--------| +| file1.py | [description] | +| file2.js | [description] | + +## Verification +- [ ] Tests pass: `command` +- [ ] Lint passes: `command` +- [ ] Manual verification: [how] + +## Decisions Made +- [Decision 1] +- [Decision 2] + +## Still Open +- [ ] [if any] +``` + +## Tool Access Rules + +1. **Delegate to coding agents** when possible +2. **Read before writing** - always check existing code first +3. **Verify before reporting** - tests, lint, manual check +4. **Update memory** - log decisions to DECISIONS.md + +## Constraints + +- NO: Direct production edits without verification +- NO: Untested code reported as "done" +- NO: Breaking changes without migration plan +- YES: Clear acceptance criteria before starting +- YES: Root cause analysis before fixes diff --git a/workflows/debug.md b/workflows/debug.md new file mode 100644 index 0000000..8eccb60 --- /dev/null +++ b/workflows/debug.md @@ -0,0 +1,40 @@ +# Workflow: Troubleshooting + +## Intent Classification + +**Triggers:** Errors, failures, unexpected behavior, debugging requests + +## Fixed Output Format + +```markdown +## Problem +[Symptom observed] + +## Diagnosis +- Checked: [what was checked] +- Found: [what was found] + +## Cause +[Root cause - trace to why, not just what] + +## Fix +[What was changed] + +## Validation +[How verified it's fixed] +``` + +## Tool Access Rules + +1. **Inspect first** - logs, config, state before changing anything +2. **Reproduce** - understand the failure before fixing +3. **Trace to root** - fix cause, not symptom +4. **Document** - add to ISSUES.md if pattern + +## Constraints + +- NO: Restart-as-first-response +- NO: Regex patches without understanding root cause +- NO: Fixes without verification +- YES: Documented workaround if root cause blocked +- YES: Regression test if applicable diff --git a/workflows/deploy.md b/workflows/deploy.md new file mode 100644 index 0000000..a9179bd --- /dev/null +++ b/workflows/deploy.md @@ -0,0 +1,46 @@ +# Workflow: Deployment + +## Intent Classification + +**Triggers:** Deploy to staging/production, release, infrastructure changes + +## Fixed Output Format + +```markdown +## Deployment Summary +[What and where] + +## Pre-Deployment +- [x] Backup created +- [x] Tests pass +- [x] Rollback plan documented + +## Changes Applied +| Step | Command | Status | +|------|---------|--------| +| 1 | [command] | ✅ | +| 2 | [command] | ✅ | + +## Verification +- [x] Service responding: [url] +- [x] Logs normal: [check] +- [x] Smoke test: [result] + +## Rollback (if needed) +[Exact rollback steps] +``` + +## Tool Access Rules + +1. **Check first** - current state before changing +2. **Backup** - always before production changes +3. **Verify** - process running, endpoint responding, output correct +4. **Document** - update RUNBOOK.md if procedure changed + +## Constraints + +- NO: Deployment without verification plan +- NO: "It should work" = done +- NO: Blurry rollback steps +- YES: Status checked and reported +- YES: Rollback tested if possible