Document rustfs, Qdrant, NocoDB infrastructure access

Adds infrastructure endpoints from vault:
- rustfs (S3-compatible) for object storage
- Qdrant for vector database
- NocoDB for structured data
- Metabase for analytics
- Odoo for business data

ARCHITECTURE.md

@@ -0,0 +1,206 @@
+# OpenClaw Agent Architecture
+
+## Core Philosophy
+
+**LLM = Execution Engine (stateless)**  
+**Orchestrator = Brain (decides what LLM sees)**  
+**Database = Source of Truth**  
+**Router = Workflow Controller**  
+**Validator = Quality Gate**
+
+---
+
+## Three-Layer Context System
+
+### Layer 1: Behavior Rules (Always Injected)
+
+These are **non-negotiable** and loaded every session. They live in:
+
+- `SOUL.md` - Core operating principles
+- `AGENTS.md` - OpenClaw operating instructions
+- `IDENTITY.md` - Role scope and responsibilities
+- `workflow/*.md` - Workflow-specific rules
+
+**Priority: ABSOLUTE** — These override everything else.
+
+### Layer 2: Persistent Facts (Database)
+
+User preferences, confirmed decisions, validated fixes. Stored in:
+
+- **Structured Memory** (`memory/items/`) - JSON with metadata
+- **Project Memory** (`Projects/*/memory/`) - Per-project facts
+- **Vector DB** (future) - Semantic search for similar tasks
+
+**Priority: HIGH** — Loaded at session start, refreshed as needed.
+
+### Layer 3: Ephemeral Context
+
+Current task, conversation history, tool outputs. This is:
+
+- Session transcript (limited history)
+- Current tool results
+- Active workflow state
+
+**Priority: LOW** — Constantly changing, not relied upon for consistency.
+
+---
+
+## Preprocessing Pipeline
+
+Every request goes through this pipeline **before** reaching the LLM:
+
+```
+Step 1: LOAD RULES
+├── Inject SOUL.md
+├── Inject AGENT.md
+├── Inject IDENTITY.md
+└── Inject active workflow rules
+
+Step 2: LOAD PREFERENCES
+├── Query structured memory (high-importance items)
+├── Load user preferences (format, communication style)
+└── Load project-specific context
+
+Step 3: LOAD RELEVANT MEMORY
+├── Vector search for similar past tasks
+├── Load project STATUS.md
+└── Load recent DECISIONS.md entries
+
+Step 4: BUILD CONTEXT PACKET
+├── Priority order: Rules → Prefs → Memory → Task
+└── Truncate to fit context window
+```
+
+---
+
+## Priority Enforcement
+
+```
+1. SYSTEM RULES (absolute)
+   └── Safety, format, tool constraints
+
+2. USER PREFERENCES
+   └── Communication style, format preferences
+
+3. TASK INSTRUCTIONS
+   └── Current goal, acceptance criteria
+
+4. CHAT INPUT
+   └── User's current message
+```
+
+**If lower priority conflicts with higher priority → Higher wins.**
+
+---
+
+## Workflow Router
+
+Before responding, classify intent and route to fixed workflow:
+
+| Intent | Workflow | Fixed Template |
+|--------|----------|----------------|
+| coding | `workflows/coding.md` | Code structure, tests, docs |
+| troubleshooting | `workflows/debug.md` | Diagnose → Fix → Verify |
+| planning | `workflows/planning.md` | Breakdown → Dependencies → Timeline |
+| deployment | `workflows/deploy.md` | Check → Backup → Execute → Verify |
+| audit | `workflows/audit.md` | Inventory → Assess → Report |
+
+Each workflow has:
+- **Fixed prompt template**
+- **Fixed output format**
+- **Fixed tool access rules**
+
+---
+
+## Validation Layer
+
+After LLM responds, run validation:
+
+```python
+validator.check(response, against={
+    "rules_followed": bool,
+    "format_compliance": bool,
+    "workflow_adherence": bool,
+    "safety_constraints": bool
+})
+
+if not validator.passed:
+    response = regenerate_with_feedback(validator.errors)
+```
+
+---
+
+## Memory Write Policy
+
+**Only store:**
+1. ✅ Repeated preferences (observed 2+ times)
+2. ✅ Confirmed fixes (user verified it worked)
+3. ✅ Validated decisions (documented in DECISIONS.md)
+4. ✅ Explicit "remember this" commands
+
+**Never store:**
+1. ❌ Random conversation text
+2. ❌ Guesses or speculation
+3. ❌ Partial ideas
+4. ❌ Tool outputs (ephemeral)
+
+---
+
+## Format Locking
+
+Enforce structure at system level:
+
+### Code Changes
+```markdown
+## Summary
+[What changed]
+
+## Files Modified
+- file1.py: [change description]
+- file2.py: [change description]
+
+## Verification
+- [ ] Tests pass
+- [ ] Lint passes
+- [ ] Manual verification complete
+```
+
+### Debug Reports
+```markdown
+## Problem
+[Symptom]
+
+## Cause
+[Root cause]
+
+## Fix
+[What was changed]
+
+## Validation
+[How verified]
+```
+
+---
+
+## Implementation Status
+
+| Component | Status | Location |
+|-----------|--------|----------|
+| Behavior Rules (Layer 1) | ✅ Implemented | SOUL.md, AGENTS.md, IDENTITY.md |
+| Persistent Facts (Layer 2) | 🔄 Partial | memory/items/, Projects/*/memory/ |
+| Ephemeral Context (Layer 3) | ✅ Built-in | Session transcript |
+| Preprocessing Pipeline | 🔄 In Progress | architecture/pipeline.js |
+| Workflow Router | ❌ Not Started | workflows/ |
+| Validation Layer | ❌ Not Started | architecture/validator.js |
+| Memory Write Policy | 🔄 Documented | This file + MEMORY.md |
+| Format Locking | ❌ Not Started | templates/ |
+
+---
+
+## Next Steps
+
+1. Implement preprocessing pipeline
+2. Create workflow router with 5 core workflows
+3. Build validation layer
+4. Add format templates
+5. Migrate memory system to enforce write policy

TOOLS.md

@@ -89,6 +89,35 @@ Secrets are organized under `kv/data/api/`:
 | `media` | Pexels, YouTube, Freepik, JSON2Video |
 | `credentials` | DB configs, collection names, paths |
 
+## Available Infrastructure
+
+### rustfs (S3-Compatible Object Storage)
+- **Endpoint:** `http://192.168.19.17:9002`
+- **Bucket:** `duma-memories`
+- **Access Key:** `rustfsadmin`
+- **Secret Key:** `[in vault: rustfs-secret-key]`
+- **Usage:** Agent memory storage, file backups
+
+### Qdrant (Vector Database)
+- **URL:** `http://192.168.19.17:6333`
+- **API Key:** `[in vault: qdrant-api-key]`
+- **Usage:** Semantic memory, experience retrieval
+
+### NocoDB (Database GUI)
+- **URL:** `http://192.168.25.5:8080`
+- **Token:** `[in vault: nocodb-token]`
+- **Table ID:** `ml5pemji4c85n6e`
+- **Usage:** Structured data, agent runs logging
+
+### Metabase
+- **URL:** `http://metabase.beawit.net:3000`
+- **Username:** `jc.beasley@beawit.net`
+- **Password:** `[in vault: metabase-password]`
+
+### Odoo
+- **URL:** `http://192.168.16.4:8069`
+- **API Key:** `[in vault: odoo-api-key]`
+
 ### Service-Specific Access
 
 For read-only access to specific paths, use the appropriate service role:

architecture/pipeline.js

@@ -0,0 +1,195 @@
+#!/usr/bin/env node
+/**
+ * Context Pipeline - Preprocessing Layer
+ * 
+ * Every request flows through:
+ * 1. Load Rules → 2. Load Preferences → 3. Load Memory → 4. Build Packet
+ */
+
+const fs = require('fs');
+const path = require('path');
+
+class ContextPipeline {
+  constructor() {
+    this.rules = [];
+    this.preferences = [];
+    this.memory = [];
+    this.context = {};
+  }
+
+  /**
+   * Step 1: Load Behavior Rules (always injected)
+   */
+  async loadRules() {
+    const ruleFiles = [
+      'SOUL.md',
+      'AGENTS.md', 
+      'IDENTITY.md',
+      'MEMORY.md'
+    ];
+
+    for (const file of ruleFiles) {
+      const content = await this.readFile(file);
+      if (content) {
+        this.rules.push({
+          source: file,
+          content: content,
+          priority: 100  // Absolute priority
+        });
+      }
+    }
+
+    // Load active workflow rules if specified
+    const workflow = process.env.ACTIVE_WORKFLOW;
+    if (workflow) {
+      const workflowRules = await this.readFile(`workflows/${workflow}.md`);
+      if (workflowRules) {
+        this.rules.push({
+          source: `workflows/${workflow}.md`,
+          content: workflowRules,
+          priority: 95
+        });
+      }
+    }
+
+    return this;
+  }
+
+  /**
+   * Step 2: Load Persistent Preferences (from structured memory)
+   */
+  async loadPreferences() {
+    const prefFiles = [
+      'memory/items/mem_pref:communication_style.json',
+      'memory/items/mem_pref:format.json',
+      'memory/items/mem_pref:memory_persistence.json',
+      'memory/items/mem_identity:name.json',
+      'memory/items/mem_identity:role.json'
+    ];
+
+    for (const file of prefFiles) {
+      const data = await this.readJson(file);
+      if (data) {
+        this.preferences.push({
+          source: file,
+          content: data.content,
+          priority: data.importance || 7,
+          tags: data.tags || []
+        });
+      }
+    }
+
+    return this;
+  }
+  
+  /**
+   * Step 3: Load Relevant Memory
+   */
+  async loadRelevantMemory(query, maxResults = 5) {
+    // For now, load recent episodic memories
+    // Future: implement vector search
+    
+    const memoryFiles = [
+      'memory/2026-07-04.md',
+      'memory/2026-07-03.md'
+    ];
+
+    for (const file of memoryFiles) {
+      const content = await this.readFile(file);
+      if (content) {
+        this.memory.push({
+          source: file,
+          content: content.substring(0, 2000),  // Truncate
+          priority: 5
+        });
+      }
+    }
+
+    // Load project memory if PROJECT context set
+    const project = process.env.CURRENT_PROJECT;
+    if (project) {
+      const projectStatus = await this.readFile(`Projects/${project}/memory/STATUS.md`);
+      if (projectStatus) {
+        this.memory.push({
+          source: `Projects/${project}/memory/STATUS.md`,
+          content: projectStatus,
+          priority: 8
+        });
+      }
+    }
+
+    return this;
+  }
+
+  /**
+   * Step 4: Build Context Packet
+   */
+  buildPacket(taskInput) {
+    // Sort by priority (highest first)
+    const allContext = [
+      ...this.rules,
+      ...this.preferences.sort((a, b) => b.priority - a.priority),
+      ...this.memory
+    ];
+
+    const packet = {
+      system: allContext.filter(c => c.priority >= 90).map(c => c.content).join('\n\n---\n\n'),
+      preferences: allContext.filter(c => c.priority >= 5 && c.priority < 90).map(c => c.content).join('\n\n'),
+      memory: allContext.filter(c => c.priority < 5).map(c => c.content).join('\n\n'),
+      task: taskInput,
+      metadata: {
+        rules_count: this.rules.length,
+        prefs_count: this.preferences.length,
+        memory_count: this.memory.length
+      }
+    };
+
+    return packet;
+  }
+
+  /**
+   * Utility: Read file from workspace
+   */
+  async readFile(filepath) {
+    const fullPath = path.join('/home/jcbeasley/.openclaw/workspace', filepath);
+    try {
+      return fs.readFileSync(fullPath, 'utf8');
+    } catch (err) {
+      return null;
+    }
+  }
+
+  /**
+   * Utility: Read JSON file
+   */
+  async readJson(filepath) {
+    const fullPath = path.join('/home/jcbeasley/.openclaw/workspace', filepath);
+    try {
+      const content = fs.readFileSync(fullPath, 'utf8');
+      return JSON.parse(content);
+    } catch (err) {
+      return null;
+    }
+  }
+}
+
+// Export for use
+module.exports = ContextPipeline;
+
+// CLI usage
+if (require.main === module) {
+  const pipeline = new ContextPipeline();
+  
+  pipeline
+    .loadRules()
+    .then(() => pipeline.loadPreferences())
+    .then(() => pipeline.loadRelevantMemory(process.argv[2] || ''))
+    .then(() => {
+      const packet = pipeline.buildPacket(process.argv[2] || 'No task specified');
+      console.log(JSON.stringify(packet, null, 2));
+    })
+    .catch(err => {
+      console.error('Pipeline error:', err);
+      process.exit(1);
+    });
+}

architecture/validator.js

@@ -0,0 +1,222 @@
+#!/usr/bin/env node
+/**
+ * Validation Layer - Post-Response Quality Gate
+ * 
+ * After LLM responds, validate against:
+ * - Rules followed
+ * - Format compliance  
+ * - Workflow adherence
+ * - Safety constraints
+ */
+
+class ResponseValidator {
+  constructor(workflow) {
+    this.workflow = workflow;
+    this.errors = [];
+    this.warnings = [];
+  }
+
+  /**
+   * Validate a response against all checks
+   */
+  validate(response) {
+    this.errors = [];
+    this.warnings = [];
+
+    this.checkRulesFollowed(response);
+    this.checkFormatCompliance(response);
+    this.checkWorkflowAdherence(response);
+    this.checkSafetyConstraints(response);
+
+    return {
+      passed: this.errors.length === 0,
+      errors: this.errors,
+      warnings: this.warnings,
+      summary: this.buildSummary()
+    };
+  }
+
+  /**
+   * Check 1: Rules Followed
+   */
+  checkRulesFollowed(response) {
+    // Check for prohibited phrases
+    const prohibited = [
+      { pattern: /it should work/i, msg: "Assumed working without verification" },
+      { pattern: /probably|maybe|likely/i, msg: "Speculative language detected" },
+      { pattern: /i think/i, msg: "Unverified assumption" }
+    ];
+
+    for (const check of prohibited) {
+      if (check.pattern.test(response)) {
+        this.errors.push({
+          category: 'rules',
+          message: check.msg,
+          suggestion: 'Verify before stating'
+        });
+      }
+    }
+
+    // Check for required phrases (workflow dependent)
+    const required = this.getRequiredPhrases();
+    for (const phrase of required) {
+      if (!response.includes(phrase)) {
+        this.warnings.push({
+          category: 'rules',
+          message: `Missing required element: "${phrase}"`,
+          suggestion: `Add "${phrase}" to response`
+        });
+      }
+    }
+  }
+
+  /**
+   * Check 2: Format Compliance
+   */
+  checkFormatCompliance(response) {
+    const checks = [
+      { pattern: /^##\s+/m, name: 'Section headers', required: true },
+      { pattern: /\|.*\|.*\|/m, name: 'Tables (optional)', required: false },
+      { pattern: /^- \[x\]|^- \[ \]/m, name: 'Checkbox lists (optional)', required: false }
+    ];
+
+    for (const check of checks) {
+      if (check.required && !check.pattern.test(response)) {
+        this.errors.push({
+          category: 'format',
+          message: `Missing required format: ${check.name}`,
+          suggestion: `Add ${check.name.toLowerCase()}`
+        });
+      }
+    }
+  }
+
+  /**
+   * Check 3: Workflow Adherence
+   */
+  checkWorkflowAdherence(response) {
+    const workflowChecks = {
+      'coding': [
+        { pattern: /## Verification/, msg: 'Missing verification section' },
+        { pattern: /## Files Modified/, msg: 'Missing files modified section' }
+      ],
+      'debug': [
+        { pattern: /## Cause|## Root Cause/, msg: 'Missing cause/root cause' },
+        { pattern: /## Fix|## Solution/, msg: 'Missing fix section' }
+      ],
+      'deploy': [
+        { pattern: /## Pre-Deployment|## Backup/, msg: 'Missing pre-deployment/backup' },
+        { pattern: /## Verification/, msg: 'Missing verification section' }
+      ],
+      'audit': [
+        { pattern: /## Findings/, msg: 'Missing findings section' },
+        { pattern: /## Recommendations/, msg: 'Missing recommendations' }
+      ]
+    };
+
+    const checks = workflowChecks[this.workflow] || [];
+    for (const check of checks) {
+      if (!check.pattern.test(response)) {
+        this.warnings.push({
+          category: 'workflow',
+          message: check.msg,
+          suggestion: `Follow ${this.workflow} workflow format`
+        });
+      }
+    }
+  }
+
+  /**
+   * Check 4: Safety Constraints
+   */
+  checkSafetyConstraints(response) {
+    const safetyChecks = [
+      { 
+        pattern: /rm -rf|dd if=|mkfs\.\w+|>\s*\/dev\/\w+/, 
+        msg: 'Potentially destructive command',
+        severity: 'error'
+      },
+      {
+        pattern: /ALTER\s+TABLE\s+.*DROP|DELETE\s+FROM\s+\w+\s+WHERE/i,
+        msg: 'Database destructive operation',
+        severity: 'error'
+      },
+      {
+        pattern: /chmod\s+777|chown\s+-R/i,
+        msg: 'Overly permissive permissions',
+        severity: 'warning'
+      }
+    ];
+
+    for (const check of safetyChecks) {
+      if (check.pattern.test(response)) {
+        const entry = {
+          category: 'safety',
+          message: check.msg,
+          suggestion: 'Review for safety'
+        };
+        
+        if (check.severity === 'error') {
+          this.errors.push(entry);
+        } else {
+          this.warnings.push(entry);
+        }
+      }
+    }
+  }
+
+  /**
+   * Get required phrases for current workflow
+   */
+  getRequiredPhrases() {
+    const phrases = {
+      'coding': ['Verification'],
+      'debug': ['Cause', 'Fix'],
+      'deploy': ['Backup', 'Verification'],
+      'audit': ['Findings']
+    };
+    return phrases[this.workflow] || [];
+  }
+
+  /**
+   * Build validation summary
+   */
+  buildSummary() {
+    const parts = [];
+    
+    if (this.errors.length === 0 && this.warnings.length === 0) {
+      parts.push('✅ All checks passed');
+    } else {
+      if (this.errors.length > 0) {
+        parts.push(`❌ ${this.errors.length} error(s)`);
+      }
+      if (this.warnings.length > 0) {
+        parts.push(`⚠️ ${this.warnings.length} warning(s)`);
+      }
+    }
+    
+    return parts.join(', ');
+  }
+}
+
+// Export for use
+module.exports = ResponseValidator;
+
+// CLI usage
+if (require.main === module) {
+  const workflow = process.argv[2] || 'coding';
+  const responseFile = process.argv[3];
+  
+  if (!responseFile) {
+    console.error('Usage: node validator.js <workflow> <response-file>');
+    process.exit(1);
+  }
+
+  const fs = require('fs');
+  const response = fs.readFileSync(responseFile, 'utf8');
+  
+  const validator = new ResponseValidator(workflow);
+  const result = validator.validate(response);
+  
+  console.log(JSON.stringify(result, null, 2));
+}

templates/code-change.md

@@ -0,0 +1,17 @@
+## Summary
+[What changed]
+
+## Files Modified
+- `file1.py`: [description]
+- `file2.js`: [description]
+
+## Verification
+- [ ] Tests pass
+- [ ] Lint passes  
+- [ ] Manual verification complete
+
+## Decisions Made
+- [decision]
+
+## Still Open
+- [if any]

templates/debug-report.md

@@ -0,0 +1,15 @@
+## Problem
+[symptom]
+
+## Diagnosis
+- Checked: [what]
+- Found: [result]
+
+## Cause
+[root cause]
+
+## Fix
+[what was done]
+
+## Validation
+[how verified]

workflows/audit.md

@@ -0,0 +1,43 @@
+# Workflow: Audit
+
+## Intent Classification
+
+**Triggers:** Security reviews, inventory checks, compliance, "what's the state of..."
+
+## Fixed Output Format
+
+```markdownn## Scope
+[What was audited]
+
+## Findings
+| Item | Status | Notes |
+|------|--------|-------|
+| [item1] | ✅/⚠️/❌ | [notes] |
+| [item2] | ✅/⚠️/❌ | [notes] |
+
+## Risks Identified
+- [Risk 1]
+- [Risk 2]
+
+## Recommendations
+- [Rec 1]
+- [Rec 2]
+
+## Verification
+[How findings were confirmed]
+```
+
+## Tool Access Rules
+
+1. **Enumerate** - list everything in scope
+2. **Inspect** - check state of each item
+3. **Assess** - evaluate against standards
+4. **Report** - structured findings with evidence
+
+## Constraints
+
+- NO: Assumptions without verification
+- NO: Partial coverage reported as complete
+- NO: Risks without severity assessment
+- YES: Evidence cited for each finding
+- YES: Actionable recommendations

workflows/coding.md

@@ -0,0 +1,45 @@
+# Workflow: Coding
+
+## Intent Classification
+
+**Triggers:** Code changes, development tasks, bug fixes, feature implementation
+
+## Fixed Output Format
+
+```markdown
+## Summary
+[What changed - 1-2 sentences]
+
+## Files Modified
+| File | Change |
+|------|--------|
+| file1.py | [description] |
+| file2.js | [description] |
+
+## Verification
+- [ ] Tests pass: `command`
+- [ ] Lint passes: `command`
+- [ ] Manual verification: [how]
+
+## Decisions Made
+- [Decision 1]
+- [Decision 2]
+
+## Still Open
+- [ ] [if any]
+```
+
+## Tool Access Rules
+
+1. **Delegate to coding agents** when possible
+2. **Read before writing** - always check existing code first
+3. **Verify before reporting** - tests, lint, manual check
+4. **Update memory** - log decisions to DECISIONS.md
+
+## Constraints
+
+- NO: Direct production edits without verification
+- NO: Untested code reported as "done"
+- NO: Breaking changes without migration plan
+- YES: Clear acceptance criteria before starting
+- YES: Root cause analysis before fixes

workflows/debug.md

@@ -0,0 +1,40 @@
+# Workflow: Troubleshooting
+
+## Intent Classification
+
+**Triggers:** Errors, failures, unexpected behavior, debugging requests
+
+## Fixed Output Format
+
+```markdown
+## Problem
+[Symptom observed]
+
+## Diagnosis
+- Checked: [what was checked]
+- Found: [what was found]
+
+## Cause
+[Root cause - trace to why, not just what]
+
+## Fix
+[What was changed]
+
+## Validation
+[How verified it's fixed]
+```
+
+## Tool Access Rules
+
+1. **Inspect first** - logs, config, state before changing anything
+2. **Reproduce** - understand the failure before fixing
+3. **Trace to root** - fix cause, not symptom
+4. **Document** - add to ISSUES.md if pattern
+
+## Constraints
+
+- NO: Restart-as-first-response
+- NO: Regex patches without understanding root cause
+- NO: Fixes without verification
+- YES: Documented workaround if root cause blocked
+- YES: Regression test if applicable

workflows/deploy.md

@@ -0,0 +1,46 @@
+# Workflow: Deployment
+
+## Intent Classification
+
+**Triggers:** Deploy to staging/production, release, infrastructure changes
+
+## Fixed Output Format
+
+```markdown
+## Deployment Summary
+[What and where]
+
+## Pre-Deployment
+- [x] Backup created
+- [x] Tests pass
+- [x] Rollback plan documented
+
+## Changes Applied
+| Step | Command | Status |
+|------|---------|--------|
+| 1 | [command] | ✅ |
+| 2 | [command] | ✅ |
+
+## Verification
+- [x] Service responding: [url]
+- [x] Logs normal: [check]
+- [x] Smoke test: [result]
+
+## Rollback (if needed)
+[Exact rollback steps]
+```
+
+## Tool Access Rules
+
+1. **Check first** - current state before changing
+2. **Backup** - always before production changes
+3. **Verify** - process running, endpoint responding, output correct
+4. **Document** - update RUNBOOK.md if procedure changed
+
+## Constraints
+
+- NO: Deployment without verification plan
+- NO: "It should work" = done
+- NO: Blurry rollback steps
+- YES: Status checked and reported
+- YES: Rollback tested if possible