Add feedback loop for cross-project pattern registry

- Create scripts/utils/propose-pattern.js to propose new patterns or
  extend existing ones based on a newly applied workaround/fix.
- Default to dry-run preview; --apply writes files only after human review.
- Matches against existing patterns via keyword overlap and proposes an
  update when the shape is similar enough, or a new pattern file otherwise.
- Update patterns/README.md with feedback-loop instructions.
- Update MEMORY.md to document the plasticity loop.
- Update CONTEXT.md decisions log and current tasks.
This commit is contained in:
JC Beasley
2026-08-06 12:52:47 -07:00
parent 0bd719ab91
commit 66215eaf76
4 changed files with 329 additions and 2 deletions
+2
View File
@@ -124,10 +124,12 @@ node architecture/orchestrator.js "your request here" --verbose
- [x] Deploy format locker with auto-fix - [x] Deploy format locker with auto-fix
- [x] Integrate all components via orchestrator - [x] Integrate all components via orchestrator
- [x] Implement cross-project pattern registry for retrieval-augmented generalization - [x] Implement cross-project pattern registry for retrieval-augmented generalization
- [x] Implement feedback loop (`scripts/utils/propose-pattern.js`) so new workarounds can propose pattern updates
### Decisions Log ### Decisions Log
| Date | Decision | Rationale | | Date | Decision | Rationale |
|------|----------|-----------| |------|----------|-----------|
| 2026-08-06 | Create a centralized `patterns/` registry and wire it into `architecture/pipeline.js` | Captures recurring technical lessons across projects; enables agents to generalize known solutions to new tasks as the first concrete component of a continual-learning layer. | | 2026-08-06 | Create a centralized `patterns/` registry and wire it into `architecture/pipeline.js` | Captures recurring technical lessons across projects; enables agents to generalize known solutions to new tasks as the first concrete component of a continual-learning layer. |
| 2026-08-06 | Add `scripts/utils/propose-pattern.js` with dry-run default and `--apply` gate | Agents need a lightweight, auditable way to grow the registry from real fixes without silently rewriting shared knowledge. |
| 2026-07-03 | Create dedicated dev team | Need scalable capacity for multiple internal app projects | | 2026-07-03 | Create dedicated dev team | Need scalable capacity for multiple internal app projects |
| 2026-07-03 | Python/FastAPI + HTMX stack | Matches existing skills, FastAPI's type safety, HTMX keeps frontend simple | | 2026-07-03 | Python/FastAPI + HTMX stack | Matches existing skills, FastAPI's type safety, HTMX keeps frontend simple |
+2
View File
@@ -75,6 +75,8 @@ See:
Patterns are automatically loaded by `architecture/pipeline.js` when the task input matches a pattern's tags or affected projects. This is the generalization layer of the agent continual-learning system. Patterns are automatically loaded by `architecture/pipeline.js` when the task input matches a pattern's tags or affected projects. This is the generalization layer of the agent continual-learning system.
**Feedback loop:** when an agent applies a reusable workaround or fix, it should run `scripts/utils/propose-pattern.js` to propose a new pattern or extend an existing one. The helper is dry-run by default; use `--apply` only after human review. This is the plasticity layer — the registry grows from real work instead of being reconstructed from memory later.
Established: 2026-08-06. Established: 2026-08-06.
### Small Local Ollama Models and Malformed JSON ### Small Local Ollama Models and Malformed JSON
+26 -2
View File
@@ -60,9 +60,33 @@ Links to other patterns that often appear together.
1. **Manual reference** — read the registry before designing a new integration or workflow. 1. **Manual reference** — read the registry before designing a new integration or workflow.
2. **Automatic retrieval**`architecture/pipeline.js` loads relevant patterns into the context packet based on keyword matching against the task input. 2. **Automatic retrieval**`architecture/pipeline.js` loads relevant patterns into the context packet based on keyword matching against the task input.
3. **Skill/worfklow design** — when a new workaround is applied, propose a new pattern entry if the underlying shape is reusable. 3. **Skill/workflow design** — when a new workaround is applied, propose a new pattern entry if the underlying shape is reusable.
## Adding or Updating a Pattern ## Feedback Loop: Proposing New Patterns
When an agent applies a workaround or fixes a recurring failure, it should ask: *"is this lesson reusable across projects?"* If yes, run the proposal helper:
```bash
node scripts/utils/propose-pattern.js \
--task "n8n workflow sends LLM-generated text to LinkedIn" \
--fix "JSON.stringify the post body and validate with JSON.parse before the HTTP Request node" \
--projects "linkedin-automation" \
--symptom "LinkedIn API returns malformed JSON payload errors" \
--root-cause "LLM output contains unescaped quotes and newlines"
```
The helper:
1. Loads the existing `patterns/patterns.json` manifest.
2. Compares the new lesson against existing patterns using keyword overlap.
3. Proposes either:
- **An update** to an existing pattern (e.g., add an affected project), or
- **A new pattern** file + manifest entry.
4. Outputs a preview. By default it is **dry-run only**.
5. If the preview looks right, run the same command with `--apply` to write the files.
After applying, run `node architecture/pipeline.js "sample query"` to verify the new or updated pattern is retrievable.
## Adding or Updating a Pattern Manually
1. Create or edit the `.md` file. 1. Create or edit the `.md` file.
2. Update `patterns.json` with id, tags, related patterns, and affected projects. 2. Update `patterns.json` with id, tags, related patterns, and affected projects.
+299
View File
@@ -0,0 +1,299 @@
#!/usr/bin/env node
/**
* propose-pattern.js — Feedback loop for the cross-project pattern registry.
*
* Use this when an agent applies a workaround, fix, or design choice that is
* structurally reusable across projects. It compares the new lesson against
* existing patterns and either proposes an update to an existing pattern or a
* brand-new pattern file + manifest entry.
*
* Default mode is dry-run / preview. Pass --apply to write changes.
*
* Examples:
* node scripts/utils/propose-pattern.js \
* --task="n8n workflow sends LLM-generated text to LinkedIn" \
* --fix="JSON.stringify the post body and validate with JSON.parse before the HTTP Request node" \
* --projects="linkedin-automation" \
* --symptom="LinkedIn API returns malformed JSON payload errors" \
* --root-cause="LLM output contains unescaped quotes and newlines"
*
* node scripts/utils/propose-pattern.js --apply --file=/tmp/proposed-pattern.json
*/
const fs = require('fs');
const path = require('path');
const WORKSPACE = '/home/jcbeasley/.openclaw/workspace';
const PATTERNS_DIR = path.join(WORKSPACE, 'patterns');
const MANIFEST_PATH = path.join(PATTERNS_DIR, 'patterns.json');
function slugify(name) {
return name
.toLowerCase()
.replace(/[^a-z0-9]+/g, '-')
.replace(/^-+|-+$/g, '');
}
function tokenSet(text) {
return new Set(
(text || '')
.toLowerCase()
.split(/[^a-z0-9_-]+/)
.filter(t => t.length > 2)
);
}
function jaccard(a, b) {
if (a.size === 0 || b.size === 0) return 0;
const intersection = new Set([...a].filter(x => b.has(x)));
return intersection.size / (a.size + b.size - intersection.size);
}
function loadManifest() {
try {
return JSON.parse(fs.readFileSync(MANIFEST_PATH, 'utf8'));
} catch (err) {
return { version: new Date().toISOString().slice(0, 10), patterns: [] };
}
}
function findBestMatch(taskText, fixText, manifest) {
const queryTokens = new Set([...tokenSet(taskText), ...tokenSet(fixText)]);
let best = { pattern: null, score: 0 };
for (const pattern of manifest.patterns || []) {
const patternTokens = new Set([
...tokenSet(pattern.name),
...tokenSet(pattern.id),
...(pattern.tags || []).flatMap(t => [...tokenSet(t)]),
...(pattern.affected_projects || []).flatMap(p => [...tokenSet(p)])
]);
const score = jaccard(queryTokens, patternTokens);
if (score > best.score) best = { pattern, score };
}
return best;
}
function buildPatternMarkdown(opts) {
const related = (opts.related_patterns || [])
.filter(id => id && id.trim())
.map(id => `- \`${id}\``)
.join('\n') || '- None yet.';
return `# Pattern: ${opts.name}
## Symptom
${opts.symptom || 'TODO: describe the symptom'}
## Affected Projects
${(opts.projects || []).map(p => `- ${p}`).join('\n') || '- TODO'}
## Root Cause
${opts.root_cause || 'TODO: describe the root cause'}
## Standard Fix
${opts.fix || 'TODO: describe the standard fix'}
## When to Apply
${opts.when_to_apply || 'TODO: describe when to apply this pattern'}
## Verification
${opts.verification || 'TODO: describe how to verify the fix'}
## Related Patterns
${related}
`;
}
function buildProposal(argv) {
const manifest = loadManifest();
const match = findBestMatch(argv.task, argv.fix, manifest);
const threshold = 0.15;
if (match.pattern && match.score >= threshold) {
// Propose update to existing pattern
const updatedProjects = Array.from(new Set([
...(match.pattern.affected_projects || []),
...(argv.projects || [])
]));
return {
mode: 'update',
score: match.score,
existing: match.pattern,
proposal: {
id: match.pattern.id,
name: match.pattern.name,
affected_projects: updatedProjects,
rationale: `Extend pattern "${match.pattern.id}" with newly observed project(s) and/or refined fix.`,
additions: {
affected_projects: argv.projects || []
}
}
};
}
// Propose new pattern
const name = argv.name || argv.task.split(/[.!?]/)[0].slice(0, 60);
const id = argv.id || slugify(name);
const tags = argv.tags || tokenSet(`${argv.task} ${argv.fix} ${argv.symptom} ${argv.root_cause}`);
return {
mode: 'create',
score: match.score,
nearest_existing: match.pattern,
proposal: {
id,
name,
file: `patterns/${id}.md`,
tags: Array.from(tags),
affected_projects: argv.projects || [],
related_patterns: argv.related || [],
markdown: buildPatternMarkdown({
name,
symptom: argv.symptom,
projects: argv.projects || [],
root_cause: argv.root_cause,
fix: argv.fix,
when_to_apply: argv.when_to_apply,
verification: argv.verification,
related_patterns: argv.related || []
})
}
};
}
function printProposal(proposal) {
if (proposal.mode === 'update') {
console.log('\n=== PROPOSED PATTERN UPDATE (dry-run) ===\n');
console.log(`Pattern: ${proposal.existing.id}`);
console.log(`Match score: ${(proposal.score * 100).toFixed(1)}%`);
console.log(`Rationale: ${proposal.proposal.rationale}`);
console.log('\nExisting affected projects:');
for (const p of proposal.existing.affected_projects || []) console.log(` - ${p}`);
console.log('\nProjects to add:');
for (const p of proposal.proposal.additions.affected_projects) console.log(` + ${p}`);
console.log('\nRun with --apply to update patterns/patterns.json.');
} else {
console.log('\n=== PROPOSED NEW PATTERN (dry-run) ===\n');
console.log(`ID: ${proposal.proposal.id}`);
console.log(`Name: ${proposal.proposal.name}`);
console.log(`File: ${proposal.proposal.file}`);
console.log(`Tags: ${proposal.proposal.tags.join(', ')}`);
console.log(`Projects: ${proposal.proposal.affected_projects.join(', ') || '(none)'}`);
console.log(`Match score: ${proposal.score > 0 ? (proposal.score * 100).toFixed(1) + '%' : 'none'}`);
if (proposal.nearest_existing) {
console.log(`Nearest existing pattern: ${proposal.nearest_existing.id} (${(proposal.score * 100).toFixed(1)}% match)`);
}
console.log('\n--- Markdown preview ---\n');
console.log(proposal.proposal.markdown);
console.log('\nRun with --apply to write the file and update patterns/patterns.json.');
}
}
function applyProposal(proposal) {
const manifest = loadManifest();
if (proposal.mode === 'update') {
const idx = manifest.patterns.findIndex(p => p.id === proposal.proposal.id);
if (idx === -1) throw new Error(`Pattern ${proposal.proposal.id} not found in manifest`);
manifest.patterns[idx].affected_projects = proposal.proposal.affected_projects;
fs.writeFileSync(MANIFEST_PATH, JSON.stringify(manifest, null, 2) + '\n');
console.log(`\nUpdated manifest: ${proposal.proposal.id} affected_projects`);
return;
}
// Create new pattern
const filePath = path.join(WORKSPACE, proposal.proposal.file);
if (fs.existsSync(filePath)) {
throw new Error(`Pattern file already exists: ${proposal.proposal.file}`);
}
fs.writeFileSync(filePath, proposal.proposal.markdown);
const manifestEntry = {
id: proposal.proposal.id,
name: proposal.proposal.name,
file: proposal.proposal.file,
tags: proposal.proposal.tags,
affected_projects: proposal.proposal.affected_projects,
related_patterns: proposal.proposal.related_patterns
};
manifest.patterns.push(manifestEntry);
fs.writeFileSync(MANIFEST_PATH, JSON.stringify(manifest, null, 2) + '\n');
console.log(`\nCreated ${proposal.proposal.file}`);
console.log(`Updated patterns/patterns.json`);
}
function parseArgv() {
const argv = { _: [] };
for (let i = 2; i < process.argv.length; i++) {
const arg = process.argv[i];
if (arg === '--apply') argv.apply = true;
else if (arg === '--help' || arg === '-h') argv.help = true;
else if (arg.startsWith('--')) {
const key = arg.slice(2).replace(/-/g, '_');
const next = process.argv[i + 1];
if (next && !next.startsWith('--')) {
if (['projects', 'tags', 'related'].includes(key)) {
argv[key] = next.split(',').map(s => s.trim()).filter(Boolean);
} else {
argv[key] = next;
}
i++;
} else {
argv[key] = true;
}
} else {
argv._.push(arg);
}
}
return argv;
}
function main() {
const argv = parseArgv();
if (argv.help) {
console.log(`Usage: node scripts/utils/propose-pattern.js [options]
Options:
--task "..." Description of the task/problem
--fix "..." The workaround or fix applied
--projects p1,p2 Comma-separated affected project identifiers
--symptom "..." What the user/agent sees
--root-cause "..." Why it happens
--when-to-apply "..." When to apply this pattern to new tasks
--verification "..." How to verify the fix
--related p1,p2 Comma-separated related pattern IDs
--name "..." Override the generated pattern name
--id "..." Override the generated pattern ID
--apply Write files after review (default is dry-run)
--file path.json Load proposal inputs from JSON file
-h, --help Show this help
`);
process.exit(0);
}
let inputs = argv;
if (argv.file) {
inputs = { ...JSON.parse(fs.readFileSync(argv.file, 'utf8')), ...argv };
}
if (!inputs.task && !inputs.fix && !inputs.symptom) {
console.error('Error: at least one of --task, --fix, or --symptom is required.');
process.exit(1);
}
const proposal = buildProposal(inputs);
printProposal(proposal);
if (argv.apply) {
applyProposal(proposal);
console.log('\nDone. Run `node architecture/pipeline.js "sample query"` to verify retrieval.');
}
}
if (require.main === module) {
main();
}
module.exports = { buildProposal, applyProposal, findBestMatch, loadManifest };