Add cross-project pattern registry for retrieval-augmented generalization
- Create patterns/ directory with README, manifest, and 10 initial patterns covering Ollama JSON fallback, API escaping, deprecation, PTY auth, queue-poll, LLM-as-parser, credential rotation, reverse proxy binding, human approval gates, and transient retry. - Wire pattern loading into architecture/pipeline.js based on task tags. - Update architecture/orchestrator.js to load patterns and surface them in the system prompt. - Update MEMORY.md, ARCHITECTURE.md, and CONTEXT.md to document the registry and record the decision.
This commit is contained in:
@@ -0,0 +1,33 @@
|
||||
# Pattern: LLM-as-Parser with Structured Fallback
|
||||
|
||||
## Symptom
|
||||
A workflow asks an LLM to parse, classify, or extract information from unstructured input. Sometimes the output is correct but poorly formatted; sometimes it is wrong or inconsistent. Downstream nodes cannot rely on it without a validation step.
|
||||
|
||||
## Affected Projects
|
||||
- Cyber Tips Newsletter pipeline
|
||||
- Proxmox VE snapshot summarizer
|
||||
- Email inbox triage agent
|
||||
- Monthly IT newsletter generator
|
||||
|
||||
## Root Cause
|
||||
Using an LLM as a parser combines the power of fuzzy reasoning with the fragility of probabilistic output. Without a structured fallback, the pipeline is brittle.
|
||||
|
||||
## Standard Fix
|
||||
1. Ask the LLM for structured output (JSON/schema) when possible.
|
||||
2. Add a validation layer that checks required fields and value ranges.
|
||||
3. Add a regex or rule-based extraction fallback for common failure modes.
|
||||
4. For classification tasks, maintain a small deterministic ruleset for high-confidence cases and use the LLM only for ambiguous cases.
|
||||
5. Log parse failures to identify when the LLM is drifting.
|
||||
|
||||
## When to Apply
|
||||
- Any workflow where an LLM extracts or classifies data that downstream nodes consume.
|
||||
- Any pipeline where consistency matters more than creative interpretation.
|
||||
|
||||
## Verification
|
||||
- Test with malformed, ambiguous, and adversarial inputs.
|
||||
- Confirm downstream nodes receive validated, structured data.
|
||||
- Review parse-failure logs periodically.
|
||||
|
||||
## Related Patterns
|
||||
- `ollama-structured-output-fallback`
|
||||
- `json-escaping-downstream-api`
|
||||
Reference in New Issue
Block a user