JC Beasley
0bd719ab91
- Create patterns/ directory with README, manifest, and 10 initial patterns covering Ollama JSON fallback, API escaping, deprecation, PTY auth, queue-poll, LLM-as-parser, credential rotation, reverse proxy binding, human approval gates, and transient retry. - Wire pattern loading into architecture/pipeline.js based on task tags. - Update architecture/orchestrator.js to load patterns and surface them in the system prompt. - Update MEMORY.md, ARCHITECTURE.md, and CONTEXT.md to document the registry and record the decision.
36 lines
1.8 KiB
Markdown
36 lines
1.8 KiB
Markdown
# Pattern: Ollama Structured Output Fallback
|
|
|
|
## Symptom
|
|
A workflow calls a local Ollama model (e.g., `gemma3:4b`, small quantized models) with a structured-output or JSON schema request, but the returned text is not valid JSON, misses required fields, wraps the JSON in prose, or otherwise fails schema validation.
|
|
|
|
## Affected Projects
|
|
- Cyber Tips Newsletter pipeline
|
|
- Proxmox VE snapshot summarizer
|
|
- AI video generation pipeline (ComfyUI + WAN 2.1 payloads)
|
|
- Forex trading analysis workflow
|
|
|
|
## Root Cause
|
|
Small local instruction-tuned models have weaker schema adherence than frontier APIs. They may produce JSON-like text that does not strictly conform to the requested schema, especially under complex prompts or when asked to combine generation with strict formatting.
|
|
|
|
## Standard Fix
|
|
Add a downstream JavaScript Code node (or equivalent parser) in n8n that:
|
|
1. Attempts a strict `JSON.parse()` first.
|
|
2. On failure, applies regex-based JSON extraction to pull the first `{...}` or `[...]` block from the text.
|
|
3. Optionally sanitizes common issues (trailing commas, unescaped newlines, code fences).
|
|
4. Falls back to a safe default or error flag if extraction still fails.
|
|
|
|
This is a deliberate workaround, not a substitute for fixing the model. Document it as such wherever applied.
|
|
|
|
## When to Apply
|
|
- Any new n8n workflow that uses a local Ollama model for structured extraction, classification, or JSON generation.
|
|
- Any integration where the downstream node requires strict JSON and the LLM is under ~8B parameters or known to drift.
|
|
|
|
## Verification
|
|
- Test the fallback with intentionally malformed LLM output.
|
|
- Confirm downstream nodes receive valid parsed JSON.
|
|
- Log fallback events so model quality can be monitored separately.
|
|
|
|
## Related Patterns
|
|
- `json-escaping-downstream-api`
|
|
- `llm-as-parser-fallback`
|