Add cross-project pattern registry for retrieval-augmented generalization
- Create patterns/ directory with README, manifest, and 10 initial patterns covering Ollama JSON fallback, API escaping, deprecation, PTY auth, queue-poll, LLM-as-parser, credential rotation, reverse proxy binding, human approval gates, and transient retry. - Wire pattern loading into architecture/pipeline.js based on task tags. - Update architecture/orchestrator.js to load patterns and surface them in the system prompt. - Update MEMORY.md, ARCHITECTURE.md, and CONTEXT.md to document the registry and record the decision.
This commit is contained in:
@@ -0,0 +1,32 @@
|
||||
# Pattern: Queue-Poll Async Job Pattern
|
||||
|
||||
## Symptom
|
||||
An operation takes longer than a synchronous HTTP timeout allows (video generation, large LLM generation, batch job). The caller either times out, retries redundantly, or loses track of the job state.
|
||||
|
||||
## Affected Projects
|
||||
- AI video generation pipeline (ComfyUI + WAN 2.1)
|
||||
- Long-running Ollama generations in n8n
|
||||
- Any workflow that submits work to a queue and must wait for completion
|
||||
|
||||
## Root Cause
|
||||
The work is genuinely asynchronous, but the integration is designed as if it were synchronous. Without explicit job-state tracking, the system cannot wait, retry, or recover correctly.
|
||||
|
||||
## Standard Fix
|
||||
1. Submit the job and immediately capture a job/prompt ID.
|
||||
2. Poll a status endpoint on a fixed interval with exponential backoff.
|
||||
3. Define terminal states (completed, failed, cancelled) and a max poll duration.
|
||||
4. Store intermediate state so a restart does not lose the job ID.
|
||||
5. Surface progress to the user if the operation is user-facing.
|
||||
|
||||
## When to Apply
|
||||
- Any integration where the expected duration exceeds a reasonable HTTP timeout (~30-60 seconds).
|
||||
- Any service that returns a job ID or queue position instead of the final result.
|
||||
|
||||
## Verification
|
||||
- Job submission returns an ID.
|
||||
- Polling correctly detects completion and failure.
|
||||
- No duplicate work is triggered by retries.
|
||||
- Progress/state survives a brief restart of the polling service.
|
||||
|
||||
## Related Patterns
|
||||
- `transient-failure-retry`
|
||||
Reference in New Issue
Block a user