Implement Vault secret cache system
New scripts: - vault-cache.sh: Main cache manager (sync/get/list/status/invalidate) - vault-env.sh: Sourceable environment with quick accessor functions - vault-api-cached.sh: Drop-in replacement for vault_api.sh using cache - nocodb-api.sh: NocoDB API wrapper using cached token - qdrant-api.sh: Qdrant API wrapper using cached key - README-vault-cache.md: Documentation Updated scripts: - vault_grist_api.sh: Now uses vault cache - checks/check-memory-table.sh: Uses vault cache - utils/get-columns.sh: Uses vault cache - utils/update-type-col.sh: Uses vault cache Features: - 29 secrets cached from Vault (6 categories) - Auto-refresh on stale entries (1hr TTL) - Vault token cached for 24h - Restricted permissions (700/600) on cache files - Fallback to direct Vault API if cache miss
This commit is contained in:
@@ -0,0 +1,101 @@
|
||||
# Vault Secret Cache System
|
||||
|
||||
## Overview
|
||||
|
||||
Local cache of all Vault secrets to avoid repeated Vault API queries. Secrets are stored in `~/.cache/vault/` with restricted permissions (700/600).
|
||||
|
||||
## Components
|
||||
|
||||
| Script | Purpose |
|
||||
|--------|---------|
|
||||
| `vault-cache.sh` | Main cache manager - sync, get, list, status, invalidate |
|
||||
| `vault-env.sh` | Sourceable environment with quick accessor functions |
|
||||
| `vault_grist_api.sh` | Grist API wrapper using cached token |
|
||||
| `nocodb-api.sh` | NocoDB API wrapper using cached token |
|
||||
| `qdrant-api.sh` | Qdrant API wrapper using cached key |
|
||||
|
||||
## Quick Start
|
||||
|
||||
```bash
|
||||
# 1. Sync all secrets from Vault to local cache
|
||||
./vault-cache.sh sync
|
||||
|
||||
# 2. Get a specific secret (auto-fetches if not cached)
|
||||
./vault-cache.sh get api/data/nocodb api_token
|
||||
./vault-cache.sh get api/data/qdrant api-key
|
||||
./vault-cache.sh get api/business/grist token
|
||||
|
||||
# 3. Check cache status
|
||||
./vault-cache.sh status
|
||||
./vault-cache.sh list
|
||||
```
|
||||
|
||||
## Using in Scripts
|
||||
|
||||
```bash
|
||||
# Source the environment helpers
|
||||
source vault-env.sh
|
||||
|
||||
# Use quick accessor functions
|
||||
NOCO_TOKEN=$(vault_nocodb_token)
|
||||
QDRANT_KEY=$(vault_qdrant_api_key)
|
||||
GRIST_TOKEN=$(vault_grist_token)
|
||||
ODOO_KEY=$(vault_odoo_api_key)
|
||||
METABASE_PASS=$(vault_metabase_password)
|
||||
SSH_KEY=$(vault_ssh_ed25519_private)
|
||||
|
||||
# Or export all fields from a service
|
||||
# eval $(vault_env_export data nocodb NOCODB_)
|
||||
# → Sets: NOCODB_API_TOKEN, NOCODB_TABLE_ID, NOCODB_INTERNAL_URL, etc.
|
||||
```
|
||||
|
||||
## Cache Behavior
|
||||
|
||||
- **Auto-refresh**: Secrets older than TTL (default: 1 hour) are re-fetched on next access
|
||||
- **Vault token**: Cached for 24 hours, then auto-refreshed via AppRole login
|
||||
- **Security**: Cache files have 600 permissions, directory has 700
|
||||
- **Selective sync**: `vault-cache.sh sync infrastructure` only syncs one category
|
||||
|
||||
## Available Categories
|
||||
|
||||
| Category | Services |
|
||||
|----------|----------|
|
||||
| `business` | gitea, grist, invoiceninja, kimai, quickbooks, snipeit |
|
||||
| `communication` | email, n8n, passpush, telegram |
|
||||
| `data` | metabase, nocodb, postgres, qdrant, rustfs |
|
||||
| `infrastructure` | homeassistant, nextcloud, odoo, ssh, uptimekuma |
|
||||
| `marketing` | firecrawl, media, newsapi, pexels |
|
||||
| `research` | hunter, kokoro, ollama, serpapi, tavily |
|
||||
|
||||
## Migration from Direct Vault Access
|
||||
|
||||
Old pattern (each script queries Vault directly):
|
||||
```bash
|
||||
VAULT_TOKEN=$(curl -sk -X POST ... approle/login)
|
||||
NOCO_TOKEN=$(curl -sk -H "X-Vault-Token:$VAULT_TOKEN" ... kv/data/api/infrastructure)
|
||||
```
|
||||
|
||||
New pattern (use cache):
|
||||
```bash
|
||||
source vault-env.sh
|
||||
NOCO_TOKEN=$(vault_nocodb_token)
|
||||
```
|
||||
|
||||
## Cache Location
|
||||
|
||||
```
|
||||
~/.cache/vault/
|
||||
├── .vault_token # Cached Vault session token
|
||||
├── meta.json # Cache metadata (timestamps, TTL)
|
||||
├── api_data_nocodb.json # Cached secrets (encrypted by file perms)
|
||||
├── api_data_qdrant.json
|
||||
└── ...
|
||||
```
|
||||
|
||||
## Cron Setup (Optional)
|
||||
|
||||
To auto-refresh cache hourly:
|
||||
```bash
|
||||
# Add to crontab
|
||||
echo "0 * * * * /home/jcbeasley/.openclaw/workspace/scripts/vault-cache.sh sync >/dev/null 2>&1" | crontab -
|
||||
```
|
||||
Reference in New Issue
Block a user