JC Beasley
6d2105d206
New scripts: - vault-cache.sh: Main cache manager (sync/get/list/status/invalidate) - vault-env.sh: Sourceable environment with quick accessor functions - vault-api-cached.sh: Drop-in replacement for vault_api.sh using cache - nocodb-api.sh: NocoDB API wrapper using cached token - qdrant-api.sh: Qdrant API wrapper using cached key - README-vault-cache.md: Documentation Updated scripts: - vault_grist_api.sh: Now uses vault cache - checks/check-memory-table.sh: Uses vault cache - utils/get-columns.sh: Uses vault cache - utils/update-type-col.sh: Uses vault cache Features: - 29 secrets cached from Vault (6 categories) - Auto-refresh on stale entries (1hr TTL) - Vault token cached for 24h - Restricted permissions (700/600) on cache files - Fallback to direct Vault API if cache miss
102 lines
3.0 KiB
Markdown
102 lines
3.0 KiB
Markdown
# Vault Secret Cache System
|
|
|
|
## Overview
|
|
|
|
Local cache of all Vault secrets to avoid repeated Vault API queries. Secrets are stored in `~/.cache/vault/` with restricted permissions (700/600).
|
|
|
|
## Components
|
|
|
|
| Script | Purpose |
|
|
|--------|---------|
|
|
| `vault-cache.sh` | Main cache manager - sync, get, list, status, invalidate |
|
|
| `vault-env.sh` | Sourceable environment with quick accessor functions |
|
|
| `vault_grist_api.sh` | Grist API wrapper using cached token |
|
|
| `nocodb-api.sh` | NocoDB API wrapper using cached token |
|
|
| `qdrant-api.sh` | Qdrant API wrapper using cached key |
|
|
|
|
## Quick Start
|
|
|
|
```bash
|
|
# 1. Sync all secrets from Vault to local cache
|
|
./vault-cache.sh sync
|
|
|
|
# 2. Get a specific secret (auto-fetches if not cached)
|
|
./vault-cache.sh get api/data/nocodb api_token
|
|
./vault-cache.sh get api/data/qdrant api-key
|
|
./vault-cache.sh get api/business/grist token
|
|
|
|
# 3. Check cache status
|
|
./vault-cache.sh status
|
|
./vault-cache.sh list
|
|
```
|
|
|
|
## Using in Scripts
|
|
|
|
```bash
|
|
# Source the environment helpers
|
|
source vault-env.sh
|
|
|
|
# Use quick accessor functions
|
|
NOCO_TOKEN=$(vault_nocodb_token)
|
|
QDRANT_KEY=$(vault_qdrant_api_key)
|
|
GRIST_TOKEN=$(vault_grist_token)
|
|
ODOO_KEY=$(vault_odoo_api_key)
|
|
METABASE_PASS=$(vault_metabase_password)
|
|
SSH_KEY=$(vault_ssh_ed25519_private)
|
|
|
|
# Or export all fields from a service
|
|
# eval $(vault_env_export data nocodb NOCODB_)
|
|
# → Sets: NOCODB_API_TOKEN, NOCODB_TABLE_ID, NOCODB_INTERNAL_URL, etc.
|
|
```
|
|
|
|
## Cache Behavior
|
|
|
|
- **Auto-refresh**: Secrets older than TTL (default: 1 hour) are re-fetched on next access
|
|
- **Vault token**: Cached for 24 hours, then auto-refreshed via AppRole login
|
|
- **Security**: Cache files have 600 permissions, directory has 700
|
|
- **Selective sync**: `vault-cache.sh sync infrastructure` only syncs one category
|
|
|
|
## Available Categories
|
|
|
|
| Category | Services |
|
|
|----------|----------|
|
|
| `business` | gitea, grist, invoiceninja, kimai, quickbooks, snipeit |
|
|
| `communication` | email, n8n, passpush, telegram |
|
|
| `data` | metabase, nocodb, postgres, qdrant, rustfs |
|
|
| `infrastructure` | homeassistant, nextcloud, odoo, ssh, uptimekuma |
|
|
| `marketing` | firecrawl, media, newsapi, pexels |
|
|
| `research` | hunter, kokoro, ollama, serpapi, tavily |
|
|
|
|
## Migration from Direct Vault Access
|
|
|
|
Old pattern (each script queries Vault directly):
|
|
```bash
|
|
VAULT_TOKEN=$(curl -sk -X POST ... approle/login)
|
|
NOCO_TOKEN=$(curl -sk -H "X-Vault-Token:$VAULT_TOKEN" ... kv/data/api/infrastructure)
|
|
```
|
|
|
|
New pattern (use cache):
|
|
```bash
|
|
source vault-env.sh
|
|
NOCO_TOKEN=$(vault_nocodb_token)
|
|
```
|
|
|
|
## Cache Location
|
|
|
|
```
|
|
~/.cache/vault/
|
|
├── .vault_token # Cached Vault session token
|
|
├── meta.json # Cache metadata (timestamps, TTL)
|
|
├── api_data_nocodb.json # Cached secrets (encrypted by file perms)
|
|
├── api_data_qdrant.json
|
|
└── ...
|
|
```
|
|
|
|
## Cron Setup (Optional)
|
|
|
|
To auto-refresh cache hourly:
|
|
```bash
|
|
# Add to crontab
|
|
echo "0 * * * * /home/jcbeasley/.openclaw/workspace/scripts/vault-cache.sh sync >/dev/null 2>&1" | crontab -
|
|
```
|