From d3fd57449213c97aa6604fec8820ed708f2c4035 Mon Sep 17 00:00:00 2001 From: Oleg Proskurin Date: Sat, 27 Dec 2025 19:47:32 +0700 Subject: [PATCH] feat: add MinIO public access + deployment docs MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Add mc anonymous set download for CDN public read access - Create docs/url-fix-vps-site.md with VPS deployment instructions - Create docs/url-fix-cloudflare-site.md with Cloudflare caching config 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 --- docs/url-fix-cloudflare-site.md | 158 ++++++++++++++++++++++++++ docs/url-fix-vps-site.md | 152 +++++++++++++++++++++++++ infrastructure/docker-compose.vps.yml | 3 + 3 files changed, 313 insertions(+) create mode 100644 docs/url-fix-cloudflare-site.md create mode 100644 docs/url-fix-vps-site.md diff --git a/docs/url-fix-cloudflare-site.md b/docs/url-fix-cloudflare-site.md new file mode 100644 index 0000000..b017112 --- /dev/null +++ b/docs/url-fix-cloudflare-site.md @@ -0,0 +1,158 @@ +# CDN URL Architecture Fix - Cloudflare Configuration + +This document describes the Cloudflare configuration for the new CDN URL architecture. + +## Domain Structure + +| Domain | Purpose | Cloudflare Proxy | +|--------|---------|------------------| +| cdn.banatie.app | CDN for images | Yes (orange cloud) | +| api.banatie.app | API server | Yes (orange cloud) | +| banatie.app | Landing page | Yes (orange cloud) | + +## Cache Rules + +### Rule 1: Cache UUID Images (High Priority) + +Cache static images with UUID filenames for maximum performance. + +**When:** Custom filter expression +``` +(http.host eq "cdn.banatie.app" and http.request.uri.path matches "^/[^/]+/[^/]+/img/[0-9a-f-]{36}$") +``` + +**Then:** +- Cache eligibility: Eligible for cache +- Edge TTL: Override origin, 7 days +- Browser TTL: Override origin, 1 year (31536000 seconds) +- Cache Key: Include query string = No + +### Rule 2: Bypass Cache for Aliases + +Aliases need dynamic resolution, bypass cache. + +**When:** Custom filter expression +``` +(http.host eq "cdn.banatie.app" and http.request.uri.path matches "^/[^/]+/[^/]+/img/@") +``` + +**Then:** +- Cache eligibility: Bypass cache + +### Rule 3: Bypass Cache for Live URLs + +Live URLs need dynamic generation, bypass cache. + +**When:** Custom filter expression +``` +(http.host eq "cdn.banatie.app" and http.request.uri.path matches "^/[^/]+/[^/]+/live/") +``` + +**Then:** +- Cache eligibility: Bypass cache + +## Page Rules (Alternative) + +If not using Cache Rules, use Page Rules: + +### Page Rule 1: Cache UUID Images +- URL: `cdn.banatie.app/*/img/*` +- Cache Level: Cache Everything +- Edge Cache TTL: 7 days +- Browser Cache TTL: 1 year + +### Page Rule 2: Bypass Aliases and Live +- URL: `cdn.banatie.app/*/img/@*` +- Cache Level: Bypass + +- URL: `cdn.banatie.app/*/live/*` +- Cache Level: Bypass + +## DNS Configuration + +Ensure DNS records point to your VPS: + +``` +cdn.banatie.app A YOUR_VPS_IP Proxied (orange cloud) +``` + +## SSL/TLS Configuration + +- SSL Mode: Full (strict) +- Always Use HTTPS: On +- Automatic HTTPS Rewrites: On +- Minimum TLS Version: TLS 1.2 + +## Performance Settings + +- Auto Minify: CSS, JavaScript (not HTML for API responses) +- Brotli: On +- Early Hints: On +- Rocket Loader: Off (may break API responses) + +## Security Settings + +- Security Level: Medium +- Challenge Passage: 30 minutes +- Browser Integrity Check: On + +## Rate Limiting (Optional) + +Create rate limiting rule for live URL abuse prevention: + +**Rule Name:** Live URL Rate Limit + +**When:** +``` +(http.host eq "cdn.banatie.app" and http.request.uri.path matches "^/[^/]+/[^/]+/live/") +``` + +**Then:** +- Rate limit: 10 requests per minute per IP +- Action: Block for 1 minute + +## Verification + +After configuration: + +1. **Test UUID caching:** + ```bash + curl -I "https://cdn.banatie.app/org/proj/img/uuid-here" + # Check for: cf-cache-status: HIT (on second request) + ``` + +2. **Test alias bypass:** + ```bash + curl -I "https://cdn.banatie.app/org/proj/img/@alias" + # Check for: cf-cache-status: DYNAMIC or BYPASS + ``` + +3. **Test live URL bypass:** + ```bash + curl -I "https://cdn.banatie.app/org/proj/live/scope?prompt=test" + # Check for: cf-cache-status: DYNAMIC or BYPASS + ``` + +## Troubleshooting + +### Images not caching +- Verify the URL matches UUID pattern (36 character UUID) +- Check Cache Rules order (UUID rule should be first) +- Purge cache and retry + +### Alias/Live URLs being cached +- Verify bypass rules are active +- Check rule order (bypass rules should run before catch-all) +- Development mode may disable caching + +### Slow first requests +- Expected behavior for cache MISS +- Subsequent requests from same edge should be HIT +- Consider using Cache Reserve for longer edge retention + +## Notes + +- UUID pattern ensures only static, immutable images are cached at edge +- Aliases and live URLs are always fresh from origin +- 1-year browser cache is safe because UUID = immutable content +- Cloudflare caches at edge, browser caches locally diff --git a/docs/url-fix-vps-site.md b/docs/url-fix-vps-site.md new file mode 100644 index 0000000..1221040 --- /dev/null +++ b/docs/url-fix-vps-site.md @@ -0,0 +1,152 @@ +# CDN URL Architecture Fix - VPS Deployment + +This document describes the changes needed on VPS to support the new CDN URL architecture. + +## Problem + +Previous URL structure used presigned URLs with 24-hour expiry, which doesn't work for permanent image embedding on websites. + +## Solution + +New URL structure with direct CDN access: +- `cdn.banatie.app/{org}/{proj}/img/{imageId}` - Direct MinIO access for static images +- `cdn.banatie.app/{org}/{proj}/img/@{alias}` - API-mediated alias resolution +- `cdn.banatie.app/{org}/{proj}/live/{scope}?prompt=...` - API-mediated live generation + +## Storage Path Format + +``` +Old: {orgSlug}/{projectSlug}/{category}/{timestamp-filename.ext} +New: {orgSlug}/{projectSlug}/img/{imageId} +``` + +Where `imageId` = UUID (same as `images.id` in database). + +## VPS Deployment Steps + +### 1. Update Caddy Configuration + +Add the following routing rules to your Caddy config: + +```caddyfile +# CDN Domain +cdn.banatie.app { + # UUID pattern - direct to MinIO (no extension in URL) + @uuid path_regexp uuid ^/([^/]+)/([^/]+)/img/([0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12})$ + handle @uuid { + reverse_proxy banatie-minio:9000 { + # Rewrite to bucket path + header_up Host cdn.banatie.app + rewrite * /banatie{uri} + } + } + + # Alias pattern (@name) - proxy to API + @alias path_regexp alias ^/([^/]+)/([^/]+)/img/@(.+)$ + handle @alias { + reverse_proxy banatie-api:3000 { + rewrite * /cdn{uri} + } + } + + # Live URL pattern - proxy to API + @live path_regexp live ^/([^/]+)/([^/]+)/live/(.+)$ + handle @live { + reverse_proxy banatie-api:3000 { + rewrite * /cdn{uri} + } + } + + # Fallback for other patterns + handle { + reverse_proxy banatie-minio:9000 { + header_up Host cdn.banatie.app + rewrite * /banatie{uri} + } + } +} +``` + +### 2. Update Environment Variables + +Add to `/opt/banatie/.env`: + +```env +CDN_BASE_URL=https://cdn.banatie.app +``` + +### 3. Reset Database and MinIO Storage + +Since this is a breaking change to the storage path format: + +```bash +# Stop services +cd /opt/banatie +docker compose down + +# Clean database (WARNING: deletes all data) +rm -rf /opt/banatie/data/postgres/* + +# Clean MinIO storage (WARNING: deletes all files) +rm -rf /opt/banatie/data/minio/drive{1,2,3,4}/* + +# Rebuild and start services +docker compose up -d --build +``` + +### 4. Run Storage Initialization + +After rebuild, the `banatie-storage-init` container will: +1. Create the `banatie` bucket +2. Configure service user with readwrite access +3. Enable public anonymous download access for CDN + +Verify public access is enabled: + +```bash +docker exec banatie-minio mc anonymous get local/banatie +# Should show: Access permission for `local/banatie` is `download` +``` + +## Verification + +### Test Direct UUID Access + +```bash +# After generating an image, get its UUID from database or API response +# Then test direct CDN access: +curl -I "https://cdn.banatie.app/{orgSlug}/{projectSlug}/img/{uuid}" + +# Expected: HTTP 200 with Content-Type: image/png (or similar) +``` + +### Test Alias Resolution + +```bash +# Assign an alias to an image via API, then test: +curl -I "https://cdn.banatie.app/{orgSlug}/{projectSlug}/img/@hero" + +# Expected: HTTP 200 (API resolves alias and streams image) +``` + +### Test Live URL Generation + +```bash +curl -I "https://cdn.banatie.app/{orgSlug}/{projectSlug}/live/test?prompt=mountain" + +# Expected: HTTP 200 (generates or returns cached image) +``` + +## Rollback + +If issues occur: + +1. Revert code changes +2. Rebuild API container +3. Regenerate any images (old storage paths won't work) + +## Notes + +- `filename = image.id` (UUID) ensures consistent identification across DB, storage, and URLs +- Files are stored without extension; Content-Type is served from MinIO metadata +- Cloudflare caching can be enabled for UUID patterns (see url-fix-cloudflare-site.md) diff --git a/infrastructure/docker-compose.vps.yml b/infrastructure/docker-compose.vps.yml index bf41add..9c687cc 100644 --- a/infrastructure/docker-compose.vps.yml +++ b/infrastructure/docker-compose.vps.yml @@ -129,6 +129,9 @@ services: {"Rules":[{"ID":"temp-cleanup","Status":"Enabled","Filter":{"Prefix":"temp/"},"Expiration":{"Days":7}}]} LCEOF mc ilm import storage/banatie < /tmp/lifecycle.json || echo 'Lifecycle policy may already exist' + # Enable public read access for CDN + mc anonymous set download storage/banatie || echo 'Anonymous access may already be set' + echo 'Public read access enabled for CDN' echo '=== Storage Initialization Completed ===' exit 0 restart: "no"