usulpro
/
banatie-service
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
banatie-service/infrastructure.md

995 lines
23 KiB
Markdown
Raw Blame History

# Banatie Service Infrastructure Documentation
## Overview
This document defines the complete containerization and deployment architecture for the Banatie AI image generation service with MinIO object storage and PostgreSQL database integration.
## Architecture Summary
### Core Principles
- **Complete Service Isolation**: Banatie ecosystem is fully isolated from core VPS services
- **Dedicated Database**: Separate PostgreSQL container exclusively for Banatie
- **S3-Compatible Storage**: MinIO for scalable object storage with multi-tenant support
- **Container-First Approach**: All components run as Docker containers
- **Network Segregation**: Internal communication via dedicated Docker networks
### Service Components
```
Banatie Ecosystem (Isolated)
├── Banatie App Container (Node.js/TypeScript/Express)
├── PostgreSQL Container (Dedicated instance)
└── MinIO Container (S3-compatible object storage)
```
## Network Architecture
### Production Networks
```yaml
networks:
banatie-network:
driver: bridge
internal: true # No external internet access
proxy-network:
external: true # Existing Caddy reverse proxy network
```
### Network Access Matrix
| Service | banatie-network | proxy-network | External Access |
| ----------- | --------------- | --------------- | --------------- |
| Banatie App | ✅ Internal | ✅ HTTP only | ❌ Direct |
| PostgreSQL | ✅ Internal | ❌ None | ❌ None |
| MinIO | ✅ Internal | ✅ Console only | ❌ Direct |
| Caddy Proxy | ❌ None | ✅ Routing | ✅ Internet |
### VPS Integration Points
**Isolation from Core Services:**
- **NO** shared resources with existing PostgreSQL (`/opt/services/`)
- **NO** access to NextCloud, Gitea, or other core services
- **ONLY** connection point: Caddy reverse proxy for HTTP routing
**Shared Infrastructure:**
- Caddy reverse proxy for SSL termination and routing
- Host filesystem for persistent data storage
- UFW firewall rules and security policies
## Directory Structure
### Development Environment
```
banatie-service/
├── src/ # Application source code
├── Dockerfile # Multi-stage container build
├── docker-compose.yml # Local development setup
├── .env.example # Environment template
├── .dockerignore # Build exclusions
└── data/ # Local development data
├── postgres/ # PostgreSQL data
└── minio/ # MinIO data
```
### Production Environment (VPS)
```
/opt/banatie/ # Isolated deployment directory
├── docker-compose.yml # Production configuration
├── .env # Production environment (secure)
├── Dockerfile # Production Dockerfile
├── data/ # Persistent data storage
│ ├── postgres/ # PostgreSQL data
│ └── minio/ # MinIO data
├── configs/ # Service configurations
├── logs/ # Application logs
└── scripts/ # Deployment scripts
```
## Container Specifications
### 1. Banatie Application Container
**Base Image**: `node:20-alpine`
**Build Strategy**: Multi-stage for optimization
**Runtime**: Node.js with TypeScript compilation
```dockerfile
# Multi-stage build for production
FROM node:20-alpine AS builder
WORKDIR /app
COPY package*.json pnpm-lock.yaml ./
RUN npm install -g pnpm && pnpm install --frozen-lockfile
COPY . .
RUN pnpm build
FROM node:20-alpine
WORKDIR /app
RUN npm install -g pnpm
COPY --from=builder /app/dist ./dist
COPY --from=builder /app/package*.json ./
COPY --from=builder /app/pnpm-lock.yaml ./
RUN pnpm install --prod --frozen-lockfile
EXPOSE 3000
CMD ["node", "dist/server.js"]
```
**Key Features:**
- Hot-reload support in development
- Production-optimized build with dependencies pruning
- Health check endpoints for monitoring
- Structured logging to stdout
### 2. PostgreSQL Container
**Image**: `postgres:15-alpine`
**Purpose**: Metadata storage for user sessions, image metadata, organization data
**Data**: User accounts, image metadata, upload sessions, organization settings
**Database Schema:**
- `users` - User authentication and profiles
- `organizations` - Multi-tenant organization data
- `images` - Generated image metadata and references
- `sessions` - User session management
- `uploads` - Temporary upload tracking
### 3. MinIO Container
**Image**: `minio/minio:latest`
**Purpose**: S3-compatible object storage for images and files
**Ports**: 9000 (S3 API), 9001 (Web Console)
**Storage Strategy:**
- Persistent volumes for data durability
- Bucket-per-organization architecture
- Lifecycle policies for temporary file cleanup
- Presigned URLs for direct browser uploads
## Multi-Tenant Storage Architecture
### MinIO Bucket Organization
```
banatie-{org-id}/ # Organization bucket (e.g., banatie-demo)
├── users/
│ └── {user-id}/ # User-specific namespace
│ ├── generated/ # AI-generated images
│ │ ├── 2024/01/ # Date-based organization
│ │ └── thumbnails/ # Generated thumbnails
│ ├── references/ # User-uploaded reference images
│ └── temp/ # Temporary processing files (7-day TTL)
├── shared/ # Organization-wide shared resources
│ ├── templates/ # Shared image templates
│ └── public/ # Public organization assets
└── metadata/ # JSON metadata files
├── users.json # User directory metadata
└── policies.json # Access control policies
```
### Demo Organization Setup
**Default Configuration:**
- **Organization**: `demo` (org-id: demo)
- **User**: `guest` (user-id: guest)
- **Bucket**: `banatie-demo`
- **Access**: Public read for demo images
**Demo Bucket Structure:**
```
banatie-demo/
├── users/
│ └── guest/
│ ├── generated/ # Guest user's generated images
│ └── references/ # Guest user's reference uploads
└── shared/
└── examples/ # Example images for UI
```
## Environment Configuration
### Development Environment (.env)
```env
# Application Configuration
NODE_ENV=development
PORT=3000
LOG_LEVEL=debug
# Database Configuration
DB_HOST=postgres
DB_PORT=5432
DB_NAME=banatie_db
DB_USER=banatie_user
DB_PASSWORD=development_password
# MinIO Configuration
STORAGE_TYPE=minio
MINIO_ENDPOINT=minio:9000
MINIO_ACCESS_KEY=minioadmin
MINIO_SECRET_KEY=minioadmin
MINIO_USE_SSL=false
MINIO_BUCKET_PREFIX=banatie
# AI Service Configuration
GEMINI_API_KEY=your_gemini_api_key_here
# Multi-tenancy Configuration
DEFAULT_ORG_ID=demo
DEFAULT_USER_ID=guest
```
### Production Environment (.env.production)
```env
# Application Configuration
NODE_ENV=production
PORT=3000
LOG_LEVEL=info
# Database Configuration
DB_HOST=banatie-postgres
DB_PORT=5432
DB_NAME=banatie_db
DB_USER=banatie_user
DB_PASSWORD=${SECURE_DB_PASSWORD}
# MinIO Configuration
STORAGE_TYPE=minio
MINIO_ENDPOINT=banatie-minio:9000
MINIO_ACCESS_KEY=${SECURE_MINIO_ACCESS_KEY}
MINIO_SECRET_KEY=${SECURE_MINIO_SECRET_KEY}
MINIO_USE_SSL=false
MINIO_BUCKET_PREFIX=banatie
# AI Service Configuration
GEMINI_API_KEY=${SECURE_GEMINI_API_KEY}
# Multi-tenancy Configuration
DEFAULT_ORG_ID=demo
DEFAULT_USER_ID=guest
# Security
JWT_SECRET=${SECURE_JWT_SECRET}
SESSION_SECRET=${SECURE_SESSION_SECRET}
```
## Docker Compose Configurations
### Development Configuration
```yaml
# docker-compose.yml (Local Development)
version: '3.8'
services:
app:
build:
context: .
target: development
container_name: banatie-app-dev
ports:
- '3000:3000'
volumes:
- ./src:/app/src # Hot reload
- ./logs:/app/logs
networks:
- banatie-dev
depends_on:
- postgres
- minio
environment:
- NODE_ENV=development
env_file:
- .env
postgres:
image: postgres:15-alpine
container_name: banatie-postgres-dev
ports:
- '5433:5432' # Avoid conflicts with system PostgreSQL
volumes:
- ./data/postgres:/var/lib/postgresql/data
- ./scripts/init-db.sql:/docker-entrypoint-initdb.d/01-init.sql
networks:
- banatie-dev
environment:
POSTGRES_DB: banatie_db
POSTGRES_USER: banatie_user
POSTGRES_PASSWORD: development_password
healthcheck:
test: ['CMD-SHELL', 'pg_isready -U banatie_user -d banatie_db']
interval: 30s
timeout: 10s
retries: 3
minio:
image: minio/minio:latest
container_name: banatie-minio-dev
ports:
- '9000:9000' # S3 API
- '9001:9001' # Web Console
volumes:
- ./data/minio:/data
networks:
- banatie-dev
environment:
MINIO_ROOT_USER: minioadmin
MINIO_ROOT_PASSWORD: minioadmin
command: server /data --console-address ":9001"
healthcheck:
test: ['CMD', 'curl', '-f', 'http://localhost:9000/minio/health/live']
interval: 30s
timeout: 10s
retries: 3
networks:
banatie-dev:
driver: bridge
```
### Production Configuration
```yaml
# docker-compose.yml (Production on VPS)
version: '3.8'
services:
banatie-app:
build:
context: .
target: production
container_name: banatie-app
restart: unless-stopped
networks:
- banatie-network
- proxy-network
depends_on:
banatie-postgres:
condition: service_healthy
banatie-minio:
condition: service_healthy
environment:
- NODE_ENV=production
env_file:
- .env
volumes:
- ./logs:/app/logs
healthcheck:
test: ['CMD', 'curl', '-f', 'http://localhost:3000/health']
interval: 30s
timeout: 10s
retries: 3
start_period: 60s
banatie-postgres:
image: postgres:15-alpine
container_name: banatie-postgres
restart: unless-stopped
networks:
- banatie-network
volumes:
- ./data/postgres:/var/lib/postgresql/data
- ./scripts/init-db.sql:/docker-entrypoint-initdb.d/01-init.sql
environment:
POSTGRES_DB: banatie_db
POSTGRES_USER: banatie_user
POSTGRES_PASSWORD: ${DB_PASSWORD}
healthcheck:
test: ['CMD-SHELL', 'pg_isready -U banatie_user -d banatie_db']
interval: 30s
timeout: 10s
retries: 3
banatie-minio:
image: minio/minio:latest
container_name: banatie-minio
restart: unless-stopped
networks:
- banatie-network
- proxy-network
volumes:
- ./data/minio:/data
environment:
MINIO_ROOT_USER: ${MINIO_ACCESS_KEY}
MINIO_ROOT_PASSWORD: ${MINIO_SECRET_KEY}
command: server /data --console-address ":9001"
healthcheck:
test: ['CMD', 'curl', '-f', 'http://localhost:9000/minio/health/live']
interval: 30s
timeout: 10s
retries: 3
networks:
banatie-network:
driver: bridge
internal: true # No external internet access
proxy-network:
external: true # Existing Caddy network
```
## Caddy Integration
### Caddy Configuration Addition
Add to existing Caddyfile in `/opt/services/configs/caddy/Caddyfile`:
```caddy
# Banatie App - Main API
banatie.app {
reverse_proxy banatie-app:3000
# Security headers
header {
Strict-Transport-Security "max-age=31536000; includeSubDomains"
X-Content-Type-Options "nosniff"
X-Frame-Options "DENY"
X-XSS-Protection "1; mode=block"
Referrer-Policy "strict-origin-when-cross-origin"
}
# Rate limiting
rate_limit {
zone banatie_api {
key {remote_host}
events 100
window 1m
}
}
# Logging
log {
output file /opt/services/logs/banatie_access.log
format json
}
}
# MinIO Console - Admin Interface
minio.banatie.app {
reverse_proxy banatie-minio:9001
# Security headers for console
header {
Strict-Transport-Security "max-age=31536000; includeSubDomains"
X-Content-Type-Options "nosniff"
X-Frame-Options "SAMEORIGIN"
}
# Logging
log {
output file /opt/services/logs/minio_console_access.log
format json
}
}
# MinIO S3 API - Direct file access
s3.banatie.app {
reverse_proxy banatie-minio:9000
# CORS for browser uploads
header {
Access-Control-Allow-Origin "*"
Access-Control-Allow-Methods "GET, POST, PUT, DELETE, OPTIONS"
Access-Control-Allow-Headers "Content-Type, Authorization"
}
# Rate limiting for uploads
rate_limit {
zone banatie_s3 {
key {remote_host}
events 50
window 1m
}
}
# Logging
log {
output file /opt/services/logs/minio_s3_access.log
format json
}
}
```
## Service Implementation Strategy
### 1. Storage Service Abstraction
Create `src/services/StorageService.ts`:
```typescript
export interface StorageService {
// Bucket management
createBucket(orgId: string): Promise<void>;
deleteBucket(orgId: string): Promise<void>;
listBuckets(): Promise<string[]>;
// File operations
uploadFile(
orgId: string,
userId: string,
fileName: string,
buffer: Buffer,
contentType: string,
): Promise<string>;
downloadFile(orgId: string, userId: string, fileName: string): Promise<Buffer>;
deleteFile(orgId: string, userId: string, fileName: string): Promise<void>;
// URL generation
getPresignedUploadUrl(
orgId: string,
userId: string,
fileName: string,
expirySeconds: number,
): Promise<string>;
getPresignedDownloadUrl(
orgId: string,
userId: string,
fileName: string,
expirySeconds: number,
): Promise<string>;
// File management
listUserFiles(orgId: string, userId: string): Promise<FileMetadata[]>;
moveFile(fromPath: string, toPath: string): Promise<void>;
copyFile(fromPath: string, toPath: string): Promise<void>;
}
```
### 2. MinIO Implementation
Create `src/services/MinioStorageService.ts`:
```typescript
import { Client as MinioClient } from 'minio';
import { StorageService } from './StorageService';
export class MinioStorageService implements StorageService {
private client: MinioClient;
private bucketPrefix: string;
constructor(
endpoint: string,
accessKey: string,
secretKey: string,
useSSL: boolean = false,
bucketPrefix: string = 'banatie',
) {
this.client = new MinioClient({
endPoint: endpoint.replace(/^https?:\/\//, ''),
port: useSSL ? 443 : 9000,
useSSL,
accessKey,
secretKey,
});
this.bucketPrefix = bucketPrefix;
}
private getBucketName(orgId: string): string {
return `${this.bucketPrefix}-${orgId}`;
}
private getFilePath(
userId: string,
category: 'generated' | 'references' | 'temp',
fileName: string,
): string {
const now = new Date();
const year = now.getFullYear();
const month = String(now.getMonth() + 1).padStart(2, '0');
return `users/${userId}/${category}/${year}/${month}/${fileName}`;
}
// Implementation methods...
}
```
### 3. Database Schema
Create `scripts/init-db.sql`:
```sql
-- Banatie Database Initialization
CREATE EXTENSION IF NOT EXISTS "uuid-ossp";
-- Organizations table
CREATE TABLE organizations (
id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
name VARCHAR(255) NOT NULL,
slug VARCHAR(100) UNIQUE NOT NULL,
settings JSONB DEFAULT '{}',
created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
updated_at TIMESTAMP WITH TIME ZONE DEFAULT NOW()
);
-- Users table
CREATE TABLE users (
id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
organization_id UUID REFERENCES organizations(id) ON DELETE CASCADE,
username VARCHAR(100) NOT NULL,
email VARCHAR(255),
role VARCHAR(50) DEFAULT 'user',
settings JSONB DEFAULT '{}',
created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
updated_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
UNIQUE(organization_id, username)
);
-- Images table
CREATE TABLE images (
id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
user_id UUID REFERENCES users(id) ON DELETE CASCADE,
filename VARCHAR(255) NOT NULL,
file_path VARCHAR(500) NOT NULL,
original_prompt TEXT,
enhanced_prompt TEXT,
model_used VARCHAR(100),
file_size BIGINT,
content_type VARCHAR(100),
metadata JSONB DEFAULT '{}',
created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW()
);
-- Upload sessions table (for tracking multi-part uploads)
CREATE TABLE upload_sessions (
id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
user_id UUID REFERENCES users(id) ON DELETE CASCADE,
session_data JSONB NOT NULL,
expires_at TIMESTAMP WITH TIME ZONE,
created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW()
);
-- Create indexes
CREATE INDEX idx_users_org_id ON users(organization_id);
CREATE INDEX idx_images_user_id ON images(user_id);
CREATE INDEX idx_images_created_at ON images(created_at);
CREATE INDEX idx_upload_sessions_user_id ON upload_sessions(user_id);
CREATE INDEX idx_upload_sessions_expires_at ON upload_sessions(expires_at);
-- Insert demo organization and user
INSERT INTO organizations (id, name, slug) VALUES
('00000000-0000-0000-0000-000000000001', 'Demo Organization', 'demo');
INSERT INTO users (id, organization_id, username, role) VALUES
('00000000-0000-0000-0000-000000000001', '00000000-0000-0000-0000-000000000001', 'guest', 'user');
```
## Development Workflow
### Local Development Setup
1. **Clone Repository**
```bash
git clone <repository-url>
cd banatie-service
```
2. **Environment Configuration**
```bash
cp .env.example .env
# Edit .env with development settings
```
3. **Start Development Environment**
```bash
docker-compose up -d
```
4. **Verify Services**
```bash
# Check service status
docker-compose ps
# View logs
docker-compose logs -f app
# Access services
# App: http://localhost:3000
# MinIO Console: http://localhost:9001 (minioadmin/minioadmin)
# PostgreSQL: localhost:5433
```
5. **Development with Hot Reload**
```bash
# Edit source files in src/
# Changes automatically reload in container
```
### Testing Workflow
```bash
# Run tests in development environment
docker-compose exec app pnpm test
# Run with coverage
docker-compose exec app pnpm test:coverage
# Run linting
docker-compose exec app pnpm lint
```
## Production Deployment
### VPS Deployment Process
1. **Prepare VPS Directory**
```bash
# SSH to VPS
ssh usul-vps
# Create Banatie directory
sudo mkdir -p /opt/banatie
sudo chown usul:usul /opt/banatie
cd /opt/banatie
```
2. **Clone and Configure**
```bash
# Clone repository
git clone <repository-url> .
# Create production environment
cp .env.example .env
# Edit .env with production settings and secure passwords
```
3. **Generate Secure Credentials**
```bash
# Generate secure passwords
DB_PASSWORD=$(openssl rand -base64 32 | tr -d '\n\r ')
MINIO_ACCESS_KEY=$(openssl rand -base64 20 | tr -d '\n\r ')
MINIO_SECRET_KEY=$(openssl rand -base64 40 | tr -d '\n\r ')
JWT_SECRET=$(openssl rand -base64 64 | tr -d '\n\r ')
# Add to .env file
echo "DB_PASSWORD=$DB_PASSWORD" >> .env
echo "MINIO_ACCESS_KEY=$MINIO_ACCESS_KEY" >> .env
echo "MINIO_SECRET_KEY=$MINIO_SECRET_KEY" >> .env
echo "JWT_SECRET=$JWT_SECRET" >> .env
# Secure the file
chmod 600 .env
```
4. **Update Caddy Configuration**
```bash
# Add Banatie routes to Caddyfile
sudo nano /opt/services/configs/caddy/Caddyfile
# Add the Banatie configuration from above
# Reload Caddy
cd /opt/services
./manage-services.sh caddy reload
```
5. **Deploy Services**
```bash
cd /opt/banatie
# Build and start services
docker-compose up -d
# Verify deployment
docker-compose ps
docker-compose logs -f
```
6. **Verify Deployment**
```bash
# Check health endpoints
curl https://banatie.app/health
curl https://minio.banatie.app/minio/health/live
# Check logs
docker-compose logs banatie-app
docker-compose logs banatie-postgres
docker-compose logs banatie-minio
```
### Manual Update Process
```bash
# SSH to VPS
ssh usul-vps
cd /opt/banatie
# Pull latest changes
git pull origin main
# Rebuild and restart
docker-compose build banatie-app
docker-compose up -d
# Verify update
docker-compose logs -f banatie-app
curl https://banatie.app/health
```
## Monitoring and Maintenance
### Health Checks
**Application Health Check** (`/health`):
```json
{
"status": "healthy",
"timestamp": "2024-01-01T12:00:00.000Z",
"services": {
"database": "connected",
"minio": "connected",
"gemini_api": "accessible"
},
"version": "1.0.0"
}
```
**MinIO Health Check** (`/minio/health/live`):
- Returns 200 OK when MinIO is operational
- Used by Docker healthcheck and monitoring
### Log Management
**Application Logs**:
- Location: `/opt/banatie/logs/`
- Format: Structured JSON
- Rotation: Daily with 30-day retention
**Access Logs**:
- Caddy logs: `/opt/services/logs/banatie_*.log`
- Format: JSON with request/response details
### Backup Strategy
**Database Backup**:
```bash
# Create backup
docker exec banatie-postgres pg_dump -U banatie_user banatie_db > banatie_db_backup.sql
# Restore backup
docker exec -i banatie-postgres psql -U banatie_user banatie_db < banatie_db_backup.sql
```
**MinIO Data Backup**:
```bash
# Backup MinIO data
sudo tar -czf banatie_minio_backup.tar.gz -C /opt/banatie/data minio/
# Restore MinIO data
sudo tar -xzf banatie_minio_backup.tar.gz -C /opt/banatie/data
```
## Security Considerations
### Container Security
- Non-root users in containers
- Read-only root filesystems where possible
- Resource limits (memory, CPU)
- Health checks for automatic restart
### Network Security
- Internal network isolation
- No direct external access to database or MinIO
- Rate limiting at proxy level
- HTTPS-only external access
### Data Security
- Encrypted environment variables
- Secure secret generation
- Regular credential rotation
- Audit logging
### Access Control
- MinIO bucket policies per organization
- PostgreSQL row-level security
- JWT-based API authentication
- Role-based access control
## Future Scalability
### Horizontal Scaling Options
- Multiple Banatie app containers behind load balancer
- MinIO distributed mode for storage scaling
- PostgreSQL read replicas for read scaling
- Redis for session storage and caching
### Multi-Region Deployment
- Regional MinIO clusters
- Database replication
- CDN integration for static assets
- Geo-distributed deployment
## Troubleshooting
### Common Issues
**Container Won't Start**:
```bash
# Check logs
docker-compose logs [service-name]
# Check resource usage
docker stats
# Rebuild container
docker-compose build [service-name]
docker-compose up -d [service-name]
```
**Database Connection Issues**:
```bash
# Test database connectivity
docker exec banatie-postgres pg_isready -U banatie_user -d banatie_db
# Check environment variables
docker exec banatie-app env | grep DB_
# Restart database
docker-compose restart banatie-postgres
```
**MinIO Access Issues**:
```bash
# Check MinIO status
docker exec banatie-minio mc admin info local
# Test S3 API
curl -f http://localhost:9000/minio/health/live
# Check access keys
docker exec banatie-app env | grep MINIO_
```
**Network Connectivity**:
```bash
# Check networks
docker network ls
docker network inspect banatie_banatie-network
# Test internal connectivity
docker exec banatie-app ping banatie-postgres
docker exec banatie-app ping banatie-minio
```
---
**Document Version**: 1.0
**Last Updated**: 2024-09-26
**Maintained By**: Banatie Development Team
Reference in New Issue View Git Blame Copy Permalink