23 KiB

Raw Blame History

Banatie Service Infrastructure Documentation

Overview

This document defines the complete containerization and deployment architecture for the Banatie AI image generation service with MinIO object storage and PostgreSQL database integration.

Architecture Summary

Core Principles

Complete Service Isolation: Banatie ecosystem is fully isolated from core VPS services
Dedicated Database: Separate PostgreSQL container exclusively for Banatie
S3-Compatible Storage: MinIO for scalable object storage with multi-tenant support
Container-First Approach: All components run as Docker containers
Network Segregation: Internal communication via dedicated Docker networks

Service Components

Banatie Ecosystem (Isolated)
├── Banatie App Container (Node.js/TypeScript/Express)
├── PostgreSQL Container (Dedicated instance)
└── MinIO Container (S3-compatible object storage)

Network Architecture

Production Networks

networks:
  banatie-network:
    driver: bridge
    internal: true              # No external internet access

  proxy-network:
    external: true              # Existing Caddy reverse proxy network

Network Access Matrix

Service	banatie-network	proxy-network	External Access
Banatie App	✅ Internal	✅ HTTP only	❌ Direct
PostgreSQL	✅ Internal	❌ None	❌ None
MinIO	✅ Internal	✅ Console only	❌ Direct
Caddy Proxy	❌ None	✅ Routing	✅ Internet

VPS Integration Points

Isolation from Core Services:

NO shared resources with existing PostgreSQL (/opt/services/)
NO access to NextCloud, Gitea, or other core services
ONLY connection point: Caddy reverse proxy for HTTP routing

Shared Infrastructure:

Caddy reverse proxy for SSL termination and routing
Host filesystem for persistent data storage
UFW firewall rules and security policies

Directory Structure

Development Environment

banatie-service/
├── src/                        # Application source code
├── Dockerfile                  # Multi-stage container build
├── docker-compose.yml          # Local development setup
├── .env.example               # Environment template
├── .dockerignore              # Build exclusions
└── data/                      # Local development data
    ├── postgres/              # PostgreSQL data
    └── minio/                 # MinIO data

Production Environment (VPS)

/opt/banatie/                  # Isolated deployment directory
├── docker-compose.yml         # Production configuration
├── .env                       # Production environment (secure)
├── Dockerfile                 # Production Dockerfile
├── data/                      # Persistent data storage
│   ├── postgres/              # PostgreSQL data
│   └── minio/                 # MinIO data
├── configs/                   # Service configurations
├── logs/                      # Application logs
└── scripts/                   # Deployment scripts

Container Specifications

1. Banatie Application Container

Base Image: node:20-alpine Build Strategy: Multi-stage for optimization Runtime: Node.js with TypeScript compilation

# Multi-stage build for production
FROM node:20-alpine AS builder
WORKDIR /app
COPY package*.json pnpm-lock.yaml ./
RUN npm install -g pnpm && pnpm install --frozen-lockfile
COPY . .
RUN pnpm build

FROM node:20-alpine
WORKDIR /app
RUN npm install -g pnpm
COPY --from=builder /app/dist ./dist
COPY --from=builder /app/package*.json ./
COPY --from=builder /app/pnpm-lock.yaml ./
RUN pnpm install --prod --frozen-lockfile
EXPOSE 3000
CMD ["node", "dist/server.js"]

Key Features:

Hot-reload support in development
Production-optimized build with dependencies pruning
Health check endpoints for monitoring
Structured logging to stdout

2. PostgreSQL Container

Image: postgres:15-alpine Purpose: Metadata storage for user sessions, image metadata, organization data Data: User accounts, image metadata, upload sessions, organization settings

Database Schema:

users - User authentication and profiles
organizations - Multi-tenant organization data
images - Generated image metadata and references
sessions - User session management
uploads - Temporary upload tracking

3. MinIO Container

Image: minio/minio:latest Purpose: S3-compatible object storage for images and files Ports: 9000 (S3 API), 9001 (Web Console)

Storage Strategy:

Persistent volumes for data durability
Bucket-per-organization architecture
Lifecycle policies for temporary file cleanup
Presigned URLs for direct browser uploads

Multi-Tenant Storage Architecture

MinIO Bucket Organization

banatie-{org-id}/              # Organization bucket (e.g., banatie-demo)
├── users/
│   └── {user-id}/             # User-specific namespace
│       ├── generated/         # AI-generated images
│       │   ├── 2024/01/       # Date-based organization
│       │   └── thumbnails/    # Generated thumbnails
│       ├── references/        # User-uploaded reference images
│       └── temp/              # Temporary processing files (7-day TTL)
├── shared/                    # Organization-wide shared resources
│   ├── templates/             # Shared image templates
│   └── public/                # Public organization assets
└── metadata/                  # JSON metadata files
    ├── users.json             # User directory metadata
    └── policies.json          # Access control policies

Demo Organization Setup

Default Configuration:

Organization: demo (org-id: demo)
User: guest (user-id: guest)
Bucket: banatie-demo
Access: Public read for demo images

Demo Bucket Structure:

banatie-demo/
├── users/
│   └── guest/
│       ├── generated/         # Guest user's generated images
│       └── references/        # Guest user's reference uploads
└── shared/
    └── examples/              # Example images for UI

Environment Configuration

Development Environment (.env)

# Application Configuration
NODE_ENV=development
PORT=3000
LOG_LEVEL=debug

# Database Configuration
DB_HOST=postgres
DB_PORT=5432
DB_NAME=banatie_db
DB_USER=banatie_user
DB_PASSWORD=development_password

# MinIO Configuration
STORAGE_TYPE=minio
MINIO_ENDPOINT=minio:9000
MINIO_ACCESS_KEY=minioadmin
MINIO_SECRET_KEY=minioadmin
MINIO_USE_SSL=false
MINIO_BUCKET_PREFIX=banatie

# AI Service Configuration
GEMINI_API_KEY=your_gemini_api_key_here

# Multi-tenancy Configuration
DEFAULT_ORG_ID=demo
DEFAULT_USER_ID=guest

Production Environment (.env.production)

# Application Configuration
NODE_ENV=production
PORT=3000
LOG_LEVEL=info

# Database Configuration
DB_HOST=banatie-postgres
DB_PORT=5432
DB_NAME=banatie_db
DB_USER=banatie_user
DB_PASSWORD=${SECURE_DB_PASSWORD}

# MinIO Configuration
STORAGE_TYPE=minio
MINIO_ENDPOINT=banatie-minio:9000
MINIO_ACCESS_KEY=${SECURE_MINIO_ACCESS_KEY}
MINIO_SECRET_KEY=${SECURE_MINIO_SECRET_KEY}
MINIO_USE_SSL=false
MINIO_BUCKET_PREFIX=banatie

# AI Service Configuration
GEMINI_API_KEY=${SECURE_GEMINI_API_KEY}

# Multi-tenancy Configuration
DEFAULT_ORG_ID=demo
DEFAULT_USER_ID=guest

# Security
JWT_SECRET=${SECURE_JWT_SECRET}
SESSION_SECRET=${SECURE_SESSION_SECRET}

Docker Compose Configurations

Development Configuration

# docker-compose.yml (Local Development)
version: '3.8'

services:
  app:
    build:
      context: .
      target: development
    container_name: banatie-app-dev
    ports:
      - "3000:3000"
    volumes:
      - ./src:/app/src              # Hot reload
      - ./logs:/app/logs
    networks:
      - banatie-dev
    depends_on:
      - postgres
      - minio
    environment:
      - NODE_ENV=development
    env_file:
      - .env

  postgres:
    image: postgres:15-alpine
    container_name: banatie-postgres-dev
    ports:
      - "5433:5432"                 # Avoid conflicts with system PostgreSQL
    volumes:
      - ./data/postgres:/var/lib/postgresql/data
      - ./scripts/init-db.sql:/docker-entrypoint-initdb.d/01-init.sql
    networks:
      - banatie-dev
    environment:
      POSTGRES_DB: banatie_db
      POSTGRES_USER: banatie_user
      POSTGRES_PASSWORD: development_password
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U banatie_user -d banatie_db"]
      interval: 30s
      timeout: 10s
      retries: 3

  minio:
    image: minio/minio:latest
    container_name: banatie-minio-dev
    ports:
      - "9000:9000"                 # S3 API
      - "9001:9001"                 # Web Console
    volumes:
      - ./data/minio:/data
    networks:
      - banatie-dev
    environment:
      MINIO_ROOT_USER: minioadmin
      MINIO_ROOT_PASSWORD: minioadmin
    command: server /data --console-address ":9001"
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
      interval: 30s
      timeout: 10s
      retries: 3

networks:
  banatie-dev:
    driver: bridge

Production Configuration

# docker-compose.yml (Production on VPS)
version: '3.8'

services:
  banatie-app:
    build:
      context: .
      target: production
    container_name: banatie-app
    restart: unless-stopped
    networks:
      - banatie-network
      - proxy-network
    depends_on:
      banatie-postgres:
        condition: service_healthy
      banatie-minio:
        condition: service_healthy
    environment:
      - NODE_ENV=production
    env_file:
      - .env
    volumes:
      - ./logs:/app/logs
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:3000/health"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 60s

  banatie-postgres:
    image: postgres:15-alpine
    container_name: banatie-postgres
    restart: unless-stopped
    networks:
      - banatie-network
    volumes:
      - ./data/postgres:/var/lib/postgresql/data
      - ./scripts/init-db.sql:/docker-entrypoint-initdb.d/01-init.sql
    environment:
      POSTGRES_DB: banatie_db
      POSTGRES_USER: banatie_user
      POSTGRES_PASSWORD: ${DB_PASSWORD}
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U banatie_user -d banatie_db"]
      interval: 30s
      timeout: 10s
      retries: 3

  banatie-minio:
    image: minio/minio:latest
    container_name: banatie-minio
    restart: unless-stopped
    networks:
      - banatie-network
      - proxy-network
    volumes:
      - ./data/minio:/data
    environment:
      MINIO_ROOT_USER: ${MINIO_ACCESS_KEY}
      MINIO_ROOT_PASSWORD: ${MINIO_SECRET_KEY}
    command: server /data --console-address ":9001"
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]
      interval: 30s
      timeout: 10s
      retries: 3

networks:
  banatie-network:
    driver: bridge
    internal: true                  # No external internet access

  proxy-network:
    external: true                  # Existing Caddy network

Caddy Integration

Caddy Configuration Addition

Add to existing Caddyfile in /opt/services/configs/caddy/Caddyfile:

# Banatie App - Main API
banatie.app {
    reverse_proxy banatie-app:3000

    # Security headers
    header {
        Strict-Transport-Security "max-age=31536000; includeSubDomains"
        X-Content-Type-Options "nosniff"
        X-Frame-Options "DENY"
        X-XSS-Protection "1; mode=block"
        Referrer-Policy "strict-origin-when-cross-origin"
    }

    # Rate limiting
    rate_limit {
        zone banatie_api {
            key {remote_host}
            events 100
            window 1m
        }
    }

    # Logging
    log {
        output file /opt/services/logs/banatie_access.log
        format json
    }
}

# MinIO Console - Admin Interface
minio.banatie.app {
    reverse_proxy banatie-minio:9001

    # Security headers for console
    header {
        Strict-Transport-Security "max-age=31536000; includeSubDomains"
        X-Content-Type-Options "nosniff"
        X-Frame-Options "SAMEORIGIN"
    }

    # Logging
    log {
        output file /opt/services/logs/minio_console_access.log
        format json
    }
}

# MinIO S3 API - Direct file access
s3.banatie.app {
    reverse_proxy banatie-minio:9000

    # CORS for browser uploads
    header {
        Access-Control-Allow-Origin "*"
        Access-Control-Allow-Methods "GET, POST, PUT, DELETE, OPTIONS"
        Access-Control-Allow-Headers "Content-Type, Authorization"
    }

    # Rate limiting for uploads
    rate_limit {
        zone banatie_s3 {
            key {remote_host}
            events 50
            window 1m
        }
    }

    # Logging
    log {
        output file /opt/services/logs/minio_s3_access.log
        format json
    }
}

Service Implementation Strategy

1. Storage Service Abstraction

Create src/services/StorageService.ts:

export interface StorageService {
  // Bucket management
  createBucket(orgId: string): Promise<void>;
  deleteBucket(orgId: string): Promise<void>;
  listBuckets(): Promise<string[]>;

  // File operations
  uploadFile(orgId: string, userId: string, fileName: string, buffer: Buffer, contentType: string): Promise<string>;
  downloadFile(orgId: string, userId: string, fileName: string): Promise<Buffer>;
  deleteFile(orgId: string, userId: string, fileName: string): Promise<void>;

  // URL generation
  getPresignedUploadUrl(orgId: string, userId: string, fileName: string, expirySeconds: number): Promise<string>;
  getPresignedDownloadUrl(orgId: string, userId: string, fileName: string, expirySeconds: number): Promise<string>;

  // File management
  listUserFiles(orgId: string, userId: string): Promise<FileMetadata[]>;
  moveFile(fromPath: string, toPath: string): Promise<void>;
  copyFile(fromPath: string, toPath: string): Promise<void>;
}

2. MinIO Implementation

Create src/services/MinioStorageService.ts:

import { Client as MinioClient } from 'minio';
import { StorageService } from './StorageService';

export class MinioStorageService implements StorageService {
  private client: MinioClient;
  private bucketPrefix: string;

  constructor(
    endpoint: string,
    accessKey: string,
    secretKey: string,
    useSSL: boolean = false,
    bucketPrefix: string = 'banatie'
  ) {
    this.client = new MinioClient({
      endPoint: endpoint.replace(/^https?:\/\//, ''),
      port: useSSL ? 443 : 9000,
      useSSL,
      accessKey,
      secretKey
    });
    this.bucketPrefix = bucketPrefix;
  }

  private getBucketName(orgId: string): string {
    return `${this.bucketPrefix}-${orgId}`;
  }

  private getFilePath(userId: string, category: 'generated' | 'references' | 'temp', fileName: string): string {
    const now = new Date();
    const year = now.getFullYear();
    const month = String(now.getMonth() + 1).padStart(2, '0');

    return `users/${userId}/${category}/${year}/${month}/${fileName}`;
  }

  // Implementation methods...
}

3. Database Schema

Create scripts/init-db.sql:

-- Banatie Database Initialization
CREATE EXTENSION IF NOT EXISTS "uuid-ossp";

-- Organizations table
CREATE TABLE organizations (
    id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
    name VARCHAR(255) NOT NULL,
    slug VARCHAR(100) UNIQUE NOT NULL,
    settings JSONB DEFAULT '{}',
    created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
    updated_at TIMESTAMP WITH TIME ZONE DEFAULT NOW()
);

-- Users table
CREATE TABLE users (
    id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
    organization_id UUID REFERENCES organizations(id) ON DELETE CASCADE,
    username VARCHAR(100) NOT NULL,
    email VARCHAR(255),
    role VARCHAR(50) DEFAULT 'user',
    settings JSONB DEFAULT '{}',
    created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
    updated_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(),
    UNIQUE(organization_id, username)
);

-- Images table
CREATE TABLE images (
    id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
    user_id UUID REFERENCES users(id) ON DELETE CASCADE,
    filename VARCHAR(255) NOT NULL,
    file_path VARCHAR(500) NOT NULL,
    original_prompt TEXT,
    enhanced_prompt TEXT,
    model_used VARCHAR(100),
    file_size BIGINT,
    content_type VARCHAR(100),
    metadata JSONB DEFAULT '{}',
    created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW()
);

-- Upload sessions table (for tracking multi-part uploads)
CREATE TABLE upload_sessions (
    id UUID PRIMARY KEY DEFAULT uuid_generate_v4(),
    user_id UUID REFERENCES users(id) ON DELETE CASCADE,
    session_data JSONB NOT NULL,
    expires_at TIMESTAMP WITH TIME ZONE,
    created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW()
);

-- Create indexes
CREATE INDEX idx_users_org_id ON users(organization_id);
CREATE INDEX idx_images_user_id ON images(user_id);
CREATE INDEX idx_images_created_at ON images(created_at);
CREATE INDEX idx_upload_sessions_user_id ON upload_sessions(user_id);
CREATE INDEX idx_upload_sessions_expires_at ON upload_sessions(expires_at);

-- Insert demo organization and user
INSERT INTO organizations (id, name, slug) VALUES
    ('00000000-0000-0000-0000-000000000001', 'Demo Organization', 'demo');

INSERT INTO users (id, organization_id, username, role) VALUES
    ('00000000-0000-0000-0000-000000000001', '00000000-0000-0000-0000-000000000001', 'guest', 'user');

Development Workflow

Local Development Setup

Clone Repository

git clone <repository-url>
cd banatie-service

Environment Configuration

cp .env.example .env
# Edit .env with development settings

Start Development Environment

docker-compose up -d

Verify Services

# Check service status
docker-compose ps

# View logs
docker-compose logs -f app

# Access services
# App: http://localhost:3000
# MinIO Console: http://localhost:9001 (minioadmin/minioadmin)
# PostgreSQL: localhost:5433

Development with Hot Reload

# Edit source files in src/
# Changes automatically reload in container

Testing Workflow

# Run tests in development environment
docker-compose exec app pnpm test

# Run with coverage
docker-compose exec app pnpm test:coverage

# Run linting
docker-compose exec app pnpm lint

Production Deployment

VPS Deployment Process

Prepare VPS Directory

# SSH to VPS
ssh usul-vps

# Create Banatie directory
sudo mkdir -p /opt/banatie
sudo chown usul:usul /opt/banatie
cd /opt/banatie

Clone and Configure

# Clone repository
git clone <repository-url> .

# Create production environment
cp .env.example .env
# Edit .env with production settings and secure passwords

Generate Secure Credentials

# Generate secure passwords
DB_PASSWORD=$(openssl rand -base64 32 | tr -d '\n\r ')
MINIO_ACCESS_KEY=$(openssl rand -base64 20 | tr -d '\n\r ')
MINIO_SECRET_KEY=$(openssl rand -base64 40 | tr -d '\n\r ')
JWT_SECRET=$(openssl rand -base64 64 | tr -d '\n\r ')

# Add to .env file
echo "DB_PASSWORD=$DB_PASSWORD" >> .env
echo "MINIO_ACCESS_KEY=$MINIO_ACCESS_KEY" >> .env
echo "MINIO_SECRET_KEY=$MINIO_SECRET_KEY" >> .env
echo "JWT_SECRET=$JWT_SECRET" >> .env

# Secure the file
chmod 600 .env

Update Caddy Configuration

# Add Banatie routes to Caddyfile
sudo nano /opt/services/configs/caddy/Caddyfile
# Add the Banatie configuration from above

# Reload Caddy
cd /opt/services
./manage-services.sh caddy reload

Deploy Services

cd /opt/banatie

# Build and start services
docker-compose up -d

# Verify deployment
docker-compose ps
docker-compose logs -f

Verify Deployment

# Check health endpoints
curl https://banatie.app/health
curl https://minio.banatie.app/minio/health/live

# Check logs
docker-compose logs banatie-app
docker-compose logs banatie-postgres
docker-compose logs banatie-minio

Manual Update Process

# SSH to VPS
ssh usul-vps
cd /opt/banatie

# Pull latest changes
git pull origin main

# Rebuild and restart
docker-compose build banatie-app
docker-compose up -d

# Verify update
docker-compose logs -f banatie-app
curl https://banatie.app/health

Monitoring and Maintenance

Health Checks

Application Health Check (/health):

{
  "status": "healthy",
  "timestamp": "2024-01-01T12:00:00.000Z",
  "services": {
    "database": "connected",
    "minio": "connected",
    "gemini_api": "accessible"
  },
  "version": "1.0.0"
}

MinIO Health Check (/minio/health/live):

Returns 200 OK when MinIO is operational
Used by Docker healthcheck and monitoring

Log Management

Application Logs:

Location: /opt/banatie/logs/
Format: Structured JSON
Rotation: Daily with 30-day retention

Access Logs:

Caddy logs: /opt/services/logs/banatie_*.log
Format: JSON with request/response details

Backup Strategy

Database Backup:

# Create backup
docker exec banatie-postgres pg_dump -U banatie_user banatie_db > banatie_db_backup.sql

# Restore backup
docker exec -i banatie-postgres psql -U banatie_user banatie_db < banatie_db_backup.sql

MinIO Data Backup:

# Backup MinIO data
sudo tar -czf banatie_minio_backup.tar.gz -C /opt/banatie/data minio/

# Restore MinIO data
sudo tar -xzf banatie_minio_backup.tar.gz -C /opt/banatie/data

Security Considerations

Container Security

Non-root users in containers
Read-only root filesystems where possible
Resource limits (memory, CPU)
Health checks for automatic restart

Network Security

Internal network isolation
No direct external access to database or MinIO
Rate limiting at proxy level
HTTPS-only external access

Data Security

Encrypted environment variables
Secure secret generation
Regular credential rotation
Audit logging

Access Control

MinIO bucket policies per organization
PostgreSQL row-level security
JWT-based API authentication
Role-based access control

Future Scalability

Horizontal Scaling Options

Multiple Banatie app containers behind load balancer
MinIO distributed mode for storage scaling
PostgreSQL read replicas for read scaling
Redis for session storage and caching

Multi-Region Deployment

Regional MinIO clusters
Database replication
CDN integration for static assets
Geo-distributed deployment

Troubleshooting

Common Issues

Container Won't Start:

# Check logs
docker-compose logs [service-name]

# Check resource usage
docker stats

# Rebuild container
docker-compose build [service-name]
docker-compose up -d [service-name]

Database Connection Issues:

# Test database connectivity
docker exec banatie-postgres pg_isready -U banatie_user -d banatie_db

# Check environment variables
docker exec banatie-app env | grep DB_

# Restart database
docker-compose restart banatie-postgres

MinIO Access Issues:

# Check MinIO status
docker exec banatie-minio mc admin info local

# Test S3 API
curl -f http://localhost:9000/minio/health/live

# Check access keys
docker exec banatie-app env | grep MINIO_

Network Connectivity:

# Check networks
docker network ls
docker network inspect banatie_banatie-network

# Test internal connectivity
docker exec banatie-app ping banatie-postgres
docker exec banatie-app ping banatie-minio

Document Version: 1.0 Last Updated: 2024-09-26 Maintained By: Banatie Development Team

23 KiB Raw Blame History