30 KiB

Raw Blame History

Banatie API - Comprehensive Testing Requirements

Version: 1.0
Target: Claude Code
Scope: Complete API + DB validation
Database Schema: v2.0
API Requirements: v2.0 + Final Refactoring

🎯 Purpose

This document provides comprehensive testing requirements for the Banatie API service. Tests must validate actual running service with real HTTP requests, actual file uploads/downloads, and real image generation - NO mocks, stubs, or placeholders.

Critical Requirements:

Tests call real running API service at http://localhost:3000
All file operations use real files and MinIO storage
Image generation uses real Google Gemini API
Database operations use real PostgreSQL instance
Test actual data flows from request → processing → storage → retrieval

📁 Test Structure

Tests are organized in /tests/api/ folder:

tests/api/
├── config.ts              # API configuration, endpoints, test settings
├── utils.ts               # Helper functions (api calls, file upload, etc.)
├── fixture/
│   └── test-image.png    # Test image file
├── 01-basic.ts           # Basic CRUD operations
├── 02-flows.ts           # Flow management and lifecycle
├── 03-aliases.ts         # Alias resolution and management
├── 04-live.ts            # Live URLs and caching
├── 05-edge-cases.ts      # Error handling and edge cases
└── run-all.ts            # Test runner script

Existing Patterns to Follow:

Use api() helper for HTTP requests
Use uploadFile() for multipart uploads
Use waitForGeneration() for polling
Use testContext to share data between tests
Use runTest() for consistent test execution

🔧 Test Environment Setup

Prerequisites

API service running on http://localhost:3000
PostgreSQL database initialized with schema v2.0
MinIO storage accessible and configured
Valid API key in config.ts
Google Gemini API credentials configured

Test Data Requirements

Test image file: tests/api/fixture/test-image.png (PNG, ~100KB)
Additional test images for reference scenarios
Valid project context from API key

📋 Test Coverage Matrix

Coverage Areas

Images - Upload, CRUD, alias management, resolution
Generations - Create, status tracking, regenerate, parameters
Flows - Lifecycle, lazy creation, alias management
Aliases - 3-tier resolution, technical aliases, conflicts
Live URLs - Scopes, caching, generation
Reference Images - Manual specification, auto-detection
CDN Endpoints - Image delivery, alias resolution
Error Handling - Validation, not found, conflicts

📝 Detailed Test Scenarios

Test File: 01-basic.ts

Purpose: Validate core CRUD operations and basic flows

TEST GROUP 1: Image Upload and Management

Test 1.1: Upload image with project alias

Purpose: Verify basic image upload with project-scoped alias
Steps:
1. Upload test-image.png with alias "@test-logo"
2. Verify response contains imageId, storageKey, storageUrl
3. Verify alias is set correctly
4. Verify source = "uploaded"
5. Save imageId to testContext.uploadedImageId
Expected:
- HTTP 201
- Valid image record
- File accessible in MinIO
- Alias "@test-logo" assigned

Test 1.2: Upload image without alias

Purpose: Verify upload works without alias assignment
Steps:
1. Upload test image without alias parameter
2. Verify response contains image data
3. Verify alias is null
Expected:
- HTTP 201
- Valid image record
- No alias assigned

Test 1.3: List uploaded images

Purpose: Verify image listing endpoint
Steps:
1. GET /api/v1/images
2. Verify response is array
3. Verify pagination data present
4. Verify uploaded images included
5. Check source = "uploaded" filter works
Expected:
- HTTP 200
- Array of images
- Pagination metadata

Test 1.4: Get image by ID

Purpose: Verify image retrieval by UUID
Steps:
1. GET /api/v1/images/{uploadedImageId}
2. Verify all image fields present
3. Verify storageUrl is accessible
Expected:
- HTTP 200
- Complete image data
- Accessible storage URL

Test 1.5: Get image by alias (project-scoped)

Purpose: Verify image retrieval by alias
Steps:
1. GET /api/v1/images/@test-logo
2. Verify returns same image as by ID
3. Verify alias resolution works
Expected:
- HTTP 200
- Correct image data

Test 1.6: Update image metadata

Purpose: Verify image metadata updates
Steps:
1. PUT /api/v1/images/{imageId}
2. Update description, tags, focalPoint
3. GET image again to verify changes
Expected:
- HTTP 200
- Updated fields reflected

Test 1.7: Update image alias

Purpose: Verify alias assignment/change via dedicated endpoint
Steps:
1. PUT /api/v1/images/{imageId}/alias
2. Change alias from "@test-logo" to "@new-logo"
3. GET /api/v1/images/@new-logo
4. Verify old alias no longer works
Expected:
- HTTP 200
- New alias works
- Old alias returns 404

Test 1.8: Remove image alias

Purpose: Verify alias removal
Steps:
1. PUT /api/v1/images/{imageId}/alias with { alias: null }
2. Verify image exists but has no alias
3. Verify alias query returns 404
Expected:
- HTTP 200
- Image exists without alias
- Alias query fails properly

TEST GROUP 2: Basic Image Generation

Test 2.1: Generate image without references (simple)

Purpose: Verify basic generation functionality
Steps:
1. POST /api/v1/generations
   {
     prompt: "A beautiful sunset over mountains",
     aspectRatio: "16:9"
   }
2. Verify generation record created
3. Poll for completion using waitForGeneration()
4. Verify status = "success"
5. Verify outputImageId present
6. Download and save generated image
7. Verify image exists in MinIO
Expected:
- HTTP 200 on creation
- Generation completes successfully
- Output image accessible
- processingTimeMs > 0

Test 2.2: Generate with manual reference image

Purpose: Verify generation with explicitly specified reference
Steps:
1. POST /api/v1/generations
   {
     prompt: "A product photo with the logo in corner",
     referenceImages: ["@test-logo"],
     aspectRatio: "1:1"
   }
2. Wait for completion
3. Verify referencedImages field contains correct data
4. Verify output image generated
Expected:
- HTTP 200
- Generation successful
- Referenced images tracked correctly

Test 2.3: Generate with auto-detected references

Purpose: Verify automatic alias detection in prompts
Steps:
1. POST /api/v1/generations
   {
     prompt: "Create product image using @test-logo and @brand-colors",
     aspectRatio: "4:3"
   }
   # Note: referenceImages NOT provided
2. Verify system auto-detected both aliases
3. Check referencedImages field
Expected:
- HTTP 200
- Both aliases auto-detected
- Generation uses both references

Test 2.4: List generations

Purpose: Verify generation listing with filters
Steps:
1. GET /api/v1/generations
2. Verify array returned
3. Test pagination parameters
4. Test status filter
5. Test sortBy and order
Expected:
- HTTP 200
- Filtered results
- Pagination works

Test 2.5: Get generation details

Purpose: Verify detailed generation retrieval
Steps:
1. GET /api/v1/generations/{generationId}
2. Verify includes:
   - Generation data
   - Output image data
   - Referenced images array
   - Processing metrics
Expected:
- HTTP 200
- Complete generation details

TEST GROUP 3: Generation with Aliases

Test 3.1: Generate with project alias assignment

Purpose: Verify alias parameter assigns project-scoped alias
Steps:
1. POST /api/v1/generations
   {
     prompt: "Brand header image",
     aspectRatio: "21:9",
     alias: "@header"
   }
2. Wait for completion
3. Verify output image has alias "@header"
4. GET /api/v1/images/@header
5. Verify returns generated image
Expected:
- HTTP 200
- Alias assigned to output image
- Alias resolution works

Test 3.2: Alias conflict resolution

Purpose: Verify new generation overwrites existing alias
Steps:
1. Generate image with alias "@hero"
2. Generate another image with same alias "@hero"
3. Verify second generation overwrites
4. Verify first image still exists but without alias
5. Verify "@hero" resolves to second image
Expected:
- Both generations successful
- Second image gets alias
- First image loses alias
- Both images preserved

Test File: 02-flows.ts

Purpose: Validate flow lifecycle and lazy creation patterns

TEST GROUP 4: Flow Lazy Creation

Test 4.1: Generate without flowId returns flowId

Purpose: Verify lazy flow pattern - generation without flowId gets one
Steps:
1. POST /api/v1/generations (no flowId parameter)
2. Verify response includes generated flowId
3. Verify flowId is valid UUID format
4. Verify flow NOT yet in database (lazy)
Expected:
- HTTP 200
- flowId present in response
- Flow record not created yet

Test 4.2: Second request with flowId creates flow

Purpose: Verify flow created on second use
Steps:
1. Get flowId from previous generation
2. POST /api/v1/generations with this flowId
3. Verify flow now exists in database
4. GET /api/v1/flows/{flowId}
5. Verify flow contains both generations
Expected:
- HTTP 200
- Flow record created
- Both generations linked

Test 4.3: Flow created immediately with flowAlias

Purpose: Verify eager creation when flowAlias present
Steps:
1. POST /api/v1/generations
   {
     prompt: "Hero image",
     flowAlias: "@hero"
   }
2. Verify response includes flowId
3. GET /api/v1/flows/{flowId}
4. Verify flow exists immediately
5. Verify flow.aliases contains "@hero"
Expected:
- HTTP 200
- Flow created immediately
- Flow alias set

Test 4.4: Upload with flowId and flowAlias

Purpose: Verify uploads work with flow association
Steps:
1. Upload image with:
   - flowId: (from previous test)
   - flowAlias: "@upload-test"
2. Verify image linked to flow
3. GET flow details
4. Verify image appears in flow
5. Verify flowAlias in flow.aliases
Expected:
- HTTP 201
- Image linked to flow
- Flow alias set

TEST GROUP 5: Flow Management

Test 5.1: List flows

Purpose: Verify flow listing endpoint
Steps:
1. GET /api/v1/flows
2. Verify array returned
3. Check computed fields:
   - generationCount
   - imageCount
4. Test pagination
Expected:
- HTTP 200
- Array of flows with counts

Test 5.2: Get flow details with generations

Purpose: Verify complete flow data retrieval
Steps:
1. GET /api/v1/flows/{flowId}
2. Verify response includes:
   - Flow metadata
   - All generations (chronological)
   - All images
   - Resolved aliases
Expected:
- HTTP 200
- Complete flow data
- Chronological order maintained

Test 5.3: Update flow aliases

Purpose: Verify manual alias updates
Steps:
1. PUT /api/v1/flows/{flowId}/aliases
   {
     aliases: {
       "@custom": "image-uuid",
       "@another": "image-uuid-2"
     }
   }
2. GET flow details
3. Verify aliases updated
Expected:
- HTTP 200
- Aliases persisted correctly

Test 5.4: Remove specific flow alias

Purpose: Verify alias deletion
Steps:
1. DELETE /api/v1/flows/{flowId}/aliases/@custom
2. GET flow details
3. Verify alias removed
4. Verify other aliases intact
Expected:
- HTTP 204
- Specified alias removed
- Other aliases preserved

Test 5.5: Delete flow with cascade

Purpose: Verify flow deletion behavior
Steps:
1. Create flow with:
   - Generation with no alias
   - Generation with project alias
   - Upload with no alias
   - Upload with project alias
2. DELETE /api/v1/flows/{flowId}
3. Verify:
   - Flow record deleted
   - Non-aliased images deleted
   - Aliased images preserved (flowId = null)
   - Generations deleted
Expected:
- HTTP 204
- Correct cascade behavior
- Aliased resources protected

Test File: 03-aliases.ts

Purpose: Validate 3-tier alias resolution system

TEST GROUP 6: Technical Aliases

Test 6.1: @last alias resolution

Purpose: Verify @last resolves to most recent generation
Steps:
1. Create flow with flowId
2. Generate image A
3. Generate image B
4. Generate image C
5. GET /api/v1/images/resolve/@last?flowId={flowId}
6. Verify returns image C
Expected:
- HTTP 200
- Returns most recent generation
- Correct scope indicated

Test 6.2: @first alias resolution

Purpose: Verify @first resolves to first generation
Steps:
1. Using same flow from Test 6.1
2. GET /api/v1/images/resolve/@first?flowId={flowId}
3. Verify returns image A
Expected:
- HTTP 200
- Returns first generation

Test 6.3: @upload alias resolution

Purpose: Verify @upload resolves to last upload
Steps:
1. Upload image X to flow
2. Generate image Y
3. Upload image Z to flow
4. GET /api/v1/images/resolve/@upload?flowId={flowId}
5. Verify returns image Z
Expected:
- HTTP 200
- Returns last uploaded image

Test 6.4: Technical alias in generation prompt

Purpose: Verify technical aliases work in prompt
Steps:
1. POST /api/v1/generations
   {
     prompt: "New variation based on @last",
     flowId: "{flowId}"
   }
2. Verify @last resolved correctly
3. Verify referencedImages contains correct imageId
Expected:
- HTTP 200
- Technical alias resolved
- Correct reference used

TEST GROUP 7: Alias Priority and Resolution

Test 7.1: Flow alias overrides project alias

Purpose: Verify flow-scoped takes precedence over project-scoped
Steps:
1. Create image with project alias "@logo"
2. Create flow
3. Upload different image with flowAlias "@logo" in this flow
4. GET /api/v1/images/resolve/@logo?flowId={flowId}
5. Verify returns flow-scoped image
6. GET /api/v1/images/resolve/@logo (no flowId)
7. Verify returns project-scoped image
Expected:
- Flow-scoped has priority when flowId provided
- Project-scoped used when no flowId

Test 7.2: Technical alias highest priority

Purpose: Verify technical aliases override user aliases
Steps:
1. Create flow
2. Upload image with flowAlias "@last"
3. Generate image (becomes actual @last)
4. GET /api/v1/images/resolve/@last?flowId={flowId}
5. Verify returns generated image, not uploaded
Expected:
- Technical @last takes priority
- User-assigned "@last" ignored

Test 7.3: Alias resolution without flow context

Purpose: Verify project-scoped-only resolution
Steps:
1. GET /api/v1/images/resolve/@logo (no flowId param)
2. Verify returns project-scoped alias
3. Verify scope = "project"
Expected:
- HTTP 200
- Project alias resolved

Test 7.4: Reserved alias validation

Purpose: Verify reserved aliases rejected
Steps:
1. Try to assign alias "@last" to image
2. Try to assign alias "@first" to image
3. Try to assign alias "@upload" to image
Expected:
- HTTP 400 for all
- Clear error messages
- Validation prevents reserved aliases

TEST GROUP 8: Alias Conflicts

Test 8.1: Project alias reassignment

Purpose: Verify alias can be moved between images
Steps:
1. Upload image A with alias "@product"
2. Verify A has alias
3. Upload image B with alias "@product"
4. Verify B now has alias
5. Verify A exists but alias is null
6. GET /api/v1/images/@product
7. Verify returns image B
Expected:
- Alias successfully moved
- Old image preserved without alias

Test 8.2: Flow alias reassignment

Purpose: Verify flow-scoped alias reassignment
Steps:
1. Create flow
2. Upload image X with flowAlias "@hero"
3. Upload image Y with same flowAlias "@hero"
4. GET flow details
5. Verify flow.aliases["@hero"] = imageY.id
6. Verify imageX still exists in flow
Expected:
- Flow alias reassigned
- Both images in flow

Test 8.3: Same alias in different flows

Purpose: Verify flow isolation for aliases
Steps:
1. Create flowA, upload image with flowAlias "@hero"
2. Create flowB, upload different image with flowAlias "@hero"
3. Resolve @hero in flowA
4. Resolve @hero in flowB
5. Verify different images returned
Expected:
- Same alias works independently in different flows
- Correct isolation maintained

Test File: 04-live.ts

Purpose: Validate live URL system and caching

TEST GROUP 9: Live Scope Management

Test 9.1: Create scope manually

Purpose: Verify manual scope creation
Steps:
1. POST /api/v1/live/scopes
   {
     slug: "hero-section",
     allowNewGenerations: true,
     newGenerationsLimit: 50
   }
2. Verify scope created
3. GET /api/v1/live/scopes
4. Verify scope in list
Expected:
- HTTP 201
- Scope created with settings

Test 9.2: List scopes with stats

Purpose: Verify scope listing includes usage stats
Steps:
1. GET /api/v1/live/scopes
2. Verify each scope includes:
   - currentGenerations count
   - lastGeneratedAt timestamp
   - Settings (allowNewGenerations, limit)
Expected:
- HTTP 200
- Complete scope data with stats

Test 9.3: Get scope details

Purpose: Verify detailed scope retrieval
Steps:
1. GET /api/v1/live/scopes/hero-section
2. Verify includes:
   - Scope settings
   - Usage statistics
   - List of images in scope
Expected:
- HTTP 200
- Complete scope information

Test 9.4: Update scope settings

Purpose: Verify scope configuration changes
Steps:
1. PUT /api/v1/live/scopes/hero-section
   {
     allowNewGenerations: false,
     newGenerationsLimit: 100
   }
2. GET scope details
3. Verify settings updated
Expected:
- HTTP 200
- Settings persisted

TEST GROUP 10: Live URL Generation and Caching

Test 10.1: First live URL request (cache miss)

Purpose: Verify live URL triggers generation on first hit
Steps:
1. GET /cdn/{org}/{project}/live/hero-section?prompt=sunset&aspectRatio=16:9
2. Verify:
   - Generation triggered
   - Image returned
   - Headers include X-Cache-Status: MISS
   - Content-Type: image/jpeg
3. Verify cache entry created in database
Expected:
- HTTP 200
- Image bytes returned
- Cache miss indicated
- Cache entry persisted

Test 10.2: Second live URL request (cache hit)

Purpose: Verify caching works on subsequent requests
Steps:
1. GET same URL as Test 10.1
2. Verify:
   - Same image returned immediately
   - X-Cache-Status: HIT
   - No new generation triggered
3. Check cache hit_count incremented
Expected:
- HTTP 200
- Cached image returned
- Cache hit recorded

Test 10.3: Live URL with underscores in prompt

Purpose: Verify URL encoding flexibility
Steps:
1. GET /cdn/{org}/{project}/live/test-scope?prompt=beautiful_sunset
2. Verify works same as %20 encoding
3. Generate with both formats
4. Verify same cache key used
Expected:
- HTTP 200
- Both formats work
- Same cached result

Test 10.4: Live URL scope auto-creation

Purpose: Verify new scope created if allowNewLiveScopes=true
Steps:
1. Ensure project allows new scopes
2. GET /cdn/{org}/{project}/live/new-auto-scope?prompt=test
3. Verify scope auto-created
4. GET /api/v1/live/scopes
5. Verify new-auto-scope in list
Expected:
- HTTP 200
- Scope created automatically
- Generation successful

Test 10.5: Live URL generation limit enforcement

Purpose: Verify scope limits respected
Steps:
1. Create scope with newGenerationsLimit: 2
2. Make 2 live URL requests with different prompts
3. Verify both work
4. Make 3rd request with new prompt
5. Verify rejected with 429
Expected:
- First 2 succeed
- 3rd request fails
- Error indicates limit exceeded

Test 10.6: Live URL with disabled scope

Purpose: Verify allowNewGenerations setting enforced
Steps:
1. Create scope with allowNewGenerations: false
2. Add one cached image to scope
3. Request cached image (should work)
4. Request new image (should fail)
Expected:
- Cached images accessible
- New generations blocked
- HTTP 403 for new generation

Test 10.7: Regenerate scope image

Purpose: Verify scope image regeneration
Steps:
1. POST /api/v1/live/scopes/{slug}/regenerate
   { imageId: "uuid" }
2. Verify image regenerated
3. Verify cache updated
4. GET live URL for that image
5. Verify new version returned
Expected:
- HTTP 200
- Image regenerated
- Cache updated

Test File: 05-edge-cases.ts

Purpose: Validate error handling and edge cases

TEST GROUP 11: Validation and Errors

Test 11.1: Invalid alias format

Purpose: Verify alias validation
Steps:
1. Try alias without @ symbol
2. Try alias with spaces
3. Try alias with special chars
4. Try empty alias
Expected:
- HTTP 400 for all
- Clear validation errors

Test 11.2: Invalid aspectRatio

Purpose: Verify aspect ratio validation
Steps:
1. POST generation with aspectRatio: "invalid"
2. POST generation with aspectRatio: "99:1"
3. POST generation with aspectRatio: ""
Expected:
- HTTP 400
- Validation error messages

Test 11.3: Missing required fields

Purpose: Verify required field validation
Steps:
1. POST generation without prompt
2. POST upload without file
3. POST scope without slug
Expected:
- HTTP 400
- Field-specific errors

Test 11.4: Non-existent resource access

Purpose: Verify 404 handling
Steps:
1. GET /api/v1/images/{fake-uuid}
2. GET /api/v1/generations/{fake-uuid}
3. GET /api/v1/flows/{fake-uuid}
4. GET /api/v1/images/@nonexistent
Expected:
- HTTP 404 for all
- Clear error messages

Test 11.5: File upload size limit

Purpose: Verify file size validation
Steps:
1. Create file > 5MB
2. Try to upload
Expected:
- HTTP 400
- File size error

Test 11.6: Invalid file type

Purpose: Verify MIME type validation
Steps:
1. Upload .txt file as image
2. Upload .pdf file
Expected:
- HTTP 400
- File type error

TEST GROUP 12: Regenerate Functionality

Test 12.1: Regenerate successful generation

Purpose: Verify regeneration works for any status
Steps:
1. Create successful generation
2. POST /api/v1/generations/{id}/regenerate
3. Verify:
   - Same imageId used
   - File overwritten in MinIO
   - updatedAt changed
   - createdAt preserved
Expected:
- HTTP 200
- Image updated in place

Test 12.2: Regenerate failed generation

Purpose: Verify regeneration works for failed
Steps:
1. Create generation that fails (bad prompt or force fail)
2. POST regenerate
3. Verify new attempt made
Expected:
- HTTP 200
- New generation attempt

Test 12.3: Regenerate preserves aliases

Purpose: Verify aliases maintained on regenerate
Steps:
1. Create generation with alias "@test"
2. Regenerate
3. Verify alias still "@test"
4. Verify alias resolution works
Expected:
- Alias preserved
- Resolution unchanged

Test 12.4: Regenerate flow's last generation

Purpose: Verify flow regenerate endpoint
Steps:
1. Create flow with 3 generations
2. POST /api/v1/flows/{flowId}/regenerate
3. Verify last generation regenerated
Expected:
- HTTP 200
- Last generation updated

Test 12.5: Regenerate empty flow

Purpose: Verify error handling for empty flow
Steps:
1. Create empty flow (or delete all generations)
2. POST /api/v1/flows/{flowId}/regenerate
Expected:
- HTTP 400 or 404
- Error: "Flow has no generations"

TEST GROUP 13: CDN Endpoints

Test 13.1: CDN image by filename

Purpose: Verify CDN endpoint for filename
Steps:
1. Upload image, note filename from storageKey
2. GET /cdn/{org}/{project}/img/{filename}
3. Verify image bytes returned
Expected:
- HTTP 200
- Image content
- Proper headers

Test 13.2: CDN image by alias

Purpose: Verify CDN endpoint with alias
Steps:
1. Upload image with alias "@cdn-test"
2. GET /cdn/{org}/{project}/img/@cdn-test
3. Verify image returned
Expected:
- HTTP 200
- Correct image
- Cache headers

Test 13.3: CDN alias priority

Purpose: Verify alias takes precedence over filename
Steps:
1. Create image with filename "test.png" and alias "@test"
2. Create another image with filename "test.png" but no alias
3. GET /cdn/{org}/{project}/img/@test
4. Verify returns aliased image
Expected:
- Alias resolution preferred
- Correct image returned

Test 13.4: CDN with flowId context

Purpose: Verify flow-scoped CDN resolution
Steps:
1. Create flow with flowAlias "@flow-cdn"
2. GET /cdn/{org}/{project}/img/@flow-cdn?flowId={id}
3. Verify flow-scoped image returned
Expected:
- HTTP 200
- Flow context respected

TEST GROUP 14: Concurrent Operations

Test 14.1: Concurrent generations in same flow

Purpose: Verify race conditions handled
Steps:
1. Start 3 generations simultaneously with same flowId
2. Wait for all to complete
3. Verify all succeed
4. Verify flow contains all 3
Expected:
- All generations successful
- No data corruption
- Flow updated_at correct

Test 14.2: Concurrent alias assignments

Purpose: Verify last-write-wins for aliases
Steps:
1. Start 2 uploads with same alias "@race"
2. Wait for completion
3. Verify one has alias
4. Verify other doesn't
5. Verify both images exist
Expected:
- No errors
- One image gets alias
- Both images preserved

Test 14.3: Concurrent cache access

Purpose: Verify cache hit_count increments correctly
Steps:
1. Make same live URL request 10 times concurrently
2. Verify all return image
3. Check cache hit_count
4. Verify count = 9 or 10 (first might be miss)
Expected:
- All requests succeed
- hit_count accurate

TEST GROUP 15: Data Integrity

Test 15.1: Generation with originalPrompt tracking

Purpose: Verify prompt enhancement tracking
Steps:
1. POST generation with autoEnhance: true
2. Verify response has both:
   - prompt (enhanced, used for generation)
   - originalPrompt (user input)
3. POST generation with autoEnhance: false
4. Verify prompt present, originalPrompt null
Expected:
- Correct field population
- originalPrompt only when enhanced

Test 15.2: Referenced images stored correctly

Purpose: Verify referencedImages JSONB format
Steps:
1. Generate with multiple references
2. GET generation details
3. Verify referencedImages array format:
   [{ imageId: "uuid", alias: "@name" }, ...]
Expected:
- Correct JSONB structure
- All references tracked

Test 15.3: Storage consistency check

Purpose: Verify DB and MinIO stay in sync
Steps:
1. Create generation
2. Check image in DB
3. Check file in MinIO (via storageUrl)
4. Delete image
5. Verify DB record gone
6. Verify MinIO file gone
Expected:
- DB and storage consistent
- No orphaned files
- No orphaned records

Test 15.4: Cascade delete verification

Purpose: Verify all cascade rules work
Steps:
1. Create complex structure:
   - Flow with generations and images
   - Mixed aliased/non-aliased
2. Delete flow
3. Query all related records
4. Verify correct cascade behavior
Expected:
- Proper cascade execution
- Protected resources preserved
- All specified deletions complete

🛠️ Implementation Notes for Claude Code

Test Organization Principles

Independence: Each test should be self-contained and not rely on state from other tests unless explicitly designed as a test group
Cleanup: Consider whether cleanup is needed between test groups
Data Sharing: Use testContext object to share IDs and data within test groups
Polling: Use waitForGeneration() for async operations, with appropriate timeouts
Assertions: Verify both success cases and expected data structure/values

Helper Functions to Implement/Extend

Recommended additions to utils.ts:

// Poll for flow to exist (lazy creation check)
export async function checkFlowExists(flowId: string): Promise<boolean>

// Download and verify image from URL
export async function verifyImageAccessible(url: string): Promise<boolean>

// Create test flow with specific configuration
export async function createTestFlow(config?: Partial<Flow>): Promise<Flow>

// Generate test image with known properties
export async function generateTestImage(prompt: string, options?: GenerationOptions): Promise<Generation>

// Verify cache entry exists
export async function checkCacheEntry(projectId: string, promptHash: string): Promise<boolean>

// Clean up test data (optional, for cleanup between test groups)
export async function cleanupTestData(): Promise<void>

Error Handling Strategy

Tests should:

Expect specific HTTP status codes
Verify error response structure
Check error messages are meaningful
Validate that errors don't leave system in inconsistent state

Performance Considerations

Generation tests will be slow (Gemini API calls)
Use reasonable timeouts (60s for generation)
Consider running tests in parallel where safe
Group tests to minimize redundant operations

Test Execution Order

Recommended execution:

01-basic.ts - Foundation, creates test data
02-flows.ts - Flow functionality
03-aliases.ts - Alias system (uses data from #1-2)
04-live.ts - Live URLs and caching
05-edge-cases.ts - Error cases and edge scenarios

Documentation in Tests

Each test should include:

Clear purpose comment
Step-by-step description
Expected outcomes
Any special considerations

Success Criteria

Tests are considered complete when:

✅ All endpoint combinations covered
✅ All database schema features validated
✅ All API requirements from v2.0 tested
✅ All refactoring changes verified
✅ Error cases handled appropriately
✅ Real data flows validated (no mocks)
✅ Tests run against actual service
✅ Tests are documented and maintainable

🎯 Expected Test Statistics

Total Test Files: 5
Total Test Groups: ~15
Total Individual Tests: ~80-100
Estimated Runtime: 15-30 minutes (due to actual generations)

Coverage Targets:

Endpoints: 100% (all documented endpoints)
HTTP Methods: 100% (GET, POST, PUT, DELETE)
Error Cases: 90%+ (major validation and not-found scenarios)
Data Flows: 100% (all CRUD and generation flows)

📚 Reference Documents

Tests should validate functionality described in:

banatie-database-design.md - Database schema v2.0
banatie-api-requirements.md - API specification v2.0
api-refactoring-final.md - Final refactoring decisions

Document Version: 1.0
Created: 2024-11-17
Target Audience: Claude Code
Status: Ready for Implementation

30 KiB Raw Blame History