8.2 KiB

Raw Blame History

Banatie API Test Suite - Summary & Known Issues

Last Updated: 2025-11-18 API Version: v1 Test Suite Version: 1.0

📊 Test Suite Overview

Test File	Tests	Status	Description
01-generation-basic.ts	~8	✅ Expected to pass	Basic image generation functionality
02-basic.ts	~15	✅ Expected to pass	Image upload, CRUD operations
03-flows.ts	~10	✅ Expected to pass	Flow lifecycle and management
04-aliases.ts	~12	✅ Expected to pass	3-tier alias resolution system
05-live.ts	~10	✅ Expected to pass	Live URLs, scopes, caching
06-edge-cases.ts	~15	✅ Expected to pass	Validation and error handling
07-known-issues.ts	~4	❌ Expected to fail	Known implementation issues

Total Tests: ~74 Expected Pass: ~70 Expected Fail: ~4

🚫 Skipped Tests

These tests are intentionally NOT implemented because the functionality doesn't exist or isn't needed:

1. Manual Flow Creation

Endpoint: POST /api/v1/flows Reason: Endpoint removed from implementation. Flows use lazy/eager creation pattern via generation/upload. Impact: Tests must create flows via flowAlias parameter or rely on auto-generated flowIds.

2. CDN Flow Context

Test: Get CDN image with flowId query parameter Endpoint: GET /cdn/:org/:project/img/@alias?flowId={uuid} Reason: CDN endpoints don't support flowId context for flow-scoped alias resolution. Impact: CDN can only resolve project-scoped aliases, not flow-scoped.

3. Image Transformations & Cloudflare

Tests: Any transformation-related validation Reason: No image transformation service or Cloudflare CDN in test environment. Impact: All images served directly from MinIO without modification.

4. Test 10.3 - URL Encoding with Underscores

Test: Live URL with underscores in prompt (beautiful_sunset) Reason: Edge case not critical for core functionality. Status: Add to future enhancement list if URL encoding issues arise.

5. Concurrent Operations Tests (14.1-14.3)

Tests:

Concurrent generations in same flow
Concurrent alias assignments
Concurrent cache access

Reason: Complex timing requirements, potential flakiness, not critical for initial validation. Status: Consider adding later for stress testing.

❌ Known Implementation Issues

These tests are implemented in 07-known-issues.ts and are expected to fail. They document bugs/missing features in the current implementation.

Issue #1: Project Aliases on Flow Images

Test: Generate image in flow with project-scoped alias Expected: Image should be accessible via project alias even when associated with a flow Current Behavior: AliasService.resolveProjectAlias() has isNull(images.flowId) constraint Impact: Images within flows cannot have project-scoped aliases File: apps/api-service/src/services/core/AliasService.ts:125 Fix Required: Remove isNull(images.flowId) condition from project alias resolution

Issue #2: Flow Cascade Delete - Non-Aliased Images

Test: Delete flow, verify non-aliased images are deleted Expected: Images without project aliases should be cascade deleted Current Behavior: Flow deletion only deletes flow record, leaves all images intact Impact: Orphaned images remain in database File: apps/api-service/src/services/core/FlowService.ts (delete method) Fix Required: Add cascade logic to delete images where alias IS NULL

Issue #3: Flow Cascade Delete - Aliased Images Protected

Test: Delete flow, verify aliased images are preserved Expected: Images with project aliases should remain (flowId set to null) Current Behavior: Images remain but keep flowId reference Impact: Aliased images remain associated with deleted flow File: apps/api-service/src/services/core/FlowService.ts (delete method) Fix Required: Set flowId = NULL for preserved images with aliases

Issue #4: Flow Cascade Delete - Generations

Test: Delete flow, verify generations are deleted Expected: All generations in flow should be cascade deleted Current Behavior: Generations remain with flowId intact Impact: Orphaned generations in database File: apps/api-service/src/services/core/FlowService.ts (delete method) Fix Required: Add cascade deletion for generations in flow

📋 Implementation Notes & Discrepancies

Alias Resolution Endpoint Mismatch

Test Requirements: GET /api/v1/images/@alias Actual Implementation: GET /api/v1/images/resolve/@alias Action: Tests use actual endpoint. Consider adding /images/@alias as shorthand in future.

IP Rate Limiting on Live URLs

Location: apps/api-service/src/routes/cdn.ts (live URL endpoint) Current Behavior: IP-based rate limiting (10 new generations per hour) Action Required: Remove IP rate limiting functionality from live URL endpoints Priority: Medium (functional but may cause issues in production)

Prompt Auto-Enhancement

Feature: autoEnhance parameter in generation endpoint Status: Implemented but not extensively tested Action: Add comprehensive tests for enhancement behavior:

Verify originalPrompt populated when enhanced
Verify prompt contains enhanced version
Verify enhancement doesn't occur when autoEnhance=false

Alias Assignment Endpoints

Note: Alias assignment is separated from general metadata updates Correct Behavior:

PUT /api/v1/images/:id - Update focalPoint, meta only
PUT /api/v1/images/:id/alias - Dedicated alias assignment endpoint

Benefit: Better separation of concerns, clearer API semantics

🧪 Required Test Fixtures

Current Fixtures

✅ tests/api/fixture/test-image.png (1.6MB PNG)

Additional Fixtures Needed

(To be determined during test implementation)

Small image (<1MB) for quick upload tests
Large image (>5MB) for size limit validation
JPEG file for format variety testing
Multiple distinct images for reference testing
Invalid file types (.txt, .pdf) for negative tests

Status: Will be generated/collected after initial test implementation.

🔧 Test Environment Requirements

Services Required

✅ API service running on http://localhost:3000
✅ PostgreSQL database with schema v2.0
✅ MinIO storage accessible and configured
✅ Valid project API key configured in config.ts
✅ Google Gemini API credentials (will consume credits)

Database State

Tests assume empty or minimal database
Tests do NOT clean up data (by design)
Run against dedicated test project, not production

Performance Notes

Each image generation: ~3-10 seconds (Gemini API)
Full test suite: ~20-30 minutes
Gemini API cost: ~70-80 generations @ $0.0025 each = ~$0.18-0.20

📈 Test Execution Commands

# Run full test suite (sequential)
cd tests/api
tsx run-all.ts

# Run individual test files
tsx 01-generation-basic.ts
tsx 02-basic.ts
tsx 03-flows.ts
tsx 04-aliases.ts
tsx 05-live.ts
tsx 06-edge-cases.ts
tsx 07-known-issues.ts

# Expected output: Colored console with ✓ (pass) and ✗ (fail) indicators

🎯 Success Criteria

All test files execute without crashes
Tests 01-06: ~70 tests pass (verify correct implementation)
Test 07: ~4 tests fail (document known issues)
Each test has clear assertions and error messages
Tests use real API calls (no mocks)
All generated images saved to tests/api/results/
Summary document maintained and accurate

📝 Maintenance Notes

Updating Tests

When API implementation is fixed:

Move tests from 07-known-issues.ts to appropriate test file
Update this summary document
Re-run full test suite to verify fixes

Adding New Tests

Choose appropriate test file based on feature area
Follow existing test patterns (runTest, clear assertions)
Update test count in Overview table
Document any new fixtures needed

Known Limitations

Tests are not idempotent (leave data in database)
No parallel execution support
No automated cleanup between runs
Requires manual server startup

Document Status: ✅ Complete Next Update: After test implementation and first full run

8.2 KiB Raw Blame History