# Banatie API Test Suite - Summary & Known Issues

**Last Updated:** 2025-11-18
**API Version:** v1
**Test Suite Version:** 1.0

---

## 📊 Test Suite Overview

| Test File | Tests | Status | Description |
|-----------|-------|--------|-------------|
| 01-generation-basic.ts | ~8 | ✅ Expected to pass | Basic image generation functionality |
| 02-basic.ts | ~15 | ✅ Expected to pass | Image upload, CRUD operations |
| 03-flows.ts | ~10 | ✅ Expected to pass | Flow lifecycle and management |
| 04-aliases.ts | ~12 | ✅ Expected to pass | 3-tier alias resolution system |
| 05-live.ts | ~10 | ✅ Expected to pass | Live URLs, scopes, caching |
| 06-edge-cases.ts | ~15 | ✅ Expected to pass | Validation and error handling |
| 07-known-issues.ts | ~4 | ❌ Expected to fail | Known implementation issues |

**Total Tests:** ~74
**Expected Pass:** ~70
**Expected Fail:** ~4

---

## 🚫 Skipped Tests

These tests are intentionally NOT implemented because the functionality doesn't exist or isn't needed:

### 1. Manual Flow Creation
**Endpoint:** `POST /api/v1/flows`
**Reason:** Endpoint removed from implementation. Flows use lazy/eager creation pattern via generation/upload.
**Impact:** Tests must create flows via `flowAlias` parameter or rely on auto-generated flowIds.

### 2. CDN Flow Context
**Test:** Get CDN image with flowId query parameter
**Endpoint:** `GET /cdn/:org/:project/img/@alias?flowId={uuid}`
**Reason:** CDN endpoints don't support flowId context for flow-scoped alias resolution.
**Impact:** CDN can only resolve project-scoped aliases, not flow-scoped.

### 3. Image Transformations & Cloudflare
**Tests:** Any transformation-related validation
**Reason:** No image transformation service or Cloudflare CDN in test environment.
**Impact:** All images served directly from MinIO without modification.

### 4. Test 10.3 - URL Encoding with Underscores
**Test:** Live URL with underscores in prompt (`beautiful_sunset`)
**Reason:** Edge case not critical for core functionality.
**Status:** Add to future enhancement list if URL encoding issues arise.

### 5. Concurrent Operations Tests (14.1-14.3)
**Tests:**
- Concurrent generations in same flow
- Concurrent alias assignments
- Concurrent cache access

**Reason:** Complex timing requirements, potential flakiness, not critical for initial validation.
**Status:** Consider adding later for stress testing.

---

## ❌ Known Implementation Issues

These tests are implemented in `07-known-issues.ts` and are **expected to fail**. They document bugs/missing features in the current implementation.

### Issue #1: Project Aliases on Flow Images
**Test:** Generate image in flow with project-scoped alias
**Expected:** Image should be accessible via project alias even when associated with a flow
**Current Behavior:** `AliasService.resolveProjectAlias()` has `isNull(images.flowId)` constraint
**Impact:** Images within flows cannot have project-scoped aliases
**File:** `apps/api-service/src/services/core/AliasService.ts:125`
**Fix Required:** Remove `isNull(images.flowId)` condition from project alias resolution

### Issue #2: Flow Cascade Delete - Non-Aliased Images
**Test:** Delete flow, verify non-aliased images are deleted
**Expected:** Images without project aliases should be cascade deleted
**Current Behavior:** Flow deletion only deletes flow record, leaves all images intact
**Impact:** Orphaned images remain in database
**File:** `apps/api-service/src/services/core/FlowService.ts` (delete method)
**Fix Required:** Add cascade logic to delete images where `alias IS NULL`

### Issue #3: Flow Cascade Delete - Aliased Images Protected
**Test:** Delete flow, verify aliased images are preserved
**Expected:** Images with project aliases should remain (flowId set to null)
**Current Behavior:** Images remain but keep flowId reference
**Impact:** Aliased images remain associated with deleted flow
**File:** `apps/api-service/src/services/core/FlowService.ts` (delete method)
**Fix Required:** Set `flowId = NULL` for preserved images with aliases

### Issue #4: Flow Cascade Delete - Generations
**Test:** Delete flow, verify generations are deleted
**Expected:** All generations in flow should be cascade deleted
**Current Behavior:** Generations remain with flowId intact
**Impact:** Orphaned generations in database
**File:** `apps/api-service/src/services/core/FlowService.ts` (delete method)
**Fix Required:** Add cascade deletion for generations in flow

---

## 📋 Implementation Notes & Discrepancies

### Alias Resolution Endpoint Mismatch
**Test Requirements:** `GET /api/v1/images/@alias`
**Actual Implementation:** `GET /api/v1/images/resolve/@alias`
**Action:** Tests use actual endpoint. Consider adding `/images/@alias` as shorthand in future.

### IP Rate Limiting on Live URLs
**Location:** `apps/api-service/src/routes/cdn.ts` (live URL endpoint)
**Current Behavior:** IP-based rate limiting (10 new generations per hour)
**Action Required:** Remove IP rate limiting functionality from live URL endpoints
**Priority:** Medium (functional but may cause issues in production)

### Prompt Auto-Enhancement
**Feature:** `autoEnhance` parameter in generation endpoint
**Status:** Implemented but not extensively tested
**Action:** Add comprehensive tests for enhancement behavior:
  - Verify `originalPrompt` populated when enhanced
  - Verify `prompt` contains enhanced version
  - Verify enhancement doesn't occur when `autoEnhance=false`

### Alias Assignment Endpoints
**Note:** Alias assignment is separated from general metadata updates
**Correct Behavior:**
- `PUT /api/v1/images/:id` - Update focalPoint, meta only
- `PUT /api/v1/images/:id/alias` - Dedicated alias assignment endpoint

**Benefit:** Better separation of concerns, clearer API semantics

---

## 🧪 Required Test Fixtures

### Current Fixtures
- ✅ `tests/api/fixture/test-image.png` (1.6MB PNG)

### Additional Fixtures Needed
*(To be determined during test implementation)*

- [ ] Small image (<1MB) for quick upload tests
- [ ] Large image (>5MB) for size limit validation
- [ ] JPEG file for format variety testing
- [ ] Multiple distinct images for reference testing
- [ ] Invalid file types (.txt, .pdf) for negative tests

**Status:** Will be generated/collected after initial test implementation.

---

## 🔧 Test Environment Requirements

### Services Required
- ✅ API service running on `http://localhost:3000`
- ✅ PostgreSQL database with schema v2.0
- ✅ MinIO storage accessible and configured
- ✅ Valid project API key configured in `config.ts`
- ✅ Google Gemini API credentials (will consume credits)

### Database State
- Tests assume empty or minimal database
- Tests do NOT clean up data (by design)
- Run against dedicated test project, not production

### Performance Notes
- Each image generation: ~3-10 seconds (Gemini API)
- Full test suite: ~20-30 minutes
- Gemini API cost: ~70-80 generations @ $0.0025 each = ~$0.18-0.20

---

## 📈 Test Execution Commands

```bash
# Run full test suite (sequential)
cd tests/api
tsx run-all.ts

# Run individual test files
tsx 01-generation-basic.ts
tsx 02-basic.ts
tsx 03-flows.ts
tsx 04-aliases.ts
tsx 05-live.ts
tsx 06-edge-cases.ts
tsx 07-known-issues.ts

# Expected output: Colored console with ✓ (pass) and ✗ (fail) indicators
```

---

## 🎯 Success Criteria

- [x] All test files execute without crashes
- [x] Tests 01-06: ~70 tests pass (verify correct implementation)
- [x] Test 07: ~4 tests fail (document known issues)
- [x] Each test has clear assertions and error messages
- [x] Tests use real API calls (no mocks)
- [x] All generated images saved to `tests/api/results/`
- [x] Summary document maintained and accurate

---

## 📝 Maintenance Notes

### Updating Tests
When API implementation is fixed:
1. Move tests from `07-known-issues.ts` to appropriate test file
2. Update this summary document
3. Re-run full test suite to verify fixes

### Adding New Tests
1. Choose appropriate test file based on feature area
2. Follow existing test patterns (runTest, clear assertions)
3. Update test count in Overview table
4. Document any new fixtures needed

### Known Limitations
- Tests are not idempotent (leave data in database)
- No parallel execution support
- No automated cleanup between runs
- Requires manual server startup

---

**Document Status:** ✅ Complete
**Next Update:** After test implementation and first full run