test(api): improve caption API test coverage and validation

Browse Source

Add model architecture coverage tests:
- VQA model family detection for 19 architectures
- Florence special prompts test (<OD>, <OCR>, <CAPTION>, etc.)
- Moondream detection features test
- VQA architecture capabilities test
- Tagger model types and WD version comparison tests

Improve test validation:
- Add is_meaningful_answer() to reject responses like "."
- Verify parameters have actual effect (not just accepted)
- Show actual output traces in PASS/FAIL messages
- Fix prefill tests to verify keep_prefill behavior

Add configurable timeout:
- Default timeout increased to 300s for slow models
- Add --timeout CLI argument for customization

Other improvements:
- Add JoyCaption to recognized model families
- Reduce BLIP models to avoid reloading large models
- Better detection result validation for annotated images

...

This commit is contained in:

CalamitousFelicitousness

2026-01-25 04:06:17 +00:00

parent a04ba1e482

commit 3208067259

1 changed files with 1321 additions and 34 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download Patch File Download Diff File Expand all files Collapse all files

cli/test-caption-api.py

+1321 -34

File diff suppressed because it is too large Load Diff