Kareem Horstink
56 lines
2.0 KiB
Markdown
56 lines
2.0 KiB
Markdown
# SigLIP-Based Aesthetic Predictor V2.5
|
|
|
|
## Model Info
|
|
|
|
- **GitHub:** <https://github.com/discus0434/aesthetic-predictor-v2-5>
|
|
- **PyPI:** `aesthetic-predictor-v2-5`
|
|
- **License:** MIT
|
|
- **Updated:** December 2024
|
|
- **Improvement over V2:** Uses SigLIP (Google's CLIP alternative) instead of OpenAI CLIP; better at illustrations and diverse image domains
|
|
|
|
## Architecture
|
|
|
|
- **Backbone:** SigLIP (Google) — superior to OpenAI CLIP on many benchmarks
|
|
- **Head:** Linear layer on SigLIP embeddings
|
|
- **Output:** Single float score (0-10 scale)
|
|
- **Score threshold:** 5.5+ is considered "great" (vs. 6+ for V2)
|
|
|
|
## Hardware Requirements
|
|
|
|
- **SigLIP ViT:** ~2-3 GB VRAM (slightly larger than CLIP ViT-L/14)
|
|
- **Precision:** Requires BF16 for best results
|
|
- **Batch size:** ~16-32 on 24 GB GPU
|
|
|
|
## Python Usage
|
|
|
|
```python
|
|
from aesthetic_predictor_v2_5 import convert_v2_5_from_siglip
|
|
from PIL import Image
|
|
import torch
|
|
|
|
model, preprocessor = convert_v2_5_from_siglip(low_cpu_mem_usage=True)
|
|
model = model.to(torch.bfloat16).cuda()
|
|
|
|
image = Image.open("photo.jpg").convert("RGB")
|
|
pixel_values = preprocessor(images=image, return_tensors="pt").pixel_values.to(torch.bfloat16).cuda()
|
|
|
|
with torch.inference_mode():
|
|
score = model(pixel_values).logits.squeeze().float().cpu().numpy()
|
|
|
|
print(f"Aesthetics score: {score:.2f}")
|
|
```
|
|
|
|
## What It's Good At
|
|
|
|
- **Better than V2 on illustrations/art:** SigLIP trained on larger, more diverse dataset
|
|
- **Same simplicity:** One-line HuggingFace-style API
|
|
- **Same speed:** Linear head on embeddings
|
|
- **Slightly better generalization:** SigLIP's training data is more diverse than OpenAI CLIP's
|
|
|
|
## What It's Bad At
|
|
|
|
- **Same fundamental limitations as V2:** No technical quality detection, no blur/exposure detection, no genre awareness
|
|
- **Still not trained on real photography:** Same bias toward AI-generated aesthetic patterns
|
|
- **BF16 requirement:** Needs GPU with BF16 support (RTX 30-series+); not ideal for older hardware
|
|
- **Community model:** Not an official LAION release; maintained by a third party
|