Kareem Horstink
94 lines
5.5 KiB
Markdown
94 lines
5.5 KiB
Markdown
# LAION Aesthetic Predictor — Current State
|
|
|
|
## Model Info
|
|
|
|
- **GitHub (V1):** <https://github.com/LAION-AI/aesthetic-predictor>
|
|
- **GitHub (V2/Improved):** <https://github.com/christophschuhmann/improved-aesthetic-predictor>
|
|
- **HuggingFace:** `shunk031/aesthetics-predictor-v1-vit-large-patch14`, `shunk031/aesthetics-predictor-v2-vit-large-patch14`
|
|
- **PyPI:** `simple-aesthetics-predictor` (wrapper), `aesthetic-predictor` (direct)
|
|
- **Creator:** Christoph Schuhmann / LAION
|
|
- **License:** MIT (model weights)
|
|
- **Last Update:** V2 released August 2022; no V3 released as of August 2026
|
|
- **Status:** **No newer version has been released.** LAION has not published a V3 or updated model (confirmed by CMU audit paper, FAccT 2026)
|
|
|
|
## Architecture
|
|
|
|
- **V1:** Linear layer (768->1 for ViT-L/14, 512->1 for ViT-B/32) on top of CLIP image embeddings
|
|
- **V2:** Linear model on top of CLIP ViT-L/14 embeddings (despite testing MLPs with ReLU — the linear model was subjectively preferred for visual ranking quality)
|
|
- **Input:** CLIP image embeddings (not raw pixels) — must run CLIP encoder first, then the aesthetic head
|
|
- **Output:** Single float score (0-10 scale)
|
|
- **Training Data (V2):**
|
|
- SAC (Simulacra Aesthetic Captions): ~176,000 AI-generated image ratings
|
|
- LAION-Logos: 15,000 logo image-text pairs with aesthetic ratings
|
|
- AVA (Aesthetic Visual Analysis): 250,000 professional photos with 5-10 human ratings each
|
|
- **CLIP Version:** OpenAI CLIP ViT-L/14 (2021 model)
|
|
|
|
## Variants
|
|
|
|
| Variant | CLIP Backbone | Head | Score Range | Notes |
|
|
|---------|--------------|------|-------------|-------|
|
|
| V1 (ViT-L/14) | ViT-L/14 (768d) | Linear 768->1 | 0-10 | Original, trained on 5,000 SAC samples |
|
|
| V1 (ViT-B/32) | ViT-B/32 (512d) | Linear 512->1 | 0-10 | Faster, lower capacity |
|
|
| V2 (ViT-L/14) | ViT-L/14 (768d) | Linear 768->1 | 0-10 | Trained on SAC + LAION-Logos + AVA; current best |
|
|
|
|
## Hardware Requirements
|
|
|
|
- **CLIP ViT-L/14 encoder:** ~1.8 GB VRAM (FP16)
|
|
- **Aesthetic linear head:** Negligible
|
|
- **Total per image:** ~2 GB VRAM
|
|
- **Batch size:** Can handle ~32-64 images per batch on a 24 GB GPU (RTX 4090)
|
|
- **CPU inference:** Possible but slow (~1-2 seconds per image on modern CPU)
|
|
- **Estimated time for 4,000 images:** ~5-10 minutes on GPU, ~1-2 hours on CPU
|
|
|
|
## Python Usage
|
|
|
|
```python
|
|
# Via simple-aesthetics-predictor (HuggingFace Transformers-style API)
|
|
from simple_aesthetics_predictor import AestheticsPredictorV1
|
|
from transformers import CLIPProcessor
|
|
import torch
|
|
|
|
model = AestheticsPredictorV1.from_pretrained("shunk031/aesthetics-predictor-v2-vit-large-patch14")
|
|
processor = CLIPProcessor.from_pretrained("shunk031/aesthetics-predictor-v2-vit-large-patch14")
|
|
|
|
# Batch inference
|
|
scores = model.predict(images, processor) # Returns list of scores
|
|
```
|
|
|
|
```python
|
|
# Via direct aesthetic-predictor package
|
|
from aesthetic_predictor import predict_aesthetic
|
|
from PIL import Image
|
|
|
|
score = predict_aesthetic(Image.open("photo.jpg")) # Returns float score
|
|
```
|
|
|
|
## What It's Good At
|
|
|
|
- **Speed:** Extremely fast — linear head on pre-computed embeddings
|
|
- **Simplicity:** One-line inference with HuggingFace transformers
|
|
- **Open source:** MIT license, no commercial restrictions
|
|
- **Lightweight:** ~2 GB VRAM, fits on consumer GPUs
|
|
- **Good general aesthetic ranking:** Trained on diverse datasets (AI-generated + professional photos)
|
|
- **Works on real photography:** AVA dataset contains 250K professional photos, so it generalizes to wedding photos reasonably well
|
|
|
|
## What It's Bad At
|
|
|
|
- **No technical quality detection:** Does NOT detect blur, motion blur, out-of-focus, exposure issues, noise, or composition problems. It scores "aesthetic appeal," not "technical correctness."
|
|
- **Trained heavily on AI-generated images:** 176K of 441K training samples are from SAC (AI-generated). This may bias scoring toward AI-aesthetic patterns (smooth gradients, saturated colors, idealized lighting) rather than genuine photography.
|
|
- **Western/art-historical bias:** CMU FAccT 2026 audit found LAP disproportionately favors landscapes, cityscapes, portraits from Western and Japanese artists. Reinforces "imperial and male gazes" from Western art history. May underrate candid moments, cultural ceremonies, and documentary-style wedding shots.
|
|
- **No genre awareness:** Treats all images the same. A technically perfect but emotionally empty portrait scores the same as a technically imperfect but emotionally powerful candid.
|
|
- **No context understanding:** Cannot distinguish between a "keep" (peak moment, emotional reaction) and a "reject" (duplicate, transitional moment, empty frame).
|
|
- **No blink/closed-eye detection:** Irrelevant for wedding photo sorting where you need to filter technical rejects.
|
|
- **No duplicate detection:** Cannot identify burst sequences or near-duplicates.
|
|
- **Score calibration:** Scores are relative rankings, not absolute quality measures. A score of 7 on one image doesn't mean the same as 7 on another.
|
|
|
|
## Bias Concerns (FAccT 2026 Audit)
|
|
|
|
A rigorous academic audit (CMU, published at FAccT 2026) found:
|
|
|
|
- **Gender bias:** Disproportionately filters in images with captions mentioning women; filters out images with captions mentioning men or LGBTQ+ people
|
|
- **Cultural bias:** Rates realistic images of landscapes, cityscapes, and portraits from Western and Japanese artists most highly
|
|
- **Training data bias:** Aesthetic scores used to train LAP primarily come from English-speaking photographers and Western AI-enthusiasts
|
|
- **No V3 released:** As of January 2026, LAION has not released an updated model to address these concerns
|