Imrayya/photo_judgers
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

init: project structure with LAION scorer, conda setup, and research docs

Browse Source
This commit is contained in:
Kareem Horstink
2026-08-23 15:31:47 +00:00
commit 4f90f5838a
25 changed files with 1628 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/output-format.md
+65
View File
@@ -0,0 +1,65 @@
# Output Format — JSON
## Structure
The scorer outputs a JSON file with an array of results. Each entry represents one photo.
## Schema
```json
[
{
"filename": "IMG_0001.jpg",
"filepath": "/path/to/wedding-photos/IMG_0001.jpg",
"subtype": "aesthetic",
"laion_score": 7.2,
"laion_version": "v2",
"scoring_date": "2026-01-15T10:30:00Z"
}
]
```
## Fields
| Field | Type | Description |
|-------|------|-------------|
| `filename` | string | Base filename (e.g., `IMG_0001.jpg`) |
| `filepath` | string | Full absolute path to the image |
| `subtype` | string | Classifier subtype — `aesthetic` for LAION (future: `technical`, `blur`, etc.) |
| `laion_score` | float | LAION aesthetic score (0-10 scale) |
| `laion_version` | string | Version of LAION model used (e.g., `v2`) |
| `scoring_date` | string | ISO 8601 timestamp of when scoring was performed |
## Example
```json
[
{
"filename": "DSC_0421.jpg",
"filepath": "C:/wedding-photos/day1/DSC_0421.jpg",
"subtype": "aesthetic",
"laion_score": 8.1,
"laion_version": "v2",
"scoring_date": "2026-01-15T10:30:00Z"
},
{
"filename": "DSC_0422.jpg",
"filepath": "C:/wedding-photos/day1/DSC_0422.jpg",
"subtype": "aesthetic",
"laion_score": 3.4,
"laion_version": "v2",
"scoring_date": "2026-01-15T10:30:01Z"
}
]
```
## Thresholds (TBD)
Thresholds for keep/reject/review are not yet defined. Will be determined after running initial batches and reviewing score distributions.
Suggested approach:
1. Score a sample of 100-200 photos
2. Analyze the score distribution
3. Set thresholds based on the distribution (e.g., top 25% = keep, bottom 25% = reject, middle = review)
4. Adjust based on photographer feedback
docs/research-brisque-niqe.md
+159
View File
@@ -0,0 +1,159 @@
# BRISQUE / NIQE — Classical No-Reference IQA
## Model Info
- **BRISQUE:** "No-reference Image Quality Assessment in the Spatial Domain" — Mittal et al., IEEE TIP 2012
- **NIQE:** "Making a 'Completely Blind' Image Quality Analyzer" — Mittal et al., IEEE SPL 2013
- **Python packages:**
- `pip install brisque[opencv-python]` — Latest BRISQUE (supports opencv-contrib, opencv-headless)
- `pip install pybrisque` — Alternative BRISQUE (older, from Bukalapak)
- `pip install image-quality` — Contains `imquality.brisque` module
- `pip install pyiqa` — IQA-PyTorch toolbox (includes BRISQUE + NIQE + 30+ other metrics)
- `pip install nr-iqa` — Comprehensive library (BRISQUE + NIQE + PIQE)
- **License:** Academic (BRISQUE: BSD-like via LearnOpenCV; NIQE: academic use)
- **Dependencies:** OpenCV, NumPy, SciPy (BRISQUE); skvideo + NumPy (NIQE)
## Architecture
### BRISQUE (Blind/Referenceless Image Spatial Quality Evaluator)
- **Type:** Handcrafted features + Support Vector Regression (SVR)
- **Pipeline:**
1. **MSCN (Mean Subtracted Contrast Normalized) coefficients** — decorrelates image from local brightness/contrast
2. **GGD (Generalized Gaussian Distribution) fitting** to MSCN coefficients — extracts shape parameter alpha (tail heaviness)
3. **Pairwise products** in 4 orientations (horizontal, vertical, 2 diagonals) — 16 features
4. **AGGD (Asymmetric GGD) fitting** to pairwise products
5. **Multi-scale analysis** at 2 scales (original + 0.5x downsampled) — 36-D feature vector total (18 per scale)
6. **SVR prediction** maps 36-D features to quality score in [0, 100]
- **Training data:** TID2008 dataset (human-rated distorted images, MOS scores 0-100)
- **Score direction:** **Lower = better** (0 = perfect, 100 = worst quality)
### NIQE (Natural Image Quality Evaluator)
- **Type:** Handcrafted features + Multivariate Gaussian (MVG) model
- **Key difference from BRISQUE:** Completely opinion-unaware — does NOT need human-rated distorted images for training
- **Pipeline:**
1. Extract 18-D NSS features from 96x96 patches (same as BRISQUE scale-1)
2. Fit MVG model (mu_ref, Sigma_ref) to pristine images — the "naturalness" reference
3. Fit MVG (mu_test, Sigma_test) to test image patches
4. **Quality = Mahalanobis distance** between the two MVG models
- **Training data:** Only pristine (undistorted) images — no human ratings needed
- **Score direction:** **Lower = better** (0 = perfectly natural, higher = more distorted)
## Installation
```bash
# Option 1: Latest BRISQUE (recommended)
pip install brisque[opencv-python]
# Option 2: For servers/Docker (headless)
pip install brisque[opencv-python-headless]
# Option 3: Comprehensive NR-IQA library (BRISQUE + NIQE + PIQE)
pip install nr-iqa
# Option 4: IQA-PyTorch toolbox (BRISQUE + NIQE + 30+ other metrics)
pip install pyiqa
```
## Python Usage
```python
# --- BRISQUE via brisque package ---
from brisque import BRISQUE
import cv2
brisque = BRISQUE()
img = cv2.imread('photo.jpg')
score = brisque.score(img) # Returns float in [0, 100]
print(f"BRISQUE: {score:.2f}") # Lower = better
# --- BRISQUE via image-quality package ---
from imquality.brisque import BRISQUE
from PIL import Image
img = Image.open('photo.jpg')
score = BRISQUE().score(img) # Returns float in [0, 100]
# --- NIQE via nr-iqa package ---
from nr_iqa import NIQE
import cv2
niqe = NIQE()
img = cv2.imread('photo.jpg', cv2.IMREAD_GRAYSCALE)
score = niqe.score(img) # Returns float in [0, 100]
print(f"NIQE: {score:.4f}") # Lower = better
# --- Both via IQA-PyTorch toolbox ---
import pyiqa
device = 'cuda' if torch.cuda.is_available() else 'cpu'
brisque = pyiqa.create_metric('brisque', device=device)
niqe = pyiqa.create_metric('niqe', device=device)
score_b = brisque('./photo.jpg')
score_n = niqe('./photo.jpg')
print(f"BRISQUE: {score_b:.2f}, NIQE: {score_n:.4f}")
# Batch processing
import glob
for path in glob.glob('./wedding-photos/*.jpg'):
score = brisque(path)
```
## Score Interpretation for Wedding Photos
| BRISQUE Score | NIQE Score | Interpretation | Recommended Action |
|---------------|-----------|---------------|-------------------|
| 0-10 | 0-3 | Excellent technical quality | Auto-keep |
| 10-25 | 3-6 | Good technical quality | Keep |
| 25-40 | 6-10 | Acceptable, minor issues | Review |
| 40-60 | 10-15 | Poor technical quality | Likely reject |
| 60-100 | 15+ | Very poor (blurry, noisy, compressed) | Auto-reject |
**Recommended thresholds for wedding culling:**
- BRISQUE < 30 AND NIQE < 8 -> keep (good technical quality)
- BRISQUE > 60 OR NIQE > 15 -> reject (poor technical quality)
- In between -> manual review
## Performance Benchmarks
| Metric | SRCC vs Human | PLCC vs Human | Notes |
|--------|--------------|--------------|-------|
| BRISQUE (TID2008) | ~0.85 | ~0.87 | Trained on TID2008 |
| NIQE (KADID-10k) | ~0.83 | ~0.85 | Opinion-unaware, generalizes well |
| BRISQUE (live) | ~0.81 | ~0.83 | Live (no-reference) dataset |
| NIQE (live) | ~0.80 | ~0.82 | Live (no-reference) dataset |
## Hardware Requirements
- **CPU-only:** No GPU needed — purely classical (no neural networks)
- **Speed per image:** ~50-200ms per image on modern CPU (single-core)
- **Speed for 4,000 images:** ~3-15 minutes on CPU (single-core, depends on image size)
- **Multi-threading:** Can parallelize across CPU cores — ~1-3 minutes with 8+ cores
- **VRAM:** None required (BRISQUE/NIQE are pure NumPy/OpenCV)
- **Memory usage:** ~50-100 MB RAM for batch processing
## What It's Good At
- **No GPU needed:** Runs on any CPU, even low-end machines
- **Fast:** Good for quick technical quality screening — faster than any deep learning model
- **Detects blur and compression artifacts:** Excellent at detecting out-of-focus shots, motion blur, JPEG artifacts, noise
- **Simple:** Minimal dependencies (OpenCV + NumPy + SciPy)
- **Deterministic:** No randomness — same image always gets same score
- **Lightweight:** ~50 MB total install size, no model weights to download
- **Good complement to aesthetic models:** Catches technical rejects that aesthetic models miss
- **Works on any image type:** Not biased toward any particular visual style
## What It's Bad At
- **Purely technical:** Scores only technical quality (blur, noise, compression), not aesthetics at all
- **Poor on artistic/creative images:** Penalizes artistic choices — shallow depth of field, high contrast, film grain, vignette, intentional blur
- **Outdated methodology:** 2012-2013 techniques; inferior to modern deep learning approaches for complex distortions
- **Sensitive to intentional artistic effects:** May score a portrait with beautiful bokeh (shallow DOF) as "low quality"
- **Color-blind:** BRISQUE operates on luminance channel; NIQE on grayscale — ignores color information entirely
- **Not suitable as sole scorer:** Must be combined with an aesthetic model for wedding photo sorting
- **No multi-scale like MUSIQ:** Only processes at one scale (BRISQUE does 2-scale but simpler than MUSIQ's 3-scale)
- **Training data limitations:** BRISQUE trained on TID2008 (lab-distorted images); may not generalize well to real-world wedding photo distortions
docs/research-commercial-tools.md
+29
View File
@@ -0,0 +1,29 @@
# Commercial AI Photo Culling Tools — For Reference
These are not self-hosted but represent the state-of-the-art in wedding photo culling.
## FilterPixel DeepCull (2026)
- **Type:** Cloud-based, genre-specific AI
- **Speed:** 2:58 for 3,000 photos (cloud)
- **Features:** 10-parameter scoring (emotion, sharpness, composition, storytelling, etc.), memory-based learning, genre-specific models
- **Accuracy:** 85-95% agreement after training on user's past culling decisions
- **Price:** $9.99-$19/month
- **Best for:** Professional photographers needing same-day delivery
## Aftershoot
- **Type:** Local desktop application
- **Speed:** 8-12 minutes for 1,000 images (local)
- **Features:** Duplicate detection, blink detection, blur detection, highlight selection, genre selection (portrait/wedding/events)
- **Learning:** Adapts to user's editing style over time
- **Price:** $15/month
- **Best for:** Wedding photographers with multi-day turnaround
## Imagen AI
- **Type:** Cloud-based editing + culling
- **Accuracy:** 63.4% (noted as lower than FilterPixel/Aftershoot in independent testing)
- **Features:** Culling studio, AI editing, pay-per-image pricing
- **Price:** Pay per image (expensive at scale)
- **Best for:** Photographers already using Imagen for editing
docs/research-iqa-pytorch.md
+65
View File
@@ -0,0 +1,65 @@
# IQA-PyTorch Toolbox — All-in-One
## Model Info
- **GitHub:** <https://github.com/chaofengc/IQA-PyTorch>
- **PyPI:** `pip install pyiqa`
- **License:** Apache 2.0
- **Contents:** 30+ IQA metrics including MUSIQ, TOPIQ, BRISQUE, NIQE, LPIPS, FID, NIMA, DBCNN, CLIP-IQA, LIQE, Q-Align (Q-ReAlign), and more
- **Latest updates (2026):**
- Jun 2026: Added Q-ReAlign (Qwen3.5-VL backbone) — 3 sizes: mini (0.8B), lite (4B), pro (9B)
- May 2026: Added FGResQ
- Dec 2025: Added DMM, MACLIP, AFINE
- Jan 2025: Added QualiCLIP variants
- **Results calibrated:** Against official MATLAB scripts when available
- **GPU accelerated:** PyTorch implementations are faster than MATLAB counterparts
## Why It Matters
This toolbox lets you test multiple IQA models on your wedding photos without installing each separately. Useful for benchmarking and comparison. It's the single easiest way to run BRISQUE + NIQE + MUSIQ + other metrics in one codebase.
```bash
# List all available metrics
pyiqa -ls
# Test multiple metrics on a directory
pyiqa musiq niqe brisque -t ./wedding-photos/ --device cuda
# Python API — all metrics in one import
import pyiqa
import torch
device = torch.device("cuda") if torch.cuda.is_available() else torch.device("cpu")
# Create any metric
brisque = pyiqa.create_metric('brisque', device=device)
musiq = pyiqa.create_metric('musiq', device=device)
niqe = pyiqa.create_metric('niqe', device=device)
# Check scoring direction
print(f"BRISQUE lower_better: {brisque.lower_better}") # True
print(f"MUSIQ lower_better: {musiq.lower_better}") # False (higher = better)
print(f"NIQE lower_better: {niqe.lower_better}") # True
# Batch inference
import glob
for path in glob.glob('./wedding-photos/*.jpg'):
score_b = brisque(path)
score_m = musiq(path)
score_n = niqe(path)
```
## Available Metrics (relevant to wedding photo sorting)
| Metric | Type | Score Direction | GPU Required | Notes |
|--------|------|----------------|--------------|-------|
| brisque | NR-IQA | Lower better | No (CPU) | Blur/noise detection |
| niqe | NR-IQA | Lower better | No (CPU) | Blur/noise detection, opinion-unaware |
| musiq | NR-IQA (Transformer) | Higher better | Yes (~4GB) | Aesthetic + technical combined |
| toqip | NR-IQA (Transformer) | Higher better | Yes (~2GB) | Modern, self-supervised |
| clip_iqa | NR-IQA (CLIP) | Higher better | Yes (~2GB) | CLIP-based aesthetic |
| liqe | NR-IQA (CLIP) | Higher better | Yes (~2GB) | CLIP-based quality |
| lpips | FR-IQA (Perceptual) | Lower better | Yes (~1GB) | Needs reference image |
| fid | FR-IQA (Distribution) | Lower better | Yes (~5GB) | Needs reference dataset |
| nima | NR-IQA (CNN) | Higher better | Yes (~1GB) | Neural IMage Assessment |
| dbcnn | FR-IQA (CNN) | Higher better | Yes (~2GB) | Needs reference image |
docs/research-laion.md
+93
View File
@@ -0,0 +1,93 @@
# LAION Aesthetic Predictor — Current State
## Model Info
- **GitHub (V1):** <https://github.com/LAION-AI/aesthetic-predictor>
- **GitHub (V2/Improved):** <https://github.com/christophschuhmann/improved-aesthetic-predictor>
- **HuggingFace:** `shunk031/aesthetics-predictor-v1-vit-large-patch14`, `shunk031/aesthetics-predictor-v2-vit-large-patch14`
- **PyPI:** `simple-aesthetics-predictor` (wrapper), `aesthetic-predictor` (direct)
- **Creator:** Christoph Schuhmann / LAION
- **License:** MIT (model weights)
- **Last Update:** V2 released August 2022; no V3 released as of August 2026
- **Status:** **No newer version has been released.** LAION has not published a V3 or updated model (confirmed by CMU audit paper, FAccT 2026)
## Architecture
- **V1:** Linear layer (768->1 for ViT-L/14, 512->1 for ViT-B/32) on top of CLIP image embeddings
- **V2:** Linear model on top of CLIP ViT-L/14 embeddings (despite testing MLPs with ReLU — the linear model was subjectively preferred for visual ranking quality)
- **Input:** CLIP image embeddings (not raw pixels) — must run CLIP encoder first, then the aesthetic head
- **Output:** Single float score (0-10 scale)
- **Training Data (V2):**
- SAC (Simulacra Aesthetic Captions): ~176,000 AI-generated image ratings
- LAION-Logos: 15,000 logo image-text pairs with aesthetic ratings
- AVA (Aesthetic Visual Analysis): 250,000 professional photos with 5-10 human ratings each
- **CLIP Version:** OpenAI CLIP ViT-L/14 (2021 model)
## Variants
| Variant | CLIP Backbone | Head | Score Range | Notes |
|---------|--------------|------|-------------|-------|
| V1 (ViT-L/14) | ViT-L/14 (768d) | Linear 768->1 | 0-10 | Original, trained on 5,000 SAC samples |
| V1 (ViT-B/32) | ViT-B/32 (512d) | Linear 512->1 | 0-10 | Faster, lower capacity |
| V2 (ViT-L/14) | ViT-L/14 (768d) | Linear 768->1 | 0-10 | Trained on SAC + LAION-Logos + AVA; current best |
## Hardware Requirements
- **CLIP ViT-L/14 encoder:** ~1.8 GB VRAM (FP16)
- **Aesthetic linear head:** Negligible
- **Total per image:** ~2 GB VRAM
- **Batch size:** Can handle ~32-64 images per batch on a 24 GB GPU (RTX 4090)
- **CPU inference:** Possible but slow (~1-2 seconds per image on modern CPU)
- **Estimated time for 4,000 images:** ~5-10 minutes on GPU, ~1-2 hours on CPU
## Python Usage
```python
# Via simple-aesthetics-predictor (HuggingFace Transformers-style API)
from simple_aesthetics_predictor import AestheticsPredictorV1
from transformers import CLIPProcessor
import torch
model = AestheticsPredictorV1.from_pretrained("shunk031/aesthetics-predictor-v2-vit-large-patch14")
processor = CLIPProcessor.from_pretrained("shunk031/aesthetics-predictor-v2-vit-large-patch14")
# Batch inference
scores = model.predict(images, processor) # Returns list of scores
```
```python
# Via direct aesthetic-predictor package
from aesthetic_predictor import predict_aesthetic
from PIL import Image
score = predict_aesthetic(Image.open("photo.jpg")) # Returns float score
```
## What It's Good At
- **Speed:** Extremely fast — linear head on pre-computed embeddings
- **Simplicity:** One-line inference with HuggingFace transformers
- **Open source:** MIT license, no commercial restrictions
- **Lightweight:** ~2 GB VRAM, fits on consumer GPUs
- **Good general aesthetic ranking:** Trained on diverse datasets (AI-generated + professional photos)
- **Works on real photography:** AVA dataset contains 250K professional photos, so it generalizes to wedding photos reasonably well
## What It's Bad At
- **No technical quality detection:** Does NOT detect blur, motion blur, out-of-focus, exposure issues, noise, or composition problems. It scores "aesthetic appeal," not "technical correctness."
- **Trained heavily on AI-generated images:** 176K of 441K training samples are from SAC (AI-generated). This may bias scoring toward AI-aesthetic patterns (smooth gradients, saturated colors, idealized lighting) rather than genuine photography.
- **Western/art-historical bias:** CMU FAccT 2026 audit found LAP disproportionately favors landscapes, cityscapes, portraits from Western and Japanese artists. Reinforces "imperial and male gazes" from Western art history. May underrate candid moments, cultural ceremonies, and documentary-style wedding shots.
- **No genre awareness:** Treats all images the same. A technically perfect but emotionally empty portrait scores the same as a technically imperfect but emotionally powerful candid.
- **No context understanding:** Cannot distinguish between a "keep" (peak moment, emotional reaction) and a "reject" (duplicate, transitional moment, empty frame).
- **No blink/closed-eye detection:** Irrelevant for wedding photo sorting where you need to filter technical rejects.
- **No duplicate detection:** Cannot identify burst sequences or near-duplicates.
- **Score calibration:** Scores are relative rankings, not absolute quality measures. A score of 7 on one image doesn't mean the same as 7 on another.
## Bias Concerns (FAccT 2026 Audit)
A rigorous academic audit (CMU, published at FAccT 2026) found:
- **Gender bias:** Disproportionately filters in images with captions mentioning women; filters out images with captions mentioning men or LGBTQ+ people
- **Cultural bias:** Rates realistic images of landscapes, cityscapes, and portraits from Western and Japanese artists most highly
- **Training data bias:** Aesthetic scores used to train LAP primarily come from English-speaking photographers and Western AI-enthusiasts
- **No V3 released:** As of January 2026, LAION has not released an updated model to address these concerns
docs/research-musiq.md
+126
View File
@@ -0,0 +1,126 @@
# MUSIQ — Multi-Scale Image Quality Transformer
## Model Info
- **Paper:** ICCV 2021 — "MUSIQ: Multi-Scale Image Quality Transformer" (Ke et al.)
- **Source:** Google Research (Pengchuan Zhang, Xiujun Li, Ping Luo, Kai Wang, Yu-Xiong Wang)
- **Official Google Repo:** <https://github.com/google-research/google-research/tree/master/musiq>
- **Google Blog:** <https://ai.googleblog.com/2021/07/musiq-assessing-image-aesthetic-and.html>
- **TensorFlow Hub:** Models available at `tfhub.dev` (official hosted checkpoints)
- **PyPI (IQA-PyTorch toolbox):** `pip install pyiqa` -> `pyiqa create_metric 'musiq'`
- **PyTorch unofficial impl:** <https://github.com/anse3832/MUSIQ> (PyTorch, works on KonIQ-10k)
- **License:** Apache 2.0 (via IQA-PyTorch reimplementation)
- **Status:** Actively maintained in IQA-PyTorch; official TensorFlow code archived but functional
## Architecture
- **Type:** Patch-based Vision Transformer (ViT) with multi-scale input
- **Backbone:** ResNet50 (ImageNet-pretrained) for feature extraction -> Transformer encoder for aggregation
- **Multi-scale processing:** Processes image at 3 scales simultaneously — native resolution, 224x224, and 384x384
- **Input:** Full-resolution images (no fixed-size constraint) — handles any aspect ratio natively
- **Output:** Single float score 0-100 (technical + aesthetic combined)
- **Training checkpoints:**
- `ava_ckpt.npz` — trained on AVA dataset (aesthetic quality)
- `koniq_ckpt.npz` — trained on KonIQ-10k (technical quality)
- `paq2piq_ckpt.npz` — trained on PaQ-2-PiQ (technical quality)
- `spaq_ckpt.npz` — trained on SPAQ (technical quality)
- `imagenet_pretrain.npz` — ImageNet pretraining only
- **Key innovation:** Patch-based design bypasses CNN fixed-size constraint; hash-based 2D spatial embedding + scale embedding for positional encoding
- **MUSIQ-single variant:** Processes only native resolution (faster, slightly lower accuracy)
## Installation
```bash
# Option 1: IQA-PyTorch toolbox (recommended — easiest, supports all metrics)
pip install pyiqa
# Option 2: Official TensorFlow code
git clone https://github.com/google-research/google-research.git
cd google-research/musiq
pip install -r requirements.txt
# Option 3: Unofficial PyTorch implementation
git clone https://github.com/anse3832/MUSIQ.git
cd MUSIQ
pip install torch torchvision einops scipy tqdm
```
## Python Usage
```python
# Via IQA-PyTorch (recommended — single API for all metrics)
import pyiqa
import torch
device = torch.device("cuda") if torch.cuda.is_available() else torch.device("cpu")
# Create MUSIQ metric (auto-downloads checkpoint on first use)
musiq = pyiqa.create_metric('musiq', device=device)
# Single image
score = musiq('./photo.jpg')
print(f"MUSIQ score: {score:.2f} / 100")
# Directory of images
import glob
scores = []
for path in glob.glob('./wedding-photos/*.jpg'):
scores.append(musiq(path))
```
```python
# Via official TensorFlow code
cd google-research/musiq
python3 -m musiq.run_predict_image \
--ckpt_path=/path/to/spaq_ckpt.npz \
--image_path=/path/to/photo.jpg
```
## Performance Benchmarks (from IQA-PyTorch docs & Google Research)
| Dataset | SRCC (rank correlation) | PLCC (Pearson correlation) |
|---------|------------------------|---------------------------|
| KonIQ-10k | ~0.90 | ~0.92 |
| PaQ-2-PiQ | ~0.85 | ~0.87 |
| SPAQ | ~0.88 | ~0.90 |
| AVA | ~0.75 | ~0.78 (lower — aesthetic-only dataset) |
## Hardware Requirements
- **VRAM:** ~3-4 GB (FP16), ~6-8 GB (FP32)
- **Batch size:** ~8-16 on 24 GB GPU (RTX 4090)
- **Speed per image:** ~50-150ms on GPU (FP16), ~200-500ms on CPU
- **Estimated time for 4,000 images:** ~15-25 minutes on GPU (FP16, batch 16)
- **Native resolution handling:** No resizing needed — processes full-resolution images directly
## Score Interpretation for Wedding Photos
| Score Range | Interpretation | Recommended Action |
|-------------|---------------|-------------------|
| 85-100 | Excellent quality | Auto-keep |
| 70-84 | Good quality | Keep, manual review if borderline |
| 50-69 | Acceptable | Review — may have minor issues |
| 30-49 | Poor quality | Likely reject |
| 0-29 | Very poor | Auto-reject |
**Recommended threshold for wedding culling:** Score >= 50 (i.e., not in the bottom third). This typically reduces 4,000 images to ~800-1,200 for manual review.
## What It's Good At
- **Technical quality detection:** Better than LAION predictors at detecting blur, noise, compression artifacts, over/under exposure
- **Full-resolution input:** No resizing artifacts — important for wedding photos with varying aspect ratios (portrait, landscape, square)
- **Combined aesthetic + technical scoring:** Not just "pretty" but also "technically sound" — single model for both dimensions
- **Google-backed research:** Well-documented, peer-reviewed, actively maintained in IQA-PyTorch
- **Apache 2.0 license:** Fully permissive for any use
- **Multiple training checkpoints:** Can choose checkpoint based on use case (aesthetic vs. technical focus)
- **Handles diverse aspect ratios:** Native resolution processing means no distortion from forced resizing
## What It's Bad At
- **Still no genre awareness:** General-purpose model, not wedding-specific
- **No context understanding:** Same as LAION — cannot distinguish emotional moments from empty frames
- **Slower than CLIP-based:** Transformer architecture is heavier (~50-150ms vs ~10-30ms for LAION V2)
- **Score range 0-100:** Different from LAION's 0-10 scale (requires threshold adjustment if combining)
- **Limited community adoption:** Less community testing and discussion compared to LAION predictors
- **Checkpoint selection matters:** Different checkpoints optimized for different datasets — no single "best" checkpoint for all use cases
- **TF/HF ecosystem split:** Official code is TensorFlow; PyTorch users need IQA-PyTorch or unofficial impl
docs/research-overview.md
+40
View File
@@ -0,0 +1,40 @@
# Research — Overview & Use Case Context
**Date:** 2026-08-23
**Source:** GitHub, HuggingFace, arXiv, LAION Blog, PyPI, Commercial Tool Benchmarks, Reddit, IQA Toolboxes
## Use Case Context
Sorting ~4,000 wedding photos — need to separate keeps from rejects based on aesthetic/technical quality. Requirements: batch processing, self-hosted or lightweight, no cloud dependency, fast inference on 4,000 images.
## Models Covered
1. **LAION Aesthetic Predictor** — CLIP-based aesthetic scoring (V1 & V2)
2. **SigLIP-Based Aesthetic Predictor V2.5** — SigLIP alternative to OpenAI CLIP
3. **MUSIQ** — Multi-Scale Image Quality Transformer (Google Research)
4. **Q-Align** — Qwen-based Visual Scorer
5. **BRISQUE / NIQE** — Classical no-reference IQA (CPU-only)
6. **Commercial Tools** — FilterPixel, Aftershoot, Imagen AI (for reference)
7. **IQA-PyTorch Toolbox** — All-in-one metric library
## Key Takeaways
1. **LAION Aesthetic Predictor V2 is the simplest starting point** — one-line Python API, MIT license, ~2 GB VRAM, fast inference. But it scores "aesthetic appeal," not "technical quality." It will NOT detect blur, out-of-focus shots, or bad exposure.
2. **Best approach: Combine models.** Use BRISQUE/NIQE for technical quality (blur/noise detection) + LAION V2 or MUSIQ for aesthetic scoring. A photo should be kept only if it passes BOTH thresholds.
3. **MUSIQ is the best single-model option** if you want one model that captures both aesthetic and technical quality. It's from Google Research, handles full-resolution images, and has Apache 2.0 license.
4. **SigLIP V2.5 is a modest improvement over LAION V2** — better on illustrations/art, but same fundamental limitations for real photography. Worth trying if you have BF16 GPU support.
5. **No open-source model understands wedding context.** None of these can distinguish a "peak moment" from a "transitional shot" or detect emotional content. For that, you need commercial tools like FilterPixel DeepCull (genre-specific AI) or Aftershoot (learning-based).
6. **For 4,000 wedding photos on a consumer GPU (RTX 4090):**
- LAION V2 batch: ~5-10 minutes
- MUSIQ batch: ~15-20 minutes
- BRISQUE (CPU): ~10-20 minutes
- Combined approach (LAION + BRISQUE): ~15-25 minutes total
7. **Bias warning:** LAION V2 has documented Western/cultural bias. For culturally diverse weddings (e.g., Indonesian, South Asian, African), scores may not accurately reflect the quality of ceremony shots, cultural attire, or traditional poses.
8. **Practical recommendation:** Start with LAION V2 + BRISQUE as a first-pass filter. Score all 4,000 images, set thresholds (e.g., LAION > 5.5 AND BRISQUE < 30), and manually review the borderline cases. This reduces 4,000 images to ~500-800 for manual review.
docs/research-q-align.md
+37
View File
@@ -0,0 +1,37 @@
# Q-Align — Qwen-based Visual Scorer
## Model Info
- **GitHub:** <https://github.com/Q-Future/Q-Align>
- **Paper:** ICML 2024 — "Teaching LMMs for Visual Scoring via Discrete Text-Defined Levels"
- **HuggingFace:** Multiple checkpoints available
- **License:** Apache 2.0
- **Latest:** Q-ReAlign (Qwen3.5-VL backbone) — June 2026 update in IQA-PyTorch
## Architecture
- **Type:** Vision-Language Model (VLM) fine-tuned for visual scoring
- **Backbone:** Qwen LLM + vision encoder
- **Output:** Discrete score levels (text-defined)
- **Innovation:** Uses reinforcement learning to align model scoring with human perception
## Hardware Requirements
- **Mini (0.8B):** ~3 GB VRAM
- **Lite (4B):** ~10 GB VRAM
- **Pro (9B):** ~20 GB VRAM
- **Batch size:** 1-4 (VLMs are memory-intensive)
## What It's Good At
- **Human-aligned scoring:** Trained to match human subjective ratings
- **Explainable:** Can generate text descriptions of why an image scored high/low
- **Multi-task:** Can do IQA, IAA (Image Aesthetic Assessment), and VQA
- **Fine-tunable:** Can be adapted to specific domains (e.g., wedding photos)
## What It's Bad At
- **Slower inference:** VLM architecture is significantly slower than CLIP-based models
- **Higher VRAM:** Even the mini variant needs 3 GB; not ideal for batch processing thousands of images
- **Still no wedding-specific training:** General-purpose model
- **Complex setup:** Requires more dependencies and configuration than LAION predictors
docs/research-siglip-v2-5.md
+55
View File
@@ -0,0 +1,55 @@
# SigLIP-Based Aesthetic Predictor V2.5
## Model Info
- **GitHub:** <https://github.com/discus0434/aesthetic-predictor-v2-5>
- **PyPI:** `aesthetic-predictor-v2-5`
- **License:** MIT
- **Updated:** December 2024
- **Improvement over V2:** Uses SigLIP (Google's CLIP alternative) instead of OpenAI CLIP; better at illustrations and diverse image domains
## Architecture
- **Backbone:** SigLIP (Google) — superior to OpenAI CLIP on many benchmarks
- **Head:** Linear layer on SigLIP embeddings
- **Output:** Single float score (0-10 scale)
- **Score threshold:** 5.5+ is considered "great" (vs. 6+ for V2)
## Hardware Requirements
- **SigLIP ViT:** ~2-3 GB VRAM (slightly larger than CLIP ViT-L/14)
- **Precision:** Requires BF16 for best results
- **Batch size:** ~16-32 on 24 GB GPU
## Python Usage
```python
from aesthetic_predictor_v2_5 import convert_v2_5_from_siglip
from PIL import Image
import torch
model, preprocessor = convert_v2_5_from_siglip(low_cpu_mem_usage=True)
model = model.to(torch.bfloat16).cuda()
image = Image.open("photo.jpg").convert("RGB")
pixel_values = preprocessor(images=image, return_tensors="pt").pixel_values.to(torch.bfloat16).cuda()
with torch.inference_mode():
score = model(pixel_values).logits.squeeze().float().cpu().numpy()
print(f"Aesthetics score: {score:.2f}")
```
## What It's Good At
- **Better than V2 on illustrations/art:** SigLIP trained on larger, more diverse dataset
- **Same simplicity:** One-line HuggingFace-style API
- **Same speed:** Linear head on embeddings
- **Slightly better generalization:** SigLIP's training data is more diverse than OpenAI CLIP's
## What It's Bad At
- **Same fundamental limitations as V2:** No technical quality detection, no blur/exposure detection, no genre awareness
- **Still not trained on real photography:** Same bias toward AI-generated aesthetic patterns
- **BF16 requirement:** Needs GPU with BF16 support (RTX 30-series+); not ideal for older hardware
- **Community model:** Not an official LAION release; maintained by a third party
docs/research-summary.md
+40
View File
@@ -0,0 +1,40 @@
# Alternatives Comparison & Summary
## Comparison Table
| Model | Type | Score Scale | Technical Quality | Aesthetic Quality | GPU Required | Speed (4K images) | Self-Hosted | License | Wedding-Specific |
|-------|------|-------------|-------------------|-------------------|--------------|-------------------|-------------|---------|------------------|
| **LAION V2 (CLIP)** | CLIP+Linear | 0-10 | No | Basic | Yes (~2GB) | ~5-10 min | Yes | MIT | No |
| **SigLIP V2.5** | SigLIP+Linear | 0-10 | No | Better | Yes (~3GB) | ~5-10 min | Yes | MIT | No |
| **MUSIQ** | Transformer | 0-100 | Yes | Combined | Yes (~4GB) | ~15-20 min | Yes | Apache 2.0 | No |
| **Q-Align Mini** | VLM (0.8B) | Text levels | Yes | Human-aligned | Yes (~3GB) | ~30-60 min | Yes | Apache 2.0 | No |
| **BRISQUE** | Handcrafted | Inverted | Yes | No | No (CPU) | ~10-20 min | Yes | Academic | No |
| **NIQE** | Handcrafted | Direct | Yes | No | No (CPU) | ~10-20 min | Yes | Academic | No |
| **FilterPixel DeepCull** | Cloud AI | 10 params | Yes | Genre-aware | N/A | ~5 min | No | Commercial | Yes |
| **Aftershoot** | Desktop AI | Stars | Yes | Genre-aware | Yes | ~10 min | Yes (local) | Commercial | Yes |
## Key Takeaways for Wedding Photo Sorting
1. **LAION Aesthetic Predictor V2 is the simplest starting point** — one-line Python API, MIT license, ~2 GB VRAM, fast inference. But it scores "aesthetic appeal," not "technical quality." It will NOT detect blur, out-of-focus shots, or bad exposure.
2. **Best approach: Combine models.** Use BRISQUE/NIQE for technical quality (blur/noise detection) + LAION V2 or MUSIQ for aesthetic scoring. A photo should be kept only if it passes BOTH thresholds.
3. **MUSIQ is the best single-model option** if you want one model that captures both aesthetic and technical quality. It's from Google Research, handles full-resolution images, and has Apache 2.0 license.
4. **SigLIP V2.5 is a modest improvement over LAION V2** — better on illustrations/art, but same fundamental limitations for real photography. Worth trying if you have BF16 GPU support.
5. **No open-source model understands wedding context.** None of these can distinguish a "peak moment" from a "transitional shot" or detect emotional content. For that, you need commercial tools like FilterPixel DeepCull (genre-specific AI) or Aftershoot (learning-based).
6. **For 4,000 wedding photos on a consumer GPU (RTX 4090):**
- LAION V2 batch: ~5-10 minutes
- MUSIQ batch: ~15-20 minutes
- BRISQUE (CPU): ~10-20 minutes
- Combined approach (LAION + BRISQUE): ~15-25 minutes total
7. **Bias warning:** LAION V2 has documented Western/cultural bias. For culturally diverse weddings (e.g., Indonesian, South Asian, African), scores may not accurately reflect the quality of ceremony shots, cultural attire, or traditional poses.
8. **Practical recommendation:** Start with LAION V2 + BRISQUE as a first-pass filter. Score all 4,000 images, set thresholds (e.g., LAION > 5.5 AND BRISQUE < 30), and manually review the borderline cases. This reduces 4,000 images to ~500-800 for manual review.
## Update Log
- **2026-08-23:** Initial research for wedding photo sorting use case (~4,000 images). Covered LAION V1/V2, SigLIP V2.5, MUSIQ, Q-Align, BRISQUE, NIQE, commercial tools (FilterPixel, Aftershoot, Imagen AI), and IQA-PyTorch toolbox.
docs/scripts.md
+85
View File
@@ -0,0 +1,85 @@
# Project Scripts
## Setup Scripts
### `setup.bat` (Windows) / `setup.sh` (Linux/Mac)
Creates a new conda environment and installs all required packages from `requirements.txt`.
**What it does:**
1. Checks that conda is installed
2. Creates a conda environment named `photo_judgers` with Python 3.11
3. Installs all packages from `requirements.txt`:
- `torch`, `torchvision`, `torchaudio` (PyTorch with CUDA 12.1)
- `transformers` (HuggingFace)
- `simple-aesthetics-predictor` (LAION aesthetic predictor)
- `Pillow` (image processing)
- `tqdm` (progress bars)
- `numpy` (numerical operations)
**Usage:**
```bash
# Windows
setup.bat
# Linux/Mac
chmod +x setup.sh
./setup.sh
```
## Run Scripts
### `run.bat` (Windows) / `run.sh` (Linux/Mac)
Activates the conda environment and runs the scorer.
**What it does:**
1. Activates the `photo_judgers` conda environment
2. Runs `python -m src.scorer` with any additional arguments passed through
**Usage:**
```bash
# Windows
run.bat
# Linux/Mac
chmod +x run.sh
./run.sh
```
**Optional arguments:**
- `--input <path>` — Directory or file to score
- `--output <path>` — Output JSON file (default: `output.json`)
- `--threshold <value>` — Score threshold for keep/reject (default: TBD)
## Project Structure
```
photo_judgers/
├── src/
│ ├── __init__.py
│ ├── scorer.py # Menu + proxy (entry point)
│ └── scorers/
│ ├── __init__.py
│ ├── base.py # BaseScorer abstract class
│ └── laion.py # LAION V2 implementation
├── requirements.txt
├── setup.bat / setup.sh
├── run.bat / run.sh
├── output.json # Generated by scorer
├── docs/
├── test/
└── .pi/
```
## Adding a New Scorer
1. Create `src/scorers/<name>.py` with a class that follows the LAION pattern
2. Register it in `src/scorer.py` by adding to the `SCORERS` list
3. Add any new dependencies to `requirements.txt`
4. Update `src/scorers/__init__.py` to export the new class