init: project structure with LAION scorer, conda setup, and research docs

2026-08-23 15:31:47 +00:00
commit 4f90f5838a
25 changed files with 1628 additions and 0 deletions
@@ -0,0 +1,40 @@
+# Research — Overview & Use Case Context
+
+**Date:** 2026-08-23
+**Source:** GitHub, HuggingFace, arXiv, LAION Blog, PyPI, Commercial Tool Benchmarks, Reddit, IQA Toolboxes
+
+## Use Case Context
+
+Sorting ~4,000 wedding photos — need to separate keeps from rejects based on aesthetic/technical quality. Requirements: batch processing, self-hosted or lightweight, no cloud dependency, fast inference on 4,000 images.
+
+## Models Covered
+
+1. **LAION Aesthetic Predictor** — CLIP-based aesthetic scoring (V1 & V2)
+2. **SigLIP-Based Aesthetic Predictor V2.5** — SigLIP alternative to OpenAI CLIP
+3. **MUSIQ** — Multi-Scale Image Quality Transformer (Google Research)
+4. **Q-Align** — Qwen-based Visual Scorer
+5. **BRISQUE / NIQE** — Classical no-reference IQA (CPU-only)
+6. **Commercial Tools** — FilterPixel, Aftershoot, Imagen AI (for reference)
+7. **IQA-PyTorch Toolbox** — All-in-one metric library
+
+## Key Takeaways
+
+1. **LAION Aesthetic Predictor V2 is the simplest starting point** — one-line Python API, MIT license, ~2 GB VRAM, fast inference. But it scores "aesthetic appeal," not "technical quality." It will NOT detect blur, out-of-focus shots, or bad exposure.
+
+2. **Best approach: Combine models.** Use BRISQUE/NIQE for technical quality (blur/noise detection) + LAION V2 or MUSIQ for aesthetic scoring. A photo should be kept only if it passes BOTH thresholds.
+
+3. **MUSIQ is the best single-model option** if you want one model that captures both aesthetic and technical quality. It's from Google Research, handles full-resolution images, and has Apache 2.0 license.
+
+4. **SigLIP V2.5 is a modest improvement over LAION V2** — better on illustrations/art, but same fundamental limitations for real photography. Worth trying if you have BF16 GPU support.
+
+5. **No open-source model understands wedding context.** None of these can distinguish a "peak moment" from a "transitional shot" or detect emotional content. For that, you need commercial tools like FilterPixel DeepCull (genre-specific AI) or Aftershoot (learning-based).
+
+6. **For 4,000 wedding photos on a consumer GPU (RTX 4090):**
+   - LAION V2 batch: ~5-10 minutes
+   - MUSIQ batch: ~15-20 minutes
+   - BRISQUE (CPU): ~10-20 minutes
+   - Combined approach (LAION + BRISQUE): ~15-25 minutes total
+
+7. **Bias warning:** LAION V2 has documented Western/cultural bias. For culturally diverse weddings (e.g., Indonesian, South Asian, African), scores may not accurately reflect the quality of ceremony shots, cultural attire, or traditional poses.
+
+8. **Practical recommendation:** Start with LAION V2 + BRISQUE as a first-pass filter. Score all 4,000 images, set thresholds (e.g., LAION > 5.5 AND BRISQUE < 30), and manually review the borderline cases. This reduces 4,000 images to ~500-800 for manual review.