init: project structure with LAION scorer, conda setup, and research docs
This commit is contained in:
Kareem Horstink
@@ -0,0 +1,159 @@
|
||||
# BRISQUE / NIQE — Classical No-Reference IQA
|
||||
|
||||
## Model Info
|
||||
|
||||
- **BRISQUE:** "No-reference Image Quality Assessment in the Spatial Domain" — Mittal et al., IEEE TIP 2012
|
||||
- **NIQE:** "Making a 'Completely Blind' Image Quality Analyzer" — Mittal et al., IEEE SPL 2013
|
||||
- **Python packages:**
|
||||
- `pip install brisque[opencv-python]` — Latest BRISQUE (supports opencv-contrib, opencv-headless)
|
||||
- `pip install pybrisque` — Alternative BRISQUE (older, from Bukalapak)
|
||||
- `pip install image-quality` — Contains `imquality.brisque` module
|
||||
- `pip install pyiqa` — IQA-PyTorch toolbox (includes BRISQUE + NIQE + 30+ other metrics)
|
||||
- `pip install nr-iqa` — Comprehensive library (BRISQUE + NIQE + PIQE)
|
||||
- **License:** Academic (BRISQUE: BSD-like via LearnOpenCV; NIQE: academic use)
|
||||
- **Dependencies:** OpenCV, NumPy, SciPy (BRISQUE); skvideo + NumPy (NIQE)
|
||||
|
||||
## Architecture
|
||||
|
||||
### BRISQUE (Blind/Referenceless Image Spatial Quality Evaluator)
|
||||
|
||||
- **Type:** Handcrafted features + Support Vector Regression (SVR)
|
||||
- **Pipeline:**
|
||||
1. **MSCN (Mean Subtracted Contrast Normalized) coefficients** — decorrelates image from local brightness/contrast
|
||||
2. **GGD (Generalized Gaussian Distribution) fitting** to MSCN coefficients — extracts shape parameter alpha (tail heaviness)
|
||||
3. **Pairwise products** in 4 orientations (horizontal, vertical, 2 diagonals) — 16 features
|
||||
4. **AGGD (Asymmetric GGD) fitting** to pairwise products
|
||||
5. **Multi-scale analysis** at 2 scales (original + 0.5x downsampled) — 36-D feature vector total (18 per scale)
|
||||
6. **SVR prediction** maps 36-D features to quality score in [0, 100]
|
||||
- **Training data:** TID2008 dataset (human-rated distorted images, MOS scores 0-100)
|
||||
- **Score direction:** **Lower = better** (0 = perfect, 100 = worst quality)
|
||||
|
||||
### NIQE (Natural Image Quality Evaluator)
|
||||
|
||||
- **Type:** Handcrafted features + Multivariate Gaussian (MVG) model
|
||||
- **Key difference from BRISQUE:** Completely opinion-unaware — does NOT need human-rated distorted images for training
|
||||
- **Pipeline:**
|
||||
1. Extract 18-D NSS features from 96x96 patches (same as BRISQUE scale-1)
|
||||
2. Fit MVG model (mu_ref, Sigma_ref) to pristine images — the "naturalness" reference
|
||||
3. Fit MVG (mu_test, Sigma_test) to test image patches
|
||||
4. **Quality = Mahalanobis distance** between the two MVG models
|
||||
- **Training data:** Only pristine (undistorted) images — no human ratings needed
|
||||
- **Score direction:** **Lower = better** (0 = perfectly natural, higher = more distorted)
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
# Option 1: Latest BRISQUE (recommended)
|
||||
pip install brisque[opencv-python]
|
||||
|
||||
# Option 2: For servers/Docker (headless)
|
||||
pip install brisque[opencv-python-headless]
|
||||
|
||||
# Option 3: Comprehensive NR-IQA library (BRISQUE + NIQE + PIQE)
|
||||
pip install nr-iqa
|
||||
|
||||
# Option 4: IQA-PyTorch toolbox (BRISQUE + NIQE + 30+ other metrics)
|
||||
pip install pyiqa
|
||||
```
|
||||
|
||||
## Python Usage
|
||||
|
||||
```python
|
||||
# --- BRISQUE via brisque package ---
|
||||
from brisque import BRISQUE
|
||||
import cv2
|
||||
|
||||
brisque = BRISQUE()
|
||||
img = cv2.imread('photo.jpg')
|
||||
score = brisque.score(img) # Returns float in [0, 100]
|
||||
print(f"BRISQUE: {score:.2f}") # Lower = better
|
||||
|
||||
# --- BRISQUE via image-quality package ---
|
||||
from imquality.brisque import BRISQUE
|
||||
from PIL import Image
|
||||
|
||||
img = Image.open('photo.jpg')
|
||||
score = BRISQUE().score(img) # Returns float in [0, 100]
|
||||
|
||||
# --- NIQE via nr-iqa package ---
|
||||
from nr_iqa import NIQE
|
||||
import cv2
|
||||
|
||||
niqe = NIQE()
|
||||
img = cv2.imread('photo.jpg', cv2.IMREAD_GRAYSCALE)
|
||||
score = niqe.score(img) # Returns float in [0, 100]
|
||||
print(f"NIQE: {score:.4f}") # Lower = better
|
||||
|
||||
# --- Both via IQA-PyTorch toolbox ---
|
||||
import pyiqa
|
||||
device = 'cuda' if torch.cuda.is_available() else 'cpu'
|
||||
|
||||
brisque = pyiqa.create_metric('brisque', device=device)
|
||||
niqe = pyiqa.create_metric('niqe', device=device)
|
||||
|
||||
score_b = brisque('./photo.jpg')
|
||||
score_n = niqe('./photo.jpg')
|
||||
|
||||
print(f"BRISQUE: {score_b:.2f}, NIQE: {score_n:.4f}")
|
||||
|
||||
# Batch processing
|
||||
import glob
|
||||
for path in glob.glob('./wedding-photos/*.jpg'):
|
||||
score = brisque(path)
|
||||
```
|
||||
|
||||
## Score Interpretation for Wedding Photos
|
||||
|
||||
| BRISQUE Score | NIQE Score | Interpretation | Recommended Action |
|
||||
|---------------|-----------|---------------|-------------------|
|
||||
| 0-10 | 0-3 | Excellent technical quality | Auto-keep |
|
||||
| 10-25 | 3-6 | Good technical quality | Keep |
|
||||
| 25-40 | 6-10 | Acceptable, minor issues | Review |
|
||||
| 40-60 | 10-15 | Poor technical quality | Likely reject |
|
||||
| 60-100 | 15+ | Very poor (blurry, noisy, compressed) | Auto-reject |
|
||||
|
||||
**Recommended thresholds for wedding culling:**
|
||||
|
||||
- BRISQUE < 30 AND NIQE < 8 -> keep (good technical quality)
|
||||
- BRISQUE > 60 OR NIQE > 15 -> reject (poor technical quality)
|
||||
- In between -> manual review
|
||||
|
||||
## Performance Benchmarks
|
||||
|
||||
| Metric | SRCC vs Human | PLCC vs Human | Notes |
|
||||
|--------|--------------|--------------|-------|
|
||||
| BRISQUE (TID2008) | ~0.85 | ~0.87 | Trained on TID2008 |
|
||||
| NIQE (KADID-10k) | ~0.83 | ~0.85 | Opinion-unaware, generalizes well |
|
||||
| BRISQUE (live) | ~0.81 | ~0.83 | Live (no-reference) dataset |
|
||||
| NIQE (live) | ~0.80 | ~0.82 | Live (no-reference) dataset |
|
||||
|
||||
## Hardware Requirements
|
||||
|
||||
- **CPU-only:** No GPU needed — purely classical (no neural networks)
|
||||
- **Speed per image:** ~50-200ms per image on modern CPU (single-core)
|
||||
- **Speed for 4,000 images:** ~3-15 minutes on CPU (single-core, depends on image size)
|
||||
- **Multi-threading:** Can parallelize across CPU cores — ~1-3 minutes with 8+ cores
|
||||
- **VRAM:** None required (BRISQUE/NIQE are pure NumPy/OpenCV)
|
||||
- **Memory usage:** ~50-100 MB RAM for batch processing
|
||||
|
||||
## What It's Good At
|
||||
|
||||
- **No GPU needed:** Runs on any CPU, even low-end machines
|
||||
- **Fast:** Good for quick technical quality screening — faster than any deep learning model
|
||||
- **Detects blur and compression artifacts:** Excellent at detecting out-of-focus shots, motion blur, JPEG artifacts, noise
|
||||
- **Simple:** Minimal dependencies (OpenCV + NumPy + SciPy)
|
||||
- **Deterministic:** No randomness — same image always gets same score
|
||||
- **Lightweight:** ~50 MB total install size, no model weights to download
|
||||
- **Good complement to aesthetic models:** Catches technical rejects that aesthetic models miss
|
||||
- **Works on any image type:** Not biased toward any particular visual style
|
||||
|
||||
## What It's Bad At
|
||||
|
||||
- **Purely technical:** Scores only technical quality (blur, noise, compression), not aesthetics at all
|
||||
- **Poor on artistic/creative images:** Penalizes artistic choices — shallow depth of field, high contrast, film grain, vignette, intentional blur
|
||||
- **Outdated methodology:** 2012-2013 techniques; inferior to modern deep learning approaches for complex distortions
|
||||
- **Sensitive to intentional artistic effects:** May score a portrait with beautiful bokeh (shallow DOF) as "low quality"
|
||||
- **Color-blind:** BRISQUE operates on luminance channel; NIQE on grayscale — ignores color information entirely
|
||||
- **Not suitable as sole scorer:** Must be combined with an aesthetic model for wedding photo sorting
|
||||
- **No multi-scale like MUSIQ:** Only processes at one scale (BRISQUE does 2-scale but simpler than MUSIQ's 3-scale)
|
||||
- **Training data limitations:** BRISQUE trained on TID2008 (lab-distorted images); may not generalize well to real-world wedding photo distortions
|
||||
Reference in New Issue
Block a user