Phase 9: Evaluation, Production & Capstone · 55 min · Python · PyTorch · CLIP

Image Quality Metrics

FID, CLIPScore, Aesthetic Score, ImageReward — if you can't measure quality, you can't engineer it.

Hiring signal: Image quality metrics knowledge (FID, CLIPScore, Aesthetic Score, ImageReward) is tested in quality evaluation exercises — a common interview format for generative media roles.

What you will learn

Implement FID (Fréchet Inception Distance): distribution-level quality comparing generated vs real image distributions
Implement CLIPScore: text-image alignment measuring how well image matches prompt
Implement Aesthetic Score: LAION-trained predictor for visual appeal (1-10 scale)
Implement ImageReward/HPS: human preference-trained reward models

The Problem

A team generates 100 images with FLUX and 100 with SDXL. Which model produces better images? Without metrics, it's subjective opinion. With metrics, it's data-driven engineering.

FID (Fréchet Inception Distance): distribution-level quality (lower = better)
CLIPScore: text-image alignment (higher = better, 0-1 scale)
Aesthetic Score: visual appeal (1-10 scale, LAION-trained)
ImageReward: human preference model (higher = better)

What you'll build
Implement FID for distribution comparison, CLIPScore for prompt-image alignment, Aesthetic Score for visual appeal, and ImageReward for human preference prediction. Build a multi-metric quality evaluator.

FID: Fréchet Inception Distance

What It Measures

FID compares the distribution of generated images to real images:

Extract features from both sets using InceptionV3
Compute Fréchet distance between the two Gaussian distributions
Lower FID = better (generated images are closer to real images)

# FID requires a reference dataset of "real" images
# Compare: 1000 generated images vs 1000 real images

# Using pytorch-fid: https://github.com/mseitzer/pytorch-fid
# Command line:
# python -m pytorch_fid path/to/generated path/to/real

# Programmatic:
from pytorch_fid.fid_score import calculate_fid_given_paths

fid_score = calculate_fid_given_paths(
    paths=["generated_images/", "real_images/"],
    batch_size=50,
    device="cuda",
    dims=2048,  # InceptionV3 feature dimension
)
print(f"FID: {fid_score:.2f}")  # Lower is better

FID Interpretation

FID Score	Quality	Interpretation
< 10	Excellent	Generated ≈ real
10-30	Good	High quality, minor artifacts
30-50	Fair	Noticeable artifacts
50-100	Poor	Significant quality issues
> 100	Bad	Clearly synthetic

FID Limitations
FID measures distribution similarity, NOT individual image quality. A model with low FID can still produce bad individual images. FID also requires a reference dataset of "real" images, which may not always be available. Use FID for model comparison, not per-image quality gates.

Unlock the full lesson

You've read the first 2 sections. The rest of this lesson covers CLIPScore: Text-Image Alignment, Aesthetic Score (LAION), ImageReward, Multi-Metric Quality Evaluator, Metric Comparison, Key Takeaways, What's Next — plus a hands-on lab, quiz, and project artifact.

Create a free account to unlock Phase 0 and Phase 1 of every course — no credit card.