Phase 8: Serverless GPU Deployment & Infrastructure · 50 min · Python · Hugging Face Transformers

GPU Resource Management

H100 for images, H200 for video, B200 for multi-model parallel. NVFP4 quantization: 3x faster, 60% less VRAM. Know your GPUs.

Hiring signal: GPU resource management (selection by modality, VRAM, quantization, cold starts) is a specialized infrastructure skill that commands premium salaries in generative media.

What you will learn

Select GPUs by modality: H100 80GB (images), H200 141GB (video), B200 (multi-model parallel)
Manage VRAM: model loading, offloading, concurrent request handling
Apply quantization: NVFP4 (4-bit) and NVFP8 (8-bit) — 3x faster, 60% less VRAM, minimal quality loss
Optimize batch processing and cold starts: warm pools, model preloading, keep-alive

The Problem

A team needs to select GPUs for different generative media workloads:

Image generation (FLUX, SDXL): needs 20-40GB VRAM, 5-15s per image
Video generation (Runway, Kling): needs 60-80GB VRAM, 60-180s per video
Multi-model parallel (image + video + audio): needs 100+GB VRAM
Batch processing (1000 images): needs efficient VRAM management

Choosing the wrong GPU means OOM errors, slow generation, or wasted money.

What you'll build
Select GPUs by modality (H100 for images, H200 for video, B200 for multi-model), manage VRAM (loading, offloading, concurrent requests), apply quantization (NVFP4/NVFP8), and optimize cold starts with warm pools.

GPU Selection by Modality

GPU Comparison

GPU	VRAM	Bandwidth	Cost (per hour)	Best For
RTX 4090	24GB	1,008 GB/s	~$0.40	Image generation (FLUX, SDXL)
A100 80GB	80GB	2,039 GB/s	~$1.50	Image batch, video generation
H100 80GB	80GB	3,350 GB/s	~$2.50	Fast image generation, video
H200 141GB	141GB	4,800 GB/s	~$4.00	Video generation, large models
B200 192GB	192GB	8,000 GB/s	~$6.00	Multi-model parallel, largest models

Selection Framework

from dataclasses import dataclass

@dataclass
class GPUSpec:
    """GPU specification."""
    name: str
    vram_gb: int
    bandwidth_gbs: int
    cost_per_hour: float
    best_for: str

GPUS = {
    "RTX 4090": GPUSpec("RTX 4090", 24, 1008, 0.40, "Image generation"),
    "A100 80GB": GPUSpec("A100 80GB", 80, 2039, 1.50, "Image batch, video"),
    "H100 80GB": GPUSpec("H100 80GB", 80, 3350, 2.50, "Fast image, video"),
    "H200 141GB": GPUSpec("H200 141GB", 141, 4800, 4.00, "Video, large models"),
    "B200 192GB": GPUSpec("B200 192GB", 192, 8000, 6.00, "Multi-model parallel"),
}

def select_gpu(modality: str, model_vram: int, batch_size: int = 1) -> str:
    """Select GPU based on modality and VRAM requirements."""
    required_vram = model_vram * batch_size
    
    if modality == "image":
        if required_vram <= 24:
            return "RTX 4090"  # Cheapest, sufficient for single image
        elif required_vram <= 80:
            return "H100 80GB"  # Fast, good for batch
        else:
            return "H200 141GB"  # Large batch
    
    elif modality == "video":
        if required_vram <= 80:
            return "H100 80GB"  # Fast video generation
        else:
            return "H200 141GB"  # Large video models
    
    elif modality == "multi-model":
        return "B200 192GB"  # Load multiple models simultaneously
    
    elif modality == "batch":
        # Cost-optimized: cheapest GPU that fits
        for name, gpu in sorted(GPUS.items(), key=lambda x: x[1].cost_per_hour):
            if gpu.vram_gb >= required_vram:
                return name
    
    return "H100 80GB"  # Default

Which GPU should you select for video generation with a model that requires 60GB VRAM?

B200 — it has the most VRAM

Unlock the full lesson

You've read the first 2 sections. The rest of this lesson covers VRAM Management, Quantization: NVFP4 and NVFP8, Cold Start Optimization, Key Takeaways, What's Next — plus a hands-on lab, quiz, and project artifact.

Create a free account to unlock Phase 0 and Phase 1 of every course — no credit card.

Key terms in this lesson

Quantization