Phase 8: Serverless GPU Deployment & Infrastructure · 45 min · Python · boto3 · FFmpeg

Storage & Delivery Infrastructure

S3 for storage, CDN for delivery, WebP/AVIF for images, H.264 for video — the delivery layer is as engineered as the generation layer.

Hiring signal: Storage and delivery infrastructure (S3, CDN, format conversion) demonstrates you understand the full production pipeline from generation to end-user delivery.

What you will learn

Implement object storage: S3, GCS, Cloudflare R2 for generated assets
Tag metadata: prompt, seed, model, parameters, generation timestamp
Configure CDN delivery: CloudFront, Cloudflare for low-latency asset delivery
Implement format conversion: WebP/AVIF for images, H.264/H.265 for video, MP3/Opus for audio

The Problem

A generative media pipeline produces 10,000 images and 500 videos per day. Where do they go? How do users access them? What formats are delivered?

Storage: S3, GCS, Cloudflare R2 — where generated assets live
Metadata: prompt, seed, model, parameters — tagged for search and reproducibility
CDN: CloudFront, Cloudflare — low-latency delivery to global users
Format conversion: WebP/AVIF for images, H.264/H.265 for video, MP3/Opus for audio

The delivery layer is as engineered as the generation layer.

What you'll build
Implement object storage (S3/R2) with metadata tagging, CDN delivery configuration, and format conversion (WebP/AVIF for images, H.264 for video, MP3/Opus for audio) using FFmpeg and Pillow.

Object Storage

S3 (AWS)

import boto3
from datetime import datetime

s3 = boto3.client("s3",
    aws_access_key_id=os.environ["AWS_ACCESS_KEY_ID"],
    aws_secret_access_key=os.environ["AWS_SECRET_ACCESS_KEY"],
)

def upload_to_s3(image_bytes: bytes, bucket: str, key: str,
                 metadata: dict = None) -> str:
    """Upload generated asset to S3 with metadata."""
    extra_args = {
        "ContentType": "image/png",
        "Metadata": {
            "prompt": metadata.get("prompt", ""),
            "seed": str(metadata.get("seed", "")),
            "model": metadata.get("model", ""),
            "generated_at": datetime.utcnow().isoformat(),
        },
    }
    
    s3.put_object(Bucket=bucket, Key=key, Body=image_bytes, **extra_args)
    return f"https://{bucket}.s3.amazonaws.com/{key}"

Cloudflare R2 (S3-compatible, no egress fees)

# R2 uses S3-compatible API
s3 = boto3.client("s3",
    endpoint_url=os.environ["R2_ENDPOINT"],  # https://<account>.r2.cloudflarestorage.com
    aws_access_key_id=os.environ["R2_ACCESS_KEY"],
    aws_secret_access_key=os.environ["R2_SECRET_KEY"],
    region_name="auto",
)

# Same API as S3 — just different endpoint
s3.put_object(Bucket="generated-assets", Key=key, Body=image_bytes)

Storage Selection

Storage	Egress Cost	Pros	Best For
AWS S3	$0.09/GB	Mature ecosystem, lifecycle rules	General purpose
Cloudflare R2	$0 (free)	No egress fees, S3-compatible	Cost-sensitive, CDN integration
Google GCS	$0.12/GB	Integrated with GCP	GCP-native stacks

Why is Cloudflare R2 often preferred over AWS S3 for generative media storage?

R2 is faster

Unlock the full lesson

You've read the first 2 sections. The rest of this lesson covers Metadata Tagging, CDN Delivery, Format Conversion, Delivery Pipeline, Key Takeaways, What's Next — plus a hands-on lab, quiz, and project artifact.

Create a free account to unlock Phase 0 and Phase 1 of every course — no credit card.

Key terms in this lesson

Latency