Phase 7: Multi-Model Pipeline Orchestration · 55 min · Python · asyncio · Celery

Pipeline Patterns

Sequential, parallel fan-out, conditional branching, multi-modal — four patterns that cover every production pipeline.

Hiring signal: Pipeline pattern knowledge (sequential, parallel fan-out, conditional branching, multi-modal) is tested in system design interviews for generative media services.

What you will learn

Implement sequential pipeline: image → bg removal → upscale → style LoRA → delivery
Implement parallel fan-out: generate 10 variants simultaneously, score them, pick the best
Implement conditional branching: if quality < threshold → regenerate with different parameters
Implement multi-modal pipeline: LLM → image → video → audio → music → post-production → final output

The Problem

A team needs to build different types of generative media pipelines:

Product images: generate → bg removal → upscale → style → deliver (sequential)
A/B testing: generate 10 variants simultaneously, score, pick best (parallel fan-out)
Quality control: generate → check quality → if low, regenerate (conditional)
Full campaign: LLM → image → video → audio → music → post → deliver (multi-modal)

Each requires a different pipeline pattern. Knowing which pattern to use when is a core architecture skill.

What you'll build
Implement four pipeline patterns in Python with asyncio: sequential, parallel fan-out, conditional branching, and multi-modal. These cover every production generative media pipeline.

Pattern 1: Sequential Pipeline

Each step depends on the previous step's output:

Step A → Step B → Step C → Step D → Output

import asyncio

async def sequential_pipeline(prompt: str) -> dict:
    """Sequential pipeline: image → bg removal → upscale → delivery."""
    results = {}
    
    # Step 1: Generate image
    results["image"] = await generate_image(prompt)
    
    # Step 2: Remove background (depends on step 1)
    results["no_bg"] = await remove_background(results["image"])
    
    # Step 3: Upscale (depends on step 2)
    results["upscaled"] = await upscale(results["no_bg"])
    
    # Step 4: Deliver (depends on step 3)
    results["url"] = await upload_to_cdn(results["upscaled"])
    
    return results

async def generate_image(prompt: str) -> str:
    """Mock: generate image via FLUX API."""
    await asyncio.sleep(2)  # Simulate API call
    return f"image_generated_from_{prompt}"

async def remove_background(image: str) -> str:
    """Mock: remove background."""
    await asyncio.sleep(1)
    return f"nobg_{image}"

async def upscale(image: str) -> str:
    """Mock: upscale image."""
    await asyncio.sleep(2)
    return f"4k_{image}"

async def upload_to_cdn(image: str) -> str:
    """Mock: upload to CDN."""
    await asyncio.sleep(1)
    return f"https://cdn.example.com/{image}.png"

Total time: 2 + 1 + 2 + 1 = 6s (sum of all steps)

Unlock the full lesson

You've read the first 2 sections. The rest of this lesson covers Pattern 2: Parallel Fan-Out, Pattern 3: Conditional Branching, Pattern 4: Multi-Modal Pipeline, Pattern Selection Guide, Key Takeaways, What's Next — plus a hands-on lab, quiz, and project artifact.

Create a free account to unlock Phase 0 and Phase 1 of every course — no credit card.

Key terms in this lesson

LoRA (Low-Rank Adaptation)