Build a Local AI Dev Environment · 45 min · Qdrant · Open WebUI · nomic-embed-text

Local RAG — Chat With Your Own Documents

Cited answers from your own real documents, running entirely on your own machine, is the actual payoff this whole course has been building toward.

Hiring signal: Local RAG over your own private documents -- with zero data leaving your machine -- is exactly the kind of privacy-first capability driving real enterprise and individual adoption of local AI in 2026.

What you will learn

Add Qdrant to the Docker Compose stack
Configure Open WebUI to use Qdrant instead of its default vector store
Load your own documents and verify cited answers

What You're Building

The reason this whole stack exists: your own real documents, chunked and embedded locally, queryable in Open WebUI with actual citations back to your files — no API call, no upload, nothing leaving your machine.

Add Qdrant to the same compose file

services:
  open-webui:
    # ... unchanged from Lesson 3 ...
    environment:
      - OLLAMA_BASE_URL=http://host.docker.internal:11434
      - VECTOR_DB=qdrant
      - QDRANT_URI=http://qdrant:6333
      - RAG_EMBEDDING_ENGINE=ollama
      - RAG_EMBEDDING_MODEL=nomic-embed-text
      - RAG_OLLAMA_BASE_URL=http://host.docker.internal:11434
    depends_on:
      - qdrant

  qdrant:
    image: qdrant/qdrant:latest
    ports:
      - "6333:6333"
    volumes:
      - qdrant-data:/qdrant/storage
    restart: unless-stopped

volumes:
  open-webui-data:
  qdrant-data:

VECTOR_DB=qdrant + QDRANT_URI=http://qdrant:6333 is genuinely simpler than it used to be -- Open WebUI now has first-class Qdrant support via these two environment variables, replacing what was previously a more involved manual config swap away from its default ChromaDB. Note qdrant:6333 (not localhost) as the hostname: open-webui and qdrant are both containers on the SAME Docker Compose network, so they reach each other by service name directly -- this is different from Lesson 3's host.docker.internal, which was specifically for reaching something OUTSIDE Docker (Ollama on the host).

Without these 3 lines, nomic-embed-text is pulled in Lesson 2 for nothing
VECTOR_DB/QDRANT_URI only tell Open WebUI where to store vectors -- they say nothing about which model creates them. Without RAG_EMBEDDING_ENGINE=ollama and RAG_EMBEDDING_MODEL=nomic-embed-text set, Open WebUI silently falls back to its own bundled sentence-transformers/all-MiniLM-L6-v2 running inside the container -- a real embedding model, but not the one this course pulled in Lesson 2, and not one you chose. Verify which one actually ran: docker exec <open-webui-container> python3 -c "import sqlite3; c=sqlite3.connect('/app/backend/data/webui.db').cursor(); c.execute(\"SELECT key,value FROM config WHERE key IN ('rag.embedding_engine','rag.embedding_model')\"); print(c.fetchall())" should show ollama / nomic-embed-text, not the sentence-transformers default. One more gotcha: this config is written to Open WebUI's database on first boot -- if you add these env vars to a compose file whose open-webui-data volume already exists from an earlier run, they won't retroactively apply. You'd need to change the embedding model in Admin Settings, or start from a fresh volume.

Two different networking patterns in one file, and knowing which applies where
Ollama runs on your host machine -> containers reach it via host.docker.internal. Qdrant runs as a container in the SAME compose file -> Open WebUI reaches it via the service name qdrant directly, because Docker Compose puts all its services on one shared network automatically. Mixing these up (trying to reach Qdrant via host.docker.internal, or Ollama via a made-up service name) is the single most common Docker Compose networking mistake in a stack this size.

Unlock the full lesson

You've read the first 2 sections. The rest of this lesson covers The pipeline: chunk, embed, store, retrieve, generate, Load your own documents and verify — don't trust, check, What you're building today — plus a hands-on lab, quiz, and project artifact.

Create a free account to unlock Phase 0 and Phase 1 of every course — no credit card.