Build a Local AI Dev Environment · 45 min · Qdrant · Open WebUI · nomic-embed-text
Objective
What you're building
The reason this whole stack exists: your own real documents, chunked and embedded locally, queryable in Open WebUI with actual citations back to your files — no API call, no upload, nothing leaving your machine.
Add qdrant to the same compose file
services:
open-webui:
# ... unchanged from Lesson 3 ...
environment:
- OLLAMA_BASE_URL=http://host.docker.internal:11434
- VECTOR_DB=qdrant
- QDRANT_URI=http://qdrant:6333
- RAG_EMBEDDING_ENGINE=ollama
- RAG_EMBEDDING_MODEL=nomic-embed-text
- RAG_OLLAMA_BASE_URL=http://host.docker.internal:11434
depends_on:
- qdrant
qdrant:
image: qdrant/qdrant:latest
ports:
- "6333:6333"
volumes:
- qdrant-data:/qdrant/storage
restart: unless-stopped
volumes:
open-webui-data:
qdrant-data:
VECTOR_DB=qdrant + QDRANT_URI=http://qdrant:6333 is genuinely simpler than it used to be -- Open WebUI now has first-class Qdrant support via these two environment variables, replacing what was previously a more involved manual config swap away from its default ChromaDB. Note qdrant:6333 (not localhost) as the hostname: open-webui and qdrant are both containers on the SAME Docker Compose network, so they reach each other by service name directly -- this is different from Lesson 3's host.docker.internal, which was specifically for reaching something OUTSIDE Docker (Ollama on the host).
Without these 3 lines, nomic-embed-text is pulled in Lesson 2 for nothing
VECTOR_DB/QDRANT_URI only tell Open WebUI where to store vectors -- they say nothing about which model creates them. Without RAG_EMBEDDING_ENGINE=ollama and RAG_EMBEDDING_MODEL=nomic-embed-text set, Open WebUI silently falls back to its own bundled sentence-transformers/all-MiniLM-L6-v2 running inside the container -- a real embedding model, but not the one this course pulled in Lesson 2, and not one you chose. Verify which one actually ran: docker exec <open-webui-container> python3 -c "import sqlite3; c=sqlite3.connect('/app/backend/data/webui.db').cursor(); c.execute(\"SELECT key,value FROM config WHERE key IN ('rag.embedding_engine','rag.embedding_model')\"); print(c.fetchall())" should show ollama / nomic-embed-text, not the sentence-transformers default. One more gotcha: this config is written to Open WebUI's database on first boot -- if you add these env vars to a compose file whose open-webui-data volume already exists from an earlier run, they won't retroactively apply. You'd need to change the embedding model in Admin Settings, or start from a fresh volume.
Two different networking patterns in one file, and knowing which applies where
Ollama runs on your host machine -> containers reach it via host.docker.internal. Qdrant runs as a container in the SAME compose file -> Open WebUI reaches it via the service name qdrant directly, because Docker Compose puts all its services on one shared network automatically. Mixing these up (trying to reach Qdrant via host.docker.internal, or Ollama via a made-up service name) is the single most common Docker Compose networking mistake in a stack this size.
The pipeline: chunk, embed, store, retrieve, generate
This is the same RAG pipeline from any cloud-based system, just with every component running locally
Chunk your documents into passages, embed each chunk with nomic-embed-text (Lesson 2's 3rd model, pulled specifically for this), store the vectors in Qdrant, retrieve the most relevant chunks for a query via similarity search, and generate an answer using the retrieved chunks as context. Nothing about this pipeline is architecturally different from a cloud RAG system -- the only thing that changed is where each step runs.
Load your own documents and verify — don't trust, check
You upload 10 of your own documents to Open WebUI's knowledge base and ask a question. The answer sounds plausible and cites "Document 3." How do you actually verify this is correct, not just confident-sounding?
Citations are a real improvement over ungrounded generation, but they don't guarantee correctness -- a model can still misread or misrepresent what a cited chunk actually says, or retrieve a chunk that's only superficially related to the query. "Cited" is not the same as "verified" -- the actual verification step (opening the source and checking) is still yours to do, especially for anything you'd act on.
What you're building today
Qdrant added to docker-compose.yml, VECTOR_DB=qdrant configured on Open WebUI, 10+ of your own real documents (notes, PDFs, markdown — not sample data) uploaded to Open WebUI's knowledge base, and 5 real questions asked and verified against the actual source documents, not just trusted at face value.