AI & ML glossary

Plain-English definitions of 65 AI and machine learning engineering terms — from attention and tokenization to RAG, evals, and deployment. Each term has its own page with the DeVenture Academy lessons that teach it.

Architecture

• Attention Mechanism — Computes weighted relationships between all token pairs in a sequence. Q·K^T produces attention scores, softmax normalizes them, and multiplication with V gives context-aware representations. O(n²) in sequence length.
• Transformer — Neural network architecture based on self-attention (Vaswani et al., 2017). Processes all tokens in parallel (not sequential like RNNs). Foundation of GPT, BERT, and all modern LLMs. Key innovation: attention is all you need.
• RAG (Retrieval-Augmented Generation) — Architecture that retrieves relevant documents from an external store and includes them as context for LLM generation. Provides fresh knowledge, citations, and reduces hallucination without fine-tuning.
• Encoder-Decoder — Two-part transformer architecture: encoder processes input into latent representations, decoder generates output autoregressively. Used in T5, BART. GPT is decoder-only.
• Mixture of Experts (MoE) — Architecture with multiple specialized sub-networks (experts) and a gating network that routes tokens to the best expert(s). Increases parameters without proportional compute. Used in Mixtral, GPT-4.
• KV Cache — Stores computed Key and Value tensors from previous tokens during autoregressive generation. Eliminates recomputation of past attention — O(n) per new token instead of O(n²). Memory grows linearly with sequence length.

Training

• Backpropagation — Algorithm for computing gradients of the loss with respect to all parameters by recursively applying the chain rule through the computation graph in reverse topological order.
• Batch Size — Number of training examples processed in one forward/backward pass. Larger batches give stable gradients but use more memory. Common: 16-512 for fine-tuning, 1-8 for LLM training.
• Fine-Tuning — Continuing to train a pre-trained model on task-specific data to change its behavior, style, or capabilities. Methods: full fine-tuning, LoRA, QLoRA. Changes weights permanently.
• Gradient Descent — Optimization algorithm that iteratively updates parameters in the direction opposite to the gradient of the loss: θ = θ - lr × ∂L/∂θ. Variants: SGD, Adam, AdamW.
• LoRA (Low-Rank Adaptation) — Parameter-efficient fine-tuning that freezes base weights and trains small rank-decomposition matrices (A, B). Reduces trainable params by 99%+. Rank 8-64 typical. Merged for inference.
• RLHF — Reinforcement Learning from Human Feedback. Process: (1) collect human preference comparisons, (2) train reward model, (3) optimize LLM policy via PPO/DPO. Aligns model outputs with human preferences.
• DPO (Direct Preference Optimization) — Simpler alternative to RLHF that skips the reward model. Directly optimizes the policy on preference data using a closed-form loss. Easier to implement, equally effective in many cases.
• QLoRA — Quantized LoRA: fine-tunes in 4-bit precision with LoRA adapters on top. Enables fine-tuning of 70B models on a single 24GB GPU. NF4 quantization + double quantization + paged attention.
• Learning Rate — Step size for parameter updates during gradient descent. Too high: diverge. Too low: slow or stuck. Typical: 1e-5 to 1e-3 for fine-tuning. Often uses warmup + cosine decay schedule.
• Gradient Accumulation — Technique to simulate larger batch sizes by accumulating gradients over multiple mini-batches before updating weights. Effective batch = mini_batch × accumulation_steps.
• Overfitting — When a model learns training data patterns too specifically, failing to generalize to new data. Signs: training loss decreasing while validation loss increases. Fixes: regularization, dropout, early stopping, more data.

LLM

• Hallucination — When a model generates plausible-sounding but factually incorrect content. Caused by: training data patterns, no real-world grounding, high temperature. Mitigated by: RAG, citations, lower temperature.
• Temperature — Scaling factor applied to logits before softmax. T=0: deterministic (argmax). T=0.7: balanced creativity. T=1.0: default sampling. T>1: more random. Controls output diversity vs. consistency.
• Token — Subword unit that LLMs process. "ChatGPT is great" ≈ 4 tokens. Models have context windows (4K-128K tokens). Input tokens are cheaper than output tokens. 1 token ≈ 0.75 words.
• Context Window — Maximum number of tokens a model can process in a single request. GPT-4: 128K, Claude 3: 200K, Gemini 1.5: 1M. Longer windows enable document analysis but increase cost and latency.
• Top-p (Nucleus Sampling) — Sampling method that only considers tokens within the smallest set whose cumulative probability exceeds p (e.g. 0.9). Filters out unlikely tokens while preserving diversity. Often combined with temperature.
• In-Context Learning — Model learns from examples provided in the prompt without any weight updates. Also called few-shot learning. Limited by context window size. Not the same as fine-tuning.
• System Prompt — Hidden instruction at the start of a conversation that defines the model's persona, rules, and behavior. Not visible to the user. Critical for safety, formatting, and role-playing.
• Function Calling — Structured output capability where the model generates a JSON function call with arguments. Enables tool use, API integration, and agent workflows. More reliable than free-text parsing.

RAG

• Chunking — Splitting documents into smaller segments for embedding and retrieval. Strategies: fixed-size, sentence-boundary, semantic, parent-child. Chunk size affects retrieval precision and context relevance.
• Reranking — Second-stage retrieval where a cross-encoder model re-scores retrieved documents for query relevance. More accurate than bi-encoder similarity alone. Adds 50-200ms latency.
• Semantic Search — Search based on meaning rather than keyword matching. Uses embeddings to find conceptually similar content. "How do I deploy?" matches "shipping to production" even without shared words.
• Hybrid Search — Combines semantic (vector) search with keyword (BM25) search for better retrieval. Keyword search catches exact matches (names, IDs, code) that embeddings might miss.

Prompting

• Chain of Thought (CoT) — Prompting technique where the model is asked to show reasoning steps before giving a final answer. Improves accuracy on math, logic, and multi-step tasks by 10-40%.
• Few-Shot Prompting — Including 2-5 input/output examples in the prompt to demonstrate desired behavior. More effective than zero-shot for format adherence and domain-specific tasks.
• Zero-Shot — Asking a model to perform a task without any examples in the prompt. Works well for simple tasks. For complex or formatted outputs, few-shot is usually better.
• Self-Consistency — Sampling multiple CoT reasoning paths and taking the majority answer. Reduces variance and improves accuracy on reasoning tasks. Cost: N× more compute for N samples.
• Prompt Engineering — The practice of designing effective prompts to elicit desired behavior from LLMs. Includes system prompt design, few-shot examples, output format specification, and chain-of-thought structuring.

Safety

• Guardrails — Input/output validation layers that prevent harmful, off-topic, or policy-violating content. Include: input sanitization, output filtering, topic classification, PII detection, toxicity scoring.
• Prompt Injection — Attack where adversarial text in user input attempts to override system instructions. Types: direct (ignore above), indirect (hidden in retrieved docs). Defense: input validation, separate channels.
• Red Teaming — Systematic testing of AI systems by adversarial actors trying to elicit harmful, biased, or unexpected outputs. Critical for deployment safety. Includes automated, human, and model-driven red teaming.
• Constitutional AI — Anthropic's alignment method where the model critiques and revises its own outputs based on a set of principles (constitution). Reduces need for human feedback labels. Used in Claude training.
• PII Detection — Identifying and redacting personally identifiable information (names, emails, SSNs, addresses) from text before sending to LLMs or storing in logs. Critical for HIPAA/GDPR compliance.

Math

• Softmax — Function that converts a vector of real numbers into a probability distribution: softmax(x_i) = e^(x_i) / Σe^(x_j). Used in attention scores and classification outputs. Temperature scales logits before softmax.
• Cosine Similarity — Measures the angle between two vectors: cos(θ) = A·B / (|A|·|B|). Range [-1, 1]. Preferred for embeddings because it is magnitude-invariant — focuses on direction (meaning) not length.
• Cross-Entropy Loss — Standard loss function for classification: L = -Σ y_i log(p_i). Measures difference between predicted probability distribution and true distribution. Used in language modeling (next-token prediction).
• Perplexity — Exponentiated cross-entropy loss: PPL = exp(loss). Measures how "surprised" a model is by the next token. Lower = better. Range: 1 (perfect) to vocab_size (random). Useful for comparing models on same dataset.

Representation

• Embedding — Dense vector representation of text/images in continuous space where similar items are nearby. Produced by encoder models. Dimensions: 384-3072. Used for search, clustering, classification.

Infrastructure

• Vector Store — Database optimized for storing and querying high-dimensional vectors via approximate nearest neighbor (ANN) search. Examples: Qdrant, Pinecone, ChromaDB, Weaviate. Core infrastructure for RAG.
• ANN (Approximate Nearest Neighbor) — Algorithms that find nearby vectors quickly by trading exactness for speed. Methods: HNSW, IVF, LSH. 10-100× faster than brute-force search with <1% recall loss.
• Inference — Running a trained model to produce outputs. Distinct from training. Optimizations: quantization, batching, KV cache, speculative decoding. Cost measured in $/M tokens.
• Quantization — Reducing model precision from FP16 to INT8, INT4, or lower. Reduces memory and speeds up inference with minimal quality loss. Methods: GPTQ, AWQ, GGUF, bitsandbytes. Essential for local deployment.
• vLLM — High-throughput inference engine using PagedAttention for efficient KV cache management. 2-4× faster than HuggingFace transformers. Supports continuous batching, tensor parallelism, and quantization.
• Speculative Decoding — Inference optimization where a small draft model proposes tokens that the large model verifies in parallel. Reduces latency by 2-3× for greedy decoding. No quality loss.

Agents

• Agent — AI system that uses an LLM to reason about goals, select tools, and take actions in a loop (observe → think → act). Examples: Claude Code, Cursor, Devin. Key capability: autonomous multi-step execution.
• ReAct — Reasoning + Acting pattern for agents. The model alternates between thinking (Thought), choosing an action (Action), and processing results (Observation). Foundation of most agent frameworks.
• Tool Use — Capability where an LLM calls external functions/APIs to gather information or perform actions. Enabled by function calling. Tools extend the model's knowledge and capabilities beyond training data.
• MCP (Model Context Protocol) — Open standard by Anthropic for connecting AI models to external tools, resources, and data sources. Defines a JSON-RPC protocol for servers to expose capabilities to any MCP-compatible client.
• Multi-Agent System — System where multiple AI agents collaborate on tasks. Patterns: sequential pipeline, parallel fan-out, supervisor dispatch, hierarchical delegation. Each agent has a specialized role.
• Orchestration — Coordination layer that manages agent communication, task routing, and result aggregation. Can be centralized (supervisor) or decentralized (peer-to-peer). Critical for multi-agent reliability.
• Sandboxing — Running agent code/actions in an isolated environment with restricted permissions. Prevents unintended side effects. Methods: Docker containers, WASM, firecracker microVMs, E2B sandboxes.

Evaluation

• LLM-as-Judge — Using a strong LLM (e.g. GPT-4) to evaluate outputs of another model. Cheaper than human eval, scalable. Risks: bias toward own style, position bias, verbosity bias. Mitigate with rubrics and multiple judges.
• Golden Set — Curated set of test cases with known correct answers used for regression testing. Should cover edge cases, common cases, and adversarial inputs. Updated as new failure modes are discovered.
• Regression Testing — Re-running evaluation suite after model or prompt changes to detect quality degradation. Automated in CI/CD. Blocks deployment if score drops below threshold.
• A/B Testing — Comparing two model versions or prompt variants on real user traffic. Statistical significance requires enough samples. Online metric (e.g. thumbs up) vs offline metric (e.g. BLEU).
• BLEU Score — N-gram precision metric for machine translation: measures overlap between generated and reference text. Range 0-1. Limitations: doesn't capture meaning, sensitive to tokenization. ROUGE is recall-focused variant.

Deployment

• Latency — Time from request to first token (TTFT) or full response. TTFT depends on prompt processing. Token generation rate (tokens/s) determines total latency. Optimized by batching, quantization, speculative decoding.
• Throughput — Number of tokens or requests processed per second. Increased by batching, continuous batching (vLLM), and tensor parallelism. Trade-off with latency: higher throughput often means higher per-request latency.
• Streaming — Returning tokens as they are generated rather than waiting for the full response. Improves perceived latency (TTFT). Implemented via SSE (Server-Sent Events) or WebSocket.

All courses · Pricing · About · FAQ · Glossary