Objective
What you're building
A structured business config — services, pricing, hours, common questions and answers — injected into the assistant so it can answer real business questions, without turning the system prompt into a wall of text the model (and the latency budget) has to wade through on every single turn.
Structured, not stuffed
{
"businessName": "Apex Consulting",
"hours": "Monday-Friday, 9am-5pm Eastern",
"services": [
{ "name": "Strategy Consultation", "price": "$150/hour" },
{ "name": "Full Engagement", "price": "Custom quote after initial consultation" }
],
"faq": [
{ "q": "Do you offer free consultations?", "a": "We offer a free 15-minute intro call to see if we're a good fit." },
{ "q": "Where are you located?", "a": "We're fully remote, working with clients nationwide." }
]
}
Structured data like this, injected as a clearly delimited block in the system prompt ("BUSINESS INFO: ..."), is easier for the model to reference precisely than the same information written as flowing prose -- and easier for YOU to update later (change a price in one JSON field, not hunt through paragraphs of prompt text).
Short prompts matter more in voice than in text -- this isn't the same tradeoff as a text chatbot
A text-based RAG chatbot can afford a fairly long system prompt with lots of context, because there's no live human waiting through the processing time. A voice agent's system prompt gets re-processed (at least in part) on every turn of a live phone call -- a bloated prompt with 40 FAQ entries the caller will never ask about is real, avoidable latency, not just messier prompt engineering.
Test against real questions, tune for how they're actually asked
You test your knowledge base with the exact phrasing from your FAQ ("Do you offer free consultations?") and it answers perfectly. A real caller instead asks "is there, like, a free trial or something before I commit to anything?" and the assistant doesn't connect this to the free-consultation FAQ entry. What's the issue?
Modern LLMs are generally good at connecting paraphrased questions to the right stored answer -- but only if you actually TEST with paraphrased, realistically-worded questions, not just the FAQ's own exact text. Testing exclusively with your own carefully-worded FAQ entries is testing an easier problem than what real callers will actually say, exactly the same trap as testing a chatbot only with your own phrasing instead of a real user's.
What you're building today
business_knowledge_base.json: services, pricing, hours, and FAQ entries for your business, injected into the system prompt as a structured block — tested against 10 realistically-phrased (not FAQ-verbatim) caller questions, tuned until the responses sound natural spoken aloud, not just factually correct on paper.