Why Your Chatbot Says "I Don't Know" (And How Crawling Fixes It)

Frustrated founders blame the AI model when the bot says "I don't know." In reality, IDK is usually a knowledge gap—the answer exists somewhere your assistant never indexed. Fix the library, not the LLM.

Retrieval-Augmented Generation (RAG) bots answer from what they can find. No crawl of your pricing page? IDK. PDF-only spec sheet? IDK. Launched v2 docs last week but never recrawled? IDK. This guide gives you a diagnostic flowchart, recrawl workflow, and test questions to close gaps systematically.

Top Causes of "I Don't Know"

• Page never crawled: Staging URLs, `/app` routes behind login, PDF-only content.
• Stale index: Pricing, features, or policies changed; bot still has last month's crawl.
• Phrasing mismatch: User says "cancel sub" but docs say "terminate subscription."
• Conflicting sources: Old blog post contradicts new pricing page—bot plays safe with IDK.
• Question outside scope: Account-specific data, legal/medical advice, real-time inventory.

Diagnostic Flowchart

Walk through this numbered checklist when a visitor or tester hits IDK:

1. Reproduce the question exactly as the user asked it. Screenshot or copy transcript.
2. Search your site manually for the answer. If YOU cannot find it in public docs, the bot cannot either—add content or upload a PDF.
3. If the answer exists on a public page: Check Chirps Knowledge → Pages indexed. Is that URL listed? If no → run crawl or add URL manually.
4. If the page is new (< 7 days): Run Recrawl on the assistant. Wait for training to complete.
5. If the answer is in a PDF or internal doc: Upload file → Train. Remove outdated versions.
6. If the page is indexed but IDK persists: Check for login walls, JavaScript-rendered content, or `noindex` tags blocking the crawler.
7. If phrasing is the issue: Add an FAQ paragraph on your site using natural language customers actually use.
8. If the question needs live data (order status, account balance): IDK is correct—configure API artifact or human handoff instead of crawl.
9. Retest the exact question. Still IDK? Review system instructions—overly cautious prompts cause false refusals.
10. Still stuck? Enable dispatch on low confidence so humans catch edge cases while you fix knowledge.

Recrawl Workflow After Product Updates

Treat recrawl as part of your ship checklist, not a one-time setup:

1. Before launch: Publish docs and pricing to production (not staging-only).
2. In Chirps → assistant → Knowledge → Recrawl (uses monthly recrawl quota per plan).
3. Wait for crawl job to finish—check page count increased.
4. Click Train if uploads changed separately from crawl.
5. Run the 10-question regression test below against the assistant.
6. Embed or purge CDN cache if you use aggressive static caching on marketing site.

Sample Test Questions (Copy-Paste Regression Suite)

After every crawl or upload, ask your assistant these—customize for your business:

• "How much does [your main plan] cost per month?"
• "How do I cancel / get a refund?"
• "What integrations do you support?"
• "Do you offer a free trial—and how long?"
• "What's your SLA or uptime guarantee?"
• "How do I contact support?"
• "Do you ship to [your top country]?"
• "What's included in [tier name] vs [tier name]?"
• "How do I reset my password?" (should answer or route to app, not hallucinate)
• "I need to talk to a human about billing." (should escalate, not IDK loop)

Log pass/fail in a spreadsheet. Any IDK on questions you know are documented = knowledge gap to fix, not model swap.

When IDK Is the Correct Answer

Train the bot to hand off instead of guessing on:

• Legal, medical, or financial advice for individual situations
• Account-specific billing disputes or security resets
• Real-time inventory, shipping tracking, or calendar availability without integrations
• Questions about competitors' products (unless you have explicit comparison content)

Good IDK behavior

"I don't have that information in my current knowledge base. I can connect you with our team—what's the best email to reach you?" beats a confident wrong answer every time.

Prevent IDK Regressions Long-Term

1. Add recrawl to your release checklist alongside changelog publish.
2. Version PDF uploads with dates; delete superseded files.
3. Review dispatch logs monthly—recurring escalations reveal missing FAQ topics.
4. Use Harvest (Pro) to learn from real chats that stumped the bot.