AI chatbot deployment checklist: what should be ready before launch?

A website chatbot should not launch just because the model can answer questions in a demo. A reliable deployment needs a defined job, approved knowledge, boundaries, handoff rules, privacy decisions, testing and an improvement process after launch.

1. Define the chatbot’s actual job.

Start with a narrow business outcome. Should the chatbot answer service questions, help visitors find the right page, explain office information, support product discovery, collect a lead or guide someone toward booking? A bot that tries to do everything is harder to test and easier to make unreliable.

Write the supported use cases in plain language. Then write what is explicitly out of scope. That boundary becomes the basis for prompts, tools, testing and human handoff.

OpenAI’s practical guide to building agents recommends starting with clear use cases and layering in guardrails and evaluation as systems become more capable. The same principle applies to a website chatbot even when it is simpler than a full agent.

Reference: OpenAI: A Practical Guide to Building Agents ↗

2. Choose approved knowledge sources.

A service chatbot should know where its answers are supposed to come from. Approved sources might include published service pages, FAQs, policies, hours, locations, booking instructions, product documentation or a curated internal knowledge base.

Do not feed the bot a random collection of old documents and assume the model will know which one is current. Remove duplicates, clearly identify authoritative sources and define how updates are handled.

If the website is the public source of truth, connect the chatbot maintenance process to website maintenance. When a service page changes, the chatbot knowledge should change too.

3. Define boundaries and unsupported questions.

Every chatbot needs a safe response for questions it should not answer. The right behaviour may be to explain the limitation, point to an approved resource or hand the visitor to a person.

For regulated or high-stakes industries, boundaries are especially important. A chatbot can help with navigation, published information and general service discovery without pretending to replace a qualified professional.

Guardrails should be layered with normal application security. OpenAI’s guidance notes that guardrails are important but should work alongside authentication, authorization and other controls where tools or sensitive data are involved.

4. Design human handoff before launch.

A useful chatbot should make it easy to reach a person when the conversation requires one. Handoff can be a contact form, phone link, booking link, live-chat escalation or clear instruction about when staff are available.

Do not make the user fight the chatbot to find contact information. If a visitor says the answer did not help, asks for a person or reaches an unsupported topic, the next step should be obvious.

Track how often handoff happens and why. Repeated handoff for the same simple question may reveal missing knowledge. Repeated handoff for complex requests may confirm that the boundary is working correctly.

5. Decide what data the chatbot should collect.

Collect only what the workflow genuinely needs. If the bot can answer a public service question without asking for personal information, do not turn the conversation into a lead form by default.

When the chatbot does collect names, email, phone numbers or other information, decide where the data goes, who can access it, how long it is retained and what privacy disclosures apply. If tools connect to a CRM, booking system or account data, access controls become more important.

Privacy requirements vary by business and jurisdiction, so legal and compliance decisions should be reviewed appropriately rather than copied from a generic chatbot template.

6. Design the conversation for mobile first.

A floating chatbot can easily cover important page content or create a poor experience on a small screen. Test the launcher size, close button, keyboard behaviour, scrolling, focus order and whether the widget overlaps other sticky controls.

Keep answers readable. Long walls of text are difficult inside a chat window. The bot should link visitors to the full page when the answer requires depth rather than copying an entire service page into the conversation.

Test with the on-screen keyboard open. A chat interface that works on desktop can become nearly unusable when half the mobile screen is occupied by the keyboard.

7. Test the questions people will actually ask.

Build a test set before launch. Include the obvious questions, but also include misspellings, vague questions, multi-part questions, wrong assumptions and questions that should trigger a handoff.

8. Test tools and actions separately from conversation quality.

If the chatbot can create bookings, send leads, search a database or call another service, test the tool behaviour independently. A fluent answer can hide a failed action.

Verify required fields, error handling, retries, duplicate submissions and confirmation messages. For tools that can change data, permissions and validation should be explicit.

Do not allow a conversational layer to bypass the same business rules that would apply to a normal form or application workflow.

9. Define evaluation criteria.

“The bot sounds good” is not enough. Decide what success means: answer accuracy, correct routing, handoff quality, successful lead capture, booking completion, low failure rate, user feedback or reduction in repetitive staff questions.

OpenAI’s current agent materials emphasize evaluation for reliable production systems. Create a repeatable test set and rerun it when prompts, models, knowledge or tools change.

The goal is not to eliminate every imperfect answer. It is to understand performance well enough to make controlled improvements.

10. Monitor real failures after launch.

Review conversation patterns in a privacy-appropriate way. Look for unanswered questions, invented information, broken links, failed handoffs and confusing UI moments.

Group failures. If several visitors ask about the same service detail, the knowledge base may need an update. If the bot repeatedly misunderstands a phrase, prompt or retrieval changes may be appropriate. If the issue is actually unclear website content, fix the source page rather than teaching the bot to compensate forever.

Record major changes so the team can see whether a new prompt or source improved results or introduced regressions.

11. Keep the knowledge base current.

Chatbot maintenance should be tied to website changes. New services, updated policies, new hours, changed staff, pricing rules or booking changes can make previously correct answers outdated.

Assign ownership for updates. A chatbot with no maintenance process slowly becomes a source of stale information even if the original launch was excellent.

12. Treat accessibility as part of the widget design.

Keyboard focus, readable contrast, meaningful labels, predictable close/open behaviour and screen-reader-friendly controls matter inside a chatbot just as they do elsewhere on the site. Avoid custom interactions that trap focus or hide the page behind an inaccessible overlay.

The chatbot should also respect reduced-motion preferences where animations are used and avoid creating a barrier to the main website content.

Deployment-readiness checklist.

What should happen after the checklist?

Start with a controlled launch, monitor the questions people actually ask, then improve the system based on evidence. A smaller chatbot with clear boundaries can be more useful than a broad bot that answers confidently outside its knowledge.

PWS’s AI Chatbot Launch Newsroom story documents an earlier clinic-specific implementation. For businesses ready to explore implementation, AI Chatbot Development is the appropriate next step.

Plan for degraded or unavailable service.

Decide what visitors see if the chatbot service is unavailable, rate-limited or cannot reach a required tool. The website should still expose normal navigation, contact information and booking paths. A chatbot should enhance the site, not become a single point of failure for basic customer access.

Test the fallback state during deployment. A simple message with direct links to important pages or human contact can be more useful than a broken widget that continues accepting questions it cannot answer.

Assign a named owner after launch.

Someone should be responsible for reviewing failures, approving knowledge changes and deciding when the system needs retesting. Without ownership, chatbot quality can drift as the business changes and nobody notices until customers report incorrect answers.