Todd Blevins
Business Problem:
Currently, the only way to test an AI Agent is manually — you type each message yourself in the Test AI Agent panel, one message at a time. There is no way to run full test conversations automatically. This makes it slow to check the agent across many situations, and hard to re-test after editing instructions or knowledge sources. Problems often surface only once real customers hit them.
Desired Outcome:
Let users set up test conversations that run against the AI Agent automatically, then review the results. The system plays the customer side and sends the messages, so testing does not depend on someone typing each message by hand.
Use Cases:
  • Re-run a set of saved test conversations automatically after changing instructions or knowledge sources, to catch problems before publishing.
  • Check that escalation and actions (e.g. handover, booking) work across many customer situations at once.
Current Workaround:
The Test AI Agent panel can test a conversation manually, one message at a time, but there is no way to run test conversations automatically. Read more here