Regression Testing for bots?

I just finished building my first bot using Facebook Messenger and wit.ai. I built out about 15 different intents. Over time, as I continued to train my wit.ai instance, some intents that previously worked started failing. I wouldn’t know this until I manually tested it out again. Has anyone else had this problem, or found a good solution for regression testing for your bots?

I’ve had similar problems while training wit.ai
It does seem to get confused easily but its most likely due to some incorrect training on our part.

we developed several functional tests using xunit framework

we have several ‘asserting’ conversations

which language do you use ?

Currently using Node.js

It’s not released yet, but this seems promising: http://chatic.io/

We open sourced our testing framework, https://github.com/fyndme/messenger-bot-tester

The tests run on node but it should be able to test any bot…