In-the-wild chatbot corpus: from opinion analysis to interaction problem detection
Résumé
The past few years have seen growing interests in the development of online virtual assistants. In this paper, we present a system built on chatbot data corresponding to conversations between customers and a virtual assistant provided by a French energy supplier company. We aim at detecting in this data the expressions of user's opinions that are linked to interaction problems. The collected data contain a lot of "in-the-wild" features such as ungrammatical constructions and misspelling. The detection system relies on a hybrid approach mixing hand-crafted linguistic rules and unsupervised representation learning approaches. It takes advantage of the dialogue history and tackles the challenging issue of the opinion detection in "in-the-wild" conversational data. We show that the use of unsupervised representation learning approaches allows us to noticeably improve the performance (F-score = 74.3%) compared to the sole use of hand-crafted linguistic rules (F-score = 67,7%).