Few-shot task-oriented dialogue generation with in-context learning

Abstract

Purpose In the construction of task-oriented dialogue systems, collecting data in specific domains is difficult. This has made the automatic generation of a large amount of dialogue data a highly promising research direction. Traditional methods lack sufficient information diversity, which consequently leads to poor quality of generated dialogues and insufficient annotation accuracy. The purpose of this paper is to propose methods to generate higher-quality dialogues with more accurate annotations under few-shot conditions. Design/methodology/approach This paper puts forward a dialogue retriever specifically designed for task-oriented dialogue generation. This retriever conducts retrieval based on two retrieval methods, namely, those based on user utterances and system responses. It screens out the training prompts that have a higher degree of fit with the test prompts as examples, thereby strengthening the in-context learning ability of large language models and effectively improving the quality of generated user utterances and system responses. Moreover, this paper proposes an automatic annotation correction mechanism, which overcomes the uncertainty of large language models and enhances the accuracy of annotations. This method requires almost no human involvement and can quickly generate dialogues at minimal cost. Findings This paper conducts experiments on two data sets. The results show that compared with previous methods, under the condition of few samples, the method proposed in this paper can generate dialogues with better quality and more accurate annotations. Originality/value In this research, this paper has proposed a turn-based dialogue retriever based on user utterances and system responses and designed a new similarity calculation method, as well as an automatic annotation correction mechanism for task-oriented dialogue generation. The calculation results have demonstrated the effectiveness of this method.