KnHiGAN: Knowledge-enhanced Hierarchical Generative Adversarial Network for Fine-grained Text-to-Image Synthesis

Abstract

To generate fine-grained images with greater authenticity, in this paper, we propose a Knowledge-enhanced Hierarchical Generative Adversarial Network (KnHiGAN) for text-to-image synthesis. KnHiGAN sets up a Knowledge Enhancement Module to expand conditions for the limited text descriptions by combining with the knowledge graph, as a result, it can provide richer fine-grained details to the generative network. Moreover, a Hierarchical Generative Adversarial Network is designed to generate the foreground and background separately, and the two are integrated together to composite the final result. Experiments on CUB-200 and Oxford-102 datasets show that our KnHiGAN can not only generate the fine-grained images which are more like those that exist in the real world, but also can maintain a high degree of consistency with the original text input.