HiTS: Hierarchical Text-Guided Stylization for Face Sketch-to-Photo Synthesis

Abstract

Face sketch-to-photo synthesis is crucial in law enforcement, converting forensic sketches into RGB images for criminal database matching. A major challenge is ensuring accurate color representation in synthesized images to avoid identification error caused by mismatched skin or eye color. However, direct sketch-to-photo translation struggles with proper color representation as it relies solely on grayscale sketches. While recent text-guided generative methods show promise for style adjustment based on text descriptions, they often produce mixed or exaggerated colors due to using a single representation for text prompts containing multiple entangled attributes. To address these challenges, we propose Hierarchical Text-guided Stylization (HiTS), a novel identity-preserving face sketch-to-photo synthesis method. HiTS categorizes text descriptions into intrinsic and mutable attributes, capturing both global and local color features. Using an encoder-decoder architecture, the encoder extracts global features from intrinsic attributes, while the decoder refines local styles via a semantic-textual embedding map. This map integrates text embeddings with facial parsing masks, enabling precise style adjustments for each facial component, even in small regions. Both quantitative and qualitative results demonstrate that HiTS achieves fine-grained stylization while preserving identity, leading to improved face recognition accuracy.