Region-Aware Double-Tail GAN with AdaIN–LADE Adaptive Style Transfer for Cinematic Anime Background Stylization in Makoto Shinkai Style

Abstract

Purpose: Background stylization in a specific anime art direction remains challenging because global style transfer often yields inconsistent stylization across semantic regions. Our prior Double-Tail GAN (DTGAN) with Adaptive Instance Normalization (AdaIN) and Linearly Adaptive Denormalization (LADE) can produce Shinkai-like backgrounds, but it still exhibits region-specific failures such as unstable sky gradients, over-stylized vegetation textures, and reduced edge clarity in buildings.

Methods: We propose a region-aware extension of DTGAN by conditioning the generator on semantic masks (sky, vegetation, and buildings) and optimizing with coverage-aware, region-weighted objectives. Semantic masks are generated automatically using a lightweight transformer-based semantic segmentation model and refined via simple morphological filtering to stabilize mask boundaries during training.

Result: Experiments on real photographs and Makoto Shinkai-style background frames show that region-aware conditioning improves both global and region-level quality compared with DTGAN without masks. The proposed method reduces global FID from 74.5 to 65.8 and LPIPS from 0.505 to 0.448, while improving sky-gradient similarity and overall palette consistency.

Novelty: This work contributes (i) a practical mask-conditioned DTGAN formulation for local controllability in cinematic anime background stylization, and (ii) a coverage-aware region-weighting strategy that mitigates over-stylization and style leakage when semantic regions occupy imbalanced areas, without requiring manual mask annotation.