TerraFusion: Joint Generation of Terrain Geometry and Texture Using Latent Diffusion Models
Published in Virtual Reality & Intelligent Hardware (Proc. of Computer Graphics International 2025), Vol. 7, No. 6, pp. 560-576, 2025
Background
Three-dimensional terrain models are essential in domains such as video game development and film production. Because surface color is often correlated with terrain geometry, capturing this relationship is critical for generating realistic results. However, most existing methods synthesize either a heightmap or a texture without adequately modeling their inherent correlation.
Methods
We propose a method that jointly generates terrain heightmaps and textures using a latent diffusion model. First, we train the model in an unsupervised manner to randomly generate paired heightmaps and textures. Then, we perform supervised learning on an external adapter to enable user control via hand-drawn sketches.
Results
Experiments demonstrate that our approach supports intuitive terrain generation while preserving the correlation between heightmaps and textures.
Conclusion
Our method outperforms the two-stage and GAN-based baselines by ensuring structural coherence, in which textures naturally align with geometry, successfully accommodating both realistic landscapes and extreme user-defined shapes.