TerraFusion: Joint Generation of Terrain Geometry and Texture Using Latent Diffusion Models

Kazuki Higo, Toshiki Kanai, Yuki Endo, Yoshihiro Kanamori

University of Tsukuba
CGI 2025 (Virtual Reality & Intelligent Hardware Journal)

We introduce TerraFusion, a novel diffusion-based framework for jointly generating terrain geometry and textures. Our method not only enables the random synthesis of plausible terrain models (left), but also supports intuitive user control through rough sketches (right), where valleys, ridgelines, and cliffs are indicated by red, green, and blue lines, respectively.

Abstract

3D terrain models are essential in fields such as video game development and film production. Since surface color often correlates with terrain geometry, capturing this relationship is crucial to achieving realism. However, most existing methods generate either a heightmap or a texture, without sufficiently accounting for the inherent correlation. Methods In this paper, we propose a method that jointly generates terrain heightmaps and textures using a latent diffusion model. First, we train the model in an unsupervised manner to randomly generate paired heightmaps and textures. Then, we perform supervised learning of an external adapter to enable user control via hand-drawn sketches. Results Experiments show that our approach allows intuitive terrain generation while preserving the correlation between heightmaps and textures.

Qualitative comparison of terrain generated by our method and the two-stage baselines.

Qualitative comparison of terrain generated by our method and PSGAN.

Results of terrain generation from sketch inputs. Red lines represent valleys, green lines indicate ridgelines, and blue lines denote cliffs.

Results of terrain generation from sketch inputs representing extreme terrain geometry not found in the real world.

BibTeX


    @article{higo2025terrafusion,
      title={TerraFusion: Joint Generation of Terrain Geometry and Texture Using Latent Diffusion Models},
      author={Kazuki Higo and Toshiki Kanai and Yuki Endo and Yoshihiro Kanamori},
      journal={Virtual Reality & Intelligent Hardware Journal},
      volume={},
      number={},
      pages={0-0},
      year={2025},
    }

Concurrent Work

Paul Borne-Pons, Mikolaj Czerkawski, Rosalie Martin, and Romain Rouffet. "MESA: Text-Driven Terrain Generation Using Latent Diffusion and Global Copernicus Data." MORSE Workshop at CVPR 2025, 2025.