The Sands of Dune

Adapted from the classic science fiction novel by Frank Herbert and directed by Denis Villeneuve, Dune transports audiences to the planet Arrakis, where feuding factions compete for control of the desert world’s priceless resource: spice. 

Vast in scale and scope, Dune presented the DNEG FX department with a series of monumental challenges. The work was divided between three teams, working primarily in SideFX Houdini. In Vancouver, FX leads Gero Grimm, Chris Phillips, Mike Marcuzzi, and FX Supervisors Lisa Nolan and Albert Szostkiewicz who oversaw work on sand simulations and city destruction, with leads in Montreal handling tasks including ocean simulations for the planet Caladan.

Foremost among the challenges for Grimm’s team were shots showing giant sandworms ploughing through desert dunes. “The big thing for us was to sell the scale,” said Grimm. “Sand looks simple but there is so much complexity.”

For most sandworm shots, the effects team began with a desert landscape provided by the environments team. “They created their dunes entirely in Houdini,” Grimm noted.

Simulations were triggered by blocking animation of the sandworm, the motion of which drove the collapse of the dune’s height field. Artists used Houdini tools for the erosion, deformation and simulation of height fields, alongside a custom ‘dune solver’ developed by FX TD Ivan Larinin “The dune solver allowed us to generate dunes shapes quickly without any extensive simulation,” said Larinin. “We tried to implement that tool in VEX and C++ but it wasn't fast enough to hit the speed we needed to generate the enormous amount of dunes we needed for this movie. We ended up creating it using OpenCL, which gave us more than a ten-times boost in speed. 

Artists then used Vellum to simulate the gross movement of the dunes, with further simulations adding progressively more detail. “We transferred the velocity of the worm onto the height fields,” explained Grimm. “We used that to drive and trigger the Vellum sim and dune collapse. Then we ran it through a secondary particle sim that released clumps of sand from the Vellum sim over time, afterwards adding more particles to increase detail and complexity in motion. We also had a dust simulation.”

Videos of dune buggies and ploughs provided essential visual reference. The team also discovered scientific papers detailing the unique physical properties of sand dunes. For example, dry sand has an ‘angle of repose’ of around 32 degrees — when the side of a dune reaches this angle, it settles into a state of equilibrium. However, the structure is fragile. “With dunes, the sand has been rolled around so much that each grain is perfectly round,” Grimm commented. “So they’re very unstable. When you take a handful of sand out of the base, the whole thing starts to collapse.” 

Research into sand explosions showed that they occur in clumps. However, air friction causes the outer surface of a clump to collapse more quickly than its interior. The DNEG team wrote Houdini scripts to simulate this. “We calculated the density of the sand every frame,” said Grimm. “Where the sand was low density, we gave it high drag. That meant the bigger clumps would fall down with normal gravity, but where there was lower density the sand would start streaming off and dissipate.” At bigger scales, internal sand collisions were less important than the effects of air turbulence, evident in the form of undulations, breakup and swirling vortices. “With a normal particle sim, you don’t get all those eddies. We built our own solver to calculate those things – a combination of a particles DOPsim and tools like VDB Project Non-Divergent – which made particles behave more like a volume rather then single independent particles. This gave us really nice details and made sure the sand didn’t just fall straight down to the ground. On top of that, we had a dust sim lingering around and creating this nice soft layer that held it all together.” 

Early sand simulation tests proved that a point-based approach was the only option — volume rendering and geometry rendering simply did not deliver. “Nothing looked good unless we had a lot of points,” remarked Grimm. This presented the effects team with a challenge. “With all those dunes there would have been hundreds of billions of points, and there’s no way you can render that.”

Optimization was the key. Grimm wrote a Houdini tool that identified points hidden either behind geometry — a sandworm or spice harvester, for example — or behind points in the foreground. “The script would see the worm and the first layer of particles,” said Grimm, “and then delete everything behind them – kind of how a raytracer works, but built in SOPs. We deleted everything that would not be seen by the primary rays of a render. That culled a lot of points.”

This approach enabled effects artists to render everything in Mantra for approval by the visual effects supervisor, then export a relatively lightweight VDB point cloud. DNEG’s R&D department developed a point replication tool using VEX syntax to transfer shot-specific Houdini settings down the pipeline for lighting and rendering in Isotropix Clarisse, ensuring nothing was lost in translation. At final render time, the point count increased by up to one hundred times. For the largest sandworm shots, this count approached 500 million points. 

Despite these efficiencies, iteration on simulation options was a slow process. “We didn’t have daily updates,” Grimm admitted. “It was more like three or four days, sometimes a week, just because of the huge amount of data.” To speed things up, the team broke big simulations into smaller clusters. “If you do things in a clever way, you don’t see that the clusters aren’t interacting with each other — it looks like one massive sim even though there are actually twenty different little sims. Each one took maybe half an hour to simulate instead of thirty hours to do the whole thing.” 

As effects lead, Grimm would often help individual artists to navigate their way past roadblocks. “Sometimes an artist gets stuck. They’re moving things around, trying to find the right settings, but the simulation isn’t really progressing.” In such situations, Grimm encouraged artists to focus on a single key element. “With a shot of a worm going through the sand, maybe part of the lip is really important. So let’s just simulate that, adjust all the forces until it works, then apply those changes to the whole thing. I call it ‘detangling the mind’!”

Reflecting on the project, Gero Grimm agreed that Houdini provided the DNEG effects team with most of the tools it needed to create the spectacular sands of Dune. “We had to use R&D for the point replication tool,” said Grimm, “but that’s because we were using DNEG’s proprietary OpenVDB data system. Apart from that, we did pretty much everything in Houdini. Every artist developed their own toolkits within the software for certain things for their shots, and we shared those toolkits between artists. It was a great collaboration.”