Smurfs: The Lost Village

Many people already know the benefits of Houdini in crafting procedural animation and effects. But in recent times the software has become an even more vital tool in the pipeline of a visual effects, animation or games studio to produce the imagery they are called on the create.

One studio in which Houdini has been a major player for sometime is Sony Pictures Imageworks. For the recent Smurfs: The Lost Village, artists working on the dramatic levitating ribbon of water sequence, the luminescent vegetation and the magical effects capitalized on Houdini’s ability to interact with animation and other tools in the pipe. FX supervisor Spencer Lueders breaks down how Imageworks carried out the impressive work.

The River Wild

In search of a lost village, a group of Smurfs encounter a magical river that hurtles them downstream. But this river is different: it levitates above the ground. The process of building the river began in animation. “They had a rig which was basically just tubes,” explains Lueders, “and they would set the size of the tubes, the length of them, and then they were able to animate them moving around as they saw fit.”

Once that animation was approved, the tubes would be sent to the FX department at Imageworks. Lueders was then able to extract the parametric detail of the tubes in order to get the direction they were flowing. That meant the system would know which way the river was flowing, after which artists could create some vectors off the tubes that would define the gravity.

A clip from the river chase sequence.

“This river was meant to be kind of like a semi zero G river,” notes Lueders. “Obviously the Smurfs were attached to it, so real world gravity applied, but there was also kind of a separate force that was holding it together in its tube shape.”

“We were able to use the normals of the tube to define that initial gravity,” adds Lueders, “but something we had to play with was how gravity on the simulations would behave when, say if there was a splash or a tear-away of water, then that would break off. Real world gravity needed to apply. It needed to fall and drop. We had to incorporate all that kind of logic in there.”

Lueders and his team, with Viktor Lundqvist and Reinhold Rittinger being the other two main artists behind the work, then carried out what they called a base fluid simulation using the Houdini FLIP solver. The fluid - water - was pushed in a general direction but remained bound to the tube via the gravitational forces that had been applied.

One of the biggest challenges of the ribbon river sequence lay in how close the camera would be to the water surface.“ Animation would actually animate their characters to the waterline that was defined by the actual tube geometry, not by our sims,” says Lueders. “We had to also respect that, so we did some tests where we were like, ‘Well, we can try to maintain the water line with our base simulation as best as possible,’ but after trying some tests we realized we needed a more robust solution.”

That solution was to warp the characters into a space so that they were flat. They would be animated on a flat plane, and then FX would do a separate FLIP sim at a much higher resolution to produce the detail needed and for extra splashes.

“Once that sim worked,” continues Lueders, “we had to then bind the sim back to the tube geometry that we got from anim. We would also have to remove that section out of the base sim, so we had our hero simulation that would be applied around where all the characters lived and were animated, and then that would be stitched in with our base sim. Because it was all point data - it allowed us some flexibility as far as blending these sims together using various point cloud techniques that Houdini offers and blending of velocities and getting all of that stuff in the right space.”

Other water and effects simulations were part of the film, too.

The next step was surfacing. Imageworks used VDB to surface all the geometry (which were really points) that the FLIP sims had generated. “The tricky part with surfacing this,” suggests Lueders, “was again that the camera was super close-up, so we needed the mesh to be high enough resolution to support the detail that was in the sim. But because the tubes were tens if not hundreds of metres long that we would see all the way in the distance, we couldn't just do it brute force, we had to make the whole mesh high res.”

“VDB offered us some ways of doing camera-based meshing,” offers Lueders. “So we would partition up the simulation, and because of the way Houdini has set up the mesher, we could easily stitch these low res meshes that were further back in space from the higher res. That allowed a lot of flexibility once we got that working. That was one of those unforeseen big technical challenges that we were up against.”

The simulation process required a three-step approval process. This was because in order to efficiently push each shot through, Imageworks needed to have checkpoints, not just wait until the base sim was ready before doing the character hero sim and then the secondary elements on top. “So we had the base sim as a step, and then we had the character sim with the surface stitching as the next kind of approval, then once that was good to go we would push this through our secondaries. What we refer to the secondaries is the foam and the white water elements, the kind of more ballistic particles that just kind of fly off and splash, but then also have a volumetric element underneath the surface which was what we call a churn element which kind of gave us colour variation underneath the water.”

The film has an array of different magical, pyro and other effects simulations.

Another element - and another challenge - was the bioluminescence element of the ribbon of water. This gave the river a magical quality. It was achieved with invected particles that were invected using the velocity field from the original FLIP sims, but they also had emissive properties to provide purples and greens and blues.

For the foam, artists carried out a POP sim - “because foam typically needs millions if not tens of millions of points,” says Lueders. “Again, because the scale of the river was quite large, we would basically run our resources to try to push that much particles through. We had the option of partitioning it up and running multiple sims and things like that, but we actually here have a proprietary tool that we've written that uses VDB and VDB volumes. What we would do is take our particles that the foam simulation would create, and then I created a bunch of triangles from those particles. Those triangles would just be a bunch of small triangles and they would just be connecting small polys from adjacent points. From each point we'd do a search and it would find its five nearest neighbours and it would create triangles from those.”

“When all these triangles start overlapping, you get this kind of nice webby look to it,” adds Lueders. “But we couldn't just render these triangles, so we fed this into one of our proprietary tools, which what it does is, it creates specks, like little points, and you can populate this with lots and lots and lots of points. We got into the billions of points. Then those points get latticed into a custom VDB volume, so we were basically able to get one voxel per pixel kind of detail in the foam. We did that with all the foam elements using that technique. You're able to apply some noise and stuff to kind of break it up so it wouldn't have this kind of perfect triangular look and we were able to have attributes to modify the density based on how dense an area was of foam versus how sparse it might be.”

Foliage Forever

In their search for the lost village the Smurfs pass through a multitude of vegetation. That was modeled in 3D, with Houdini used to propagate the foliage and trees and other vegetation. “Basically,” describes Lueders, “we used various point clouds and we would import the terrain that we would get from the modeling department and then we would be able to paint where we wanted grassy patches or clovers, moss mounds, things like that.”

Watch this Imageworks behind the scenes look at their environments work for the film.

Houdini was crucial in adding to the vegetation an element that Lueders calls ‘keep alive’ - the idea that branches and leaves and grasses had to be shown moving and swaying in a realistic manner. “What we would do is, in Houdini we're able to do cycles of just deforming leaves and things like that, and then we could apply those deformations to the instances in Katana.

“The main architect of this was Sam Rickles,” adds Lueders. “He developed the whole vegetation pipeline, but then we also had a lot of interaction, so the Smurfs are running around in the forest and they're kicking the leaves or interacting with clovers, so we had a way to say we need to de-instance small patches that they need to actually interact with. You could quickly callout a section, say this is what we want to have interacted with, and then we could run our simulations using it would either be with noises but we also used Wire Solver to do a lot of that deformation work, especially to get that residual back and forth as something's getting kicked around.”

An incredible amount of vegetation was required for the Smurf forest environments.

Part of the Pipe

Lueders is adamant this complex work for the river ribbon and other effects work (Houdini’s volumetric pyro solvers were also used to generate magic effects, for example) could not have been done without Houdini having been integrated so tightly into the studio’s workflow. “Here at Imageworks, Houdini is very much embedded in our pipeline and it has been for decades. Fluid sims are no different. We rely on them across all out projects, like our whole effects department uses Houdini explicitly for all our effects needs, whether it's fluid or bullets, rigid body stuff, or any procedural elements effects.”

Houdini also provided Imageworks with a great degree of flexibility in solving the ribbon challenges. This came from having layers of simulations and various velocity fields. “That's the great thing about Houdini,” says Lueders. “We can have one element, kind of our base simulation, define the rest of the elements below it. Our base simulation really drove everything else on down the line. When we needed to go back, if we needed to go all the way back to revise the base simulation, theoretically we would update that simulation and then that would plug all the data that comes out of that, we would just pipe through and we could kick it off to our farm and in a few hours or the next day we would have images back of the updates.”

“Houdini really gave us that flexibility to ingest animation updates as animation needed to refine something. Sometimes you would pick stuff up before it was fully approved, so once we got it working as long as things didn't just radically change it was easy to just push stuff through again and again as needed. For the amount of time that we had to do it, it was impressive that we were able to push all of that through.”