David Fincher is no stranger to pushing the limits of Visual Effects. The photogrammetry work he did with Digital Domain on ‘Fight Club’, arguably his most well known movie, pushed Digital Set technology forward by light years. And carved out a now indispensable corner of the VFX toolbox. But over Christmas break, they out did themselves with the film, ‘The Curious Case of Benjamin Button’.
I had been waiting for this film for quite some time. And I was not let down. Not only was the story touching and compelling, but the visual effects were also absolutely breathtaking. Ranging from the standard 2.5D/3D Digital Matte Paintings all the way to more exotic digital aging techniques developed at D&D.
All of which is covered in full by Bill Desowitz at VFXWorld. Who posted a great article yesterday talking about all of the work done to bring this ground shattering movie.
LINK: Bringing Benjamin Button to Life
Bill Desowitz uncovers the curious vfx case of Benjamin Button in this in-depth report with Digital Domain, Asylum, Hydraulx, Lola VFX and Matte World Digital.
It’s a long article to say the least, so I’m going to talk about the highlights. Or at least the parts that were the most interesting for a Nuke Compositor to read about.
___________________________________________
The process of developing the tech needed to make the impossible possible, all started with the creepy and controversial Orville Redenbacher commercial back in 2007.
The strangest thing…that’s Marty McFly’s Dad, otherwise known as Crispin Glover behind all that digital makeup. But I digress…
LINK: Ground-Breaking Orville Redenbacher’s Ads
The ads, scheduled to premier during the 64th annual Golden Globe awards, employ computer-generated imaging that allows directors to create and use completely believable digitally created actors in live-action settings. While movie audiences may be familiar with animation of large crowds or stunts, such as the thousands of troops in “Flags of our Fathers,” or the character Gollum from “Lord of the Rings,” this new technology — being used publicly for the first time — advances current techniques tenfold to create an authentic digital human with detail, personality and close-up realism.
Starting the R&D process in the commercial world with a truncated schedule forced D&D to automate the pipeline as much as possible. But some of the constraints made the project fall just short.
As Fincher worked on Zodiac, Digital Domain made the controversial Orville Redenbacher spot, which, though turning out more creepy than uncanny, provided another R&D opportunity. “David may have had confidence from the beginning but we didn’t,” admits Character Supervisor Steve Preeg. “Having worked in rigging in this industry for quite some time, it’s definitely a scary thing to even think about [achieving this feat]. In fact, everyone we tried to hire to work on it, felt the same way. It was actually quite hard to staff up for the show, which was surprising. But during the Orville Redenbacher [experience], we had a certain number of constraints put on us to test certain aspects, most of which didn’t work out, but there were some little [kernels] of optimism.”
Barba adds, “With the bleeding edge of technology, you certainly get cut and hurt and that was the case with Redenbacher. Commercial budgets and schedules are tiny compared to features and we put together a fantastic team, but schedule and the technology maybe let us down a tiny bit, and maybe some of our own plans. But if we hadn’t done that, I don’t think we would’ve gotten the head start on Benjamin Button. For me, as much as I hate failing, it was absolutely necessary to go through this process. It left a scar but it didn’t kill us.”
Sometimes failing is just as important as succeeding though, and the lessons learned from the commercial world made people around the halls to start and believe the work could actually be done.
Nuke was of course the tool of choice. But not just for the compositing. It’s powerful 2D and 3D compositing work flow allowed the D&D artists to use existing tools to accomplish new challenges.
Tight collaboration, therefore, was crucial between compositing and lighting. “A lot of the setup was done in Nuke, which, as everyone knows, is very powerful in both 2D and 3D,” Litt suggests. “Teams of compositors would be the first to touch the shots by ingesting this data and bring it up in Nuke. We started off by using straight HDR spheres but slowly came to realize that it wasn’t getting us the subtleties we needed in terms of localization of lighting. With a head moving around in a room or gradations of light across the head itself, some of the light sources are very close.
They also developed code to integrate the Nuke process into areas of the pipeline you wouldn’t normally see it pop up in.
“But we had a pretty new drill at the beginning of the pipeline and part of it was getting the grades into place, like how David wanted the scenes to look,” adds Compositing Supervisor Janelle Croshaw. “We were setting the color and the tone. Nuke is the backbone of everything here, so every single department ended up having a check, a script or a comp. An image sequence was basically an auto comp of the head and we had track check, anim check, roto check. We figured out on Zodiac that digital data can get really messy and there can be a lot of it. And so we worked on automating on Benjamin, so everyone could see it immediately. We were able to automate the process going out as well and reverse all the grading we did because David likes to do all the grading in sequences up in DI with the shots that don’t have visual effects. It was important that tracking needed a good way to view what they were doing over the plates, so we set up track tracks where Marco and his team could hit a button out of Maya and it would run an automated Nuke script that would grab the graded plates and the roto and do mini-comps so they could see how their tracked head fit within the shot. So it helped all the departments a lot.”
___________________________________________
There were HUGE leaps made in the tracking and motion capture process as well. Without which the outstanding compositing and 3D work would have been for naught.
Instead of relying on traditional marker-based motion capture and hand animation techniques to bring a live actor’s performance to a digital character, Digital Domain went in a completely different direction, and developed a system called “emotion” (led by Barba and Preeg). This proprietary process incorporated several different technologies — using them in ways that they weren’t intended — and brand new techniques.
Mova’s Contour system was a key component in this part of the process, and it was used to volumetrically capture Pitt’s expressions. Digital Domain has been working with Mova’s Steve Perlman for years on adapting their technology for this application. Meanwhile, Image Metrics‘ technology was used to provide primary animation curves and timing data from Pitt’s live-action performance captured from four angles using Viper HD cameras. This gave artists a head start for a number of shots that saved them production time during animation.
___________________________________________
Just as important as the digital aging was to the story and film, the other star of the VFX work was the digital de-aging. And that’s where LOLA comes in.
LINK: LOLA
LOLA cut their teeth in the commercial world as well, pioneering digital youthful techniques for makeup ads. But they had their big break when they were awarded the opening scenes for ‘X-Men: The Last Stand’ where they de-aged Ian Mcclellan and Patrick Stewart by a good 30 years each.
Since then, LOLA has refined their techniques even more and gotten things down to a science.
In fact, Lola broke it down to a science, using Flame for compositing, Maya for 3D and Bijou for 3D tracking.
In the dance studio scene, Pitt dramatically emerges from the shadows, and is 20 years younger. This is the first time Daisy sees Button as a young man, and the effect required strong audience reaction. Senior Animator Casey Allen was tasked with creating new techniques. “In the end, after removing and/or repositioning every fold and crease of the eyelids, Casey was able to create new geometry of the eye though re-lighting and resetting the structure of the entire eyelid,” Williams explains. “This created an eye socket structure that Brad possibly never had, but the overall effect, combined with the re-shaping of the rest of his face, was that of extreme age rejuvenation. Casey worked closely with David Fincher and me to fine tune the look of Brad’s eyes, and Casey devoted close to 60 hours for this one shot.
___________________________________________
All these new amazing techniques has made Ben Ulbrich a believer that Digital Domain has finally crossed the so called ‘Uncanny Valley’. But one thing still is lacking. One of the most difficult things in fact in ANY form of art…the eyes.
“The eyes were a different story: those were a pain in the butt. Unfortunately, the system that we used to get the movement of Brad’s skin ended at certain parts of the face because of the nature of the capturing, and that included the eyes. There was a lot of work on the eyes by hand and I think we put some stuff into the eyes on Benjamin that I haven’t seen in characters before.”
___________________________________________
But they’re close. And I wouldn’t be surprised if D&D’s next project, that will incorporate the technology developed for ‘Button’, actually crosses the divide. Which I think you’ll agree isn’t too small of a limb to go out on, once you go and see the quantum leap that is ‘Benjamin Button’.
