Ncam: Making The Nutcracker and the Four Realms Come to Life

Based on the classic festive fairytale by ETA Hoffman, The Nutcracker and the Four Realms follows the adventures of a young girl, Clara, who discovers a delightful new world of adorable mice, living toys, handsome soldiers – and something a little darker. The beautifully realized film boasts some incredible visual effects work, for which a significant amount of blue/green screen was used. Read on to find out how Ncam’s augmented reality solution played a key role both on set and in post-production.

“The Nutcracker had many scenes shot against blue/green screens which needed to be replaced with our virtual world, so one of the first things we did was look for a pre-visualization system,” says Max Wood, VFX Supervisor. “Firstly, this would help the director to frame more accurately – we didn’t want to shoot blind and find out later that we’d cropped the top of the palace off; we knew it would save time and money in post to have a better idea of the geography while on set. Secondly, it would give us a great head start when it came to production screenings to have a representation of the backgrounds while the final VFX were still being built – people would be able to understand what the movie would look like.”

As a VFX Supervisor at MPC, Max had had experience of receiving data from pre-vis systems but had never used one on set, so the team tested several different systems before choosing Ncam. “All the systems we tried had pros and cons, but Ncam had the best all-round functionality. We also found it very flexible, reliable and, most importantly, wouldn’t impact on production times. It was important that factors outside of VFX's control did not hinder us in having an accurately tracked live set extension.”

The ability to make CG geographical changes on the fly was also key for the team. “We knew that our pre-vis landscape matched to the art department layout, but also we knew from experience that things change very quickly whilst filming! The flexibility of the Ncam setup gave us the peace of mind that it would be possible to make quick changes.”

The Ncam system was primarily used for set extensions, from palace walls to vast landscapes going off into the distance with waterfalls and rotating water wheels.

“The visuals provided in real time by the Ncam system were extremely useful to give the director and DoP the ability to frame for specific key elements, and also to give the actors an understanding of the geography, which was vital in many scenes,” Max explains. “In one particular scene, the actors were standing at the edge of a cliff in front of a blue screen. Story-wise, Philip is pointing out key elements of the VFX world to Clara, but on the set, he was just pointing at different areas on a large blue screen. We used the Ncam system to accurately place markers onto the blue screen to represent the areas being described in the dialogue. Without the Ncam system, we would have had to guesstimate where Philip needed to point and then retrofit our world to match. Not only could we see the shots in real time, but we saved a large amount of post-production time in not having to adjust our world for one shot.” 

The post-production team also used Quicktime files captured from the Ncam system as backgrounds for screenings while the final VFX were being created. “This saved us even more time in post-production as we had backgrounds already done to quickly place into the edit and help us to flesh out the movie. It also meant that we had very good starting points for our match moves in post-vis - both huge advantages for the first screening of the movie,” says Max.

A new piece of equipment on set can be hard to integrate when people aren’t accustomed to using it, but Max and the Ncam team ensured from the start that the system wouldn’t be intrusive.

The production had four Ncam systems available – two systems were used for the A and B Panaflex Millennium XL2 cameras on the main shoot, with two additional available for a splinter unit that filmed additional elements. “We rigged the Ncam bar to our main cameras, a handheld Arriflex 235 and crane-mounted cameras,” says Max. “We had done prep work with Panavision in advance to ensure the Ncam bars would mount securely as we needed them to, so by the time we got on set we could rig and derig efficiently.

“By the end of the principal photography, both the director and DoP were big fans of the system,” he adds. “The ability to have a glimpse into the future of their movie was huge to them, they were definitely converted by the end! Once people see what it can do and how it can make things easier, they are more accommodating to a new workflow.”

Max’s best piece of advice for a successful shoot with a system like Ncam? Make friends with the camera team. “If something is not working for them or they are getting frustrated, come up with a better solution going forward,” he says. “Also, think carefully where to place the Ncam operators’ workstations and make them easy to move. Things change quickly on a film set, and in a stage with space at a premium, VFX needs are often overlooked. Having one eye looking towards the next set-up and where we may need to move to is vital, so as to not miss a take. We kept in constant communication with ADs to stay one step ahead.”

Another key tip from Max is meticulous preparation. “It was vital for us to have prep time where our Ncam team and pre-vis asset team could check the assets - not only that the assets worked within the Ncam system, but also that they were grouped properly with correct pivots to enable us to make changes efficiently whilst filming,” he says. “It’s absolutely essential not to hold up filming, and with proper preparation, this can be avoided.”

The Nutcracker and the Four Realms was released around the world in November 2018.