Really nice VFX Breakdown by Industrial Light & Magic about their great work on the New York sequence for DOCTOR STRANGE:

WANT TO KNOW MORE?DOCTOR STRANGE: My interview of Richard Bluff, VFX Supervisor.

How was this collaboration with Director Sam Raimi?

All things considered, it was a little bit of a crazy ride. So, it was great to work with a director like Sam who possesses an indy, guerrilla style can-do attitude, to deal with what felt like one road block after another at times. He is also the type of director who welcomes input from all departments, as opposed to coming in with a rigid, fully-formed plan that folks simply execute. This created a great opportunity for the VFX team, both on the production side and with the vendors, to flex some creative muscle and see where the movie might go visually. With subject material like Dr. Strange, literally anything can happen, and you’re only really limited by your imagination. Having a partner in crime who was open to a whole bunch of often wacko explorations was a real boon.

How did it feel to be involved with such an iconic director?

Multiverse of Madness wasn’t strictly my first rodeo with Sam. We were both previously in development on a show that ultimately didn’t get a green light, but that still meant we got to spend several days crammed together in a helicopter, scouting all over New Zealand for locations. Clearly, we were destined for one another at some point.

And I’m be lying if there wasn’t a touch of fan boi appreciation to it all. Sam’s Evil Dead pictures were pretty much on continuous late night video rotation during my college days, and so now finally, many years later, I get to be involved in a project with the same kind of wild artistic sensibilities I enjoyed watching so much.

Can you tell us how you split the work amongst the vendors?

Gone are the days when productions could simply pick and choose vendors, precisely matching skillsets and specialties to the needs of specific VFX sequences. Vendors are being overwhelmed with work from multiple clients, which has resulted in modern tentpole VFX movies routinely booking vendor capacity far in advance of knowing what the work will actually entail.

So it’s more about working with trusted vendors who we’ve established good relationships with in the past, knowing that together we can sort things out and divide and conquer once things are clarified later. As sequences start to resolve, the trick is to create a healthy balance of work for all the vendors on board, and not saddle any particular one with all the hard/thankless grunt work. It’s better to spread the load both technically and creatively so that we maintain as much flexibility as possible over the course of the show. Things are always in a constant state of flux these days, right up until final delivery, so it pays to be nimble.

In terms of vendors, we started out with 6 major players – Framestore, Sony Imageworks, ILM UK, WETA, Luma, and Digital Domain – all veterans of past Marvel shows, so they knew the drill going in. That history also meant we had some idea about how best to carve up the show to everybody’s mutual satisfaction once we actually knew where we heading. Later in the production, the workload inevitably expanded and we brought on Spin, Perception, Trixter, ILM SF, Capital T, and Crafty Apes to relieve the strain.

How was filmed the impressive sequence in which we cross lots of universe?

That sequence was 100% practical…. no wait.… I mean 100% digital.

We actually started out with the noble intention of trying to capture whatever sections we could in a practical fashion, such as shooting elements in a sky-diving training device. However, it quickly become obvious that we desired something far too choreographed in terms of actor performance and camera work to achieve in that manner. Coupled with the fact that the sequence was actually a single unbroken shot/performance, passing through multiple worlds and lighting environments, and it was back to the methodology drawing board.

Round two had us testing shooting just actor facial performances with carefully choreographed camera moves and interactive lighting changes but this also proved to be impractical, not to mention downright dangerous, having pieces of fast-moving machinery moving around the cast. Ultimately we settled on shooting the actors, safely acting in nice comfy chairs, with multiple sync’d cameras capturing their performances from multiple angles, and using this material to drive highly-detailed digital doubles. Better to have the actors be able to do what they do best, i.e. act, than to also have to worry how about we’re capturing their performances!

Having such a choreographed/specific goal meant approaching the sequence in the reverse order to what one would normally do on set, where performances are captured first, and then the rest of the VFX world filled in around them. We had to design, animate, and light the sequence to a high degree of polish first, enough for folks to buy off on, and then go in and capture specific facial performances for the key sections, as needed.

Which universe was the most complicate to create and why?

In terms of the portal ride sequence where Strange and America tumble through the different worlds, it was actually the more “normal” looking worlds that were the most difficult to pull off, simply because everybody understands what they should look like, at least to some degree, based upon their everyday lives or media they’ve been exposed to. There’s far more leeway in the execution of a world built from cubes, or composed of paint, as nobody (at least that I know) has direct experience of such things. The challenges in creating those worlds are more aesthetic than technical, on the whole.

Also, many of the worlds in that particular sequence flash by in seconds, sometime just frames, so there’s a delicate balancing act to be done in detailing out those worlds and not burning up resources that will (always) be needed elsewhere. Fully animating the sequence as part of the development work really helped in this regard as it meant the worlds could be built and dressed specifically to camera vs. more creating more generously overbuilt digital “playgrounds”

There are plenty of other fully-digital worlds elsewhere in the movie though. For example, the opening action sequence where Strange and America are pursued by the fire demon creature started off as the actors on a purely green stage. And Wanda’s fight with the Illuminati takes place inside a grandiose digital space that was based on the interior of the British Museum in London.

COVID restrictions during filming also greatly curtailed the use of practical locations that were intended to play as different universes. Instead we had to resort to a lot of repurposing the same sets and enhancing them digitally to make them feel different from one another.

How did you help the actors to visualize their magical powers on set?

Previs is invaluable when it comes to this sort of thing but it typically only gets you so far – it’s only what you think you’re going to shoot, not necessarily what you’ll actually end up doing. A skilled actor like Benedict is always going to bring something valuable and interesting to the table that you hadn’t expected, and that’s going to necessarily spill over into how he reacts to any VFX content that’s typically going on around him. Being able to improvise and roll with the punches is crucial.

I try to come prepared with as much visual reference as possible, not just the previs/storyboards but also imagery pulled from other movies, or from the web in general, that I think might help the cast visualize the “missing” content on set. More often than not, though, I find myself resorting to play-acting out in front of the actors what I think I should be happening at any given moment. I highly doubt I’ll be winning any awards for my thespian efforts but it’s often the most agile approach, especially when folks are still working through scenes, and there’s a lot of on-the-fly changes and new ideas occurring.

How did you use interactive lights on set for the magical powers?

Interactive lights on set are a bit of a damned if you do, damned if you don’t situation when it comes to gags like magical powers. My general rule of thumb is that if we know an event is going to happen with 100% certainty, and the timing of that event doesn’t require absolute surgical precision, then it’s worth doing interactive lighting gags on set. Otherwise, you’re better off horse trading the inherent realism you get from those in camera gags for more flexibility in post and faking them digitally. The lighting gags that accompany magical powers are almost always driven by actor performance – the hand-waving stuff, to put it in glib terms – and that’s something that often varies from take to take as the cast explore options and the action/blocking develops organically on set. Those variations in performance mean that dimmer board operators often have to anticipate an unpredictable, moving target for their lighting cues, not always any easy task by any means.

In terms of the lights themselves, programmable RGB LEDs are the go-to solution these days, ranging from large fixtures such as Skypanels, down to much smaller, custom-built clusters of battery-powered, individual LEDs that can be worn by an actor. All these lights can be controlled wirelessly these days via iPad dimmer board apps, and that means that they can all be orchestrated together live, on-set, as the action plays out.

Can you elaborates about the fights between the two doctors strange and the use of music elements?

The idea of a musical note-based fight actually originated way back in pre-production days from one of our storyboard artists, Doug Lefler, but was actually the last thing to be shot, toward the end of post-production in a round of additional photography. I guess the world just wasn’t ready for it until we were really pressed for time! Once we were all creatively (back) on board, it then became a question of how to take a simple concept – weaponized notes – and use it in a variety of ways, with ever-increasing stakes, to create a short but varied sequence that built to a suitably dramatic climax. I likened the challenge to a Jackie Chan fight where the question was how many ways could a set of stepladders be used to fend off the bad guys?

The final “form” of Doctor Strange in the final battle is impressive! Can you explain in detail about his design and creation?

The Zombie Strange character really just emerged organically from story writing process. In a Multiversal movie it was clear from the outset that we would end up meeting other versions of Stephen Strange. So as soon as we decided to kill one of them (after much debate to be sure, but then there’s plenty of them to go round in an infinite number of universes) it was almost pre-ordained that we’d have some sort of Sam Raimi wheelhouse dead character to play with.

Once that concept clicked into place, it was a chance to pay homage to the look of Sam’s Evil Dead movies, and that provided the inspiration for the basic look of Zombie Strange… the sunken eyes, the rotting flesh, and so on. Compared to those movies, however, now we had the technology to incorporate digital FX into the practical makeup and appliance work that Sam likes to work with, creating a look for Benedict that would have been impossible years ago.

If you’re the kind of person that tracks these sort of things you’ll (hopefully) notice that the the cuts and slashes the movie-opening Strange incurs are the ones that degrade and rot into the deeper festering wounds that Zombie Strange sports in the final battle, the most of extreme of which is a big missing chunk of his jaw and cheek.

Zombie Strange’s cloak was borne out of necessity, to be honest – we found ourselves with Zombie Strange needing to travel up to the top of a nearby mountain but without his usual Cloak of Levitation to aid him. So we were at a bit of impasse until we hit upon the bright idea of not just having Zombie Strange battle demon guardians of the underworld but then also corral them to do his building in the form of a Cloak of Souls.

WETA FX handled the digital design and build assets of the Zombie Strange character and his Cloak of Souls. On set, Benedict wore a full set of prosthetic appliances with areas that were to be digitally replaced by WETA painted green. The cloak was always a 100% digital addition due to its fully animating nature. I don’t think we even had any practical reference for the cloak as we really didn’t know what kind of ephemeral substance it was going to be composed of at the time of shooting. BTW, there were plenty of other Dr. Stranges that didn’t make the grade as the story developed.

There are a lot of easter egg in the movie. Can you reveal which one is your favorite?

I suppose the Illuminati are really just one big basket of Easter eggs when it comes down to it, and if I had to choose from that basket it would be Professor X’s yellow hover chair, and accompanying theme tune, that are riffing on the X-Men animated TV series from the 90’s. That said, I’m still not sure if it’s a love it or love to hate it thing as the huge yellow roller coaster car design is the total antithesis of typical MCU movie tech.

It’s entertaining reading online what other Easter eggs folks have picked up on…or at least they think they have. Some things are definitely pure coincidence, but we’ll happily take retroactive credit for them. And there’s still the odd one that I haven’t seen mentioned anywhere yet. Many people correctly pointed out the green furry Minotaur character as being the sorcerer Rintrah who featured in the original Dr. Strange comics. He actually had a much more significant role in earlier versions of the movie but ultimately was reduced down to Easter egg status as the edit developed. Who knows…maybe he’ll make a comeback in an extended cut at some point?

A big thanks for your time.