Behind-the-scenes of the traumatic Spider-Man scene!

Behind the scenes of the biggest effects in ‘Spider-Man: No Way Home’

‘No Way Home’s’ VFX supervisor on crafting the Mirrorverse, orchestrating 3 different Spideys, and whether they could access any old 3D models from the past films (hint: no).

With Spider-Man: No Way Home, not only was production visual effects supervisor Kelly Port tasked with overseeing a dazzling array of complex VFX sequences—moments like the Mirrorverse, a freeway battle between Spider-Man and Doc Ock, a fight atop the Statue of Liberty involving several super-heroes, and many more—Port also had to navigate the fact that of course a number of beloved characters previously crafted with the aid of visual effects would be returning to the screen in some form or another.

In this interview with befores & afters, Port, who was supported by visual effects producer Julia Neighly and a bevy of VFX studios, including Digital Domain, Imageworks, Framestore, Luma Pictures, Crafty Apes, Cinesite, Folks VFX, Mr. X, SSVFX, MARZ, Secret Lab, Perception and more, talks about how the VFX teams made those returns and elaborate sequences possible.

b&a: First up, I wanted to say I really loved the transitions between live action actors and CG swinging digi-doubles, especially at the beginning where Peter and MJ are swinging around trying to get away from the spotlight. Was there anything new you felt you were doing there with those kinds of shots?

Kelly Port: Well, it’s really just attention to the detail of it all. For example, when they first come out of the manhole cover and walk and then kind of run and then jump, normally, we would need a digi-double when we can’t use the live action anymore. But in order to blend seamlessly into that CG digi-double for both MJ and Peter, sometimes we would need to get into the live action part of it earlier. So, they’re actually digi-doubles quite a bit sooner, I think, than you would think. We’ll have to change their cadence a little bit before the jump up to get the right weight. We are actually changing some of the animation leading up into it.

b&a: On these big films, of course, the lead actors and often the secondary actors are scanned and photographed and textured for stunt doubles and also close-up work. What was your methodology here?

Kelly Port: We scanned everything and everybody, high-resolution, proxy-resolution, props, actors, background extras. Because we had a scanning booth on set, we just took advantage of it throughout the course of the production. We ended up processing a lot, but we scanned literally everything.

b&a: Obviously, someone like Tom Holland, he’s there without his mask and with mask sometimes. Does that mean you’d scan him in normal clothing, and then on-set suit as well?

Kelly Port: Yeah, we did. That’s another thing we could take advantage of; we would scan him and get high-resolution textures, polarized, non-polarized, you name it, in every single iteration of that costume, too. So with and without mask. We also scanned stunt doubles, because oftentimes you need to body track them and they have a slightly different body. So, if we’re tracking somebody who’s a stunt performer, we would need to swap out the Spidey for that guy’s scan, just to track it better. If it was using a slightly different body, it wouldn’t track.

b&a: One of the scenes I wanted to ask you about is the bridge attack with Doc Ock. I just thought that was a brilliant example, I assume, of shooting so much on some kind of backlot or set in Georgia and then implementing a New York environment. Tell me first about the planning for that sequence.

Kelly Port: Digital Domain was tasked with doing that sequence, which was a few hundred VFX shots. Also, the Digital Domain previs team did the previs for the whole movie, which was really cool, led by Matt McClurg. They just did such a great job. There’s so many cool iterations of that sequence that will never see the light of day. It was way longer. At one point in its longest iteration, it was 15 minutes long.

We shot in Atlanta at Trilith Studios, which was called Pinewood Atlanta. It was a backlot pad that was made specifically for us. We had 40-foot bluescreens on three sides of that pad, a little bit of roadway with the exit where the assistant vice chancellor was exiting and where most of the action took place. We’d redress the set for when Spidey first lands on a sign, and then he jumps off the sign and tumbles and he is looking for her. He goes through some traffic before he gets to her, so we would redress that same section for up until he gets to where she is. And then that became our main action set piece. But beyond that little piece of road section, Digital Domain built an entire, fully-CG digital environment around that that would include all the other surrounding bridges, the other side of the bridge that they’re on, all the city of Brooklyn and all the area around it, the river, and all the trees, the fall foliage. It just looked great.

b&a: When you were shooting that, what sort of rigs and other things helped? There’s some crazy stuff that happens with that, apart from just Doc Ock throwing cars around. There’s cars over the edge and with actors in them.

Kelly Port: Dan Sudick and his special effects team built a few specialized rigs. One was for when the assistant vice chancellor was inside the car, it would be able to tilt down 90 degrees, that is, go from flat to 90 degrees. And so when you see her expression, well, she was not a fan of that rig. I felt bad for her! But that was authentic fear in her. She was quite high up and it just basically tilted straight down.

We then put the environment in around her. Also, even though it was a live action car with her in it, we had to replace a lot of that exterior surface on the car to get all the proper reflections. One of my favorite shots is when Spidey swings down, upside down and then he turns around towards camera to see Ock, and I just love all the reflections in there and of that whole environment. And then of course, there’s those big, big explosions that Sudick is great with, like when Goblin throws the bombs. Those were some really big explosions that were fun to watch on set.

b&a: I feel like that’s quite bold to do real fiery and dirty explosions just on a bluescreen set like. I mean, I know that Dan has done a lot of that work before, but just as a compositing challenge for the VFX team. But I’m guessing, Kelly, you just head straight into this stuff, and just having it for real gives you the best results straight away.

Kelly Port: I couldn’t agree more. My rule of thumb is, if there’s any way possible to shoot something for real, something physical, then we do. We made an Iron Spider bust from chest to head and shoulders, which was just so, so helpful to get those bright pings for reference. We shot this outdoors, and so it was all about, how does this read on an Alexa camera if we were to actually shoot it Otherwise, you just don’t know. You just guess. And just actually having that there, having physical cars there, too, will help inform our digital cars.

Having a practical Iron Spider bust will help inform our fully-animated CG version. We also had pieces of Doc Ock’s claws and tentacles that we would bring out as reference. Anything physical would be super, super helpful. It’s the same thing with the explosions. I love the idea of tying physical elements into the digital world. It just always, always makes them look more believable.

b&a: When you had to do Doc Ock shots, I’m curious about the approach there. How were you filming Alfred Molina when he was live action, hovering, effectively?

Kelly Port: We had a couple different rigs. The one that we probably used the most was a platform rig. It was like a crane on a pivot, which meant he could rise or fall. It would enable him to be much freer with his upper body versus any wire work. He had that big coat on, too, so any kind of wire work where he would have to wear a big harness and it would pull the coat, we could tell almost right away that that really wasn’t going to work as well.

The platform would work great for some of the shots where you just had to deal with above the waist kind of framing. For the wider shots, where we used that, we would have to replace his legs of course because they would be dangling.

Essentially, we would use what we could from the live action. Most of the time, it would just be his head, though, but we’d try to keep his performance. If we had to roto-mate his performance, we did so. But especially if he’s moving quite a bit or there’s an explosion next to him or something like that, we’d have to add cloth simulation and things like that to get him into that world better.

b&a: For Spider-Man 2, when we see Doc Ock, that was such a breakthrough for Imageworks in terms of digital humans back then. Obviously this film has Doc Ock, and the past Spider-Men—was there any asset retrieval that could occur?

Kelly Port: Absolutely not [laughs]. None of those digital assets, as far as we were able to find, just simply don’t exist anymore. They’ve vanished from the world, I think. But we did have physical assets that we were able to access and I think were really helpful, like a portion of Doc Ock’s claw and tentacles. We actually based our tentacles and claws on those. We were able to take those out of the physical archive and do a tonne of great reference photos for those.

And then we scanned the Goblin’s costume. It wasn’t in very good shape, but we scanned that and modified that a little bit. But none, really, were digital assets that we could retrieve because they just don’t exist. A lot of that wouldn’t have been very useful anyway because systems that were used then, in terms of look development or shaders, they would all be different by now, anyway.

b&a: When it came time to feature Tobey and Andrew and Tom’s Spider-Men together in scenes, I’m really curious about whether you did feel there needed to be very different looks to their suits to match their films, but also different behaviors and movement, like particular poses or anything? I felt like there were and it was really nice to see those subtle differences.

Kelly Port: Well, we did. And in fact, the suits are different. In the end battle, there’s a lot of passing and things like that. Editorially, we did everything we could to just make sure that it was as clear as possible, whether through handovers, meaning you saw more than one Spidey in a frame handing something off and tossing something to somebody else, so that you know who’s who for the most part, in addition to what the different suits look like.

And, as you mentioned, stylistically, they each have their own kind of iconic poses. This is especially the case where after the sandstorm and regrouping, you get that amazing theme music where even the scores are intertwined. You get some of the Danny Elfman stuff in there, and it’s really, really cool, because each have their own themes.

When all three jumping onto the top of the statue head, backlit by the moon, you get those iconic poses as well in the air. We just took great pains to even get their running styles and the gaits and their iconic poses in the air and their swinging styles, things like that. We had a few animators, both at Imageworks and DD and I think a few other places, where a lot of those artists and animators had worked on those previous films, so it was fun to get their firsthand experience on that, too. We had animators animating Doc Ock walking who had done the original Doc Ock walking. So that was cool. Imageworks handled that end-sequence on the Statue of Liberty, with Cinesite doing the shield fight with Green Goblin.

b&a: Even the suits themselves, did they match the suits from those Tobey films and Andrew films?

Kelly Port: Well, we based our digital suits on what our costume department made. And I think for the most part, they based them on the original suits. However, there could be a little bit of variation in them. The fact that Tobey and Andrew came from a slightly later timeline, they’re older, so time has passed, and they’ve lived their lives and probably have changed their suits or augmented them a little bit.

b&a: I really liked Sandman in this film. There was something nice about the throw-back, but something new as well. How was he achieved?

Kelly Port: There were two parts to him, where he’s more humanoid and talking, and when he is in the sanctum. Digital Domain did a lot of that work. And then Luma did the sequence that we called the power line corridor, which is where Sandman is first introduced. And Electro is also introduced and they have that little battle before they all meet and then get sent to the sanctum. I have to say, if you’re able to see that in extended dynamic range, for both that sequence and the end battle, it really looks cool in that higher dynamic range. That electricity just pops.

Then Imageworks, for the end battle, they did the much bigger Sandman with big FX sims. There were a lot of big sims in the end battle, not only with Sandman, but for the shield falling and crashing through all the scaffolding and millions and millions of pieces all interacting with each other, the crane falling through Sandman’s head and all that good stuff. I remember when we were first interviewing companies, there’s a little bit of collective post-traumatic stress disorder with anyone who had previously worked on Sandman. And so I knew that going in, I had a feeling this was going to be challenging both technically and creatively, just to get that character looking right and behaving right. But I think we were successful in the end. I think it looked pretty cool.

b&a: Did you have the benefit of the actor for any facial cap for Sandman?

Kelly Port: Well, we had his voice. We weren’t able to get a lot of visuals on him, but we were able to get his voice for sure and we got scans and textures and things like that.

b&a: For that Mirrorverse sequence, I wanted to ask you about going from script or early concept to ‘how you’re going to shoot’.

Kelly Port: That sequence went through quite a few ideas and just different iterations as well. And again, a huge thanks to Matt McClurg and his team because they came up with a lot of those cool, fun ideas, those little action beats. Initially, it was mostly just a city and Dr. Strange/mirrorverse kind of New York, bending buildings, things like that. But we did want to try something new and fresh for that. Framestore did the visual effects for this scene.

Now, basically, this whole chase is meant to be Strange putting obstacles in Peter’s way. And Peter, being smart and adaptable, is just adapting to everything that he’s throwing at him. So Strange is continually getting frustrated and trying to do all sorts of weird and different things. You know, ‘Okay, now I’m going to turn the whole world upside down. Okay, if you’re sticking to buildings, I’m going to make it the Grand Canyon.’ And so we came up with this idea of doing some kind of a hybrid city/Grand Canyon, which ended up looking really cool, too.

And then you get into playing a lot with different portals. Because he’s basically throwing his arsenal of magic at Peter, and Peter’s, in a fun way, adapting to it all. So you get portals upon portals, and you end up in this ‘mirrors reflecting each other’ kind of situation where we would say, ‘Well, what would happen if that is the case?’

There was definitely a little bit of head scratching going on—‘So, what if you’re looking through a portal, how would…etc etc.’ You have to really figure out the physics of it all. If he webs that way into a portal but then there’s another portal here and then he would web his own foot. But how would he hold onto it, exactly? Well, if there’s enough mass, because it’s an infinite number of Spider-Men and webs, then maybe there’s just so much mass that even though they’re theoretically just hanging in the air, but there’s so much mass because it’s an infinite number of them. Sure. It’s, like, we can make sense of that.