In The Mandalorian on Disney+, the titular bounty hunter takes out a group of irksome Jawas with his blaster rifle, then scales their massive Sandcrawler vehicle with a grappling hook while it bumps across the Arvala-7 desert.
If you think that’s cool, wait until you hear how the scene was filmed.
Industrial Light & Magic (ILM)—the Lucasfilm subsidiary George Lucas founded in 1975 to make the visual effects for Star Wars— deployed a real-time 3D projection system called “Stagecraft” on the Disney+ series that could, eventually, replace green-screen as the film industry standard for rendering virtual environments.
The company has been testing Stagecraft for five years—most recently on the Star Wars spin-off movie Solo in 2018. But The Mandalorian, the flagship series on Disney’s new streaming service, likely marks the most extensive use yet of the new system.
The entertainment blog Slashfilm attended a “Women of Lucasfilm” panel this week, where Lucasfilm president Kathleen Kennedy described how Stagecraft was used to film The Mandalorian, as well as how she believes it will revolutionize the way filmmakers plan and execute complicated shoots in the future.
Stagecraft’s chief innovation is that it can project a 3D visual environment around the actors that changes in real time to match the perspective of the camera. When the camera moves, the background moves too, simulating the experience of filming in a different location. It’s a significant upgrade from green-screen technology, which requires the filmmakers layer in a static image or footage after filming in front of the blank backdrop.
Kennedy said a Disney executive visited the set of The Mandalorian and thought that the Stagecraft-generated environment was physically there. “He had no idea he was standing in a virtual set,” Kennedy said, according to Slashfilm. “That’s how unbelievable it is.”
While that may be a tad hyperbolic, Stagecraft does allow for a much more realistic filmmaking experience. This video from Unreal Engine—the game engine made by Fortnite developer Epic Games that helped ILM film The Mandalorian—shows a smaller scale version of the technology in action:
The tech has a wide range of benefits. For starters, it can draw better performances from the actors, who don’t have to imagine the environment they are in, as they do when filming in front of green-screen. They can instantly be transported to any location, real or made-up, and feel as though they are there.
And that’s another big advantage: Stagecraft allows films and TV shows to simulate environments without actually having to send an entire production there to film. “If you want a big establishing shot in Iceland, and you don’t want to take 700 people, spend four months prepping a set because you only want to do the establishing shot and you can bring everything back to shoot interiors on a stage, that becomes very meaningful on big, huge projects and small projects,” Kennedy said.
Now, they only need to send one camera person to an exotic location to capture footage of it in order to use it as a projected environment. Locations can be recorded and then digitally manipulated in whatever ways the filmmakers want before projecting them around actors on set.
Stagecraft can also make the production process more efficient. “You can change from Iceland to the desert in one [minute] from setup to setup, so it really changes the flow of production,” Lynwen Brennan, a Lucasfilm executive, said during the panel, according to Slashfilm.
It may become so easy to render 3D environments that filming on location or with physical movie sets could become a thing of the past. Why bring a 1,000-person film crew to a remote location when you can bring the remote location to the film crew? Why spend millions on massive set pieces for the actors to be immersed in when you can simulate those things with the press of a button?
One downside is that the displays used in Stagecraft require liquid crystals that take several years to grow. “It’s quite a process,” Kennedy said. Growing and maintaining these crystals, which are the backbone of LCD (liquid crystal display) screens, can be expensive and time-consuming, perhaps complicating the attempts of other companies to adapt the technology.
Despite the obvious improvement over green-screen when it comes to the filmmaking process itself, it’s still unclear if the technology will lead to a better product for viewers. Some scenes that take place outdoors in The Mandalorian, for instance, very much look like they could have been rendered with green-screen. The technology is not yet at a point where it can match the visual splendor and immediacy of traditional filming, as cumbersome as it sometimes may be. From a viewer’s perspective, it’s still no substitute for going to a real location and filming there.