Will “Clean AI” Be Part Of Hollywood’s Transformation?

After running a gauntlet in 2023, with dual strikes thwarting the recovery from Covid, Hollywood has entered a different phase with respect to AI.

Perhaps not a radically different phase — plenty of anxiety and doubt remains in the wake of the WGA and SAG-AFTRA’s fight with studios and streamers to win protections against unethical uses of artificial intelligence. Spasms of unchecked innovation like OpenAI’s Studio Ghibli generator, which remade the internet into one big Miyazaki film, raised unsettling questions about the agency of individual artists in a world of machines. Even so, a sense of curiosity and pragmatism has gradually seeped into the creative community, especially as technology keeps improving.

“The whole industry is going through a transformation, and it’s not because of AI,” says Andrew Hevia, head of film and TV at Fabula North America (Pablo and Juan de Dios Larraín’s production company). “Theatrical is being challenged and audience viewing patterns are changing … The outsized blockbuster budgets aren’t sustainable. You have all these other, way bigger forces. If this is a tool that allows us to increase our ambition while still allowing the industry to survive and thrive and make stories engaging in a way that hadn’t really been possible before, that feels like a net good.”

In that spirit, Hevia helped spearhead a partnership between Fabula and Runway, a rising AI startup valued at $3 billion. Runway also has collaborations in place with Lionsgate and Harmony Korine’s studio EDGLRD. The firm’s tools are being used on Sundance prize-winning director Cutter Hodierne’s film The Shepherd, set to shoot in 2026 in the Andes Mountains. The filmmakers are striving for a Terrence Malick-style, nature-based aesthetic without having to fund a full-on mountaineering expedition.

“It’s not a one-size-fits-all opportunity. It’s still a very human-driven process,” Hevia says. “You really have to refine it and speak to it to understand what you want to get out of it that’s really meaningful. That’s where there’s opportunity.”

Asked at a recent CNBC conference about her view of AI, NBCUniversal chairman Donna Langley described the prevailing response to it in recent years as a “panic” that was “a bit overdone.” The industry has a choice, she said. “We could be really scared of it and run for the hills, or we could embrace it as a technology that could actually enable efficiency or just a better set of processes.”

One thing is clear: A vast amount of money is being pumped into AI. Not only are Google, Amazon, OpenAI, Meta and Nvidia planning to spend hundreds of billions on it across a broad set of areas, but AI-oriented entertainment shops are booming.

Companies like Asteria and Flawless are making their mark. Stability AI turned heads last year by getting James Cameron to join its board. Joe and Anthony Russo’s independent studio AGBO is not only implementing AI across its business, but it also hired former Apple exec and Stanford professor Dominic Hughes as chief scientific officer to oversee AI efforts.

In a recent report on the state of AI studios, consulting firm FBRC.ai found the focus shifting from gee-whiz tech to traditionally rigorous storytelling. “This re-centering around narrative within the AI space is a welcome relief from the hype and overstatements that drove some of the conversations in the last two years,” the report’s author, Rachel Joy Victor, wrote. “The gleeful ‘press a button and make a whole film’ narrative that drove conversations (both positively and negatively) for much of 2023 was spearheaded by tech companies, not by the studios they hoped would implement them.”

Innovation can happen at scale, given that a D.W. Griffith-sized period epic doesn’t have to involve a cast of thousands. That explosive capacity is one reason unions and other stakeholders are wary. But a number of optimists see virtue in that speed to market. One is Staircase Studios, a new maker of films, TV series and video games founded by Divergent producer Pouya Shahbazian. It is promising to deliver “near-major-studio” levels of quality at budgets under $500,000.

If this is a tool that allows us to increase our ambition while still allowing the industry to survive and thrive and make stories engaging in a way that hadn’t really been possible before, that feels like a net good.

Andrew Hevia, head of film and TV at Fabula North America

Staircase points to the caliber of its pipeline, which is stocked with Black List scripts and overseen by animation veterans. Former studio production chief Lorenzo di Bonaventura has come aboard as an advisor.

Shahbazian says his experience of packaging and selling some 150 projects into the traditional studio system over the past 15 years revealed “far too much inefficiency to continue the status quo.” At Staircase, he says he will be “pairing ethical AI usage with our industry’s most underutilized assets — overlooked stories waiting to be produced from fantastic writers and directors.”

One industry titan who is tracking the evolution of AI closely is Netflix co-founder Reed Hastings, who segued from co-CEO to executive chairman of the streaming giant in 2023. He recently gave $50 million to his alma mater, Bowdoin College in Maine, to establish the AI and Humanity Initiative. The effort’s objectives are broader than the entertainment industry, but they are grappling with challenges that resonate with the uncertainty in the film and TV world. “As AI becomes smarter than humans, we are going to need some deep thinking to keep us flourishing,” Hastings says.

Asteria

Asteria, the company founded by Oscar-nominated producer-director Bryn Mooser and actress-producer Natasha Lyonne, occupies the vintage Mack Sennett Studios in Los Angeles, built in the early 20th century when the movie business was in its infancy. Now, more than a century later, the industry faces revolutionary change with dynamic advances in generative artificial intelligence.

“What we’re seeing with this technology,” says Mooser, “is probably the most significant leap, certainly in my lifetime, in terms of what it’s going to mean for how we make things.”

Existing AI companies, like OpenAI, train their video generation models at least in part on “publicly available data” like YouTube videos, without compensating or acknowledging the content creators. But Asteria, working in partnership with tech startup Moonvalley, promises “the first clean and ethical AI model.”

They call their model “Marey,” after 19th century French scientist and cinema pioneer Étienne-Jules Marey. “We’ve trained it only on data that we have paid filmmakers or rights holders to license,” Mooser says. “Nobody else is training a model like this.”

It’s a filmmaker-first approach, Mooser insists, and solves a major concern for Hollywood­ copyright issues. Studios can’t slap AI-generated images on screen that have been co-opted from someone else’s work without inviting a lawsuit.

“If we can come out with a model that’s as powerful as everything else, but is trained ethically and legally, then we can get that model through the [Hollywood] legal departments,” Mooser says. “And on the studio side, we hope Marey can become the foundational model for the industry.”

The Marey model unleashes stunning possibilities. Take an animator who wants to turn 2D illustrations into a film. By ingesting artwork into the model, Marey can take care of animation and more — saving huge amounts of labor.

“The technology that we have under it, which is this clean model, is then going to help scale it,” Mooser explains. “More characters, more backgrounds, but all in the style of what the artist has done.”

In a live-action context, material shot by a filmmaker can be uploaded into Marey to generate sequences that weren’t captured in camera. “We want to work with studios to take their dailies, after they’ve wrapped production,” notes Paul Trillo, Asteria’s strategic partner and senior creative director. Based on that “data set,” Marey can create “an insert shot, get B-roll shots that they couldn’t have gotten on set, impossible camera angles, whatever, on top of the filmmaker’s [original] material.”

One fear voiced about AI is that it might somehow render the filming process obsolete. But Asteria proposes quite the opposite. “It’s about expanding that footage infinitely,” Trillo says. “And the edit room becomes, for better or worse, very malleable where it’s like all things are on the table and hopefully empowers editors to audition ideas. So, an editor could be like, ‘I think this would flow better if we cut here, if we insert this.’ The editor can have some agency in the storytelling too.”

Inferior AI models could potentially homogenize cinema by generating imagery from stale, generic prompts. “AI can really throw away a lot of craft,” Trillo observes. “There is a risk on the cultural and creative side. And so, we’re trying to show another path.”

Filmmaker Scott Mann was watching a foreign version of his 2015 Robert De Niro starrer Heist, when he noticed something odd. “All the dialogue had been changed, and the lines were very different,” he says. “I then learned that to try and get synchronicity, the dialogue is rewritten to try and fit the wrong mouth movements and I just thought, ‘Well, no wonder the international markets get hurt so badly and films don’t sell for as much outside of their home markets.’”

At the time, the British writer-director was growing increasingly frustrated at the health of the film industry — “It was getting harder and harder to make good movies because it was so expensive, so constrained and the budgets were going down” — before he stumbled upon an article about deep video portraits at major science and tech conference Siggraph in 2018. “These guys Hyeongwoo Kim and Pablo Garrido at the Max Planck Institute in Germany presented this new video-editing technique at Siggraph, which is what we now call generative AI. I knew it was going to change how we make films, and it would fix the film dubbing problem.”

Mann then joined forces with tech entrepreneur Nick Lynes to launch Flawless AI, a company that provides what they describe as “the world’s most advanced and artist-friendly film technology.” The duo, who are both co-founders and co-CEOs, say the mission is to integrate transformational, AI-powered technology into the film industry that will enhance creativity, and it believes this is the most profitable long-term approach.

Since starting in 2019, the company has spent $100 million on building out its technology platform and filmmaking tools, which currently consist of three main aspects: its visual dubbing tool TrueSync, which was named one of Time Magazine’s Best Inventions; DeepEditor, a tool that allows for dialogue and performance changes without costly reshoots, synchronizing actors’ mouths to new lines; and the rights management platform Artistic Rights Treasury.

“The difference with the track we have taken is that we’re looking at this from a filmmaker perspective,” Mann says. “We look at it from a copyright and rights perspective. There are two routes the technologies can go: It can go in the theft of copyright route and destroy things, or it can be a useful tool if these things are provided. We have taken the harder path, but we know its worth it.”

Its dubbing technology was recently used in Swedish sci-fi hit Watch the Skies, making it the first theatrical full-length feature to use immersive AI dubbing. Flawless partnered with XYZ Films to release the English-language version of the film in U.S. theaters. And this will be the first of many.

“We’ve been doing this really quietly because it’s a great test bed for our technology,” he says. So far they’ve picked up Michel Gondry’s The Book of Solutions, Judo thriller Tatami, South Korean title Smugglers, Stéphan Castang’s Vincent Must Die and Berlin Film Festival opener The Light. All will have a U.S. theatrical release after dubbing.

“The language of cinema is universal, but the barrier of a different language can sometimes hurt a film,” says Mann. “Utilizing technology to knock that barrier down and have a truly immersive experience, as if the actors performed it in English, means that you end up really watching the performance rather than reading subtitles on the bottom.”