Autodesk Media & Entertainment
Boxel Studio thrives at the intersection of technology and artistry, constantly pushing the boundaries of what’s possible in visual effects and animation. Over the past two years, as the film and TV industry underwent significant evolution, they recognized the transformative role of machine learning (ML) in streamlining production processes.
This shift inspired them to invest R&D resources in emerging tools and technologies to enhance their production capabilities. Among these innovations was Wonder Dynamics, a pioneering company whose potential was evident from its inception. Their solutions expanded Boxel Studio’s workflows and introduced a more efficient approach to streamlining processes.
Machine Learning Motion Capture
At the start of 2024, as the Hollywood strikes and negotiations came to a close, the accelerated production schedule for the final season of Superman & Lois demanded high-quality visual effects within tighter deadlines and a reduced budget. Traditional motion capture methods were no longer practical under these conditions. Recognizing an opportunity, Boxel Studio partnered with Wonder Dynamics to utilize their markerless motion capture technology, powered by machine learning (ML). This groundbreaking solution allowed them to capture actor performances from a single camera angle and swiftly convert them into motion data, streamlining the process significantly.
What previously required a three-week turnaround using traditional workflows could now be accomplished in a couple of days, allowing for immediate feedback. This breakthrough not only accelerated previsualization but also enriched Boxel Studio’s animation pipeline, giving their artists greater flexibility and control over the creation of digital doubles. The result was a seamless integration of motion data, ensuring a polished final product while meeting the demands of a fast-paced production.
In the right place at the right time
When Wonder Dynamics co-founders Nikola Todorovic and Tye Sheridan launched Autodesk Flow Studio (formerly Wonder Studio) in March 2023, Boxel Studio seized the opportunity to become one of the early beta testers of their groundbreaking technology. Freddy Chávez Olmos, Creative Director of Artificial Intelligence and Innovation at Boxel Studio, reflects on the experience:
“I still remember the surreal moment watching that first promo clip the Wonder Dynamics team created to showcase Autodesk Flow Studio’s capabilities. There was skepticism in the industry. It was so innovative that many thought it couldn’t be real. I was eager to get Boxel to join Wonder Dynamics’ early beta testing group to explore practical use cases for their tools in production.”
One of the early tests featured Juan Carlos Galindo, Head of New Technologies at Boxel Studio, filmed on an iPhone with the motion data retargeted to a proxy model of Superman and Doomsday in Unreal Engine. Galindo reflected:
“At Boxel, we thrive on constantly exploring new ways of doing things and thinking outside the box. A show like Superman & Lois was the perfect scenario to test real-time technologies alongside emerging machine learning tools for rapid previsualization”.
Technical challenges
As early beta testers, Boxel Studio understood the limitations of the technology. Understanding the need to work within these boundaries, the team collaborated closely with Wonder Dynamics’ technical team to ensure that all processed data and security protocols met the stringent requirements of their clients.
Taking a new technology beyond previsualization and into actual shot production posed additional challenges. To address these, Boxel developed custom tools to retarget proxy assets with motion capture data into production models equipped with complex rigs. Additionally, they built translator tools to integrate seamlessly with software like Maya, Unreal Engine and Cascadeur, enabling artists to refine animations with full control and flexibility. On this topic, Juan Carlos Galindo remarked:
“As soon as we saw the results of our first test with Autodesk Flow Studio within Unreal Engine’s Previz, we began collaborating with the rigging and animation departments to develop a method for transferring Wonder’s markerless motion capture data to a production rig. This iterative process involved tailoring the animation pipeline and production requirements into a Python tool that empowered animators with precise control”.
Production Requirements
As the Boxel team transitioned from Superman & Lois Season 3 to Season 4, the stakes were higher than ever. Season 3 concluded with an epic cliffhanger: Doomsday battling Superman on the moon. This set the stage for the Season 4 finale, Episode 10, which demanded even more complex creature animation than the Season 3 finale—a significant challenge under normal circumstances.
During this period, the Hollywood industry faced substantial challenges, and the team entered Season 4 with both a tighter schedule and a reduced budget. These constraints meant they could no longer rely on the same resources, such as the motion capture stage and facilities that had been instrumental in bringing Doomsday to life in Season 3. Innovation became a necessity. As Andres Reyes, Founder and Head of Studio at Boxel, explains: “We had been experimenting with markerless, machine-learning-driven motion capture systems—an innovative yet untested technology in real-world production. Implementing these systems was not only a technical leap but also a collaborative effort that depended on the trust and support of the Superman & Lois team.”
Securing the confidence of VFX producer Matt Gore and showrunners Todd Helbing and Brent Fletcher was critical. Understanding the constraints and sharing the vision for smarter, faster solutions, they placed their trust in Boxel’s ability to deliver. This trust made it possible to fully implement the groundbreaking technology, despite it being its first real-world application on television.
Reflecting on this, Matt Gore explains: “In season 4 our visual effects department and production team faced budgetary constraints at a level that we had not experienced in our previous 3 seasons. We knew entering season 4 that we wanted to keep trying to push our character animation (and our limited budgets) as far as we could. As a result, we pivoted very quickly between seasons and came up with a way to provide our animators with more time, even though we had fewer resources. Boxel rose to the challenge and helped devise a system that not only improved on the animation pipeline from our previous seasons, but also went well beyond anything we had hoped for in terms of budget, efficiency, and quality”.
The results were remarkable. In the Season 3 finale, the team delivered 104 creature animation shots using a traditional motion capture system. For the Season 4 finale, Episode 10, the stakes were even higher, with 134 creature shots required. However, the team only had six weeks to deliver compared to the eight weeks available in Season 3. This meant producing more animation in less time while maintaining the high standards expected by the show’s creators and fans.
Thanks to the implementation of markerless machine-learning motion capture technology, the Boxel team met these demands efficiently. This real-world application demonstrated that the cutting-edge solution could not only accelerate the animation pipeline but also maintain the quality necessary for a high-profile production. It stands as a testament to how innovation can transform workflows under the pressure of tight deadlines. VFX Supervisor, Weverton Alves, emphasized:
“This new approach allowed us to work quickly during the production of the offline VFX cut, helping us determine whether a shot fit seamlessly within the sequence. We could refine and define Doomsday’s action in the offline stage and then give the green light to Hector Gil, our animation supervisor, to proceed”.
Stunt department embracing new technologies
One of the most exciting aspects of implementing markerless motion capture technology for Superman & Lois Season 4 was the collaboration between Boxel Studio and stunt coordinator Rob Hayter and his team. From the outset, Rob recognized a fundamental truth about this technology: the quality of the input is directly proportional to the quality of the output. This understanding made him an invaluable collaborator in pushing the boundaries of what could be achieved. As Weverton Alves notes:
“Unlike traditional workflows, where performers wear motion capture suits on a dedicated stage, this approach eliminated those requirements entirely. Instead, we utilized a setup of four strategically placed witness cameras, capturing full-frame footage of the performers. This streamlined setup not only saved time but also gave the stunt team the freedom to focus entirely on the authenticity and precision of their performances without the constraints of technical gear”.
Rob’s adaptability and leadership in rallying his team were critical to the success of this innovative workflow. By embracing this new approach, the stunt team was able to perform in real-world environments, resulting in more natural and dynamic motion that seamlessly translated into creature animations.
The impact of this shift is profound for stunt coordinators and their teams. With just a high-quality camera and markerless motion capture technology, they can now produce hyper-realistic CG animation and motion capture data without the need for extensive technical setups. This empowers stunt teams to concentrate on their craft, refining their performances while feeding directly into a streamlined VFX pipeline. It marks a new era where the artistry of stunts and the precision of animation are more connected than ever.
This pioneering collaboration between the stunt department and Boxel Studio demonstrated how technology and artistry can merge to redefine production workflows, proving that innovation thrives when experts across disciplines work hand-in-hand.
As Rob Hayter stated, “Full Circle Action Design has always been excited to work with emerging technology. When Superman & Lois first began integrating stunts and motion capture into our VFX workflow, it was very cutting edge for television, but this took things to another level. By setting the precedent of engaging our performers at their full union rate, production opened up the opportunity to directly integrate the stunt department into the process of creating top-notch VFX action. This level of commitment ensured that the VFX team would have the specificity of movement and physical technique that you only get from using professional performers. It was a great team effort. I think the results speak for themselves”.
Animators embracing new technologies
Animators are the unsung actors behind the scenes, puppeteers who meticulously move the joints and controls of creatures and characters to inject them with life and emotion. Whether refining motion capture data or creating entirely new animation from scratch, their artistry is the foundation of the final product. Andres Reyes explains:
“As much as we value the capabilities of motion capture, it’s not always enough. There are moments in production when motion capture data is incomplete or when a scene demands a creative touch that no performance can capture. In these instances, animators step in—not just as technicians, but as artists and storytellers who bring characters to life on screen”.
In Superman & Lois Season 3, animators often relied on traditional methods for certain shots. For example, when creating entirely handmade animation, an animator would record themselves performing as reference, observing their own movements in the mirror to replicate subtle details. Crafting a high-quality five-second shot could take a senior animator up to three weeks to complete. This meticulous process required incredible skill and dedication to achieve the desired results.
Now, with the integration of AI motion capture, this process has been revolutionized. Animators still record themselves acting out performances in front of the camera, but instead of using the footage solely as reference, they can process the data directly. This enables them to extract motion capture data and use it as a foundation for their animation as explained by Weverton Alves: “Generally, most of the motion capture data produced on set by the stunt team was processed through Autodesk Flow Studio and passed to the animators for refinement. However, some shots required more significant adjustments to fit within the context of the sequence once they were edited into the cut and reviewed by the client. One of the cool discoveries during the process was that some animators, like Misael Garcia, benefited from recording their own motion capture performances and using them as a starting point for rapid visualization in dailies”.
The impact has been profound. What once took weeks of painstaking work can now be completed in a fraction of the time—often within two to three days—while maintaining the same level of quality. By reducing the time required for execution, animators can focus more on refining details and enhancing the emotional depth of their work. This blend of traditional techniques and cutting-edge technology has unlocked new levels of efficiency and creativity, empowering animators to deliver top-tier results on even the most demanding timelines.
This seamless integration of innovation and artistry underscores the evolving role of animators in modern productions. By embracing these tools, they continue to push the boundaries of what’s possible, ensuring that even the most fantastical creatures and characters feel real and emotionally engaging on screen. Juan Carlos Galindo remarked:
“Skilled animators are essential for refining Wonder’s motion capture data into polished final animations efficiently. Our animators have embraced this technology, achieving exceptional results while pushing its capabilities to the limit”.
Keeping up with innovation and moving forward
As emerging technologies continue to evolve at a rapid pace, Boxel Studio firmly believes that human creativity and storytelling should remain at the heart of every technological advancement, a core value they share with Wonder Dynamics and Autodesk. Freddy Chávez Olmos elaborates:
“We are always keeping an eye on every new tool available out there, constantly testing and evaluating new ways to tell stories more efficiently with the help of technology. As a filmmaker myself, I always approach our work from that perspective when collaborating with clients.
Emerging technologies are enabling us to become more multifaceted, experimental, and curious about other disciplines. In the past, as artists, we were often confined to a single specialty, but now we are in an era of endless opportunities, where we can wear multiple hats and optimize our time and resources. For me, as a filmmaker, this is a dream come true. I’m excited to bring a new set of innovative tools recently developed by Wonder Dynamics and Autodesk into production and see how they continue to transform our creative workflows”.
Andres Reyes concludes: “As the Founder and Head of Studio at Boxel, our DNA is clear: storytellers at heart, image engineers by craft. This ethos drives everything we do. At Boxel, we were not born to think as selfish individuals, but as a united team of professionals, passionately committed to supporting studios, directors, and showrunners—both creatively and technologically—in their mission to tell the most compelling stories.
In an industry that thrives on collaboration and innovation, we embrace the necessity of building new relationships. Disruption and transformation are constants in our world, but we believe there’s always an opportunity to approach these changes in a way that benefits the industry as a whole. Together, we can shape the future of storytelling”.
Explore how Wonder Dynamics’ AI-powered motion capture technology can speed up your VFX workflow.