Autodesk Media & Entertainment
Article written by Steve Silver, Games Industry Strategy Manager at Autodesk
Steve Silver is the Manager of Games Industry Strategy for Autodesk’s Entertainment and Media Solutions. He has worked in the gaming industry for 19 years as a Producer and QA Analyst with credits in games such as Empires and Allies, TOME: Immortal Arena, and the Battlefield series, to name a few. His current games industry focus is on growth, mentorships, and history.
Creation is a foundational ingredient to the growth and expression of our species and planet. Our ability to extend our capabilities has always been done with the usage and innovation of tools. To build or to make real what we imagine takes an understanding of our own limits in order to inspire new ways to exceed them. Imagination consistently proceeds implementation.
Making something is a form of expression seen throughout our own history, with entertainment and art being at the core of some of the best work we’ve ever done. Renaissance sculptors punching chisels to limestone would chart the course for 3D modelers to bring stylus to screen.
When we think about the role art plays in our modern entertainment, it’s difficult to fathom the amount of skill and time it takes to craft the detailed characters and effects we’ve come to expect of our AAA games and summer blockbusters. Very few would match the high expectations that artists have for their own work, and those expectations only grow as we become more efficient as a species.
From the beginning of game development as an industry, artists, designers, and programmers have had to think smarter to get more out of their experiences. As a result, the use of AI for optimizing workflows and content creation has a much longer history in video games than it does in other modern forms of entertainment.
AI in game development isn’t a new concept because it has always been a part of it, taking many forms, such as procedural generation, deep learning, decision trees, and random encounters. These tools were forged with the intention of helping bring complex experiences into reality, informed by our experiences and guided to accelerate our ability to tell an immersive and replayable story.
So, let’s dive in and take a closer look at the role of AI in modern gaming, its history, its impact, and where it’s going.
The role of AI in modern games
When making a game, the developer attempts the difficult task of designing for every possible player scenario. Accounting for outcomes is much easier when a game’s rules are obvious and served to the audience in a mutually agreed upon format, like an on-rails shooter or a side-scrolling platform game. So, when you have a detailed open-world game like Grand Theft Auto or Cyberpunk 2077, there are valid reasons something like that can take over a decade to release.
For small teams to accomplish this, they require tools to assist them. Not only to accelerate their ability to make a stable experience, but in many cases, to help automate the content that will leave players with a meaningful impression.
Enter Artificial Intelligence.
Let us look at two types of AI for games; one type where players have a larger variety of experiences, and another where developers create experiences more efficiently.
As an experience, AI in games acts as a content generation tool that uses procedural calculations to randomize the distribution of assets. These capabilities are at the heart of some of the most popular games, Like the unique characters in Spore, the randomized dungeons in Diablo, and the massive worlds in Minecraft.
AAA games today on average are about 80GB in size and much of that storage is comprised by the game’s textures and assets. Content is at the heart of a game’s experience, no matter how photorealistic or stylized. So often, physics-based rendering is utilized to simulate effects, content is created and shaped for procedural generation, and NPCs are trained through months, or sometimes years, of playtesting to adjust the behaviors of their scripted events until they feel balanced and satisfying.
Creating games with that level of complexity requires meeting it with equal or greater capability, providing the challenge to think more efficiently about a game’s production pipeline.
When I was a Quality Assurance tester, our job was to verify that everything on the critical path of our game functioned as designed, but it was also our job to think outside of the parameters of the game’s conventional intent to trick the code into behaving beyond its breaking point. Sometimes, the results were hilarious, leading to meme-able moments, while other glitches led to show stopping bugs that could be exploitative or game breaking. In either of those cases, these issues broke not just the game, but the immersion, potentially causing player engagement to suffer.
To attempt the task of accounting for every outcome before the public did, we used automated tools that would help us simulate large networks of players, to test the limits of a game’s performance under pressure. We frequently automated our manual test suites to run repetitive test cases more efficiently, leading us to flagging issues quickly. These tools provided us with the valuable bandwidth we needed to think creatively about new ways to push the game’s boundaries.
Outside of QA, this existed for asset and technical production as well. It wasn’t to cut corners, rather to round the edges, achieving our intended results much faster.
Today, AI Development tools continue to evolve with new capabilities like Generative Scheduling, which can optimize the instant adaptability of a product’s delivery process by considering all factors of a production pipeline, quickly adjusting the schedule to account for any dependencies or pivots. Sony Interactive Entertainment described their acquisition of the AI video optimization company iSIZE as a product that, “will benefit a range of our R&D efforts as well as our video and streaming services”. Tools like these are focused on workflow and rendering optimizations.
These kinds of efficiencies are all in the service of making a process more predictable and stable, so that the greatest number of gamers can enjoy a seamless experience while also minimizing the tedious complexities of game development.
The evolution of AI in gaming
The concept of a non-player character with AI behaviors can be traced back as far as 1770, when Wolfgang von Kempelen unveiled perhaps the biggest prank in the 18th century, a fraudulent AI chess-player known as the “Mechanical Turk”.
In 1809, the Turk robotically saluted Napoleon I of France before defeating him in a friendly game of chess. Napoleon, however, was unaware that hiding inside the automaton’s cockpit of prop gears and cogs, was the chess champion Johann Baptist Allgaier, who was able to checkmate the Emperor’s King in a mere 19 moves. He would be one of many chess pros who would puppeteer the robotic ruse. The truth of The Turk would not be known until 1854 when it was destroyed in a museum fire, 84-years after the hoax began.
Due to its history, calling something a “Mechanical Turk” has been used as a term to insinuate unbelievable technology that is covertly human-controlled. In perhaps a poetic contrast, I’m willing to wager that there is a skilled online gamer who is currently being accused of secretly being an AI bot.
If only Wolfgang could see us now.
At its very nature, the concept of game machines that appear to learn and adapt presented an alluring competitive challenge.
Throughout the years, developers would learn to hone and expand bot behaviors to mimic human players. With the evolution of algorithm methods like Procedural Fractals (1975) and Perlin Noise (1983), developers have enabled the creation of digital worlds in refreshingly innovative ways and sometimes randomized ways.
For perspective, the space trading video game Elite (1984) contained 256 procedurally generated planets, while No Man’s Sky (2016), contains 18 quintillion planets, each with its own unique flora, fauna, and biosphere. Exploration of its entire universe will not be possible until 580 billion years after the death of Earth’s own Sun.
Image created with Midjourney Pro
The artistry behind AI in game development
In a 2015 Polygon Article, No Man’s Sky developer, Hello Games, discussed their approach to procedural content with their process of creating seeds composed of artist-directed DNA. Seeds are stylistic boundaries that defined the essential guidelines of plants, animals, and environments were established and then placed into a package the Art Director called “a big box of maths.” That box was then attached onto AI drones programmed to plant them in various locations and send back a report of what was created, expanding the universe with each Digital Galaxy they grew. An apt metaphor for art imitating life.
While manual manipulation of entire worlds is possible, it’s a timely endeavor to make sure every tree collides with the level’s geometry in a realistic way. Fortunately, with the continued advancements in physics-based shaders and real-time simulation, artists can push the boundaries of immersive visual effects, designing environments that feel alive with texture, depth, and variety. The results of that level of detail are worth it, providing a cohesive harmony of style and purpose. Essential ingredients for creating memorable gaming experiences.
AI in art, when done responsibly, should allow the artists to focus more on their creative inspiration, using co-pilots or assistants like these to simplify the growing complexity and demand of game content. Being creative within a budget while delivering on a timeline is a moving target, the best thing we can do to support our artists, is to understand what workflows are the most desired for augmentation. Artists don’t want the art made for them, they want the autonomy to make the art they imagine, and our goal should be to make that process painless.
The impact of AI on game replayability
As gamers, we determine our enjoyment based on key factors like quality, immersion, and replayability.
We crave deep and unique experiences in games. If the game we’re playing is going to do something that has been done before, replayability will exist if it’s done well. If the game relies on randomized dungeons, loot, and depth, a game needs systems of content creation and distribution that can expand and adapt, adding pavement to unexplored pathways.
It’s also the driving-force behind the complex behaviors we see of non-player characters (NPCs) that make a game’s setting feel alive. It’s an inner dialogue of predetermined actions that provide these bots with their behavior tree of options they can pursue telling them how to react to the player’s choices. When a computer-controlled bot moves around the level, it does so at the behest of predetermined pathing. Over the years, AI Bots have been programmed to interpret their behavior within a wider range of guidelines, making their reactions seem more natural and grounded.
When playing any game, enemies and opponents alike are programmed to respond to a player’s choices, accordingly, providing the player with a challenge they can use to improve or learn from. The higher the difficulty, the more advanced the tactics a bot is allowed to perform. If you want to “get good” at a multiplayer game, you might play a local game with AI bots for enemies to practice or hone your skills. Some bots are also programmed with dynamic difficulty adjustments that enable them to react to a certain playstyle.
Replayability is more likely, when your game has dynamic experiences, giving the player the ability to explore different strategies and options. Having mechanisms in place that can expand or randomize worlds or give the game a sense of unpredictability is exciting. When done with care and direction, these types of AI tools in games add substance to the entire play experience.
The future of AI in gaming
In its very essence, generative forms of AI, much like the Mechanical Turk, are not possible without human guidance. Many diffusion models that produce artificially generated content are trained off data created by artists, writers, and engineers. Some of this data collection has been unethically utilized without knowledge or permission. Successful AI content, especially in games, is directed by artists and engineers to execute content in their desired style. There is still, at its core, that factor of human guidance grounded in purpose and consent. They know what they’re building, they know how they want it to look, and they use it as a tool to do exactly that.
When it comes to the game development pipeline, there are so many complexities and bottlenecks that are great candidates for accelerated tools. Opportunities to provide solid assistance for developers in their process. When built with a specific use or purpose, we can create AI tools that truly help creators remove the tedious parts of their work and provide the runway to stay creative and energized.
Worldbuilding for games is becoming more complex the larger the setting or the more nuanced the details. If you want environments to feel cohesive and vast or want digital experiences that feel larger than life, it will take time to build. The ability to optimize the effects in your worldbuilding with simulation or procedural creation will continue to be beneficial to creators and gamers alike.
The future of AI in games must redefine itself, to assist creators in achieving the aspirations they have for the stories they want to tell. It means building for workflows that are malleable and provide the autonomy to customize and train our data around the ways that artist specifically require. Successful and ethical AI in any field is achievable if we learn to stop and listen to what creators want in their day-to-day work. It will inspire important conversations that lead to content created with consent, so we can arrive at a future together where we’re able to stop working as hard to survive doing the work we love to do.
I’m confident we will arrive at that point as we have time and time again in games. If you have doubts, I encourage you to consider the details the next time you’re playing an open-world game – take a moment and admire the way the colors shift on a skybox, or the way the light shimmers as it reflects harmoniously on a body of water; or the subtle sway of the grass as it reacts to simulated wind. Take a moment to appreciate the effort of this beautiful dance that combines art, science, and of course, a “big box of maths”.