A16Z: Opportunities for Generative AI in Gaming

链捕手ChainCatcher · 2022-11-30T09:20:13.000Z

Author: James Gwertzman&Jack Soslow Compiled by: Alpha Rabbit A16Z recently wrote a very interesting article, talking about what they think are the opportunities for combining generative AI and games. The author translated and annotated part of the content. The article has two main parts: the first part includes A16Z’s observations and predictions on generative AI in the game field; the second part includes A16Z’s judgment on the market ecology in the game + generative AI field. Preface What is the connection between the field of gaming and generative artificial intelligence? There is an impossible triangle in the field of game design: usually only two of the three items, cost, quality or speed, can be chosen. But now, designers can use these AIGC tools to eliminate the need to spend a lot of manual production time and only need a few hours. What's truly transformative about creating high-quality images is that anyone can gain this creative ability by learning a few simple tools. These tools can create endless variations in a rapid iterative manner, and once trained, the entire process is real-time, which means that the results are almost immediately available. Since the emergence of Real-time 3D technology, there has not been a technology that has the potential to change games so much (with real-time 3D software, the entire virtual world can be digitally rendered at a faster speed, providing users with more attractive power and immersive experience) So, where is generative AI headed? How will it change the game? First, let’s review the concept of generative AI. What is generative AI? Generative AI is a category of machine learning where computers can generate original new content based on user input/prompts. At present, the most mature applications of this technology are mainly in the fields of text and images, but there are similar advances (technical applications of generative AI) in almost all creative fields, covering animation, sound effects, music, and even for people with complete personalities. Virtual characters are created. Of course, artificial intelligence is nothing new in gaming. Even early games like Atari's Pong already had computer-controlled opponents fighting against the player. (Author’s note: Atari, a game developer, was founded shortly after the birth of the microprocessor. It launched the first arcade machine Pong in 1972, establishing its status as the originator of arcade machines.In 1974, Apple's Steve Jobs joined Atari and was responsible for developing video games) However, the virtual opponents in these computers are not the same as the generative artificial intelligence we talk about today. These computer opponents are just scripts carefully designed by game designers. An AI opponent is indeed simulated, but they cannot learn and iterate at the same level as the engineers who wrote them. So, what are the changes in the underlying technology of the combination of generative AI and games? Microprocessors are faster, cloud computing and various computing capabilities are stronger, and have the potential to build large-scale neural networks, which can identify patterns and representations in highly complex fields. (Thanks to faster microprocessors and the cloud. With this power, it's possible to build large neural networks that can identify patterns and representations in highly complex domains. Author's note: What this means is that the capabilities of a single microprocessor are multiplied by faster and faster microprocessors. The scale factor of cloud computing enables the establishment to support complex pattern recognition. What is pattern recognition? Pattern recognition refers to the processing of various forms of information (numerical, textual and logical relationships) that represent things or phenomena. and analysis, the process of describing, identifying, classifying and explaining things or phenomena, is an important part of information science and artificial intelligence) Part 1: Some assumptions and industry observations

By James Gwertzman & Jack Soslow
Compiled by: Alpha Rabbit
 
A16Z recently wrote a very interesting article about what they think are the opportunities for combining generative AI and games. I translated it and annotated some of the content. The article is mainly divided into two parts: the first part includes A16Z's observations and predictions on generative AI in the game field; the second part includes A16Z's judgment on the market ecology of the game + generative AI field.
 
Preface
What is the connection between the field of games and generative AI? There is an impossible triangle in the field of game design: cost, quality or speed are usually only two of the three, and now, designers can use these AIGC tools to create high-quality images in just a few hours without spending a lot of manual production time. What is truly revolutionary is that anyone can gain this creative ability by learning a few simple tools. These tools can create endless variations in a fast and iterative way, and once trained, the whole process is real-time, that is, the results are almost instantly available. Since the advent of Real-time 3D technology, there has been no technology that has the potential to change games so much (with real-time 3D software, entire virtual worlds can be digitally rendered at a faster speed, providing users with a more engaging and immersive experience) So what is the development direction of generative AI? How will it change games? First, let's review the concept of generative AI. What is generative AI? Generative AI is a category of machine learning in which computers can generate original new content based on user input/prompts. Currently, the most mature applications of this technology are mainly in the fields of text and images, but there are similar advances in almost all creative fields (technical applications of generative AI), covering animation, sound effects, music, and even original creation of virtual characters with complete personalities. Of course, artificial intelligence is not new in games. Even early games, such as Atari's "Pong", have long had computer-controlled opponents and players fighting. (Author's note: Game developer Atari was founded shortly after the birth of the microprocessor. In 1972, it launched the first arcade machine Pong, establishing its position as the originator of arcade machines. In 1974, Apple's Steve Jobs joined Atari to develop video games) However, the virtual opponents in these computers are not the same as the generative artificial intelligence we are talking about today. These computer opponents are just scripts carefully designed by game designers. They do simulate an artificial intelligence opponent, but they cannot learn and iterate at the same level as the engineers who wrote them.So, what changes have occurred in the underlying technology when generative AI and games are combined? Faster microprocessors, stronger cloud computing and various computing capabilities have the potential to build large neural networks that can identify patterns and representations in highly complex domains. (Thanks to faster microprocessors and the cloud. With this power, it’s possible to build large neural networks that can identify patterns and representations in highly complex domains.  Author’s note: What this means is that the faster and faster single-unit capabilities of microprocessors multiplied by the scale factor of cloud computing enable the establishment of networks that can support complex pattern recognition. What is pattern recognition? Pattern recognition refers to the process of processing and analyzing various forms of information (numerical, textual, and logical) that represent things or phenomena in order to describe, identify, classify, and explain things or phenomena. It is an important part of information science and artificial intelligence) Part I: Some assumptions and industry observations
 
Some assumptions: First, let's discuss some assumptions on which the rest of the article is based: 1. The amount of research (success) in general artificial intelligence will continue to grow, and more and more effective technologies will emerge. The above figure shows the number of academic papers published in arXiv each month on machine learning or artificial intelligence. As shown in the figure, the number of papers is growing exponentially and there is no sign of slowing down. This part of the data only includes published papers. There are also many studies that are not publicly published, but directly applied to open source models or product development. These open source models and products have brought explosive innovation. 2. Among all entertainment categories, games will be the field that will be most affected by generative artificial intelligence. In terms of the types of assets currently involved (2D art, 3D art, sound effects, music, etc.), games are the most complex of the entertainment categories. At the same time, games are also the most interactive, and they place great emphasis on real-time experience. This creates a very high barrier to entry for new game developers, and a high cost to make a true AAA game. These barriers and cost issues create huge opportunities for disruptive innovation in the field of generative AI in the field of games (as shown below): For example, a game like Red Dead Redemption 2 is one of the most expensive games ever made, costing nearly $500 million to make. Red Dead Redemption is also one of the most visually stunning games on the market, taking nearly 8 years to make, with more than 1,000 game characters (each with their own personality and exclusive voice actors), a nearly 30 square mile game world, more than 100 missions in 6 chapters, and nearly 60 hours of music composed by more than 100 musicians. All the content involved in this game is very large. So, if we compare Red Dead Redemption 2 to Microsoft Flight Simulator, Microsoft Flight Simulator is even bigger... because players of Microsoft Flight Simulator can fly around the entire earth in the game, all 197 million square miles.So how did Microsoft build such a large-scale game? It was mainly done through artificial intelligence. Microsoft worked with blackshark.ai to train artificial intelligence to generate infinite realistic three-dimensional worlds from two-dimensional satellite images. What is blackshark.ai? Blackshark.ai is a company that extracts global earth infrastructure through machine learning technology, extracts data from global satellite and aerial images, and uses artificial intelligence to create digital twin scenes based on current geographic data. These results can be used for visualization, simulation, mapping, mixed reality environments and other enterprise solutions, and the cloud computing update capabilities of the technology itself can update this data in real time. This is just one example. Without the use of artificial intelligence technology, the game "Microsoft Flight Simulator" would actually be impossible to make. In addition, the success of the game is also due to the fact that these models can be continuously improved over time. For example, the "highway cloverleaf overpass" model can be strengthened, and all highway overpasses on the entire earth in the game can be improved immediately by running the entire construction process through artificial intelligence. 3. Every asset involved in game production will have a generative AI model. So far, 2D image generators like Stable Diffusion or MidJourney have occupied most of the current excitement about generative AI because of the eye-catching images they can generate. Now there are generative AI models for almost all assets in games, from 3D models to character animations, to dialogue and music. (The next article will talk about the market map of specific companies) 4. The cost of content will continue to decline, and in some cases the cost of content will drop to zero. When we talk to game developers who try to integrate generative AI into production scenes, the biggest excitement is that the time and cost of making games will be greatly reduced. One developer told us that the time to generate a concept map for an image dropped from 3 weeks to 1 hour. We believe that similar "cost reduction and efficiency improvement" can be achieved in the production process of the entire game process.It is worth noting that artists are not in danger of being replaced, which means that artists no longer need to do all the work themselves: artists and designers can set the initial creative direction, and then hand over most of the time-consuming and technical execution work to artificial intelligence. At this point, just like the painters of early hand-drawn animation, highly skilled "painting experts" outline the animation, and then relatively low-cost painters complete the time-consuming work of coloring the animated film and filling in the lines, but we are talking about applications in the field of game creation. 5. We are still in the early days of this industry transformation, and there are still many parts to be perfected. Although many people have been excited recently, we are still just at the starting line. There is still a lot of work to be done when everyone is clear about how to truly apply this new technology to the field of gaming, and there will be huge opportunities for companies that have entered this new field before and quickly. Part II: Predictions for the future Given the above assumptions, this article predicts and deduces how the gaming industry will be transformed.
1. Learning how to effectively apply generative AI will become a marketable skill.
There are already pioneers who can apply generative AI more effectively than anyone else. To make the best use of this new technology, you need to understand the various tools and techniques and know how to combine them. We predict that effectively applying generative AI will become a very promising skill in itself because it combines the creative vision of artists with the technical ability of programmers. Chris Anderson famously said, "Every abundance creates a new scarcity." As content becomes more abundant, artists who understand how to work most effectively with AI tools will be the most scarce. For example, using generative AI for the generation of artworks also brings some challenges, including:
Maintaining consistency: We need to be able to modify or edit various assets in the game, and for AI tools, this means being able to replicate (digital) assets with the same signal so that we can modify and challenge it. This can be tricky because the same prompt can produce completely different results.
Maintaining a consistent style: All artwork within a single game needs to maintain a consistent style, which means that AI tools need to be trained or connected to the artist/designer’s established style.
2. The lowering of the barrier to entry for game development will lead to more risk-taking and creative exploration We may soon enter a new "golden age" of game development, where lower barriers to entry will lead to more innovative and creative games, not just because lower production costs lead to lower risks for game makers, but also because these tools represent the ability to create high-quality content for a wider audience. 3. The rise of "micro-game studios" assisted by AI With the tools and services of generative AI, perhaps more viable commercial games will be made by small "micro-studios" with only 1 or 2 employees. Of course, small independent game studios are already common, and the hit game "Among Us" (Author's Note: Among Us is a strategy casual game produced and published by Innersloth, which can be played online by 4-10 people and released on November 17, 2018) was made by Innersloth, a studio with only 5 employees, and the scale of games that these small studios can create will grow.
4. The number of games released each year will increase
The success of Unity and Roblox shows that providing powerful creative tools will lead to more games being built. Generative AI will further lower the barrier to entry and create more games. The industry already suffers from a discovery challenge - more than 10,000 games were added to Steam last year alone - which will put even more pressure on discovery. However, we will also see... 5. New game types will be created. There will be new game types invented, like the Microsoft Flight Simulator mentioned above, but completely new game types invented, which will be combined with the real-time generation of new content. For example, Spellbrush's RPG game Arrowmancer features AI-created characters, with almost unlimited new ways to play. Other game developers are using AI to let players create their own avatars in the game: automatically generating avatar images based on the player's description. Note that from the user-side experience, letting players generate content through AI allows players to perceive greater ownership. 6. Value will be attributed to industry-specific AI tools, not just basic models The enthusiasm around basic models such as Stable Diffusion and Midjourney is generating extremely exaggerated valuations, but as new research continues to emerge, new models will emerge and continue to iterate as new technologies are refined. Judging from the website search traffic for the three popular generative AI models (Dall-E, Midjourney, and Stable Diffusion), each new model has a specific focus around it. Another approach is to build industry-specific (vertical industry) tool suites that focus on the generative AI needs of specific industries, deeply understand specific audiences, and integrate with existing production scenarios (Unity or Unreal). A typical example is Runway, which provides AI-assisted tools such as video editing, green screen removal, in-painting, and motion tracking for the needs of video creators, and such tools can add new application scenarios over time. We haven't seen gaming tools like Runway yet, but this is an area with potential. 7. Upcoming Legal Challenges What all of these generative AI models have in common is that they are trained using large datasets of content, typically created from datasets from the internet.For example, “Stable Diffusion is trained on over 5 billion images/captions, which were scraped from the web. Currently, these models claim to operate under the copyright doctrine of “fair use”, but this argument has not been clearly tested in law. Clearly, upcoming legal challenges could change the landscape of generative AI. It is possible that large film companies will seek competitive advantage by building proprietary models through the strength of their copyrights. For example, Microsoft has many studios under its umbrella, especially with its acquisition of Activision Blizzard. 8. Unlike the arts, will generative AI bring about huge changes in the programming field, at least for now. Software engineering is another major source of cost in game development, but generating code with AI models requires more testing and verification, so code generation is less productive than generating creative assets. We believe that coding tools like Copilot may provide engineers with modest performance improvements, but will not change the content field as much in the short term. Part III: Some suggestions 1. Start exploring generative AI: It will take some time to figure out how to fully leverage the power of this upcoming generative AI revolution. Companies that start developing their businesses early will have an advantage in the future, and several studios are conducting internal experimental projects to explore how these technologies can affect game production. 2. Look for opportunities in gaps in the market. Many parts of the entire track are already very crowded, such as animation, voice, and dialogue, but there are still many areas that are widely open. We encourage entrepreneurs interested in this field to focus on still undeveloped areas, such as the "game + generative AI track."