Machine Learning in Unity — Part 1 of 3
Normally, I would start with a dev blog followed by a presentation on the same topic. Due to time constraints (yesterday was our final day of the ACI/GameDev HQ Game Development Internship 2020), I have reversed the order and am following up with a dev blog of my presentation to the projects team at GameDev HQ. There was a lot of material I wanted to share, and the live feedback was helpful to expose some gaps in explanations and change the order of certain topics as I translate that experience to this three part series.
Today, in Part 1, I would like to present a bit of history of the use of Artificial Intelligence (AI) and games, and the crucial role it has played in the advancement in certain areas in AI in general.
In Part 2, some terminology and concepts that will help to understand machine learning and how it relates to AI and deep learning in conjunction with neural networks and the use of deep learning neural network brains in Unity.
Finally, Part 3 will cover the mechanics of Unity ml-agents and the training a neural network to be integrated into Unity projects and some use case examples to get started concluding with the implications of AI technology and provide some useful resources to get started or continue learning about ml-agents.
First of all, I want to bring some awareness about what has been emerging in technology, in the past few years. At this point in time, we see convergence and combining of several technologies to create smarter applications to solve more complex requirements: XR + IoT + AI (CV, etc.) + Cyber
An example in general terms would be an XR application that connects to and controls physical/mechanical objects in the real world at a distant location via IoT device, based on the results from complex AI algorithms, secured by cyber controls — is just the tip of the iceberg.
A specific example would be a user in an XR app controlling a drone, scanning every city street could be completed in the span of 15 minutes time. The video or snapshots captured, stored, and analyzed real-time by computer vision algorithms detects depth and widths of cracks and potholes in the surfaces of streets and sidewalks, and makes a decision about which ones need to be repaired and on what schedule to avoid costly damage due to extent of deterioration over the next few months. Revealing these seemingly innocent vulnerabilities to terrorist cells could create a opportunities to exploit infrastructure, so collective information gathered and analyzed for this project would require safeguarding, thus, including the security element is crucial in almost any technology area today. A high demand for these technologies in conjunction is Industrial Controls System (ICS) and Human Performance.
The above is just a practical example, but there are seemingly no limits. Check out Elon Musk’s recent achievement with Neuralink, a brain interface.
The video, above, focuses on providing solutions for a multitude of physical disabilities, and other sources cite mental health and enhanced learning, as well. It is by no means a stretch to imagine AI driving this device in the near future.
AI in conjunction with other rapidly emerging technologies is closing the gap between sci-fi and reality.
Artificial Intelligence is on the cusp of infiltrating almost every data driven process in our daily lives. Around the globe, the race is on to lead this effort.
- China set out to be the leader in AI and was declared the AI Superpower (Forbes earlier this year)
- CV (Computer Vision) — London — Cameras at intersections for facial recognition of terrorists and terrorist behavior
As game developers, XR (immersive) projects are on the rise and AI is fast becoming an integral part of what drives and controls these experiences. Only, this growth is not limited to games, and has expanded its reach into technologies in the industrial enterprise for non-game applications in every area imaginable.
Due to the great strides made in the last several years that have led to generalizing AI and proven invaluable. Although games have played a major role it is not just about games. It is important to understand a little bit of history to understand how we have arrived at this point — great deal thanks to research done on AI for games.
The following notable events in the evolution of generalizing AI stemming from efforts designed to learn to play games to win against humans and non-human players.
In 1997, IBM’s Big Blue super computer was designed to play and proved a computer could win against the chess grandmaster champion Garry Kasparov.
By 2014, Dennis Hassabis of DeepMind applied machine learning to beat the Atari game, Breakout. The AI applied to the player started out clumsier than a human beginner and in two hours learned how to play like an expert. In about 4 hours, the player figured out the most efficient way to win was to create a tunnel on either side of the game space as the most optimal way to win the game.
Google purchased DeepMind for over $500M after seeing the demonstration and extrapolating its potential.
Google’s DeepMind then took on the challenge of applying this technology to the game of Go, more complicated than chess and originally thought to be near impossible to beat by a computer in a hundred years. Witness for yourself the unexpected move at a critical moment by the computer player against the human champion Lee Sedol, 4 to 5, and the even more surprising human reaction to these events about 3/4ths the way through the film in the 2016 showdown.
Shortly after, AlphaZero then defeated AlphaGo 100 to 0. AlphaZero was also able to take in the rules of any two player game and defeat it. Provided with no rules and an ability to learn from experience alone, this proved to be a more generalized algorithm. Whereas, AlphaGo relied rules and a great deal on a massive database of human experiences on a single objective.
OpenAI was also making strides in this arena, and conquered DOTA 2 and Sonic the Hedgehog 1, 2, and 3 with AI.
In 2019, Google’s DeepMind went on to apply AI to Blizzard’s StarCraft II. The pro level player Grzegorz “MaNa” Komincz was defeated by AI, 5 to 0, in spite of his ability to make over 800 strategic moves per minute that would normally defeat other human players. This is a real-time situation, with millions of possibilities and requires current strategy and planning into the future.
The effort to create each of these types of AI models is monumental, and now the goal was to make this a more generic one, to apply to any game. Announced just last week, Google’s DeeMind MuZero accomplished this. Without being informed of the rules, MuZero defeated chess, go, shogi (Japanese chess), and the Atari 57.
Aside: The above video was released by one of the most addictive (IMHO) YouTube channels on these topics: Two Minute Papers.
Here is a summary of this evolution to generalization starting from requiring massive experience data from multiple knowledge areas on a single game, to no required knowledge areas on a multitude of games:
Here’s the punchline: These advances ultimately culminated to AlphaFold 2 for folding protein structures. Although a biological breakthrough -
“This breakthrough demonstrates the impact AI can have on scientific discovery and its potential to dramatically accelerate progress in some of the most fundamental fields that explain and shape our world.”
Building algorithms and deep neural network systems to beat human and artificial players in the challenging games of our time has led to unlocking the mysteries of biological and artificial life forms. A game starts out as a problem, and the objective is a challenge to solve it (achieving a win state), so it is no surprise that the most important breakthroughs in AI have come about in the quest to conquer games has been at the heart of recent and accelerated advances in a matter of a few years.
For further details on the two leaders on this topic, check out the research on projects by Google’s DeepMind and OpenAI.
Worth a mention is the impact deep learning is having on the visual and audible arts with Generative Adversarial Networks (GANs), originally proposed by Ian Goodfellow in 2014, since they play an integral part in creating impactful games .
- GAN Gough Creates Art — Bizarre but true creations, by two undergrads
- Improvements with StyleGAN
- 18 Impressive Applications of GAN
- Human Faces Generated by GANs (refresh browser to generate a new face)
- AI Artists — Creative Tools to Generate AI Art — Not so bizarre, after all!
- AI Generated Music on MuseNet
Videos that inspired me to further investigate applying deep reinforcement learning to game play agents:
In my next blog, I will introduce a high level introduction to the AI concepts that make this all possible, that will prepare to understand the final blog in this series describing how we will be able to use machine learning in Unity.
If you made it this far, thank you for joining me on this adventure, and see you in the next blog.
Happy New Year’s Eve 2021!
