09-06-2018, 02:21 AM
Blog: Why academics can no longer compete in game AI competitions
<div style="margin: 5px 5% 10px 5%;"><img src="http://www.sickgaming.net/blog/wp-content/uploads/2018/09/blog-why-academics-can-no-longer-compete-in-game-ai-competitions.png" width="640" height="480" title="" alt="" /></div><div><p><strong><i><small> The following blog post, unless otherwise noted, was written by a member of Gamasutras community.<br />The thoughts and opinions expressed are those of the writer and not Gamasutra or its parent company. </small></i></strong> </p>
<hr />
<p><img src="http://www.sickgaming.net/blog/wp-content/uploads/2018/09/blog-why-academics-can-no-longer-compete-in-game-ai-competitions.png" /><br /> <br />
UC Berkeley’s Overmind winning the first StarCraft AI Competition</p>
<p><a href="https://openai.com/five/" rel="noopener" target="_blank">OpenAI Five</a> is a huge step forward for AI, but it’s also really intimidating for AI researchers. Never before has there been so many open tools for building AI systems, but it also feels like the barrier to entry for academics has actually increased over recent years. I posted an open <a href="https://games.slashdot.org/story/09/11/12/1729217/starcraft-ai-competition-announced" rel="noopener" target="_blank">call</a> for any interested parties to build the best StarCraft AI possible back in 2009, and it was open to anyone interested in AI. Now, it seems like you need to have access to closed APIs, massive compute power, and historic training data to make advances in AI.</p>
<p id="737d">I’m putting this argument out there as a devil’s advocate, hoping that I can be proven wrong. I’d like to see more open competitions, and area’s where researchers without massive compute resources can continue to make advances in AI. Here’s some of the main issues I’ve seen with recent advances.</p>
<h4 id="8b28"><strong>Closed APIs</strong></h4>
<p id="8b51">OpenAI, to my knowledge, is using an API to build and train bots that is not available to academic researchers. If you’re a grad student that wants to build a bot for Dota2, then you’ll need to wait until the current competition ends and an open source version is eventually made available. For games likes Go, which have seen great progress with deep and reinforcement learning, this issue is not a problem. But if you’re a grad student that wants to work with video games that have large and active player bases your options are extremely limited. The <a href="https://deepmind.com/blog/deepmind-and-blizzard-open-starcraft-ii-ai-research-environment/" rel="noopener" target="_blank">DeepMind API</a> for StarCraft 2 seems like a great option, but you’ll still have other challenges to face.</p>
<p id="82e5">Dota 2 does provide a <a href="https://developer.valvesoftware.com/wiki/Dota_Bot_Scripting" rel="noopener" target="_blank">scripting interface</a>, enabling bots to be written in Lua. However, this limited interface does not enable bots to communicate with remote processes and save data about games played.</p>
<p id="1f78"><strong>Work Around:</strong> Find games with APIs that you can use. I used the <a href="https://github.com/bwapi/bwapi" rel="noopener" target="_blank">Brood War API</a> during grad school to write a bot for the original StarCraft. It was a huge hack and I somehow managed to get it working with Java, but I was extremely lucky that this project was not shut down by Blizzard. It’s great seeing the open source community continue to evolve both BWAPI and the <a href="https://github.com/JNIBWAPI/JNIBWAPI" rel="noopener" target="_blank">Java version</a>.</p>
<p><img src="http://www.sickgaming.net/blog/wp-content/uploads/2018/09/blog-why-academics-can-no-longer-compete-in-game-ai-competitions-1.png" /><br /> <br />
Bot debugging from an early BWAPI-Java prototype.</p>
<h4 id="6ef2"><strong>Compute is Expensive</strong></h4>
<p id="03b6">OpenAI is spending massive amounts of compute power to train bots, and efforts by DeepMind to train Go bots were also substantial. This is an issue that OpenAI had addressed in a <a href="https://blog.openai.com/ai-and-compute/" rel="noopener" target="_blank">blog post</a>, but it’s also discouraging for academics. It’s unlikely that students will have massive cloud computing resources available for training bots that perform at professional levels.</p>
<p id="756e"><strong>Work Around:</strong> Find sub-problems that you can solve. This is actually something that OpenAI did previously, by focusing on 1on1 matches and then slowly progressing up to 5v5 by relaxing more and more constraints. I included a tech-restricted matchup in the original StarCraft AI competition, but it wasn’t as popular as the full game option, in part because there weren’t human opponents knowledgeable about this version of the game for participants to train against.</p>
<p><img src="http://www.sickgaming.net/blog/wp-content/uploads/2018/09/blog-why-academics-can-no-longer-compete-in-game-ai-competitions-2.png" /><br /> <br />
One of the sub-problems in the original StarCraft AI Competition.</p>
<h4 id="30aa"><strong>Training Data</strong></h4>
<p id="c805">One of the goals with my dissertation research was to build a StarCraft bot that learns from professional players, but now that some time has passed I realize that a different approach may have been move effective. Instead of showing my bot how to perform, I could have specified a reward function for how the bot should act and then let it self train. But, I didn’t have a good simulation environment, and had to rely on the sparse training data that I could scrape from the web at the time. This is especially a problem when you are working on sub-problems, such as 1on1 in Dota2, where professional replays simply don’t exist.</p>
<p id="a505"><strong>Work around:</strong> Mine data sets from the web, such as professional replays from sites like TeamLiquid. This won’t work for sub problems, but you can always bootstrap the replay collection with your own gameplay to get things rolling. I used third party tools to parse replays and then built an export tool with BWAPI later on.</p>
<p><img src="http://www.sickgaming.net/blog/wp-content/uploads/2018/09/blog-why-academics-can-no-longer-compete-in-game-ai-competitions-3.png" /><br /> <br />
Exhibition match at the first StarCraft AI Competition.</p>
<h4 id="e914"><strong>Next Steps</strong></h4>
<p id="6554">I’ve provided a pessimistic viewpoint for academics and hobbyists in game AI. It may seem like you need connections and massive compute power to make advances in game AI. It helps, but it doesn’t mean you’ll make progress without some great direction. Here’s some recommendations I have for making AI competitions more open moving forward:</p>
<ul>
<li id="b5b9">Open up APIs for bots to play against human opponents</li>
<p> </p>
<li id="4e53">Provide a corpus of replay data</li>
<p> </p>
<li id="20f9">Integrate with open source and cloud tooling</li>
<p>
</ul>
<p id="08ca">It’s now easy to get up and running with deep learning environments thanks to tools such as Keras, and environments like <a href="https://colab.research.google.com/notebooks/welcome.ipynb#recent=true" rel="noopener" target="_blank">Colaboratory</a> that make it easier to work with other researchers on models.</p>
<p id="1dea">In a way, my initial call for research on open AI problems failed. It was an open problem, but I didn’t fully encourage participants to be open in their approach to the problem. Now, great tools exist for collaboration and it’s a great time to openly work on AI problems.</p>
<section>
<hr />
<p id="b1cd"><a href="https://www.linkedin.com/in/ben-weber-3b87482/" rel="noopener" target="_blank">Ben Weber</a> is a principal data scientist at Zynga. We are <a href="https://www.zynga.com/careers/positions/categories/data-analytics-user-research" rel="noopener" target="_blank">hiring</a>! This post is an opinion based on my experience in academia and does not represent Zynga.</p>
<p> <br />
</section>
</div>
<div style="margin: 5px 5% 10px 5%;"><img src="http://www.sickgaming.net/blog/wp-content/uploads/2018/09/blog-why-academics-can-no-longer-compete-in-game-ai-competitions.png" width="640" height="480" title="" alt="" /></div><div><p><strong><i><small> The following blog post, unless otherwise noted, was written by a member of Gamasutras community.<br />The thoughts and opinions expressed are those of the writer and not Gamasutra or its parent company. </small></i></strong> </p>
<hr />
<p><img src="http://www.sickgaming.net/blog/wp-content/uploads/2018/09/blog-why-academics-can-no-longer-compete-in-game-ai-competitions.png" /><br /> <br />
UC Berkeley’s Overmind winning the first StarCraft AI Competition</p>
<p><a href="https://openai.com/five/" rel="noopener" target="_blank">OpenAI Five</a> is a huge step forward for AI, but it’s also really intimidating for AI researchers. Never before has there been so many open tools for building AI systems, but it also feels like the barrier to entry for academics has actually increased over recent years. I posted an open <a href="https://games.slashdot.org/story/09/11/12/1729217/starcraft-ai-competition-announced" rel="noopener" target="_blank">call</a> for any interested parties to build the best StarCraft AI possible back in 2009, and it was open to anyone interested in AI. Now, it seems like you need to have access to closed APIs, massive compute power, and historic training data to make advances in AI.</p>
<p id="737d">I’m putting this argument out there as a devil’s advocate, hoping that I can be proven wrong. I’d like to see more open competitions, and area’s where researchers without massive compute resources can continue to make advances in AI. Here’s some of the main issues I’ve seen with recent advances.</p>
<h4 id="8b28"><strong>Closed APIs</strong></h4>
<p id="8b51">OpenAI, to my knowledge, is using an API to build and train bots that is not available to academic researchers. If you’re a grad student that wants to build a bot for Dota2, then you’ll need to wait until the current competition ends and an open source version is eventually made available. For games likes Go, which have seen great progress with deep and reinforcement learning, this issue is not a problem. But if you’re a grad student that wants to work with video games that have large and active player bases your options are extremely limited. The <a href="https://deepmind.com/blog/deepmind-and-blizzard-open-starcraft-ii-ai-research-environment/" rel="noopener" target="_blank">DeepMind API</a> for StarCraft 2 seems like a great option, but you’ll still have other challenges to face.</p>
<p id="82e5">Dota 2 does provide a <a href="https://developer.valvesoftware.com/wiki/Dota_Bot_Scripting" rel="noopener" target="_blank">scripting interface</a>, enabling bots to be written in Lua. However, this limited interface does not enable bots to communicate with remote processes and save data about games played.</p>
<p id="1f78"><strong>Work Around:</strong> Find games with APIs that you can use. I used the <a href="https://github.com/bwapi/bwapi" rel="noopener" target="_blank">Brood War API</a> during grad school to write a bot for the original StarCraft. It was a huge hack and I somehow managed to get it working with Java, but I was extremely lucky that this project was not shut down by Blizzard. It’s great seeing the open source community continue to evolve both BWAPI and the <a href="https://github.com/JNIBWAPI/JNIBWAPI" rel="noopener" target="_blank">Java version</a>.</p>
<p><img src="http://www.sickgaming.net/blog/wp-content/uploads/2018/09/blog-why-academics-can-no-longer-compete-in-game-ai-competitions-1.png" /><br /> <br />
Bot debugging from an early BWAPI-Java prototype.</p>
<h4 id="6ef2"><strong>Compute is Expensive</strong></h4>
<p id="03b6">OpenAI is spending massive amounts of compute power to train bots, and efforts by DeepMind to train Go bots were also substantial. This is an issue that OpenAI had addressed in a <a href="https://blog.openai.com/ai-and-compute/" rel="noopener" target="_blank">blog post</a>, but it’s also discouraging for academics. It’s unlikely that students will have massive cloud computing resources available for training bots that perform at professional levels.</p>
<p id="756e"><strong>Work Around:</strong> Find sub-problems that you can solve. This is actually something that OpenAI did previously, by focusing on 1on1 matches and then slowly progressing up to 5v5 by relaxing more and more constraints. I included a tech-restricted matchup in the original StarCraft AI competition, but it wasn’t as popular as the full game option, in part because there weren’t human opponents knowledgeable about this version of the game for participants to train against.</p>
<p><img src="http://www.sickgaming.net/blog/wp-content/uploads/2018/09/blog-why-academics-can-no-longer-compete-in-game-ai-competitions-2.png" /><br /> <br />
One of the sub-problems in the original StarCraft AI Competition.</p>
<h4 id="30aa"><strong>Training Data</strong></h4>
<p id="c805">One of the goals with my dissertation research was to build a StarCraft bot that learns from professional players, but now that some time has passed I realize that a different approach may have been move effective. Instead of showing my bot how to perform, I could have specified a reward function for how the bot should act and then let it self train. But, I didn’t have a good simulation environment, and had to rely on the sparse training data that I could scrape from the web at the time. This is especially a problem when you are working on sub-problems, such as 1on1 in Dota2, where professional replays simply don’t exist.</p>
<p id="a505"><strong>Work around:</strong> Mine data sets from the web, such as professional replays from sites like TeamLiquid. This won’t work for sub problems, but you can always bootstrap the replay collection with your own gameplay to get things rolling. I used third party tools to parse replays and then built an export tool with BWAPI later on.</p>
<p><img src="http://www.sickgaming.net/blog/wp-content/uploads/2018/09/blog-why-academics-can-no-longer-compete-in-game-ai-competitions-3.png" /><br /> <br />
Exhibition match at the first StarCraft AI Competition.</p>
<h4 id="e914"><strong>Next Steps</strong></h4>
<p id="6554">I’ve provided a pessimistic viewpoint for academics and hobbyists in game AI. It may seem like you need connections and massive compute power to make advances in game AI. It helps, but it doesn’t mean you’ll make progress without some great direction. Here’s some recommendations I have for making AI competitions more open moving forward:</p>
<ul>
<li id="b5b9">Open up APIs for bots to play against human opponents</li>
<p> </p>
<li id="4e53">Provide a corpus of replay data</li>
<p> </p>
<li id="20f9">Integrate with open source and cloud tooling</li>
<p>
</ul>
<p id="08ca">It’s now easy to get up and running with deep learning environments thanks to tools such as Keras, and environments like <a href="https://colab.research.google.com/notebooks/welcome.ipynb#recent=true" rel="noopener" target="_blank">Colaboratory</a> that make it easier to work with other researchers on models.</p>
<p id="1dea">In a way, my initial call for research on open AI problems failed. It was an open problem, but I didn’t fully encourage participants to be open in their approach to the problem. Now, great tools exist for collaboration and it’s a great time to openly work on AI problems.</p>
<section>
<hr />
<p id="b1cd"><a href="https://www.linkedin.com/in/ben-weber-3b87482/" rel="noopener" target="_blank">Ben Weber</a> is a principal data scientist at Zynga. We are <a href="https://www.zynga.com/careers/positions/categories/data-analytics-user-research" rel="noopener" target="_blank">hiring</a>! This post is an opinion based on my experience in academia and does not represent Zynga.</p>
<p> <br />
</section>
</div>