Abel

By Isaac Levin & Abel Wang

Let’s face it, as the demands of writing software increase, more pressure is put on devs to be as productive as humanly possible. And with this demand, the landscape for being a developer has never been more challenging.

With increased responsibilities and technology options, developers are asked to not just worry about the code they write, but all other aspects of the application development life cycle.

The plethora of tools at a developer’s disposal has also made developer responsibilities even more challenging. Determining what tools and platforms to develop on and with have caused a “too many options” scenario in lots of cases.

GitHub To The Rescue

As challenging as this landscape is, there are solutions. One thing that’s abundantly clear, GitHub is the place where developers go to learn and collaborate with the community.

GitHub has enabled developers to have the power of the entire Open-Source community at their disposal and the freedom to learn from the work of others to better themselves as a developer. GitHub has unveiled many tools aside from hosting source code to make the ecosystem as comfortable as possible.

One of those tools I am fired up about is “GitHub Actions”. Look, as you all know, I’m a DevOps guy , and I think about DevOps every time I put on my developer hat. I believe that no matter the app, if you are going to send it somewhere that isn’t your local machine, that app deserves to have some continuous integration and continous delivery (CI/CD), simple as that.

The question becomes, if my code is already in GitHub, what is the easiest way to hook CI/CD into my app? The answer is GitHub Actions.

GitHub Actions allow us to easily configure custom workflows, using a bounty of existing community-created workflows to fulfill the needs of our app. It isn’t just starting from scratch as there are workflows to do nearly everything, and your goal is to just build your Action in the way YOU need it. With GitHub Actions, getting your app to the Cloud is super easy.

But What About Developing My App?

Talking about where it goes before I write and compile my app seems non-sensical, but I believe that thinking about these things early will allow us to choose the technology/tools we need to be most productive.

In my opinion, one of the best tools to write the code that powers our software is Visual Studio Code. Visual Studio Code is a cross-platform, multi-language editor that provides extensive extensibility to create the perfect environment to write code.

Whether you are writing express apps in Node.js, multi-thread highly scalable applications in Go, or writing cutting-edge modules on your Raspberry Pi with Python, VS Code gives you the ability to work in the way that benefits you the most.

The best part, VS Code offers deep integrations with GitHub, which allows you to clone/pull/push your repositories as well as manage your work on issues and pull requests, without leaving the editor, HOW COOL IS THAT! Once you have your repo in VS Code, the editor gives you access to an oversupply of Open-Source extensions to do everything from configure your experience, down to your font and background of choice. Nearly anything is possible with VS Code, and with first-class GitHub support, it truly is one of the best options.

What Should Host My App?

With our code being built in VS Code and stored in GitHub and DevOps’d with GitHub Actions, the last thing we need to think about is now what?

Personally, I think Azure is the ultimate Cloud for GitHub. There is no other Cloud that best enables developers to be the most productive, and with integrations to VS Code AND GitHub, you truly have “best of breed” capabilities at your disposal.

Let’s start with VS Code Integrations, and boy there are a ton of them. Existing extensions enable developers to seamlessly connect to their Azure tenants and have full management capability of the resources in their subscriptions, including creation, scaling, configuration, debugging and if we want, even deployment!

Microsoft has built an extension pack called “Azure Tools” which includes extensions to manage every major Azure resource type as well as Azure CLI support AND Docker. There are also other extensions published by Microsoft that connect and manage nearly every Azure Resource. This means when new features come to the platform, they will be coming to the Microsoft published extensions.

Azure Loves GitHub

Finally, it is safe to say that Azure loves GitHub and the integrations are plentiful. From Azure Portal authentication with GitHub Ids, to individual services interfacing with GitHub (one great example of this is Azure Static Web Apps, which allow CI/CD to be configured on creation of the resource).

There is no better integration story between GitHub and Azure than “GitHub Actions for Azure” a set of pre-built GitHub Action workflows that helps you automate your app’s story on Azure, from deployment to monitoring and everything in between.

The team has built over 30 of these workflows and they are documented in a way that you will be able to use them without hesitation. One of my favorite examples is container image scan which allows you to scan the container images you are using for known vulnerabilities as well as linting to ensure you are using best practices.

GitHub Universe 2020

This is the messaging we’re delivering at GitHub Universe 2020. Check out our booth video:

Conclusion

GitHub + Visual Studio Code + Azure ensures you as a developer can just trust the tools and get your work done. There are tons of more features that I didn’t talk about here, so please take a look at some of the resources below to get started enhancing your developer productivity.

More Resources

GitHub Actions for Azure | Create workflows to build, test, package, release and deploy to Azure

GitHub Actions for Azure | Microsoft Docs

Automate your workflow with GitHub Actions | Microsoft Learn

Visual Studio Code Azure Extensions

Create free services with Azure free account | Microsoft Docs

Abel Wang

Principal Cloud Advocate, DevOps Lead

Follow

DONATION MILESTONE REWARDS 

INDIVIDUAL DONATION INCENTIVES 

Below you may find individual donation incentives. All in-game digital incentives will be automatically redeemed via donors’ Bungie.net ID and will appear in the Destiny 2 Collections tab on or after Tuesday, December 1. 

A $100 donation will earn you the three items above and will enter you to win an item from the Bungie Prize Pool. Prizes include GameStop/EB Games gift cards (U.S. and Canada only), Destiny 2: Beyond Light Digital Deluxe Editions, and Destiny 2: Beyond Light Physical Deluxe Editions. 

We are Light Keepers 

Those who dare to ignite hope in the face of uncertainty

By our united, global passion we protect those in need

Making a lasting difference for the Little Lights around the world

We are Light Keepers

Love, 

The Bungie Foundation 

We’re trying something new here on Gamasutra! We’re piloting a new Gamasutra Webcasts series, featuring great speakers who will share insight into the art and business of making games. These are free to attend, live broadcasts with a Q&A (and available on-demand if you miss the original broadcast).

We’re excited to announce that the very first guest in this series is indie marketing superstar Victoria Tran (@TheVTran). Now working as community director at Among Us developer Innersloth, she was previously communications director at Kitfox Games, working on titles including Boyfriend Dungeon, Dwarf Fortress, Pupperazzi, and more. She has a serious knack for building communities and knows how to get people interested in her projects–and she’s going to share a little bit of how she does that!

Tran will be giving the talk “Conveying Your Game’s Value” on Tuesday, December 1 at 2:00pm EST.

What to expect:

• A 30-minute presentation by Victoria Tran on how to convey your game’s value through marketing
• A 15-minute presentation on game publishing and distribution from our sponsor Xsolla
• A 15-minute moderated live Q&A

Here’s what you’ll learn:

• How the value of a game is calculated in a potential player’s mind
• How to create a customer value proposition
• How to use this information in your messaging
• Writing tips to help your messaging shine
• And stick around for an audience Q&A!

Featured Presentation:
Conveying Your Game’s Value

So you have a game and you’ve decided to self-publish, but now the marketing is all up to you. Most importantly, one of the biggest responsibilities is up to you: how do you craft a message that makes people understand and care about your game? It isn’t just as easy as saying “add value.”

What is valuable? How do you find your game’s value? And what are the ways you can actually create that message?  What writing tips should you keep in mind? No one will be interested in your game if you cannot tell them why it is interesting in a way that they actually care about. Let’s learn how to stand out in a noisy market!

Sponsor Presentation:
Navigating the Future of Game Publishing and Distribution

With players spread across multiple game platforms and services, many challenges have emerged for developers. At Xsolla, we believe the platform of the future is an independent, decentralized system that utilizes common, unified basic principles and standards. This system spreads to all current platforms and uses them as sources for gaining new users, building a community, and acting as sales channels.

Key takeaways: Learn how to navigate the future of game publishing and distribution with solutions that easily integrate with your existing business and allow you to grow across all platforms.

Speakers

Featured Speaker: Victoria Tran
Community Director
Innersloth

Victoria Tran is the Community Director at Among Us developer Innersloth. Previously, she worked at Kitfox Games, working on projects like Boyfriend Dungeon, Dwarf Fortress, Pupperazzi, and more. When she is not exploring ways to create kinder game community spaces and how tech and compassion interact, she co-organizes Game & Colour MTL, a group that supports and amplifies game developers of colour in Canada.

Sponsor Guest Presenter: Chris Hewish
President
Xsolla

Chris Hewish is the President of Xsolla and is an award-winning interactive executive and studio head, with experience driving operations and go-to-market strategies across all video game industry segments. He has used his skills to lead multi-team video game studios and global interactive business units with hundreds of employees at companies like Activision, DreamWorks Animation, Survios, Skydance, and Xsolla.

Moderator: Kris Graft
Editor-in-Chief and Publisher
Gamasutra

Kris Graft is editor-in-chief and publisher of Gamasutra, the leading site dedicated to the art and business of making games.

Control, the latest supernatural jaunt from Remedy, takes place in shifting brutalist structure called The Oldest House. It’s an imposing, cinematic, and impossibly beautiful skyscraper filled with secrets, and wandering its ancient halls is probably the second best thing about the otherworldly shooter. The first is how you can blow everything inside those concrete corridors to smithereens.

As anybody who’s played the game will know, the destruction system in Control is a delight. Objects cascade around rooms and splinter into fragments as players blitz through levels with the finesse of a radioactive sledgehammer, painting a canvas of chaos that Jackson Pollock would be proud of.

Explaining how that procedural destruction system was developed and implemented during a talk at GDC Summer, Remedy’s principle VFX artist Johannes Richter said the process began with understanding ‘granularity.’

That principle of granularity revolves around the concept of recreating every level of natural detail in each visual effect. That might sound overwhelming, but it’s a fairly common principle in the VFX industry — especially in film — and asks VFX artists to do their best to capture the continuum of nature.

“You try and capture nature because nature isn’t quantified. Nature is sort of a continuum from very big objects down to dust and very thin veils of smoke. And you always want to encapsulate that to create this rich gradient across all those different scales,” explains Richter.

“If you look at how to do that in a game engine, we’re looking at an overlapping representation three different layers. And the first one is you can do rigid body simulations (RBDs). You have chunks and parts of props and environments. All of that is RBDs. And the environments are a big static mesh that you can collide with.

“Then you’ve got to start representing them with something else because the smaller you get, the more of them you’ll have on screen. So now we’re looking at something like mesh particles or rigid body hierarchies or material decals even. Those can sell a certain material richness on those layers. So now we’re moving from objects to chunks, and almost to the debris already.

“The last layer is pure particles. Particle sprites, mesh particles for embers, sparks, smoke, fire, splinters, gravel, sand, and all that kind of stuff that plays a big role in filling out that kind of gradient.”

That’s the process, but why use a procedural approach to implement it? According to Richter, the VFX team on Control was restricted by its size. That meant they couldn’t start working on hundreds of assets and props individually. What they could do, however, is assign those objects with rules and metadata that would dictate how they behave.

“We had to choose a certain approach to this and we went procedural, but what does ‘procedural’ mean? It’s a rule-based processing and interpretation of world data. That’s really all it is. You get some data and you have some rules, then you apply the rules and change the data. That’s sort of the key to this,” he explains.

“What that looked like for us, for this kind of environment process, was we got the art models and the world and used them as our data. Then the data was actually amended or augmented with a bit of extra stuff, extra metadata that basically told us what everything is made of.

“So we could say ‘Hey, those cushions are fabric. That concrete is, well, concrete, and the plant is made out of plant. Once you know that about an object, you can start applying rules based on what those objects are made of. So you could say ‘well, grass spawns bits of leaves when you shoot it. Or concrete breaks into smaller concrete bits when you shoot it and then spawns dust. Shoot a metal pipe and it spawns water that dribbles out of the hole, and then you add a bend decal to sell a little bit of deformation up there.'”

In short, the Remedy VFX team created a finite set of rules that encapsulated the whole process, and ensured every object — be it a wooden chair, plant, computer, or concrete pillar — had a material-driven reaction to whatever could happen inside the world of Control.

Richter concedes that it wasn’t the easiest system to create, but once the VFX team got over that initial hurdle they had something they could unleash on a swathe unique objects and props as they came through the pipe.

The instrument platform of the 305m telescope at Arecibo Observatory in Puerto Rico fell overnight. No injuries were reported. NSF is working with stakeholders to assess the situation. Our top priority is maintaining safety. NSF will release more details when they are confirmed. pic.twitter.com/Xjbb9hPUgD

— National Science Foundation (@NSF) December 1, 2020

As artificial intelligence continues its rapid progress, equaling or surpassing human performance on benchmarks in an increasing range of tasks, researchers in the field are directing more effort to the interaction between humans and AI in domains where both are active. Chess stands as a model system for studying how people can collaborate with AI, or learn from AI, just as chess has served as a leading indicator of many central questions in AI throughout the field’s history.

AI-powered chess engines have consistently bested human players since 2005, and the chess world has undergone further shifts since then, such as the introduction of the heuristics-based Stockfish engine in 2008 and the deep reinforcement learning-based AlphaZero engine in 2017. The impact of this evolution has been monumental: chess is now seeing record numbers of people playing the game even as AI itself continues to get better at playing. These shifts have created a unique testbed for studying the interactions between humans and AI: formidable AI chess-playing ability combined with a large, growing human interest in the game has resulted in a wide variety of playing styles and player skill levels.

There’s a lot of work out there that attempts to match AI chess play to varying human skill levels, but the result is often AI that makes decisions and plays moves differently than human players at that skill level. The goal for our research is to better bridge the gap between AI and human chess-playing abilities. The question for AI and its ability to learn is: can AI make the same fine-grained decisions that humans do at a specific skill level? This is a good starting point for aligning AI with human behavior in chess.

Our team of researchers at the University of Toronto, Microsoft Research, and Cornell University has begun investigating how to better match AI to different human skill levels and, beyond that, personalize an AI model to a specific player’s playing style. Our work comprises two papers, “Aligning Superhuman AI with Human Behavior: Chess as a Model System” and “Learning Personalized Behaviors of Human Behavior in Chess,” as well as a novel chess engine, called Maia, which is trained on games played by humans to more closely match human play. Our results show that, in fact, human decisions at different levels of skill can be predicted by AI, even at the individual level. This represents a step forward in modeling human decisions in chess, opening new possibilities for collaboration and learning between humans and AI.

AlphaZero changed how AI played the game by practicing against itself with only knowledge of the rules (“self-play”), unlike previous models that relied heavily on libraries of moves and past games to inform training. Our model, Maia, is a customized version of Leela Chess Zero (an open-source implementation of AlphaZero). We trained Maia on human games with the goal of playing the most human-like moves, instead of being trained on self-play games with the goal of playing the optimal moves. In order to characterize human chess-playing at different skill levels, we developed a suite of nine Maias, one for each Elo rating between 1100 and 1900. (Elo ratings are a system for evaluating players’ relative skill in games like chess.) As you’ll see below, Maia matches human play more closely than any chess engine ever created.

If you’re curious, you can play against a few versions of Maia on Lichess, the popular open-source online chess platform. Our bots on Lichess are named maia1, maia5, and maia9, which we trained on human games at Elo rating 1100, 1500, and 1900, respectively. You can also download these bots and other resources from the GitHub repo.

Measuring human play

What does it mean for a chess engine to match human play? For our purposes, we settled on a simple metric: given a position that occurred in an actual human game, what is the probability that the engine plays the move that the human played in the game?

Making an engine that matches human play according to this definition is a difficult task. The vast majority of positions seen in real games only happen once, because the sheer number of possible positions is astronomical: after just four moves by each player, the number of potential positions enters the hundreds of billions. Moreover, people have a wide variety of styles, even at the same rough skill level. And even the same exact person might make a different move if they see the same position twice!

Creating a dataset

To rigorously compare engines in how well they match human play, we need a good test set to evaluate them with. We made a collection of nine test sets, one for each narrow rating range. Here’s how we made them:

• First, we made rating bins for each range of 100 rating points (such as 1200-1299, 1300-1399, and so on).
• In each bin, we put all games where both players are in the same rating range.
• We drew 10,000 games from each bin, ignoring games played at Bullet and HyperBullet speeds. At those speeds (one minute or less per player), players tend to play lower quality moves to not lose by running out of time.
• Within each game, we discarded the first 10 moves made by each player to ignore most memorized opening moves.
• We also discarded any move where the player had less than 30 seconds to complete the rest of the game (to avoid situations where players are making random moves).

After these restrictions we had nine test sets, one for each rating range, which contained roughly 500,000 positions each.

Differentiating our work from prior attempts

People have been trying to create chess engines that accurately match human play for decades. For one thing, they would make great sparring partners. But getting crushed like a bug every single game isn’t that fun, so the most popular attempts at engines that match human play have been some kind of attenuated version of a strong chess engine. Attenuated versions of an engine are created by limiting the engine’s ability in some way, such as reducing the amount of data it’s trained on or limiting how deeply it searches to find a move. For example, the “play with the computer” feature on Lichess is a series of Stockfish models that are limited in the number of moves they are allowed to look ahead. Chess.com, ICC, FICS, and other platforms all have similar engines. How well do these engines match human play?

Stockfish: We created several attenuated versions of Stockfish, one for each depth limit (for example, the depth 3 Stockfish can only look 3 moves ahead), and then we tested them on our test sets. In the plot below, we break out the accuracies by rating level so you can see if the engine thinks more like players of a specific skill level.

As you can see, it doesn’t work that well. Attenuated versions of Stockfish only match human moves about 35-40% of the time. And equally importantly, each curve is strictly increasing, meaning that even depth-1 Stockfish does a better job at matching 1900-rated human moves than it does at matching 1100-rated human moves. This means that attenuating Stockfish by restricting the depth it can search doesn’t capture human play at lower skill levels—instead, it looks like it’s playing regular Stockfish chess with a lot of noise mixed in.

Leela Chess Zero: Attenuating Stockfish doesn’t characterize human play at specific levels. What about Leela Chess Zero, an open-source implementation of AlphaZero, which learns chess through self-play games and deep reinforcement learning? Unlike Stockfish, Leela incorporates no human knowledge in its design. Despite this, however, the chess community was very excited by how Leela seemed to play more like human players.

In the analysis above, we looked at a number of different Leela generations, with the ratings being their relative skill (commentators noted that early Leela generations played particularly similar to humans). People were right in that the best versions of Leela match human moves more often than Stockfish. But Leela still doesn’t capture human play at different skill levels: each version is always getting better or always getting worse as the human skill level increases. To characterize human play at a particular level, we need another approach.

Maia: A better solution for matching human skill levels

Maia is an engine designed to play like humans at a particular skill level. To achieve this, we adapted the AlphaZero/Leela Chess framework to learn from human games. We created nine different versions, one for each rating range from 1100-1199 to 1900-1999. We made nine training datasets in the same way that we made the test datasets (described above), with each training set containing 12 million games. We then trained a separate Maia model for each rating bin to create our nine Maias, from Maia 1100 to Maia 1900.

As you can see, the Maia results are qualitatively different from Stockfish and Leela. First off, the move matching performance is much higher: Maia’s lowest accuracy, when it is trained on 1900-rated players but predicts moves made by 1100-rated players, is 46%—as high as the best performance achieved by any Stockfish or Leela model on any human skill level we tested. Maia’s highest accuracy is over 52%. Over half the time, Maia 1900 predicts the exact move a 1900-rated human played in an actual game.

Importantly, every version of Maia uniquely captures a specific human skill level since every curve achieves its maximum accuracy at a different human rating. Even Maia 1100 achieves over 50% accuracy in predicting 1100-rated moves, and it’s much better at predicting 1100-rated players than 1900-rated players!

This means something deep about chess: there is such a thing as “1100-rated style.” And furthermore, it can be captured by a machine learning model. This was surprising to us: it would have been possible that human play is a mixture of good moves and random blunders, with 1100-rated players blundering more often and 1900-rated players blundering less often. Then it would have been impossible to capture 1100-rated style, because random blunders are impossible to predict. But since we can predict human play at different levels, there is a reliable, predictable, and maybe even algorithmically teachable difference between one human skill level and the next.

Maia’s predictions

You can find all of the juicy details in the paper, but one of the most exciting things about Maia is that it can predict mistakes. Even when a human makes an absolute howler—“hanging” a queen, in other words letting an opponent capture it for free, for example—Maia predicts the exact mistake made more than 25% of the time. This could be really valuable for average players trying to improve their game: Maia could look at your games and tell which blunders were predictable and which were random mistakes. If your mistakes are predictable, you know what to work on to hit the next level.

Modeling individual players’ styles with Maia

In current work, we are pushing the modeling of human play to the next level: can we actually predict the moves a particular human player would make?

It turns out that personalizing Maia gives us our biggest performance gains. Whereas base Maia predicts human moves around 50% of the time, some personalized models can predict an individual’s moves with accuracies up to 75%!

We achieve these results by fine-tuning Maia. Starting with a base Maia, say Maia 1900, we update the model by continuing training on an individual player’s games. Below, you can see that for predicting individual players’ moves, the personalized models all show large improvements over the non-personalized models. The gains are so large that the personalized models are almost non-overlapping with the non-personalized ones: the personalized model for the hardest-to-predict player still gets almost 60% accuracy, whereas even the non-personalized models don’t achieve this accuracy on even the easiest-to-predict players.

The personalized models are so accurate that given just a few games, we can tell which player played them! In this stylometry task—where the goal is to recognize an individual’s playing style—we train personalized models for 400 players of varying skill levels, and then have each model predict the moves from 4 games by each player. For 96% of the 4-game sets we tested, the personalized model that achieved the highest accuracy (that is, predicted the player’s actual moves most often) was the one that was trained on the player who played the games. With only 4 games of data, we can pick out who played the games from a set of 400 players. The personalized models are capturing individual chess-playing style in a highly accurate way.

Using AI to help improve human chess play

We designed Maia to be a chess engine that predicts human moves at a particular skill level, and it has progressed into a personalized engine that can identify the games of individual players. This is an exciting step forward in our understanding of human chess play, and it brings us closer to our goal of creating AI chess-teaching tools that help humans improve. Among the many capabilities of a good chess teacher, two of them are understanding how students at different skill levels play and recognizing the playing styles of their students. Maia has shown that these capabilities are realizable using AI.

The ability to create personalized chess engines from publicly available, individual player data opens an interesting discussion on the possible uses (and misuses) of this technology. We initiate this discussion in our papers, but there is a long road ahead in understanding the full potential and implications of this line of research. As in countless times before, Chess will be one model AI system that sets the stage for this discussion.

Acknowledgments

Many thanks to Lichess.org for providing the human games that we trained on, and hosting our Maia models that you can play against. Ashton Anderson was supported in part by an NSERC grant, a Microsoft Research gift, and a CFI grant. Jon Kleinberg was supported in part by a Simons Investigator Award, a Vannevar Bush Faculty Fellowship, a MURI grant, and a MacArthur Foundation grant.