05-28-2019, 07:34 PM
Blog: Welcome to rabbit hell! Reliable AI locomotion with TDD
<div style="margin: 5px 5% 10px 5%;"><img src="http://www.sickgaming.net/blog/wp-content/uploads/2019/05/blog-welcome-to-rabbit-hell-reliable-ai-locomotion-with-tdd.jpg" width="200" height="200" title="" alt="" /></div><div><h2>Evaluation: Is Test-Driven Development Right for You?</h2>
<p>As developers it can be tempting to put <em>a little too much </em>work into testing scenes the player will never appreciate; I won’t deny that I had a lot of fun building Rabbit Hell. Internal features like this can be a big waste of time and jeopardize milestones, so we need to take a hard look at when & whether a given feature warrants a unit testing apparatus. Below I’ve identified a few criteria that, in my eyes, justified TDD for <em>ElemenTerra</em>’s creature locomotion<em>.</em></p>
<h3>1. Are your test cases time-consuming to produce manually?</h3>
<p>Before spending time on automated testing, we need to check whether we can evaluate a feature with the regular game controls. If you want to make sure your keys unlock doors, spawn a key and unlock a door with it! Creating unit tests for this feature would be an irresponsible time sink because it takes only seconds to test manually.</p>
<h3>2. Are your test cases difficult to produce manually?</h3>
<p>Automated unit tests become justified when there are known, difficult-to-produce edge cases. Rabbit Hell course #7 tests creatures stepping off ledges, a circumstance their AI works very hard to avoid. Course #12 simulates the navmesh desyncing from the floor geometry, which only occurs during extreme lag. Such situations can be difficult or impossible to contrive with the game controls, while our tests produce them effortlessly.</p>
<h3>3. Do you know the desired results won’t change?</h3>
<p>Game design is all about iteration, and the goals of individual features may change as your game is redesigned. Even small changes in intent can invalidate the metrics by which you evaluate your features, and thus any unit tests along with them. While the creatures’ eating, sleeping, and player interaction behaviors underwent several redesigns, we always needed them to get from point A to point B. Thus, the locomotion code and its unit tests remained valid throughout development.</p>
<h3>4. Are regressions likely to go unnoticed?</h3>
<p>Maybe you’ve been in this situation: you’re wrapping up one of your final tasks before shipping a game, and suddenly you find a game-breaking bug in a feature you finished ages ago. Games are giant interconnected systems, and so it’s only natural that adding New Feature B might break Old Feature A.</p>
<p>This isn’t so bad when the broken feature is ubiquitous, like the player’s ability to jump. If your core mechanic breaks, you’re bound to notice immediately. But the bug might slip under the radar if the broken feature is less frequently observed, like what happens when the player steps into a narrow crevice. Bugs detected in late development can jeopardize your schedule, and post-launch they hurt your player experience. Thus, unit tests can be great tools for maintaining edge case behavior, but are often redundant for functionality which already gets a lot of incidental testing.</p>
<h3>5. What’s the the worst-case cost of having tests, and what’s the worst-case cost of not having tests?</h3>
<p>Setting up a testing apparatus is a form of risk management. Let’s imagine that you’re deciding whether to buy insurance for a vehicle. The three questions you need to answer are:</p>
<ol>
<li>How much do the monthly premiums cost?</li>
<li>How likely is it the vehicle will be damaged?</li>
<li>How expensive would the worst-case scenario be if you were uninsured?</li>
</ol>
<p>For TDD we can imagine the monthly premiums as the production cost of maintaining our unit tests, the likelihood of vehicle damage as the likelihood of our feature breaking, and the cost of fully replacing the vehicle as the worst-case scenario for a regression bug.</p>
<ol></ol>
<p>If a feature’s tests take a lot of time to create, the feature is uncomplicated and not likely to be changed, or it would be manageable if it broke in late development, then unit tests may be more trouble than they’re worth. If the tests are simple to put together, the feature is volatile and interconnected, or its bugs would cost a lot of production time, then tests can help keep us on schedule.</p>
<h3>The limits of automation</h3>
<p>Unit tests can be a great supplement to catching and reducing bugs, but I want to emphasize that they don’t replace the need for professional quality assurance on large-scale games. Proper QA is an art that requires creativity, subjective judgment, and excellent technical communication, which means that you need skilled and well-taken-care-of humans!</p>
<h2>Testing the Waters</h2>
<p>While not the right choice for every circumstance, Test-Driven Development is a powerful tool that can and should be applied to more game development contexts. Let’s expand the horizons of what and how we test!</p>
<p>If you have opinions on the points made in this article or have stories of using TDD in your own games, please shoot us an email. If you liked what you read and are looking for a skilled group of developers to build, improve, or help finish your game, <a href="https://www.freeformlabs.xyz/contact"><u>you can get in touch with Freeform Labs here</u></a>. Until next time!</p>
<hr>
<p>Originally posted on the <a href="https://www.freeformlabs.xyz/blog/rabbit-hell-tdd-ai">Freeform Labs Development Blog</a>.</p>
<p><em><strong>Freeform Labs, Inc. </strong>is a games and software development team with a commitment to providing the highest quality of digital craftsmanship, and a mission to inspire learning and creativity through cutting-edge experience design. The team is centrally located in Los Angeles, and maintains a network of trusted partners across the globe. With specialized experience in VR, AR, Game AI, and more, Freeform Labs provides an array of services including consultation, work-for-hire software development, and on-site “ship it” assistance. The company’s portfolio includes work for Microsoft, Disney, and Starbreeze – as well as award-winning original content. </em></p>
</div>
<div style="margin: 5px 5% 10px 5%;"><img src="http://www.sickgaming.net/blog/wp-content/uploads/2019/05/blog-welcome-to-rabbit-hell-reliable-ai-locomotion-with-tdd.jpg" width="200" height="200" title="" alt="" /></div><div><h2>Evaluation: Is Test-Driven Development Right for You?</h2>
<p>As developers it can be tempting to put <em>a little too much </em>work into testing scenes the player will never appreciate; I won’t deny that I had a lot of fun building Rabbit Hell. Internal features like this can be a big waste of time and jeopardize milestones, so we need to take a hard look at when & whether a given feature warrants a unit testing apparatus. Below I’ve identified a few criteria that, in my eyes, justified TDD for <em>ElemenTerra</em>’s creature locomotion<em>.</em></p>
<h3>1. Are your test cases time-consuming to produce manually?</h3>
<p>Before spending time on automated testing, we need to check whether we can evaluate a feature with the regular game controls. If you want to make sure your keys unlock doors, spawn a key and unlock a door with it! Creating unit tests for this feature would be an irresponsible time sink because it takes only seconds to test manually.</p>
<h3>2. Are your test cases difficult to produce manually?</h3>
<p>Automated unit tests become justified when there are known, difficult-to-produce edge cases. Rabbit Hell course #7 tests creatures stepping off ledges, a circumstance their AI works very hard to avoid. Course #12 simulates the navmesh desyncing from the floor geometry, which only occurs during extreme lag. Such situations can be difficult or impossible to contrive with the game controls, while our tests produce them effortlessly.</p>
<h3>3. Do you know the desired results won’t change?</h3>
<p>Game design is all about iteration, and the goals of individual features may change as your game is redesigned. Even small changes in intent can invalidate the metrics by which you evaluate your features, and thus any unit tests along with them. While the creatures’ eating, sleeping, and player interaction behaviors underwent several redesigns, we always needed them to get from point A to point B. Thus, the locomotion code and its unit tests remained valid throughout development.</p>
<h3>4. Are regressions likely to go unnoticed?</h3>
<p>Maybe you’ve been in this situation: you’re wrapping up one of your final tasks before shipping a game, and suddenly you find a game-breaking bug in a feature you finished ages ago. Games are giant interconnected systems, and so it’s only natural that adding New Feature B might break Old Feature A.</p>
<p>This isn’t so bad when the broken feature is ubiquitous, like the player’s ability to jump. If your core mechanic breaks, you’re bound to notice immediately. But the bug might slip under the radar if the broken feature is less frequently observed, like what happens when the player steps into a narrow crevice. Bugs detected in late development can jeopardize your schedule, and post-launch they hurt your player experience. Thus, unit tests can be great tools for maintaining edge case behavior, but are often redundant for functionality which already gets a lot of incidental testing.</p>
<h3>5. What’s the the worst-case cost of having tests, and what’s the worst-case cost of not having tests?</h3>
<p>Setting up a testing apparatus is a form of risk management. Let’s imagine that you’re deciding whether to buy insurance for a vehicle. The three questions you need to answer are:</p>
<ol>
<li>How much do the monthly premiums cost?</li>
<li>How likely is it the vehicle will be damaged?</li>
<li>How expensive would the worst-case scenario be if you were uninsured?</li>
</ol>
<p>For TDD we can imagine the monthly premiums as the production cost of maintaining our unit tests, the likelihood of vehicle damage as the likelihood of our feature breaking, and the cost of fully replacing the vehicle as the worst-case scenario for a regression bug.</p>
<ol></ol>
<p>If a feature’s tests take a lot of time to create, the feature is uncomplicated and not likely to be changed, or it would be manageable if it broke in late development, then unit tests may be more trouble than they’re worth. If the tests are simple to put together, the feature is volatile and interconnected, or its bugs would cost a lot of production time, then tests can help keep us on schedule.</p>
<h3>The limits of automation</h3>
<p>Unit tests can be a great supplement to catching and reducing bugs, but I want to emphasize that they don’t replace the need for professional quality assurance on large-scale games. Proper QA is an art that requires creativity, subjective judgment, and excellent technical communication, which means that you need skilled and well-taken-care-of humans!</p>
<h2>Testing the Waters</h2>
<p>While not the right choice for every circumstance, Test-Driven Development is a powerful tool that can and should be applied to more game development contexts. Let’s expand the horizons of what and how we test!</p>
<p>If you have opinions on the points made in this article or have stories of using TDD in your own games, please shoot us an email. If you liked what you read and are looking for a skilled group of developers to build, improve, or help finish your game, <a href="https://www.freeformlabs.xyz/contact"><u>you can get in touch with Freeform Labs here</u></a>. Until next time!</p>
<hr>
<p>Originally posted on the <a href="https://www.freeformlabs.xyz/blog/rabbit-hell-tdd-ai">Freeform Labs Development Blog</a>.</p>
<p><em><strong>Freeform Labs, Inc. </strong>is a games and software development team with a commitment to providing the highest quality of digital craftsmanship, and a mission to inspire learning and creativity through cutting-edge experience design. The team is centrally located in Los Angeles, and maintains a network of trusted partners across the globe. With specialized experience in VR, AR, Game AI, and more, Freeform Labs provides an array of services including consultation, work-for-hire software development, and on-site “ship it” assistance. The company’s portfolio includes work for Microsoft, Disney, and Starbreeze – as well as award-winning original content. </em></p>
</div>