Last week, we unveiled two new RoboRepair Tutorials! This week, we’re going behind-the-scenes of what it’s like working on RoboCo with our QA Analyst, Austin Smythe!
More RoboCo Events!
A quick announcement though before we introduce Austin: we’re excited to announce that RoboCo will be taking part in the 2022 Seattle Indies Game Expo! Keep an eye out for the live broadcast on their Twitch channel coming up September 24th, 2022! Check out their website to learn more.
Meet Austin Smythe, RoboCo’s QA Analyst
Austin first joined Filament Games in February 2018. While the majority of his time is spent working on RoboCo, Austin also spends a lot of time working hand-in-hand with Filament’s other QA Analysts (Georgia Adkins, Brian Czech, and Alex Avalon) on other projects. In addition to troubleshooting games, Austin is a talented RoboArtist, particularly when it comes to RoboCo pixel art.
Nicole: Hello, Austin! Thanks for taking some time today to talk about RoboCo!
Austin: No problem, Nicole! Happy to be here!
Nicole: Let’s jump straight into it. We haven’t covered QA or their roles extensively in a devblog before, so could you explain what a QA analyst does?
Austin: So I could give you the very boring job title or job description, but basically a QA analyst is someone that tests the software that the company is creating.
Nicole: What’s the process you and your team goes through when testing a game?
Austin: The whole process involves a lot of verification testing. Oftentimes, when a developer or a designer creates a game, they have these lists of actions that are supposed to happen. As a QA analyst, I take that list and test against everything that the designer says should happen to see if it all works. This allows us to search for blindspots that a developer may have missed and make sure an end user (i.e. player) won’t run into something weird. Essentially, we play the game differently than it was intended.
Nicole: So does that mean you’re constantly playing the same game on repeat? Or is there other stuff like software involved?
Austin: Partially. We spend a lot of time in a software called Jira, which is where developers will write “stories” that have a list of all the different acceptance tests we need to perform for a feature. Once we know what branch specifically of the game we need to test, that’s when we’ll go in and repeatedly play the game. We also, whenever we find a defect, reproduce it multiple times. We then document what we find in Jira and send it back to the developer to fix, so it’s basically a constant cycling of reading, playing, and writing.
Nicole: How did you first get interested in this type of work then?
Austin: I initially started QAing at the tailend of collage. I then got a job QAing for Activision Blizzard in their Minnesota office. I started working on Call of Duty: World War II first then moved on to Sekiro: Shadows Die Twice before coming to Filament.
Nicole: I imagine going from Activision to Filament took some adjustment then, given just how big their studio is.
Austin. It was quite a culture-shock, especially since I went from a team of 200 people to (what was then) two people. On Call of Duty especially, I remember I would work my QA shift, leave my desk, and then have someone come fill my spot as soon as I left to work their night shift. Working on Sekiro was a lot of fun though, I personally love that game and was super happy to be a part of it!
Working on those games really taught me a lot about what I liked about QAing. For example, I like that a lot of things are Pass/Fail, and that it’s almost a perfect balance between programming and exploration, because a lot of the job is figuring out what you don’t know rather than what you do know.
Nicole: So what does the typical day of a QA analyst look like then?
Austin: Typically our team starts or ends the previous week with sync, which is a meeting where we go over which projects need QAing. Sometimes, we’ll divide the QA work between people or, if we need to test things more quickly, we’ll all jump in on a project. That’s again when we’ll go into Jira to look at what tickets have been submitted, which can either be “stories” from the developers like I mentioned earlier or a list of previous fixed defects. When it’s the latter, I will go through those fixed defects again to make sure nothing else popped up. Once those tickets are verified, I’ll then go through them again just to double-check I didn’t miss anything.
Nicole: And how do you and your team determine which games get tested first?
Austin: Each game goes through phases. Typically, a game goes through three phases: alpha, beta, and gold. It usually takes a game six to eight months to get through till the end of a gold phase, since that’s the last QA phase a game will go through before it’s ready to launch. Games that are at the end of their phases usually take priority over small QA items from sprints. We also have additional exit/entry criteria for games that are close to release, but that has more to do with company standards than it does anything else.
The six to eight months timeline isn’t set in stone either. RoboCo, for example, has been through more phases than any other game we’ve worked on, because we want to make sure we’re going through it as detailed as we can before it’s released to Steam Early Access later this year.
Nicole: Speaking of RoboCo, what’s been the funnest thing about working on that project?
Austin: I really enjoy that my job is sometimes just going into the sandbox and creating a robot. A lot of times I get asked to test stuff for the challenges and my whole day will be spent building a robot that can complete every aspect of it. Funnily enough, I also learned from doing this that my boss, Brian Czech, is… less enthusiastic than myself about robot-building. This means, whenever I bring him in to help on RoboCo, I get to either teach him about building or send him robot files I’ve made, leading to a super fun role-reversal.
I also, when testing, get the opportunity to build whacky robots like the F1 or the Boston-Dynamics inspired claw arm, which is super cool.
Nicole: Is there any aspect of the job that’s been challenging?
Austin: I’d say the most challenging part of QAing RoboCo is the size of the project. Normally, games at Filament consist of one designer, one engineer, and one artist. On RoboCo, we have one or two designers, three engineers, and at least one art person. We currently have over 5000 test cases for RoboCo, which is about five times more than an average project. Sometimes juggling that work within a two week testing period can be difficult, especially if there are other games that need looking at.
Nicole: What about the robot programming feature? Has that made things more complex?
Austin: Sometimes. I know the bare essentials of Python, so sometimes I’m able to figure things out on my own. Other times, I may have to have one of the engineers send me a robot for testing. The process for QAing is the same though. I just have to run through the code a bunch of times to make sure that the robot is doing what it was intended to do.
Nicole: Are there any funny glitches or bugs you’ve encountered that you can tell us about?
Austin: Hmmm. That’s a good question. There’s the one about humans knocking themselves out with clipboards, but I think Carter already talked about that one. One I fondly remember involves the Center of Mass RoboRepar Tutorial. You used to be able to move the block so it was inside the robot, which, when you pressed play, would cause the block and base to jump away from each other, thus catapulting the robot over the finish line.
Nicole: Final question, what is your biggest dream or hope for RoboCo?
Austin: I love seeing the content creators who have played RoboCo apply real-world engineering principles to building robots. I also love watching people solve each challenge in their own unique way.
And that’s it for this week’s devblog! Next week, Luke Jayapalan returns to talk about a brand new feature added to our Paint Tool to take your robot’s customizations to the next level!
For the latest news on RoboCo, follow us on Twitter, YouTube, Facebook, Instagram, and TikTok! You can also connect with other community members and us by joining our official Discord and Reddit!
Don’t forget to add RoboCo to your Steam Wishlist!