Test Plan
From Project Shrink
Contents |
Test Plan
I. Testing Goals
In testing SHRINK, our main priorities will be to obtain feedback from players regarding each aspect of the game, as well as obtaining ideas for how to improve interaction and navigation of the game levels. In analysis of the feedback we receive, we will specifically be looking for qualitative and quantitative responses on how to improve or modify the overall gameplay, the individual game components, the technical aspects, and the artistic look and feel of the game. This will be done through live play-tests followed by either an oral interview or a written survey. In each play-test session, we will attempt to bring new faces into the mix, in order to get a more comprehensive pool of results.
In our play-tests, it is also imperative that we not only get suggestions on how to improve the already-present aspects of gameplay, but also how to provide the player with results he or she may expect from certain actions. To elaborate on this, one of our main goals in producing SHRINK is to provide the player with the means to do virtually anything in the world that they can conceive of. As such, we will constantly be using our players’ expectations to help shape our level design and gameplay experience.
Of course, internal testing will also take place to ensure product quality. This process of quality assurance will consist of three phases; testing, revising, and refining. These three processes will be applied to technical and artistic aspects alike. Before the outside play-testers are given a version of the game to test, the internal play testing will have already gone through several cycles. Our goal in outside play-testing is to acquire feedback on unanticipated events and features that users would like to see incorporated into our product.
II. Test Components
A. General Components
As stated earlier, each play test will focus on different features or sections of the overall whole. For the purposes of SHRINK we have defined several key groups of features that will need individual testing. The chart below lists the macro scale categories, and the micro scale features in each category.
B. Specific Aspects
- Technical
First and foremost, the game will need to consistently be combed for bugs stemming from logic errors in the code. While each new feature added will go through extensive testing in-house, the implementation of the core mechanics will go through testing beyond the initial debugging. In particular, player movement, physics, and collision will need to go through many stages of testing in order to ensure that it is all fully functional. The reasoning that lies behind this is that these particular game features will be used much more than any others, and are absolutely essential to the gameplay. As such, the chances of a player discovering an unforeseen exploit in these mechanics are higher than would be in a less used game feature. Additionally, the impact that a bug in the collision and physics systems would have on the overall gameplay could be potentially catastrophic.
In addition to the main mechanics of the game, we will be gauging the games’ performance by load testing the engine at given points during the development process. With so much content going into the final version of the game, it is important that the game be able to run well despite the level of intricacy. This will require benchmark tests on a wide variety of machines to ensure reliably comprehensive results. As part of these load tests, we will perform operations such as loading the environment with hundreds of different renderable objects which will collide with terrain surfaces, giving us an idea of how effective our collision system is at conserving system resources to provide a seamless gameplay experience.
Other technical issues may have to be dealt with on a case-by-case basis. For example, if part of the program that loads the level breaks, development will likely have to focus entirely on the immediate problem, as any other testing cannot be completed when the game itself cannot even be loaded up. For the most part, issues such as this should be corrected during the debugging phase, but in the case that something unforeseen happens, it should be dealt with before any further testing is done.
- Fundamental Gameplay
The first round of dedicated outside testing for SHRINK will need to focus on the basic game controls. Without an intuitive control scheme, the players will be focusing more on how to make the character move around the way they intend it instead of how to accomplish more advanced game objectives. The idea here is to get the controls to a point where a player can just pick up the controller and go instead of having to spend time deciphering the control scheme first. Having these controls perfected as much as possible will give us more accurate results in the play-tests.
Following the refinement of the controls, the primary game mechanics must be tested. First and foremost under this category is going to be lateral movement and jumping. Also inherent in this stage is going to be refinement of camera behavior. We will be looking for feedback here on how well players can relate to the movement model, and how effective they find the camera in framing the scene. Similar to controls, we want the main mechanics of the game to come to players naturally rather than have them sit and ponder the most basic aspects of gameplay.
Once the movement has been refined, we will be able to start testing the more unique aspects of the gameplay. The inventory system, for example, will be vital to almost every aspect of gameplay save for movement. There are several aspects to the inventory and item systems which will need to be tested. First off, we want to make sure the player can distinguish items from non-interactive background objects. This will require some basic implementation of the artwork in order to test. Once we are satisfied that the items are distinct from the environment, we will test methods of item pickup. For instance, should an item pick up as soon as the player moves over it, or should he or she have to press a button while over it to pick it up? Does it make sense to the player if their currently selected item is replaced by an item they pick up? Questions like these will only be answered through feedback from the play-testers.
Once item implementation has been refined to our liking, we will need to test the dynamics of actually using these items. In the full version of the game, items will most often be used for combat purposes. For this reason, testing combat with item usage would be reasonable. As part of this, it would also make sense to incorporate lose conditions, level restarting, and a heads-up display into the test. This would give the players all the components they would need to fully test the core combat mechanics in the game. From this test we should get an idea of the level of difficulty needed to make combat fun, as well as help us understand how the players instinctively intend on confronting opponents.
- Usability
Once the fundamentals of gameplay are nailed down, the testing process will grow to focus on the more advanced levels of gameplay which the fundamentals make possible. Building on the movement mechanics, platforming, falling damage, and win conditions will be introduced first. This begins to get into fun factor, rather than a simple, “is this working?” Questions begin to come up, like, “Is this fun and intuitive? Does it seem fair?”
From there, it expands to level/room navigation. This is where the testing becomes more unique to SHRINK itself. The player needs to navigate the level. This further expands into a larger scale level flow. The player now has a beginning and end and he must traverse the level. Exploration is introduced. Here, we can begin to test puzzles and SHRINK-specific elements. Was the level design fun? Were the mechanics fun when put together?
The final element of basic gameplay is the item system and how it interacts with the player and environment. We can place items and have the tester use them and grow/shrink with them. Was it intuitive? Did the items stand out enough? Are the items vital to the mission easy to find? Are the secret items challenging enough to find without being frustratingly so?
On top of this is the menu navigation/GUI. Although this isn’t directly part of the gameplay, it will be vital to have a menu which the user can find agreeable. Were the menus navigable? Did they make sense? Our aim is to have menus that not only are visually appealing to users, but inherently intuitive to them as well. Even a computer illiterate should be able to find their way through our menus.
- Art and Sound
Art and sound, while admittedly not part of the actual play experience, will be important in immersing the player in the world of SHRINK. As part of one of the other tests, artwork, animations, sound effects, and music could be added in for possibly an exclusive test of the design work. What we will be looking for in feedback on the artwork is that everything fits together thematically. For instance, do our visuals fit in with the feel of the sound design? We want all of the design elements to build on one another, not contrast to each other.
III. Test Plan
In house testing has already begun, and gone through much iteration in order to make sure that the core of the gameplay works (aka. the code compiles). However, as we start to add more functionality and playability to the game, we will need to obtain feedback from our possible consumers as soon as possible, in order to actually use the results and modify SHRINK as necessary. Our starting play tests will occur as soon as we have a functional “game”. In an effort to assure fun gameplay, the first external play test session will be on the macro level, getting feedback on the overall feel our game actually communicates. We want to make sure that the atmosphere in our game is actually being presented the way we intended it to. After we are assured that the core is delivering the level of immersion we are looking for, we will implement and test more of the details that make SHRINK unique.
As each new feature and component is added to the game, it will be tested in house before being released for external testing. This is a preventative measure that allows the maximum possible time for our external play-testers to provide feedback. The reason that in house testing provides better out of house testing is simple; if we present broken code to an external play tester, they will focus merely on the large scale problem that is inherent in the code and not focus on the more minute details that we wish them to review. A situation such as this would effectively render any outside beta testing entirely useless.
In beginning play-tests on the public, the first round should simply be to make sure the basic controls of movement are intuitive. A test like this could be as simple as sending out a survey to a group of people and, giving them a brief description of the game, have them fill in which buttons they imagine being bound to which functions. After having configured the controls in a way which the player is familiar with, they can then be given working versions of the actual game to test.
Essentially, we want to release the components of the game bit by bit as they are developed, and get feedback on each individual one. In addition to reporting their own opinions about gameplay, we will encourage players to report any bugs or errors they come across in their game experience. Each time one of these errors is reported, it will be dealt with immediately and distributed to the testers over an internet repository. After the bugs have been worked out and the primary elements of gameplay tested, the focus of testing will switch from a technical focus to a more “usability” oriented one. In the usability play-tests, we will be looking less for suggestions on how to fix things like the running and jumping (though we will still be open to people’s suggestions at this point), and looking more in favor of ways to make the game more fun. Testing on this level would involve sending players a working version of an entire level, and having them navigate it themselves without any outside assistance. When the player solves the puzzle and reaches the end, we will both interview and survey them for their reactions, and modify the game mechanics accordingly.
Also important in the process of play-testing levels will be taking into account the players’ personal suggestions of different ways to complete a given level. Since it is unrealistic to expect the developers to be able to come up with every possible scenario for beating a level on their own, we will let the players themselves contribute to the list of possible scenarios. In doing this, we will need to find clean, conservative ways of integrating interaction into gameplay with a minimum amount of hard-coding. Given enough players, the number of possible scenarios should eventually grow to a level at which the vast majority of people will be able to complete a level in any way conceivable to them.
