controller trollers test report 1115

Once Legendary

Usability Testing & Analysis

[Version 0.0.1.105716]

Jeff Houchens, Dakota Daugherty, Kyle Devlin, Jon Capps November 20, 2015

1

2

Table of Contents

Table of Contents…………………………………………………………..2 Executive Summary……………………………………………………….3 Methodology………………………………………………………………...3

Testing Procedure……………………………………………………..3 Test Subject Demographics…………………………………………5 Summary of Testing Results………………………………………..5

Impression Questions…………………………………………………..10 Exploratory Questions………………………………………………….11

Prologue………………………………………………………………...11 Chapter 1………………………………………………………………..11 Chapter 2………………………………………………………………..12 Chapter 3………………………………………………………………..13

Directive Questions (Development Team)...........................13 Game Bugs Encountered by Playtesters……………………...14

Data Collection…………………………………………………………….15 Graphs of Data Acquired…………………………………………..17

Issues Encountered by Testing Team……………………………..18 Analysis……………………………………………………………………...25 Recommendations……………………………………………………….27 Post Mortem………………………………………………………………..28 References………………………………………………………………….29

3

Executive Summary

We have compiled our findings and testing notes in this document to help the development team, Team Zulu, better understand what is working in their game and what is not. We hope the data collected in this report will help prepare Team Zulu, and their game Once Legendary, for a future release. Throughout this document we will define our tester demographics, testing procedure used, issues found , bug feedback, analysis, recommendations, and more for Team Zulu.

An overview of our testing feedback proves that:

1. Players need helpful hints to show them the initial movement controls at the beginning of the prologue level.

2. Enemies do little to no damage and are not a significant threat to the player. 3. With 64% of deaths in Chapter 3 from cannonball fire, the cannonball size

and speed is a significant challenge and frustration point. 4. Each boss needs a health bar to give the player visibility to system status

(Nielsen, 1995). 5. Controlling the sword movement is unfamiliar to testers and isn’t a strong

match between the system and the real world (Nielsen, 1995).

Methodology Testing Procedure

I. Designate testing station A. Standard PC with Windows 7 Home/Professional OS B. Standard Mouse & Monitor C. Xbox controller for PC or Standard Keyboard D. Confirm Audio Source is audible

II. Ask tester Demographic Questionnaire A. “What genre of games do you normally play?” B. “How many hours per week do you play video games?” C. “Would you consider yourself a casual or hardcore gamers? Why?” D. “How well do you do in 2D games (side-scrollers)?”

III. Download Once Legendary build 0.0.1.105716 IV. Launch Once Legendary V. Instruct tester to sit in front of Monitor

VI. Instruct tester to review Game Menu and Directories A. Ensure tester does not click New Story B. Ensure tester does not click Continue Story C. Tester can click Table of Contents, but not Check Points

VII. Ask Impression Questions

4

A. “How does the menu compare to other games? Is it easy to navigate?” B. “What does the title make you think is game about?” C. “Does the naming convention of the title screen match other game

title menus?” VIII. Instruct tester to click New Story

IX. Tester should play and complete each level. A. Note progress blockers, emotions and player choices B. Ask Exploratory Questions during and after each level

1. Prologue a) “Do you find the sound effects bothersome or amusing?” b) “Is the level easy, in your opinion, to progress through

and complete? 2. Chapter 1

a) “Did you like controlling the sword instead of the player? Why or why not?”

b) “How does the difficulty of the enemies compare to similar side-scrolling games you’ve played?”

3. Chapter 2 a) “What did you find most interesting about this level?

Why?” b) “What was the most difficult part of this level? Why?”

4. Chapter 3 a) “How difficult was it to dodge the cannonballs?” b) “How difficult or frustrating was it to dodge the

cannonballs?” C. Note game bugs encountered

X. Ask Directed Questions XI. End Testing Session, thank tester

5

Test Subject Demographics

Test Subject

Age Gender Game Experience

Game Genre

Preference

Hours playing

games per week

Tested By

Michael 15 Male FPS, Sandbox,

Side Scroller

FPS 12 Jeff H

Sydney 23 Female Minimal Arcade 4 Kyle D.

Lucas 20 Male Average Simulator 20 Dakota D.

Derrick 25 Male Average FPS 8 Jon C.

Summary of Testing Results

Michael

Prologue: Michael initially laughed at the storybook pictures after he clicked “New Story”. Once the character loaded on the screen, Michael did not know how to move the character and spent the first minute or so trying to figure it out. He used the mouse and the keyboard arrow keys to no avail. The tester had to instruct him how to move the character so he could continue the level. At the first pendulum floor trigger, he simply jumped over it. When Michael died at one of the pits, and the character was reset to the previous checkpoint, he commented that the checkpoint was so far away from where he died. At the dual trigger area, Michael immediately noticed that the trigger colors correspond with the doors they opened. Opposed to the testers testing, Michael did not die at the thorn drop down area.

Chapter 1: At the first “Slam”, Michael commented that he didn’t know what to do and there was no way to do this (slam the board to break it). After a minute of pressing random buttons, he was ready to give up. The tester had to step in and give him a hint (pressing down and “K”). The tester also noticed that Michael had to move his head close to the monitor and squint his eyes to try and read the next pop-up instruction window indicating the font may be too small for smaller monitors. At the waterfall area, Michael clicked the switch to move the platform closer so he could proceed, but he fell into the pit and died. When the character was reset to the checkpoint, Michael noticed with a little frustration in his voice that he

6

had to activate the switch over again. At the platform that requires the player to jump and double tap/hold “W”, Michael was getting frustrated and was again about to give up because he was only double tapping “W” and did not jump. The tester had to again step in and give him a hint to jump so he could proceed. For the boss fight, Michael commented it’s annoying and takes too long. He didn’t understand why the flying bunnies were too high since they couldn’t be Riposted.

Chapter 2: Throughout the level, Michael was jumping randomly. When asked why, he said that it makes moving a lot faster. Michael’s body language during the level showed that he was very bored. At the jumping platforms, he kept slipping off and dying. Michael commented that, if the guy had shoes, he might have better traction.

Chapter 3: Michael hardly killed any mobs on this level, especially the large cylinders, archers and birds. He simply ran through them without trying to dodge or riposte the arrows or eggs. Michael commented that he was going to try and just speed run through the level. When he reached the first cannonballs, Michael died over and over again. He commented that the cannonballs are very frustrating and that he wishes there was a way to get on top of the platforms above. Michael also commented that it’s so hard to react to the cannonballs with as little time as he has when they appear on the screen. After 32 deaths, the character got stuck on a cylinder mob and Michael couldn’t move. He was very frustrated and quit playing here. Sydney

Prologue: Sydney had trouble figuring out how to move the character initially. She stated that there really needed to be an on-screen box that would explain the controls at first. She was also not accustomed to the “esc” key being the menu key. Once she found the controls menu, she thought that “B” for bloodlust was a bit distant from the other keys, but understood why it was B because of the name. She then proceeded to test the new controls out. She soon tripped the character and couldn’t figure out how to make him stand back up. After several minutes and looking at the controls menu several times, she had to ask for help. When told it was solved by pressing “W”, she commented that it was never explained that it was the way to stand up in the menu.

As she progressed through the level, she found the voice-overs amusing. She did laugh at the kid talking to the grandpa though. She continued to fall over, finally exclaiming that “This hero needs to learn to stay on his feet! This is so stupid, why does he have to always fall over?!” This was a major point of frustration for Sydney throughout the playtesting session. She explained afterwards that she felt it was unnecessary and would annoy any serious gamer who played the game easily.

7

She understood quickly that “L” was used to activate pillars, after listening to the voice-over. She explained it was a cool way to give controls help by doing it through a cut-scene or voice-over. She regularly wondered what the “little blue flame things…” were and why they were present. She at first thought they were trying to show her how to play, but thought they were random after seeing them enough. Finally she made it to the part where the player must fall through the double-sided walls of thorns.

The double thorn wall part was the most challenging part of the level for Sydney by far. She noticed that it was made extremely challenging because the player character, “slides around like he’s on ice even when I stop moving.”, that the jumping controls were “very delayed, which makes timing the jumps difficult.”, and that, “The air controls are loose. Which makes the thorn part nearly impossible.” After she finally beat the double thorn wall part, after roughly 30 minutes, she stated that it took most of the fun out of the level, which to that point had been amusing.

Chapter 1: After moving around for only a few seconds, Sydney commented on how weird controlling the sword felt. Next she encountered the part that expects the player to attack down to break the floor. She had to search for how to even move down, and noted that this was not explained in the controls menu. It took a minute, but she finally figured out the “S” key was down, after pressing several keys on the keyboard. She noted that the down attack, used to break floors etc., doesn’t work exactly like the explanation box says. After using the down attack a few times, she mentioned that it felt weird to do and didn’t fit any kind of control scheme she had used in previous games.

During one voice-over her character was paralyzed, but enemies were still able to attack her during the voice-over, and ended up killing the character before it was finished. Her remarks on this being possible were that it was “So stupid that I couldn’t move, but enemies could still attack me during the voice-over.” The checkpoint before the “You’re boring grandpa…” voice-over were commented as being “Way too far apart.” and the voice-over playing every time was noted to be an annoyance.

Once she reached the waterfall portion of the level, she was extremely confused on how to get across the waterfall. Since there aren’t any explanations about using double-tap moves with the “W” key, she couldn't figure out how to beat this part. It eventually had to be explained and was noted that there needed to be an indication of how to use “W” in that fashion before you entered that part of the level. Sydney stated often that, “A way to heal myself somehow would be nice, and make it a little easier to play.”

8

Sydney then reached the final boss cube at the end of the level. After dying a few times, she remarked that it was difficult to tell how injured the boss was without any HP bars. It was also never explained that she was supposed to riposte the flying small white cubes back at the boss cube to do damage. This eventually had to be explained and was noted that it should have been explained at some point in the level. Finally, after completing the level she commented, yet again, that having no way to tell if she was hitting enemies on this level was an annoyance.

Chapter 2: Sydney managed to get through this level with little difficulty. She even stated that this level was the most enjoyable one in the game due to it’s ease and it’s normalized play style and feel. The only negative thing she remarked on was the fact that the controls switched from level to level, which was confusing. The only notable glitch found was the continuous jumping animation when walking up or down stairs near the arena area.

Chapter 3: This level was the most frustrating for Sydney. She first noted that the sword being the controller was once again a major negative in the level. She played through the enemies and pits initially without any major issues, but once she reached the cannonball portions, she became irritated. It was often exclaimed, in less than professional terms, that jumping over or around the cannonballs was impossible and that they came so fast she didn’t have time to react. After roughly 25 minutes of trying, she declared that she had had enough of the game and that she wished she could “Put myself out of my misery with this game”. This was noted and she was directed past this part of the level so that the playtest could continue.

After progressing through the later parts of the level, she did enjoy the “Kill All Enemies” portion. Once she finally reached the dragon boss portion of the level, she remarked that this could be a cool boss fight. After failing a few times against the dragon boss, she finally understood that the dragon could only be attacked when it fell asleep. After a few more tries, she managed to beat it. She did say that the boss battle on this level was one of the most enjoyable parts of the game and, “These types of things should be the core of the game, because then it would be fun.”

After she completed the level the playtesting session was ended. Her overall opinion of the game was that it was majorly flawed, but had potential. She also said that, if the bugs were taken out and some of the more challenging parts were changed, she would be willing to Once Legendary again in the future. The total time playtesting was 3 hours and 42 minutes. Lucas

Prologue: Lucas thought that the story book was a lever title screen. When he got into the game he had to try a few usual movement setups such as the arrow keys. He finally figured out that wasd was to move. He then laughed at the whoopsie

9

doodle sound effect, and learned how to pick his character up quickly. The player got stuck on the part when you drop down past the spikes and said the walk/slowdown key should be in a different place. He then got confused at the part after the platform moves up because he could not see another platform that he had to make a long jump to. By the end, he said that he would definitely play with the sound off if he could.

Chapter 1: Lucas’ initial thought was “woah, what is up with this?” He was confused about controlling the sword and it took him awhile to figure out how to ground smash. At the first cutscene he stated that he thought that there should have been some dialogue at the beginning of the level. The next thing that he had trouble with was hitting the fireballs, so he just ran past a few of them. He had a lot of trouble getting the grapple to work right and eventually I had to give him some advice. Even then it took him a few attempts.

Chapter 2: In this chapter he learned that if you double tap a direction when not playing as the sword would make the character fall over. He did that a few times and thought it was amusing. He got a little frustrated at the end of the moving platform section and learned that he could long jump past most of the platforms to get past it. When he saw the part where you have to jump across multiple pits to tiny platforms, he sat up and said he had to focus for this part. He looked very bored at the parts where he was just walking straight for long periods.

Chapter 3: First, he had trouble with the beginning cannon area. He quickly learned to dash to get to the next trench. On the next cannon area the player got onto the platforms and tried to get on the second tower and fell through the bottom. He got slightly irritated that he could not go onto the tower. After dying at this part a lot I asked if he wanted to skip it and he said no he was going to beat it. During the kill all enemies area the archer started randomly shooting really fast non-stop. At the final boss battle Lucas was confused on how to dodge the fire breath attack. Derrick

Prologue: Derrick started the level normally and worked his way right. He cleared most obstacles fairly easily. He found the sounds and narration very amusing, in particular the “whoopsie doodle” when you fall. He quickly learned how to pick his character up and it didn’t seem to phase him. The vines did prove to be a little trouble for him especially the one where you drop down; it took him 11 tries to get past that one part. He did not have much trouble getting through the rest of the level.

Chapter 1: Derrick didn’t understand why his character was a ragdoll and messed around for a few minutes with the controls to figure it out. He eventually made his way right and followed the mini tutorials on how to use the various moves.

10

The only tutorials he had issues with were ground slam and grapple. He didn’t actually read the ground slam one and couldn’t get past that part so I told him how to ground slam. He was very interested in killing all the monsters and collecting the gold coins throughout the entire level. He also had a little trouble with grapple as well and it took him a few minutes to figure it out. When he reached the boss, it took him a little bit to figure out the riposte skill. The level one boss provided little challenge.

Chapter 2: He went through this level fairly easily and only missed two jumps where he fell to his death.

Chapter 3: When he started the level a cannon ball killed him before he got to the first trench. He still focused on killing all the enemies and collecting all the gold. There was a couple of times during the first cannonball encounter where the cannonballs hit him and threw him back without killing him. He eventually worked his way to the first checkpoint and reached the most challenging part of the game for him. The Second encounter with the cannon balls proved most challenging. He managed to dash under the cannon balls a couple of times. He eventually figured out that jumping over them was the most efficient way. He grew frustrated with this part of the game as he died 20+ times on this part. He ran through rest of the game without any trouble and still enjoyed killing all the NPCs. The dragon provided no challenge at all for him.

Impression Questions Question 1: How does the menu compare to other games? Is it easy to navigate? Is it visually appealing? Michael: “It doesn’t look like it’s that good of quality. The font looks like it’s from Microsoft Word. The book doesn’t really look like a book.” Sydney: “The menu is a little odd, but comparable. It’s easy to navigate, but the book flipping closed and open again every time I click a sword is annoying.” Lucas: “Basic menu layout. Easy to navigate. However, the flipping of the book is slightly confusing.” Derrick: “Easy to navigate.” Question 2: What does the title make you think is the basis of the game? Michael: “He (the main character) goes on a hero's journey and becomes a legend. It looks like a fantasy game.” Sydney: “It makes me think of a knight or someone who was legendary once and lost his power?” Lucas: “The main character was great and had power, then lost everything”

11

Derrick: “Medieval times game” Question 3: Does the naming convention match other game title menus? Is it understandable? Michael: “No, normal games aren’t going to have the same setup. There will be sub-categories and it will ask you to change the settings.” Sydney: “Yes. I understood what was happening with the menu, and it was understandable because of it being set up like a book.” Lucas: “Yes, for the most part. New game, Continue, and Quit are all you need to see.” Derrick: “It doesn’t match any games I’ve played and is understandable.”

Exploratory Questions

Prologue Question 1: Do you find the sound effects bothersome or amusing? Michael: “They’re quite amusing. (he laughs)” Sydney: “Funny, but play too often and can become annoying.” Lucas: “At first they are funny especially whoopsie doodle. However, they get bothersome when you have to jump a lot or fall down too much.” Derrick: “Both amusing and bothersome. Bothersome due to no realism and amusing because of the tripping saying ‘whoopsie doodle’.” Question 2: Is the level easy, in your opinion, to progress through and complete? Michael: “It was medium and straightforward. It took good problem solving because you had to be observant with the (switch) colors.” Sydney: “Moderate. Those vines sucked, otherwise it was enjoyable. A bit short though.” Lucas: “Yes after learning the controls and how the character moves. Also learning how close you can get to spikes helps.” Derrick: “The thorns were a difficult challenge because when you touch them you die immediately.”

Chapter 1 Question 1: Did you like controlling the sword instead of the player? Why or why not?

12

Michael: “It’s interesting. I’m glad they added something else to the game.” Sydney: “No! There was no real control. It felt more like a limp noodle than a character.” Lucas: “I liked controlling the player more. It is what I am more used to and has a lot more control.” Derrick: “I liked controlling the character more because there was more control than there was in the sword.” Question 2: How does the difficulty of the enemies compare to similar side-scrolling games you’ve played? Michael: “Most games, you jump on their head and they’re done. This one takes a lot more hits. It makes me very frustrated.” Sydney: “The only hard enemy was that last stupid block. There are no HP bars, so I had no idea how good or bad I was doing against it. Otherwise they are comparable.” Lucas: “I think it is normal for a side scroller with actual attacks, one or two hits is plenty.” Derrick: “Very easy to obliterate.”

Chapter 2 Question 1: What did you find most interesting about this level? Why? Michael: “”It wasn’t really interesting” Sydney: “The long jump with the slide involved. It wasn’t fun, but it was interesting because it was unusual.” Lucas: “Nothing really. The level was mainly walking and kind of boring.” Derrick: “This level had more obstacles in it and had no enemies to kill. It was more of an obstacle course which gave it more of an adventure game than other games.” Question 2:What was the most difficult part of this level? Why? Michael: “After the second jump point, I would land, but the character would slide off the platform. The momentum would keep him going. Sydney: “I didn’t find it difficult. It was the easiest so far, and thankfully there weren’t any thorns!” Lucas: “Timing the jumps on the smaller platforms because of the narrow window you have in order to land right.” Derrick: “The long jump after the uphill I thought was the most difficult part of the level because the character ran a lot slower going up the hill than he did running on flat ground which is accurate but made it more of a challenge to jump the distance.”

13

Chapter 3 Question 1: How difficult or frustrating was it to dodge the cannonballs? Michael: “Oh my gosh, that was max frustration because it was, like, impossible. It was so annoying.” Sydney: “Very frustrating. I can’t emphasize that enough, because it made the whole level unbearable and basically impossible to enjoy.” Lucas: “Very frustrating until I figured out how to get on top of the platforms.” Derrick: “Extremely difficult and very annoying.” Question 2: Did you enjoy the “Kill All Enemies” portion more or less than the other portions of the game? Why? Michael: Michael did not reach this part of the level Sydney: “It was interesting to be held back by kills, but I already kill most enemies so no big change for me.” Lucas: “I enjoyed this part less because the fireball creatures were too hard to hit, I would have rather ran past them.” Derrick: “Yes I did enjoy it very much because it was very entertaining to kill all the enemies.”

Directive Questions (Development Team) Question 1: How was the level difficulty scaling or difficulty change from level to level? Michael: “The levels went like this: Easy, Easy, Medium, then SUPER difficult.” Sydney: “The third level was easy, but the others were difficult in comparison. That final level sucked because it was just TOO difficult.” Lucas: “The first level was the hardest for me just because I had to learn the controls. The middle two levels were easy, and the last level was pretty hard, but at least I knew the controls by then.” Derrick: “The first three level were moderate and then jumped up to extremely difficult on the 4th level.” Question 2: How did the controls feel during each level? Michael: “There is a lot to remember from jumping to the dash. The controls are weird for gaming, but normal for standard typing.” Sydney: “Levels 1&3 felt normal, but controlling that sword was bothersome and the controls for it felt weird and not very enjoyable.” Lucas: “The walk button should be in a different location and the sword movement and timing is confusing.”

14

Derrick: “The blade dash feels kind of slow to respond especially when trying to dodge those cannon balls.”

Game Bugs Encountered by Playtesters

Tester, Issue #, Level Encountered, Description Michael, #1, Level 4 (Chapter 3): After the player died at the cannonball gauntlet on the 24th time, his character became stuck when he encountered the first large cylinder enemy. The enemies around him were still moving, but he could not jump, move forward or move backward. Sydney, #1, Level 1 (Prologue): As the player is jumping down to the floor, before jumping up platforms to reach the double wall of thorns portion of the game, they can often get caught on collision boxes while falling. This happened several times to Sydney during her play testing session. Sydney, #2, Level 1 (Prologue): The finish level “cloud” that marked the end of the level, when entered, crashed the game. This happened twice in a row, then worked the third time. Sydney, #3, Levels 2&4 (Chapter 1 & 3): The character ragdoll will go out of control and his body parts will fly around the screen randomly. This happened several times after Sydney hit the “esc” key to access the pause menu, and sometimes when she would respawn. Sydney, #4, Level 3: The character infinitely jumps when moving up and down the staircase leading into the giant open arena-looking area. This was noticed by Sydney, and was quickly commented on.

15

Data Collection

Controller Trollers Test

Subject Level Time to

complete lvl

Times Died

Death by hitting thorns

or spikes

Death by falling in

a Pit

Death by swinging

pendulum/cannon

Death by

falling object

Death by

enemies

Enemies Defeated

Times fell over

Number of Coins

Collected

Objects Destroyed

(trees, rocks, crates,

barrels)

Jeff H. Prologue 6:32 5 4 0 1 n/a n/a n/a 12 n/a n/a

Jeff H. Chapter 1 20:14 4 0 0 n/a 0 4 46 n/a 79 26

Jeff H. Chapter 2 4:42 4 n/a 4 n/a n/a n/a n/a 8 n/a n/a

Jeff H. Chapter 3 27:10 32 2 1 26 0 3 53 n/a 103 37

Kyle D. Prologue 15:24 36 30 4 2 n/a n/a n/a 12 n/a n/a

KyleD . Chapter 1 n/a 54 14 34 n/a 0 3 9 n/a 14 13

KyleD . Chapter 2 4:48 3 n/a 3 n/a n/a n/a n/a 6 n/a n/a

KyleD . Chapter 3 23:47 34 0 4 24 6 4 22 0 37 14

Dakota D. Prologue 5:20 3 1 1 1 n/a n/a n/a 18 n/a n/a

Dakota D. Chapter 1 14:10 7 0 4 n/a 0 3 27 n/a 68 28

Dakota D. Chapter 2 10:15 1 n/a 1 n/a n/a n/a n/a 4 n/a n/a

Dakota D. Chapter 3 15:45 20 2 1 17 0 0 16 0 25 3

Jon C. Prologue 6:45 3 2 1 0 n/a n/a n/a 5 n/a n/a

Jon C. Chapter 1 16:14 6 0 0 n/a 0 6 7 n/a 57 20

Jon C. Chapter 2 3:41 4 n/a 4 n/a n/a n/a n/a 6 n/a n/a

Jon C. Chapter 3 21:34 35 5 2 23 0 5 35 0 48 19

16

Test Subjects Test Subject Level Time to

complete lvl

Times Died

Death by hitting

thorns/spikes

Death by falling in a

Pit

Death by swinging

pendulum/cannon

Death by

falling object

Death by enemies

Enemies Defeated

Times fell over

Number of Coins

Collected

Objects Destroyed

(trees, rocks)

Michael Prologue 8:29 5 1 2 2 n/a n/a n/a 9 n/a n/a

Michael Chapter 1 15:06 4 0 3 n/a 0 1 51 n/a 60 16

Michael Chapter 2 6:02 8 0 8 n/a n/a n/a n/a 8 n/a n/a

Michael Chapter 3 18:31 40 6 2 24 0 8 8 n/a 14 49

Sydney Prologue 43:29 51 43 5 3 n/a n/a n/a 81 n/a n/a

Sydney Chapter 1 48:14 26 1 18 n/a 0 7 26 n/a 71 14

Sydney Chapter 2 9:36 4 0 4 n/a n/a n/a n/a 13 n/a n/a

Sydney Chapter 3 52:36 62 7 10 40 0 5 32 n/a 44 19

Lucas Prologue 20:24 17 14 2 1 n/a n/a n/a 30 n/a n/a

Lucas Chapter 1 21:22 8 0 5 n/a 0 3 42 0 91 31

Lucas Chapter 2 7:49 7 n/a 7 n/a n/a n/a n/a 13 n/a n/a

Lucas Chapter 3 21:56 28 5 2 20 0 1 30 n/a 29 8

Derrick Prologue 14:44 18 15 2 1 n/a n/a n/a 25 n/a n/a

Derrick Chapter 1 25:30 10 1 1 n/a 0 8 64 n/a 47 20

Derrick Chapter 2 5:29 2 n/a 2 n/a n/a n/a n/a 10 n/a n/a

Derrick Chapter 3 38:29 49 11 5 33 0 2 61 n/a 40 60

17

Graphs of Data Acquired

18

Issues Encountered by Testing Team

Issue (Jeff) In the Prologue level, when the player encounters the first floor switch, the character gets stuck on the floor and cannot smoothly walk onto the switch. The player has to jump to move and easily avoids the swinging pendulum from ever triggering.

Type(QA or Gameplay)

QA

Severity (Critical, Medium, Low)

Medium

Reproduction steps if QA

In the Prologue level, when the player encounters the first floor switch, the character gets stuck on the floor and cannot smoothly walk onto the switch. The player has to jump to move and easily avoids the swinging pendulum from ever triggering.

19

Issue (Jeff) In Chapter 1, for the rabbit boss encounter, the character can jump high (out of the screen) in some instances while close to the exit.


QA


Medium


During the rabbit encounter, move the character close to the right hand wall. Jump using space bar, and double tap “W”. .

Issue (Jeff) In Chapter 3, dying before “The Barricades…” checkpoint places the player in front of the checkpoint.


QA

20


Medium


Die from the cannonball while the cannon is on the screen. The character is placed at “The Barricades” checkpoint.

Issue (Jon) You can get stuck in mid air when you try to ground slam and grapple down at the same time.


QA


Critical


Character first jumps and then double taps S and press K. It happens when there is an object for ground slam to destroy.

21

Issue (Jon) There is a spot where a checkpoint is right before a lengthy dialog. When you die you are forced to go through that dialog again and can be slightly frustrating when you die multiply times in that one spot.


Gameplay


Medium

Solutions Steps if Game Play Issue

I can think of two solutions for this, one being to delete the cutscenes after it plays. The other is to move the check point after the cutscene.

Issue (Kyle) You are routinely caught on collision boxes near platforms. This can cause the player to glitch through the map floor or to fall into pits when trying to free themselves. Happens often enough to be detrimental to gameplay.


Gameplay


Medium


Go back into editor and fix collision boxes to better match blocks in game. Or edit the character controllers ability to climb slopes, in other words, make his slope climb lower to keep his avatar from trying to climb up.

22

Issue (Kyle) The game crashes if you reach the finish cloud on Chapter 2 before the voice-over ends. This has happened 100% of the times tested.


QA


Critical


Finish Chapter 2 and run to the finish point before the voice-over stops.

23

Issue (Dakota) The grapple mechanic is slightly touchy, sometimes it will connect and sometimes it won’t.


Gameplay


Medium


Jump and double tap “W” while a platform is above the character.

Issue (Dakota) The second cannonball area of chapter three is very difficult to get past due to the size of the cannonballs and the height at which the character can jump. This could cause players to get frustrated.


Gameplay


Medium


A solution would be to try scaling down the cannon ball size slightly or making it easier to get onto the platforms.

25

Analysis

Specific areas that are common points of frustration/confusion for all testers

➢ In Prologue and Chapter 2, all players occasionally fell over due to pressing movement keys multiple times. There was no clear reason why falling over was a mechanic because it didn’t appear to have any impact on the gameplay

➢ In the prologue level, the double vine/spike wall was overly difficult because of loose controls and the reaction time of the character in free fall.

➢ When initially starting gameplay, there is no movement instructions. Several of the players tried using the keyboard movement arrows and mouse clicking, but did not know what to do after those had no effect. None of the players thought of pressing ESC to look for movement instructions.

➢ The cannonball sections in the final level were easily the most frustrating part for all testers. The size of the cannonballs, the frequency, and the speed at which they travel where all points of frustration for testers and a game stopper for one tester.

➢ All players reported being confused at the beginning of Chapter 1 because there was no explanation on why they starting controlling a sword instead of the player.

➢ Bunny Boss health was not visible and players reported the time between phases took too long. The small flying cubes were higher than the character and unable to be Riposted, which caused additional frustration.

Level Scaling/Difficulty

➢ 73 of the 91 (81%) deaths in the prologue were due to thorns. ➢ 137 of the 215 (64%) deaths in Chapter 3 were due to cannonball fire ➢ Chapter 2 had the least amount of deaths with only 16 or 6% overall. ➢ Coin collection dropped dramatically for 3 out of 4 playtests due to players

realizing they can run through enemies without significant damage Mechanics Players Didn’t Use During Gameplay

➢ Bloodlust (B) ➢ Walk (Left Shift)

26

Player Emotion During Each Level Prologue

➢ Enjoyment: When reviewing the first book page ➢ Laughing: Sound effects and voice over ➢ Interested: Throughout ➢ Engaged: Throughout ➢ Annoyed: Double thorn wall drop ➢ Confused: Character movements and controls

Chapter 1

➢ Confused: why character was now a sword dragging a body ➢ Frustrated: a lot of combat keys to remember, but only brief tips

Chapter 2

➢ Bored: No meaningful choices throughout the level ➢ Frustrated: Character slides and controls are loose ➢ Enjoyable: No difficult mechanics to deal with

Chapter 3

➢ Frustrated/Rage: Cannonball portions ➢ Annoyed: Character controls as the sword ➢ Frustrated: Character movement was not consistent (dash, double “W” +

jump)

27

Recommendations

1. In the Prologue, when the player initially begins the level, explain to them how they can move the character. Other abilities weren’t used by the testers, such as Walk and Bloodlust. You should add something that points the player to the ESC menu, which contains the movement controls. There are also no instructions on how to pick the character up after they fall over.

2. For the pop-up window instructions, they should be permanent fixtures in the level. If you do keep the pop-up window, we suggest making the font larger so it is easier to read on smaller screens. Different windows also require different keys to close, but by normal standards would be “Press any key to continue”.

3. Unless falling over (Prologue/Chapter 2) has some use in the future, we feel that it’s not consistent/standard with other side scrolling game mechanics.

4. In Chapter 3, the cannonballs are too large, move too fast, and give the user little time to react to them without memorization after excessive deaths. We suggest that you either scale them down and slow their movement or simply have them do significant damage instead of instantly killing the character.

5. In the Prologue, for the drop down wall with thorns on either side, we recommend that you reduce the size of the extruding thorns to give players a better chance at successfully passing through in two or three tries.

6. A health bar for the bosses would be useful and provide the player visibility of the bosses status.

7. A “Settings” menu should be added to the Title Screen allowing the character to toggle sound effects, text size, difficulty and music (when added).

28

Post Mortem What went right

For player testing, we felt that having a procedure helped us set up the station and work with our play testers throughout the sessions. Having a solid foundation that the team prepared previously helped us focus more on active listening and note taking. The team also stayed in constant communication with one another throughout the week, allowing us to stay up-to-date on our individual progress. What went wrong

We had to wait till mid week for playtesting due to the additional levels that were added and not discussed in previous week. The team also had a hard time finding testers that fit the required demographic, which set us back on completing our data tables and graphs.

What We Would Change

We would have liked a little more advanced notice on the need for outside playtesters in week four. This would have helped the team contact potential playtesters sooner and would have allowed a smooth approach to testing.

For our testing setup, it would have been nice to have two cameras set up. One facing the tester, the other facing the game. Both could have recorded more of the players emotions during different periods in each level.

We would have liked to have had a few more testers so we could have had

larger data pool to review. One of the three testers had a significantly harder time completing several of the levels. It would be interesting to see how many players out of ten or twenty would have had the same issues.

29

References Nielsen, J. (January 1, 1995). 10 Heuristics for User Interface Design. Retrieved

from: http://www.nngroup.com/articles/ten-usability-heuristics/

Once Legendary [Computer Software]. (2015) Zulu: Team.

Screenshots (2015). Once Legendary [Computer Software]. (2015) Zulu: Team.

Zulu, T. (2015). Once Legendary GDD (Vol. 1, pg. 1-9).

controller trollers test report 1115

Data & Analytics