The preliminary test was conducted on the 22nd March 2012, and consisted of four testers playing either the achievement or control version of our game. We conducted our testing over two days; firstly we tested one person on the first day to prove that our initial set-up of the testing was correct, and we could collect data through our testing. Then after doing this, we proceeded to conduct further pre-testing on the following day in the morning, before we started the real testing.

On our first day of testing, we had our first tester playing the achievement version of the game. He seemed to be an extreme tester from his observation and interview results. The tester tried to gain every achievement in the game, however due to our first set-up, the tester realised very quickly that they were being observed. Therefore we decided that instead of just having the observers and the testers, we would also add one of us conducting the interviews, and the other person near the testers as ‘technical assistance’. This is one thing we decided to do in the real testing. Also, one problem that occurred during the testing was for some reason we had technical difficulties taking the time on the ‘Mountain Pass’ track, so we had to add this to the observation sheet for the real test. In addition from doing this pre-test, the sample seemed to discover sections in our games with bugs where we hadn’t really tested.
On the second day of testing, we had three testers who each had an hour time-slot in which they could do the testing in. Two of the testers were playing on the achievement version of the game, and one tester was playing on the control version of the game. The testing went well, however the sample seemed to get very lost within our game so as a group we had to quickly come up with a solution for the real testing.

One thing we took from the preliminary testing and were happy with was the room and testing area in which we conducted the testing in; the barricade in-between the testers seemed to stop them from getting distracted from each other. In addition the machines we were using throughout were perfect at running our game fluidly, therefore not distracting the player from the game-play. In addition, conducting the interviews with the tester seemed to flow really well, and only took about 10 minutes for each tester, so we decided to stick to this method of qualitative data gathering.

One thing we changed first was the roles in our team; we thought that taking it in turns to be an observer would be simple, however it seemed very disjointed, therefore we decided that Shauni would be the main interviewer, Reece and Jonny would be the observers, and back-up interviewers if Shauni was busy with a tester, and lastly David was technical support for the testers in case something went wrong, but to also lead the sample away from the observers.
Some of the things we decided to change or add to our game for the real testing were, we decided to add signposts into the island level to help players navigate around the large area; this was due to our second pre-testers getting very lost within the game, and therefore getting frustrated with the game. Considering the length of our testing, our group decided that it would be a good idea to have a booking system for the testing, as this way we can maximise our time with testers, as well as having enough time  to conduct a quality interview. In addition, we believe that if the tester is aware of the one hour time-slot they will take their time and not feel pressured to ‘rush’ the game in anyway, also as a group we felt that this could eliminate the issue of testers not playing the game for as long as they would normally play a racing game for. Most of our testers will be booked into our time-slots, however if we have an empty booking we will go and find some testers around the university to help us with our testing.
Due to an error/bug with our game which we could not fix we decided to add some data collection through the observation sheet as the in-game statistics wouldn’t work for the Mountain Pass level timing, so we had to make sure that our observers took this information down on their sheet, and would use a timer to track this time. When a tester goes into the Mountain pass level entrance, the observer will get the timer ready and start it as soon as the race starts.
One of the last things we took from doing the testing was that using Xbox controllers instead of the keyboard seemed to prove very successful with the testers; they all had played some sort of video games console, and were used to the layout of the Xbox controller. All the testers picked up the controls very quickly, and got straight into playing the game very quickly.

Overall, we felt that our preliminary testing went very well and we were able to gather a lot of useful information from this.


GO TO WEEK 23 – Testing Achievement group

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: