The stakeholder identified a problem with the short-term retention of Heavy Metal Machines. They didn't know why, but the game had a high churn rate before the seventh match. So one significant research was initiated to find the reason behind this problem.
Disclaimer: This research involved several people and different areas. I will talk about the entire process since I participated in their definitions and the discussion of its results. But keep in mind that I was responsible for the usability tests.
We divided this research into five parts.
The first one was a game review analysis. We took all the reviews where the player had less than two hours of gameplay and separated them by subject to create a word cloud.
The second part was an interview with our most loyal players. Our objective was to see if they remembered the problems they had when started playing the game, or if they saw new users speaking about any barrier in the game. We didn't expect any results from this part, but it was too cheap not to be done.
The third part was the implementation of a survey at the end of the users' first matches. If they closed the game, the questionnaire would also open in their web browser.
The fourth part was to analyze the behavior of the players that would leave the game before the sixth match and analyze the players who would keep playing after it. With this, we would be able to see how different actions during the gameplay improves the chances to increase customer loyalty.
The fifth part was a usability test with new users. I was responsible for this one, and I started it recruiting people. I created a survey that aimed to find people that fit in our target audience but never had any contact with the game. This survey was shared by Facebook groups with a fake account to keep the results unbiased. After I got enough answers, I filtered the users and got in touch with them to schedule the test. During it, I introduce the players, gave them some assignments and analyzed how they did the tasks. In the end, I made some questions.
After all this, we found a hypothesis. The combat and the movement didn't work well together. New players had much difficulty to do both things at the same time, so they died a lot and quit. The match time was longer than expected because of this. Some times four times longer.
We formulated some theories on how to solve this problem, but the priorities of the project changed, and the stakeholders put it on hold.
We found some significant problems in the base gameplay, but stakeholders changed the priority of the project and put the task on hold.