10 tips for testing VR games
In the recent years, VR entered the entertainment stage in a big way. Not only that we are getting bigger choice of VR sets but we also have more and more apps and games available. They offer fantastic new levels of entertainment and something called “presence” – illusion that you are somewhere else, in amazing new worlds, instead in the room with device on your head. But, how do we really test that? Good testing practices are easy to adopt cross any platforms, but VR has its own, very unique features, that require special consideration. Technology is so new, that some things we just have to figure out as we go. Following these 10 tips should help you get a good start with testing VR games:
1. With VR, it’s not enough to only “test does the software work”. Games on VR are completely immersive and let you play out the games in a new way, using only your eyes, for example or using your hands like you would “in the real world”. You should experience presence in full and really feel totally immersed in alternative world of a VR game. As a tester, there is a challenge of “testing the experience” versus “testing the requirement”. You need to answer not only “does it work” but also “how does it feel” question.
2. Testing time is much more limited. How long can tester test with a VR set? Well, it can be anything from 10 to 30 minutes. It will depend on the game you are testing as well as on your testers affinity for VR. Some people get sick easier, while some are more resistant. Whatever you do, make sure that there is someone supervising testing – meaning actually watching the testers test, as it can help prevent injuries and ensures safety.
3. Some people can get sick much easier than others. VR world has its own “disease”, virtual reality sickness, which is very much alike motion sickness. While symptoms are practically the same, cause is somewhat different – visually-induced perception of self-motion, rather than real motion is the culprit for discomfort, headache, disorientation, nausea or even vomiting. So, you think, I will just use testers who have more resilient stomachs? Wrong – you should test the game on the most sensitive members on your team, as you want to see what is the bottom line and what the level of comfort the game has.
4. Keep your set clean. It’s not something you would worry too much with other types of hardware, it’s important with VR sets. You are having it on your face, covering your eyes and it can get sweaty really quickly. Clean your set and lenses every 15 minutes or so or whenever you feel that it’s needed. The more frequent, the better.
5. Your set is still quite heavy. There might be physical strain on your testers, not only from the motion sickness. This is also something you should keep on your mind when making testing schedule – crunching time might not be an option here.
6. Bug screenshot anybody? Your tester will need to have excellent written skills as in many cases, bugs will be about overall experience fail or something not feeling right in the virtual world. This can make bug reporting cumbersome and cause strife between coders and the QA team.
7. Reproduction rate of bugs. With VR not only that you want to try to reproduce the bug few times, you also want to have a few people reproducing the bug, especially if it’s a presence related bug or one about causing discomfort. With discomfort bugs, you would want to check it with your most sensitive as well as with your most resilient tester to get good idea about how discomforting it really is.
8. Count on the extra space needed. Your tester will move (with the headset on) around, not seeing his environment. You should make it safe and obstacle free and allow for enough movement space. Your tester should have at least about 1,5 m space around him to be able to do efficient testing. If your office/lab allows for more, even better.
9. As VR sets have very specific physical limitations, make sure that you allow for enough testing time. Testers will need to take frequent breaks. You will need to plan your testing accordingly and allow much more time for the testing than it would take on mobile or PC only.
10. Automation testing will not help you that much – at least not yet. While automated tests can do marvels for testing of the basic functionality, there are still no reliable solutions for testing VR games. Besides, VR experience is very immersive and it’s hard to imagine how that could be effectively tested without human involvement.
VR is super exciting and allows us to have so much more fun with the games. On the other hand, when “reproducing” reality, some things just don’t work out as well as planned. Testing VR poses new challenges to even most experienced teams – not only that you have to re-think your test cases and your test scope, but you also have to allow for much more testing time. That makes test planning really challenging and it might take quite some time before you really learn limits of your testing team and have good idea of their velocity.
Nikolina Finska is an author of “Modern Game Testing” book that is now available in Amazon and other major online retailers. She is also available for QA consulting and workshops. For availability and prices please contact info@nikolinawork.com
