There’s no doubt: the current state of affairs has multiplied entertainment consumption at home and, in turn, streaming platforms have grown exponentially this year. According to a report from PwC, Global Entertainment & Media Outlook 2020-2024, streaming platforms have witnessed a global profit increase of 26% during this pandemic, and it’s expected to keep on growing. Accessibility and endless supply of content has been key to lead the segment.
As a downside, this unprecedented demand could make platforms malfunction, decreasing streaming quality or even collapsing. How can we solve these issues through testing?
Testing to Fix Streaming Services
If we want to have a successful streaming platform, we need a team of testers that excels, achieving excellent platform performance. This is the case of Giselle Díaz and Stefany Rodriguez, intive testers for a very popular streaming platform in the US.
Which are the challenges faced when testing platforms? Giselle and Stefany tell us everything in this interview
1- How would you describe the testing process for a service such as streaming?
During the whole process, we’re constantly carrying out an overall verification of the main functions, guaranteeing that, with every release, the quality of the app isn’t affected, and filtering every error that might appear when a new functionality or update is to be released.
2- What parameters are taken into account to measure performance?
- Seamless navigation inside the app.
- All images should load as fast as possible.
- Minimum waiting time to reproduce content.
- No errors while content is reproduced for a significant amount of time, at least 3 to 5 minutes.
- No errors when interacting with video player (play, pause, fast-forward, place subtitles, etc.)
3- Which are the most common issues that users have to deal with?
They mostly have to do with:
- Long waiting times for video to start playing.
- Poor internet connection, so messages appear signaling network error.
- Subtitles which aren’t synchronized with audio.
- Problems when sending content to certain devices, such as Google Chromecast, or in the case of Apple users, from iPhone to AppleTV.
- Crashes, maybe due to network issues or memory leaks, which commonly appear in user reviews.
We’ve got many!
- When we need to download builds in iPhone or AppleTV, we use Testfligh.
- We also use Fiddler or Charles to filter and analyze logs.
- When it’s time to test API, we use Postman.
- Since we work with more than one TV channel, we’re now using New Relic, which makes it easy to monitor performance and errors among all channels.
- Lastly, for test automation, we use Katalon, Git and BitBucket.
5- What devices do you test in?
- For mobile devices, we’ve got Android and iOS.
- For OTT devices, we use FireTV, AndroidTV, Roku, and AppleTV.
6- What’s the most challenging part of this type of job? And what’s the thing you enjoy the most?
The whole learning and adaptation process when new procedures or tools are integrated can be quite challenging, since our responsibilities evolve day by day. However, we’re able to gain new knowledge, and that’s something that lets us grow in our field.
7- How’s the interaction with the development team?
Although our interactions are 100% remote because we don’t live in the same country, communication is fluent, so we don’t really hesitate to go and ask when we need to clarify certain scenarios to design and execute the necessary test cases.
Thanks a lot, Giselle Díaz and Stefany Rodríguez, for sharing your work and your experience with us!
We hope you’ve enjoyed the interview as much as we did. As always, we’ll read you in the comments!