The intiver school —you can find out more about it here— is a training space where quality analysts can try new tools and processes, among other things. The aim is to learn and share the new knowledge with the rest of the company. Today we’d like to discuss one of the latest tools we tried, a Jira add-on: Zephyr.
As we said, Zephyr is a Jira add-on useful for creating, importing, executing, tracking, reporting and documenting test cases, as well as for linking them with their respective user stories.
It allows us to create a type of ticket called “Test” in the same way as we create any other ticket in Jira. It also allows us to identify which version and components are affected, add attachments, assign the ticket a priority, add tags to it, link it to an issue, etc.
The novelty is that Zephyr tests have fields to specify the stage we’re in (Test Step), the data needed for executing the stages (Test Data), and the expected results.
In the Argentina React Training Program we started to use Zephyr to track test cases and we carried out several experiments to see how we could best adapt it to our work routine. So we created a test battery following Zephyr’s redaction guidelines, and we generated a Jira ticket for each test case.
There are two ways to create a test: by using the “Create” button in Jira or with the “Create Zephyr Test” option in the User Story (or Sub Task). The advantage of the latter is that the test case will be automatically linked to that ticket, either the User Story or the Sub Task.
Running Test Cases
Once the test battery is created, we can begin the execution of the tests. But first, we need to create a Test Cycle for this version and the folders we’ll be using in it. We can create as many cycles and folders as needed.
Now the test case is ready for execution: we have to enter the ticket for that Zephyr test and click “Execute”. Here the flow will require us to link the execution to a Test Cycle. We select the version, the cycle, the folder and the person responsible for the execution.
In this screen, we can mark the status of each step: whether it passed or failed, whether it wasn’t executed or whether it’s in progress or blocked. In Zephyr we can leave comments, attach files, create a new bug or link to an already existing one. It also offers the possibility to create or link a bug to the entire test case rather than to just one particular step.
In the Zephyr test we can check the status, the cycle, the folders and the bugs of each execution. What are some of the benefits of this?
- We can see if the executions failed, and when.
- We can see the bugs linked to each execution.
- We can check the status of the bugs.
Zephyr displays all of this information in only one place, offering great time optimization.
Importing from Excel
An alternative for creating test cases is to import an Excel file. We select “Importer” from the menu and explore the options to select projects and types of issues. If we create an Excel file specifically for this purpose, we can match every column to the fields in the Zephyr test.
In that way, we can accelerate the process by leveraging the advantages of using an Excel file to migrate information already available.
Charts and Metrics
One of the reasons why we were encouraged to try Zephyr at the intiver school is that it enables to centralize data in one application.
In addition to helping to streamline processes, thanks to centralization we can obtain useful metrics and analytics. Zephyr has a basic summary showing the total number of tests and those that were and were not executed. We can also see two charts showing the tests’ date of creation, the executions of the last 30 days and a summary of tickets grouped by version, component and tag.
Lastly, Zephyr offers the possibility to create a custom dashboard of charts displaying the different metrics we need for evaluation.
To be completely honest, there are areas of improvements: Some of Zephyr’s functionalities didn’t turn out to be that useful, like the option to reorganize or categorize folders in the test cycles, and, on the other hand, it lacks (for now) features we’d love to have, like the possibility to clone test cases for batch processing. In spite of this, the review is positive:
- Zephyr allows us to carry out step by step executions and to visualize the whole process, so that we can document and evaluate. The entire team has access to all of the information right from the start, which optimizes communication and knowledge sharing of what’s being tested and how.
- It’s strongest point is traceability: We can visualize the entire process in a single application.
In sum, Zephyr is a tool worth using, we suggest you try it! To those who have already tried it, what’s your opinion on it?