Test repository#

Test repository acts as a storage of test documentation and is bound to a particular project.

Repository contains a collection of test cases, logically grouped into (mapped to) test suites. Suites and cases can be browsed via test cases view (Test repository → Test cases).

Test cases view combines three horizontal columns: test suites tree, test cases list and test case preview (displayed after clicking on a particular test case).

Shared steps are also part of the repository and can be accessed via a dedicated view (Test repository → Shared steps).

Working with test suites#

Test suite is a collection of test cases logically grouped by certain criteria (e.g. belonging to the same functional modules/areas of the application under test).

Info

Zebrunner supports nested suites allowing to store test documentation in a tree-like structure. You can put a suite into another suite either while creating and editing it, or by using the drag-n-drop functionality on a test cases view.

In this section, you'll find the information on how to create, update and delete test suites.

Creating a test suite#

To create a new test suite, go to the Test repository → Test cases grid and perform the following steps:

Click +Suite
Enter the suite details and select a parent suite
Click Create

The new test suite will appear on the Test repository → Test cases grid. You can now add new test cases to it.

Editing a test suite#

To edit an existing test suite, go to the Test repository → Test cases grid and perform the following steps:

Hover over the test suite and click the Edit
Make the necessary changes to the test suite fields
Click Save

The updates will be applied to the test suite.

Deleting a test suite#

To delete a test suite, go to the Test repository → Test cases grid and perform the following steps:

In the test cases column, hover over the test suite and click the Delete . The deleting dialogue will appear on the screen
Click Delete

Info

Deleting a suite will permanently delete the suite itself and all its test cases.

Info

Professional and Enterprise plans at Zebrunner Test Case Management also include a trash bin, enabling soft deletion of test cases. For more info on the trash bin, please refer to the corresponding section of the documentation.

Working with test suites#

Test case is a sequence of steps describing how to validate a particular test objective.

In this section, you'll get an overview of test cases management in Zebrunner.

Test case view#

There are two ways users can view test cases in the test repository:

Sliding view: In this view, test case data is displayed as a third column in the test repository next to the test case list. This view is convenient for quick view and is optimal for simple test cases.

Modal view: In this view, test case data is displayed in an expanded view fitting more data. This view is recommended for large and complex test cases.

You can change the way your test cases are displayed on the grid by clicking the Switch test case view icon in the top right corner of a test case.

Both views display data in tabs:

General tab contains core test case fields (such as description, steps, pre- and post-conditions)
Properties tab contains important flags and attributes, such as priority, author, automation state, etc.
Attachments tab contains files linked to the test cases (e.g. images)
Executions tab contains the last 10 test case executions (both manual and automated; ordered from latest to oldest) with test configuration details.

Creating a case#

To create a new test case, go to the Test repository → Test cases grid and perform the following steps:

Click +Case
Fill in mandatory fields and select a parent suite
Click Create

Info

Test case must always belong to a particular test suite, so make sure that there is at least one suite created for this project.

Once created, the test case will appear in the specified test suite.

Tip

You can also quickly add new test cases via the +Create quick case by only specifying the title (remaining fields will be set to defaults). It may be handy for drafting large amounts of new test cases before proper detalization.

Editing a case#

To edit a test case, go to the Test repository → Test cases grid and perform the following steps:

Hover over the test case to edit and click the Edit to the right of the case title
Make the necessary changes to the test case fields
Click Save

The updates will be applied to the test case.

Alternatively, a test case can be edited from a test case view by clicking ⋮ → Edit.

Cloning a test case#

In order to reduce time spent on creating similar test cases, you can clone them and later update according to your requirements.

To clone a test case, go to the Test repository → Test cases grid and perform the following steps:

Hover over the test case and click the Clone icon to the right of the case title
In cloning dialogue, select the parent suite and change the title if needed
Click Clone

The cloned test case will be added to the parent suite as the last one in the list, together with all the info inside it except for the execution history.

Alternatively, a test case can be cloned from a test case view by clicking ⋮ → Clone.

Deleting a case#

To delete a test case, go to the Test repository → Test cases grid and perform the following steps:

Hover over the test case to and click the Delete to the right of the case title
Confirm the action by clicking Delete in the dialogue

The test case will be permanently removed.

Alternatively, a test case can be deleted from a test case view by clicking ⋮ → Delete.

Searching and filtering cases#

Test cases can be searched by entering the query into the corresponding input on the Test repository → Test cases grid.

Filtering is supported for test case attributes - corresponding controls can be selected by clicking +Add filter button. In addition, a special attribute called “Last update date” is available for filtering, allowing users to define date ranges for any update made to the test case.

Note

Depending on the attribute type, it is possible to apply exact match predicates (combining multiple for the same attribute with a logical OR condition) or ranges (in case of dates). All the filters applied will be combined using a logical AND.

Switching repository view#

It is possible to switch between different repository views, allowing you to select the one that works best for you. In order to do that, you can use the switcher located in the top right corner of the Test repository → Test cases grid.

The following views are available:

Tree view: In this view, in the test cases panel of the repository you can see all your suites and cases as one large tree.

Suite view: In this view, in the test cases panel of the repository you can only see selected suites and cases that are direct children of that suite.

Suite with a sub-suites view: In this view, in the test cases panel of the repository you can only see selected suites and cases that are direct children of that suite, all its nested suites (sub-suites) and their test cases.

Importing test cases#

Zebrunner allows you to import test cases from 3rd party sources using .CSV files. It is currently possible to import a .CSV file produced by TestRail or a .CSV file with a predefined set of columns (the so called Generic CSV file).

Warning

It is strongly recommended to review the subsection corresponding to the source of your .CSV file before starting the import. These subsections describe the structure that your .CSV file must follow in order to be successfully imported.

Also, it is a good idea to create a separate project to test how the import of your file works. Only after making sure that everything is imported correctly, you should import the cases into desired project.

To import test cases from a .CSV file, you need to perform the following steps:

Go to the Test repository → Test cases grid
Click Import CSV icon in the top-right corner
Choose the source of the CSV file you are about to import
Select the target suite where you want to import the test cases
Choose a .CSV file with test cases from your computer or drag and drop it to the upload area
Click Import

The import may take up to several minutes depending on the number of test cases in the file. Zebrunner does not support partial imports: either all test cases or none will be imported.

TestRail#

Before starting the import, it is recommended to examine what columns are in your .CSV file. For this, open the .CSV file using any plain text editor, such as Notepad or TextEdit. This is particularly necessary if your TestRail project has custom fields or fields with custom names.

Warning

We don't recommended opening the .CSV file in handier or more advanced tools such as Excel. These tools can corrupt the structure of the .CSV file and make it unsuitable for import.

TestRail .CSV columns#

The .CSV file must contain at least 2 columns for the import to work:

Title - the title of an imported test case. The value of this column is used to distinguish test cases when the .CSV file includes separated steps on separate rows
Section Hierarchy - test suite in which the test case should reside. The value of this column may contain the '>' symbols. With it, TestRail reflects nesting of the test suites. If the test suite does not exist in Zebrunner, it will be created

There are a few other columns recognizable by Zebrunner:

Priority - priority of the test case. If a priority from the .CSV file does not exist in Zebrunner, it will be created. If the file does not contain this column, test cases will be assigned a default priority
Preconditions - preconditions of the test case
Section Description - description of the test suite. This value is used along with Section Hierarchy column when creating the test suite
Steps - column with the test case steps definition. Usually, the steps are numerated. Note: according to the default TestRail behaviour, there can be two columns with this name
Expected Result - column with step expected results. This column is processed only together with Steps column
Steps (Step) - contains step actions only
Steps (Expected Result) - contains step expected results. This column is processed only together with Steps (Step) column

If your .CSV file does not contain the columns listed above, but contains data matching the meaning of a listed above column, feel free to manually rename the column. This is especially valid for steps-related columns.

Your .CSV file may contain all the mentioned steps-related columns. It is a usual situation, so Zebrunner will do its best to extract data from those.

Troubleshooting#

Most of the time, the import works correctly. But there are several anomalies (usually unpredictable) in how TestRail exports data. This section describes some of them and gives you tips on what to do if this happens to you.

Why have my test cases been imported without steps?

Please check if your .CSV file contains valid column names. To learn more about recognizable column names, refer to TestRail .CSV columns.

Why do my imported steps have corrupted images?

By nature, .CSV files are regular text files. They were created as a human-readable column-structured data format, but not as a mechanism to transfer data between computing systems. Images in turn are binary data. While technically possible, it is not natural to encode images inside a .CSV file. That is why TestRail does not export images inside a .CSV, but only provides a relative path to the image within the TestRail workspace. Because of this, Zebrunner is not able to resolve TestRail images for imported cases.

Steps of my test cases are not properly imported. What to do?

When it comes to exporting test cases, TestRail is not usually reliable. We observed cases when the structure of the file with the same exporting data had been changed over time.

There is no a single solution to this, but it usually helps to experiment with the set of exporting steps-related columns. E.g. export only Steps column or Steps and Expected Result. In some cases, export only the Steps (Step) and Steps (Expected Result) is the best option. Also, you can try to enable or disable the "Include separated steps on separate rows" option during the export from TestRail.

Why have only the first steps of my test cases been imported?

There are some anomalies in how TestRail exports test cases and this is one of them. In this case, you can try to play around the set of exporting steps-related columns. Usually, this issue is fixed by not exporting the Steps (Step) column.

Import failed with the "Import failed" message

This can be due to many reasons, e.g. incorrect file structure. Please contact support@zebrunner.com for further assistance.

I did everything I could, but it still doesn't work. What should I do?

In this case, feel free to contact support@zebrunner.com for further assistance.

Generic CSV#

By nature, .CSV files are regular text files. They were created as a human-readable column-structured data format, but not as an mechanism to transfer data between computing systems. The very first line in such a file is a header line: it contains a list of column names. All subsequent lines starting from the second one are rows with data.

Since the list of columns can be arbitrary, Zebrunner exposes strict requirements for the column names of Generic .CSV files. Before you starting the import, it is recommended to examine what columns are in your .CSV file. For this, open the .CSV file using any plain text editor, such as Notepad or TextEdit.

Warning

We don't recommended opening the .CSV file in a handier or advanced tools such as Excel. These tools can corrupt the structure of the .CSV file and make it unsuitable for import.

As of now, Zebrunner recognizes the following columns (columns with other names will be ignored):

Title - mandatory. The title of the test case
Description - description of the test case
Suite - mandatory. The name of the test suite this test case belongs to. It is possible to reflect the nested structure of the target test suite. For this, parent and child suites must be divided by the '>' symbol. If there is no target suite in Zebrunner, then it will be created
Suite Description - description of the test suite. This value is used along with the Suite column when creating the test suite
Priority - priority of the test case. If a priority from the .CSV file does not exist in Zebrunner, it will be created. If the file does not contain this column, test cases will be assigned a default priority
Automation State - automation state of the test case. If a value from the .CSV file does not exist in Zebrunner, it will be created. If the file does not contain this column, test cases will be assigned a default automation state
Deprecated - specifies if the test case should be marked as deprecated. Positive values are +, 1, t, true, yes
Draft - specifies if the test case should be marked as a draft. Positive values are +, 1, t, true, yes
Preconditions - pre-conditions of the test case
Postconditions - post-conditions of the test case
Step - column with a test case step's actions. The column may either contain a numerated list (using 1. numbering) or one step per CSV file row (in case when CSV file includes separated steps on separate rows)
Expected Result - column with a test case step's expected results. The column may either contain a numerated list (using 1. numbering) or one step per a .CSV file row (in case a .CSV file includes separated steps on separate rows)
Steps - column containing the list of test case steps and expected results. The list should be numerated (using 1. numbering). Step's action and expected result will be divided by the Expected Result: string

If your .CSV file contains Step along with Expected Result columns and the Steps column at the same time, the Step along with Expected Result takes precedence when the file is processed.

If your .CSV file does not contain the columns listed above, but contains data matching the meaning of a listed above column, you can manually rename the column.

If you have difficulty importing the .CSV file, please contact support@zebrunner.com for further assistance.

Customizing the test case layout [Professional+]#

It is possible to alter the test case layout used for a particular project. You can enable or disable predefined test case fields and provide additional (or change default) values for some attributes. Changes to the layout will be applied for a current project only, meaning that every project can have its own layout.

Info

You need to be a project Administrator/Manager to manage test case fields.

In order to alter the test case layout, go to the project Settings ( in the sidebar) → Test case fields. From there, you can enable or disable certain default fields (excluding the Title that cannot be disabled).

Info

Disabled fields will disappear from the test case views, corresponding filters will disappear too. Nevertheless, disabling a field doesn’t delete its values, so it’ll still be available if the field is turned on later.

Additionally, it is possible to modify (edit predefined and add new) values (and icons for those) for Priority and Automation State. To do that, click on the next to the field name. The view will appear, allowing you to add, alter or re-arrange existing possible values, as well as select the Default value (it’ll be automatically selected while creating a test case until changed to another value manually).

Info

Please note that the order of values is taken into account when those fields are part of sorting conditions for test cases (for instance, for the Priority field).

Using the trash bin [Professional+]#

Deleting test cases from the test repository does not remove them completely from the system. Test cases are first placed into a trash bin from where you can restore them or purge completely.

You can access the Trash bin from test cases grid by clicking the button at the top-right corner of the page.

Restoring cases#

To restore one or several test cases, go to the Trash bin page and perform the following actions:

Select the cases that you want to restore and click Restore button
Select the parent suite to place the restored test cases
Click Restore again

Alternatively, you can restore all the test cases at once by clicking Restore all at the top of the page.

Purging cases#

To purge one or several test cases, go to the Trash bin page and perform the following actions:

Select the cases that you want to purge and click Purge button
Confirm your action by clicking Purge again

Alternatively, you can purge all the test cases at once by clicking Restore all at the top of the page.

Using shared test steps#

Shared steps represent a sequence of steps (or just one step) that can be reused (shared) between multiple test cases. Usage of shared steps aims to reduce content duplication and simplify future maintenance of test cases.

Steps are always included into other cases as a group - if you modify a sequence later by adding more steps (or removing some of them) - the changes will be reflected in test cases that use these sequences.

Creating shared steps#

To create a new shared step, go to the Test repository → Shared steps grid and perform the following actions:

Click +Step
Fill out all necessary fields, add additional steps if needed
Click Create

The new sequence will be added to the grid. You can now include your shared steps to test cases.

Using shared steps in test cases#

To insert a shared step to a test case, go to the Test repository → Test cases grid and perform the following actions:

Navigate to an existing test case or start creating a new one
Under the steps, click +Shared step
Select the needed shared step from the list and click Add

Selected steps will appear in a test case creation/edit form. Proceed with more changes if needed and save the test case.

Info

After adding a shared step to your test case, you can delete it by clicking the Delete beside it.

Info

Shared steps can be converted to a set of “regular” steps on test case creation. To do this, after adding a shared step to your test case, click the Edit to the right. This action will convert them to “regular” steps that you now can alter or re-arrange.

Editing shared steps#

To edit a shared step, go to the Test repository → Shared steps grid and perform the following actions:

Hover over the shared step to edit and click the Edit to the right of the step title
Make the necessary changes to the shared steps and click Save

The update will be applied to the shared steps (and reflected in all cases that use them).

Deleting shared steps#

To delete a shared step, go to the Test repository → Shared steps grid and perform the following actions:

Hover over the shared step to and click the Delete to the right of the step title.
Confirm action by clicking Delete

The shared step will be deleted from the system, as well as from all the test cases where they have been added.