Test repository

Test repository acts as a storage of test documentation and is bound to a particular project.

Repository contains a collection of test cases, logically grouped into (mapped to) test suites. Suites and cases can be browsed via test cases view (Test repository → Test cases).

Test cases view combines three horizontal columns: test suites tree, test cases list and test case preview (displayed after clicking on a particular test case).

Shared steps are also part of the repository and can be accessed via a dedicated view (Test repository → Shared steps).

Working with test suites

Test suite is a collection of test cases logically grouped by certain criteria (e.g. belonging to the same functional modules/areas of the application under test).

Info

Zebrunner supports nested suites allowing to store test documentation in a tree-like structure. You can put a suite into another suite either while creating and editing it, or by using the drag-n-drop functionality on a test cases view.

In this section, you'll find the information on how to create, update and delete test suites.

Creating a test suite

To create a new test suite, go to the Test repository → Test cases grid and perform the following steps:

1. Click +Suite
2. Enter the suite details and select a parent suite
3. Click Create

The new test suite will appear on the Test repository → Test cases grid. You can now add new test cases to it.

Editing a test suite

To edit an existing test suite, go to the Test repository → Test cases grid and perform the following steps:

1. Hover over the test suite and click the Edit
2. Make the necessary changes to the test suite fields
3. Click Save

The updates will be applied to the test suite.

Deleting a test suite

To delete a test suite, go to the Test repository → Test cases grid and perform the following steps:

1. In the test cases column, hover over the test suite and click the Delete . The deleting dialogue will appear on the screen
2. Click Delete

Info

Deleting a suite will permanently delete the suite itself and all its test cases.

Info

Professional and Enterprise plans at Zebrunner Test Case Management also include a trash bin, enabling soft deletion of test cases. For more info on the trash bin, please refer to the corresponding section of the documentation.

Working with test cases

Test case is a sequence of steps describing how to validate a particular test objective.

In this section, you'll get an overview of test cases management in Zebrunner.

Test case view

There are two ways users can view test cases in the test repository:

Sidebar view: In this view, test case data is displayed as a third column in the test repository next to the test case list. This view is convenient for quick view and is optimal for simple test cases.

Modal view: In this view, test case data is displayed in an expanded view fitting more data. This view is recommended for large and complex test cases. To make the process of navigating between test cases within the suite easier, we have included navigation controls within the modal window.

You can change the way your test cases are displayed on the grid by clicking the Switch to sidebar view / Switch to modal view icon in the top right corner of a test case.

Both views display data in tabs:

• General tab contains core test case fields (such as description, steps, pre- and post-conditions)
• Properties tab contains important flags and attributes, such as priority, author, automation state, etc.
• Attachments tab contains files linked to the test cases (e.g. images)
• Executions tab contains the last 10 test case executions (both manual and automated; ordered from latest to oldest) with test configuration details.

Creating a case

To create a new test case, go to the Test repository → Test cases grid and perform the following steps:

1. Click +Case
2. Fill in mandatory fields and select a parent suite
3. Click Create

Info

Test case must always belong to a particular test suite, so make sure that there is at least one suite created for this project.

Once created, the test case will appear in the specified test suite.

Tip

You can also quickly add new test cases via the +Create quick case by only specifying the title (remaining fields will be set to defaults). It may be handy for drafting large amounts of new test cases before proper detalization.

Editing a case

To edit a test case, go to the Test repository → Test cases grid and perform the following steps:

1. Hover over the test case to edit and click the Edit to the right of the case title
2. Make the necessary changes to the test case fields
3. Click Save

The updates will be applied to the test case.

Alternatively, a test case can be edited from a test case view by clicking ⋮ → Edit.

Cloning a test case

In order to reduce time spent on creating similar test cases, you can clone them and later update according to your requirements.

To clone a test case, go to the Test repository → Test cases grid and perform the following steps:

1. Hover over the test case and click the Clone icon to the right of the case title
2. In cloning dialogue, select the parent suite and change the title if needed
3. Click Clone

The cloned test case will be added to the parent suite as the last one in the list, together with all the info inside it except for the execution history.

Alternatively, a test case can be cloned from a test case view by clicking ⋮ → Clone.

Deleting a case

To delete a test case, go to the Test repository → Test cases grid and perform the following steps:

1. Hover over the test case to and click the Delete to the right of the case title
2. Confirm the action by clicking Delete in the dialogue

The test case will be permanently removed.

Alternatively, a test case can be deleted from a test case view by clicking ⋮ → Delete.

Searching and filtering cases

Users can search for test cases by entering the query into the corresponding input field on the Test repository → Test cases grid. It's possible to search by phrases from the case title, individual words or symbols within the case title, and by the case key. Please note that using double quotes is not supported as they are considered special characters.

Note

Search is case-insensitive.

The search query supports two wildcard operators:

• ? - matches any single character. For example, the ste?l query matches either steal or steel in the title, but not still or stel. The st??l query matches steal, steel and still.
• * - matches zero or more characters, including an empty one. For example, the pr*e query matches all words starting with pr and ending with e, such as pre, price, prize, prince, propose.

Filtering is supported for both system and custom test case fields - corresponding controls can be selected by clicking +Add filter button. In addition, a special attribute called “Last update date” allows users to define date ranges for any update made to the test case.

Depending on the field type, it is possible to apply exact match predicates (combining multiple for the same attribute with a logical OR condition), ranges (in case of dates), substrings (works in the same way as searching but for specific custom field of type String or Text). All the applied filters will be combined with a logical AND.

How Zebrunner handles searching and filtering under the hood

Step 1: Zebrunner splits an entered query (if present) by space character into a set of search tokens. The search tokens will be used to search in the case titles. If you need to search cases by phrase (a few words in specific order), escape the phrase in double-quotes. All the double-quotes in the middle of a resulted token will be ommited.

Step 2: Search tokens (if any) are combined with a logical AND into a single search predicate for the test case title.

Step 3: All search token conforming the test case key format (XXX-123) are translated into a search predicate for the case key. Then they are added to search predicate from Step 2 with a logical OR. In addition, all search token representing integers (e.g. 3174) are considered as part of the case key and are also added to the predicate with a logical OR. At the end of this step we derive the filtering criteria for the search query.

Step 4: All specified filters (if any) are added to the filtering criteria from Step 3 (if any) with a logical AND.

Example 1: Search query is case 431, no filters provided. This will be translated into the following predicate: (title: *case* AND title: *431*) OR key: *-431. As a result, users will receive all test cases containing case and 431 in their title, or the case whose key ends with -431.

Example 2: Search query is API-431 API-572 API-881, no filters provided. This will be translated into the following predicate: (title: *API-431* AND title: *API-572* AND title: *API-881*) OR key: API-431 OR key: API-572 OR key: API-881.

Example 3: Search query is user API-2 572, filter is Priority: Critical. This will be translated into the following predicate: ((title: *user* AND title: *API-2* AND title: *572*) OR key: API-2 OR key: *-572) AND priority: Critical.

Example 4: Search query is "user management", filter is Priority: Critical. This will be translated into the following predicate: title: *user management* AND priority: Critical. As a result, users will receive all critical test cases containing user management in their title.

Sharing test case

You can copy the link to a test case or test case ID. These buttons are designed to be visible only when the user hovers over the test case id. By simply hovering over it, the buttons will appear, allowing users to quickly copy the desired information to their clipboard for further use.

Copy Link to Test Case Button allows users to copy the link to a specific test case. This button is useful when users want to share the test case with others or refer to it later.

Copy Test Case ID Button enables users to copy the unique identifier of a test case. This ID can be helpful when referencing a test case within your project or when communicating with other team members.

Switching repository view

It is possible to switch between different repository views, allowing you to select the one that works best for you. In order to do that, you can use the switcher located in the top right corner of the Test repository → Test cases grid.

The following views are available:

Tree view: In this view, in the test cases panel of the repository you will see all your suites and cases as one large tree.

Suite view: In this view, in the test cases panel of the repository you will only see selected suite and cases that are immediate children of that suite.

Suite with a sub-suites view: In this view, in the test cases panel of the repository you will only see selected suite and cases, that are immediate children of that suite, together with all nested suites (sub-suites) and their test cases. Essentially, you will be seeing a sub-tree that starts from selected suite.

Bulk change actions

Zebrunner allows you to perform certain actions on test cases in bulk. Currently, it is possible to Move and Clone test cases across projects, as well as Delete test cases from a project.

Moving cases between projects

Moving cases between projects allows you to transfer cases from one project to another. In order to perform a move, user should have at least a role of Engineer in both (source and target) projects.

Test case layout mismatch note

Fields in the source project without a match (based on both name and type) in the target project will not be transferred during Move.

To move test cases, follow these steps:

1. Navigate to the Test repository.
2. Click the three-dot menu and choose Bulk change.
3. In the appeared modal window, select the Move option.
4. Select target project and suite to which the test cases will be moved (note, that suite should be created in target project beforehand, otherwise you will see a warning on UI and will not be able to proceed).
5. Choose the test cases you wish to move.
6. Confirm the action in the presented modal.

Once you confirm, you will be redirected to a new screen where you can monitor the progress of the move operation. Once the operation is completed, you can safely close the modal window.

Case creator and creation date

For the Move, case data will be transferred as-is, including information on creator and date when case was created.

Cloning cases between projects

Cloning cases between projects allows you to duplicate cases from one project to another. Cloning is treated as an operation, that creates brand new cases, using the cloned ones as a base. In order to perform a clone, user should have at least a role of Engineer in both (source and target) projects.

Test case layout mismatch note

Fields in the source project without a match (based on both name and type) in the target project will not be transferred during Clone.

Cloned cases note

Cloned cases will become brand new cases in target project and will not keep any connection with the cases, used for cloning.

To clone test cases, follow these steps:

1. Navigate to the Test repository.
2. Click the three-dot menu and choose Bulk change.
3. In the appeared modal window, select the Clone option.
4. Select target project and suite to which the test cases will be cloned (note, that suite should be created in target project beforehand, otherwise you will see a warning on UI and will not be able to proceed).
5. Choose the test cases you wish to clone.
6. Confirm the action in the presented modal.

Once you confirm, you will be redirected to a new screen where you can monitor the progress of the clone operation. Once the operation is completed, you can safely close the modal window.

Case creator and creation date

For the Clone, case data will be transferred as-is. However, creator and creation date will be set to user, who initiates cloning and current date respectively.

Deleting test cases from a project

The bulk deletion feature allows you to efficiently remove multiple test cases from a project at once. To perform a deletion, users must have a role of Engineer or higher.

To delete test cases, follow these steps:

1. Navigate to the Test repository.
2. Click the three-dot menu and choose Bulk change.
3. In the appeared modal window, select the Delete option.
4. Choose the test cases you wish to delete.
5. Confirm the action in the presented modal.

Once you confirm, you will be redirected to a new screen where you can monitor the progress of the deletion operation. Once the operation is completed, you can safely close the modal window.

Are cases deleted forever?

Subscribers on the Professional plan or above will have their deleted test cases sent to the trash bin, providing the possibility for restoration or permanent removal. For those on lower-tier plans, deleted test cases cannot be recovered and will be permanently removed.

Importing test cases

Zebrunner allows you to import test cases from 3rd party sources using .CSV files. It is currently possible to import a .CSV file produced by TestRail or a .CSV file with a predefined set of columns (the so called Generic CSV file).

Warning

It is strongly recommended to review the subsection corresponding to the source of your .CSV file before starting the import. These subsections describe the structure that your .CSV file must follow in order to be successfully imported.

Also, it is a good idea to create a separate project to test how the import of your file works. Only after making sure that everything is imported correctly, you should import the cases into desired project.

To import test cases from a .CSV file, you need to perform the following steps:

1. Go to the Test repository → Test cases grid
2. Click Import CSV icon in the top-right corner
3. Choose the source of the CSV file you are about to import
4. Select the target suite where you want to import the test cases
5. Choose a .CSV file with test cases from your computer or drag and drop it to the upload area
6. Click Import

The import may take up to several minutes depending on the number of test cases in the file. Zebrunner does not support partial imports: either all test cases or none will be imported.

TestRail

Before starting the import, it is recommended to examine what columns are in your .CSV file. For this, open the .CSV file using any plain text editor, such as Notepad or TextEdit. This is particularly necessary if your TestRail project has custom fields or fields with custom names.

Warning

We don't recommended opening the .CSV file in handier or more advanced tools such as Excel. These tools can corrupt the structure of the .CSV file and make it unsuitable for import.

TestRail .CSV columns

The .CSV file must contain at least 2 columns for the import to work:

• Title - the title of an imported test case. The value of this column is used to distinguish test cases when the .CSV file includes separated steps on separate rows
• Section Hierarchy - test suite in which the test case should reside. The value of this column may contain the '>' symbols. With it, TestRail reflects nesting of the test suites. If the test suite does not exist in Zebrunner, it will be created

There are a few other columns recognizable by Zebrunner:

• Priority - priority of the test case. If a priority from the .CSV file does not exist in Zebrunner, it will be created. If the file does not contain this column, test cases will be assigned a default priority
• Preconditions - preconditions of the test case
• Section Description - description of the test suite. This value is used along with Section Hierarchy column when creating the test suite
• Steps - column with the test case steps definition. Usually, the steps are numerated. Note: according to the default TestRail behaviour, there can be two columns with this name
• Expected Result - column with step expected results. This column is processed only together with Steps column
• Steps (Step) - contains step actions only
• Steps (Expected Result) - contains step expected results. This column is processed only together with Steps (Step) column

If your .CSV file does not contain the columns listed above, but contains data matching the meaning of a listed above column, feel free to manually rename the column. This is especially valid for steps-related columns.

Your .CSV file may contain all the mentioned steps-related columns. It is a usual situation, so Zebrunner will do its best to extract data from those.

Troubleshooting

Most of the time, the import works correctly. But there are several anomalies (usually unpredictable) in how TestRail exports data. This section describes some of them and gives you tips on what to do if this happens to you.

Why have my test cases been imported without steps?

Please check if your .CSV file contains valid column names. To learn more about recognizable column names, refer to TestRail .CSV columns.

Why do my imported steps have corrupted images?

By nature, .CSV files are regular text files. They were created as a human-readable column-structured data format, but not as a mechanism to transfer data between computing systems. Images in turn are binary data. While technically possible, it is not natural to encode images inside a .CSV file. That is why TestRail does not export images inside a .CSV, but only provides a relative path to the image within the TestRail workspace. Because of this, Zebrunner is not able to resolve TestRail images for imported cases.

Steps of my test cases are not properly imported. What to do?

When it comes to exporting test cases, TestRail is not usually reliable. We observed cases when the structure of the file with the same exporting data had been changed over time.

There is no a single solution to this, but it usually helps to experiment with the set of exporting steps-related columns. E.g. export only Steps column or Steps and Expected Result. In some cases, export only the Steps (Step) and Steps (Expected Result) is the best option. Also, you can try to enable or disable the "Include separated steps on separate rows" option during the export from TestRail.

Why have only the first steps of my test cases been imported?

There are some anomalies in how TestRail exports test cases and this is one of them. In this case, you can try to play around the set of exporting steps-related columns. Usually, this issue is fixed by not exporting the Steps (Step) column.

Import failed with the "Import failed" message

This can be due to many reasons, e.g. incorrect file structure. Please contact support@zebrunner.com for further assistance.

I did everything I could, but it still doesn't work. What should I do?

In this case, feel free to contact support@zebrunner.com for further assistance.

Generic CSV

By nature, .CSV files are regular text files. They were created as a human-readable column-structured data format, but not as an mechanism to transfer data between computing systems. The very first line in such a file is a header line: it contains a list of column names. All subsequent lines starting from the second one are rows with data.

Since the list of columns can be arbitrary, Zebrunner exposes strict requirements for the column names of Generic .CSV files. Before you starting the import, it is recommended to examine what columns are in your .CSV file. For this, open the .CSV file using any plain text editor, such as Notepad or TextEdit.

Warning

We don't recommended opening the .CSV file in a handier or advanced tools such as Excel. These tools can corrupt the structure of the .CSV file and make it unsuitable for import.

As of now, Zebrunner recognizes the following columns (columns with other names will be ignored):

• Title - mandatory. The title of the test case
• Description - description of the test case
• Suite - mandatory. The name of the test suite this test case belongs to. It is possible to reflect the nested structure of the target test suite. For this, parent and child suites must be divided by the '>' symbol. If there is no target suite in Zebrunner, then it will be created
• Suite Description - description of the test suite. This value is used along with the Suite column when creating the test suite
• Priority - priority of the test case. If a priority from the .CSV file does not exist in Zebrunner, it will be created. If the file does not contain this column, test cases will be assigned a default priority
• Automation State - automation state of the test case. If a value from the .CSV file does not exist in Zebrunner, it will be created. If the file does not contain this column, test cases will be assigned a default automation state
• Deprecated - specifies if the test case should be marked as deprecated. Positive values are +, 1, t, true, yes
• Draft - specifies if the test case should be marked as a draft. Positive values are +, 1, t, true, yes
• Preconditions - pre-conditions of the test case
• Postconditions - post-conditions of the test case
• Step - column with a test case step's actions. The column may either contain a numerated list (using 1. numbering) or one step per CSV file row (in case when CSV file includes separated steps on separate rows)
• Expected Result - column with a test case step's expected results. The column may either contain a numerated list (using 1. numbering) or one step per a .CSV file row (in case a .CSV file includes separated steps on separate rows)
• Steps - column containing the list of test case steps and expected results. The list should be numerated (using 1. numbering). Step's action and expected result will be divided by the Expected Result: string

If your .CSV file contains Step along with Expected Result columns and the Steps column at the same time, the Step along with Expected Result takes precedence when the file is processed.

If your .CSV file does not contain the columns listed above, but contains data matching the meaning of a listed above column, you can manually rename the column.

If you have difficulty importing the .CSV file, please contact support@zebrunner.com for further assistance.

Customizing the test case layout [Professional+]

It is possible to alter the test case layout used for a particular project. You can:

1. Enable or disable predefined test case fields and provide additional (or change default) values for them (if applicable)
2. Add custom fields having one of the available data types

Changes to the layout will be applied for a current project only, meaning that every project can have its own layout.

Info

You need to be a project Administrator/Manager to manage test case fields.

In order to alter the test case layout, go to the project Settings ( in the sidebar) → Test case fields. From there, you can enable or disable certain default fields (excluding the Title that cannot be disabled).

Info

Disabled fields will disappear from the test case views, corresponding filters will disappear too. Nevertheless, disabling a field doesn’t delete its values, so it’ll still be available if the field is turned on later.

Additionally, it is possible to modify (edit predefined and add new) values (and icons for those) for Priority and Automation State. To do that, click on the next to the field name. The view will appear, allowing you to add, alter or re-arrange existing possible values, as well as select the Default value (it’ll be automatically selected while creating a test case until changed to another value manually).

Info

Please note that the order of values is taken into account when those fields are part of sorting conditions for test cases (for instance, for the Priority field).

Test case field data types

There are multiple data types that you can use when you create custom fields. Please see field data types breakdown in a table below:

Type	Description
Date	Allows you to define a test case field that can have date as a value. Default value can be assigned. For default values, there are two possible options: default value can be set to a specific date; default value can be computed automatically relatively to the date of the test case creation. You can either configure it to use current date or define a shift relative to current date.
Dropdown	Allows you to define a test case field that can have one value from the pre-defined list. In addition to that, values can have icons, associated to them. Default value can be assigned.
Multi-Select	Allows you to define a test case field that can have multiple values from the pre-defined list. Default value or set of values can be assigned.
String	Allows you to define a test case field that can have single-line string of plain text (without formatting) as a value. Default value can be assigned.
Text	Allows you to define a test case field that can have a long multi-line rich text (with formatting) as its value. Default value can be assigned.
URI	Allows you to define a test case field that can have a URI its value. Markdown syntax for URIs is supported, allowing you to use human-friendly aliases/names for URIs. Default value can be assigned.
User	Allows you to define a test case field that can have another system user reference set as a value. Default value can be assigned.

Info

Adding a custom filed to the test case will also render a corresponding test repository filter component that can be used to narrow down test cases based on that field value.

Using the trash bin [Professional+]

Deleting test cases from the test repository does not remove them completely from the system. Test cases are first placed into a trash bin from where you can restore them or purge completely.

You can access the Trash bin from test cases grid by clicking the button at the top-right corner of the page.

Restoring cases

To restore one or several test cases, go to the Trash bin page and perform the following actions:

1. Select the cases that you want to restore and click Restore button
2. Select the parent suite to place the restored test cases
3. Click Restore again

Alternatively, you can restore all the test cases at once by clicking Restore all at the top of the page.

Purging cases

To purge one or several test cases, go to the Trash bin page and perform the following actions:

1. Select the cases that you want to purge and click Purge button
2. Confirm your action by clicking Purge again

Alternatively, you can purge all the test cases at once by clicking Restore all at the top of the page.

Using shared test steps

Shared steps represent a sequence of steps (or just one step) that can be reused (shared) between multiple test cases. Usage of shared steps aims to reduce content duplication and simplify future maintenance of test cases.

Steps are always included into other cases as a group - if you modify a sequence later by adding more steps (or removing some of them) - the changes will be reflected in test cases that use these sequences.

Creating shared steps

To create a new shared step, go to the Test repository → Shared steps grid and perform the following actions:

1. Click +Step
2. Fill out all necessary fields, add additional steps if needed
3. Click Create

The new sequence will be added to the grid. You can now include your shared steps to test cases.

Using shared steps in test cases

To insert a shared step to a test case, go to the Test repository → Test cases grid and perform the following actions:

1. Navigate to an existing test case or start creating a new one
2. Under the steps, click +Shared step
3. Select the needed shared step from the list and click Add

Selected steps will appear in a test case creation/edit form. Proceed with more changes if needed and save the test case.

Info

After adding a shared step to your test case, you can delete it by clicking the Delete beside it.

Info

Shared steps can be converted to a set of “regular” steps on test case creation. To do this, after adding a shared step to your test case, click the Edit to the right. This action will convert them to “regular” steps that you now can alter or re-arrange.

Editing shared steps

To edit a shared step, go to the Test repository → Shared steps grid and perform the following actions:

1. Hover over the shared step to edit and click the Edit to the right of the step title
2. Make the necessary changes to the shared steps and click Save

The update will be applied to the shared steps (and reflected in all cases that use them).

Deleting shared steps

To delete a shared step, go to the Test repository → Shared steps grid and perform the following actions:

1. Hover over the shared step to and click the Delete to the right of the step title.
2. Confirm action by clicking Delete

The shared step will be deleted from the system, as well as from all the test cases where they have been added.