Skip to content

Launchers#

Launcher is a pre-defined configuration associated with a particular test repository allowing to perform parametrized test executions.

Launcher

Motivation#

Nowadays, more and more companies have teams of test automation engineers. The primary responsibility of these specialists is to automate the execution of test cases using a programming language. But having suites of automated tests is only part of the story. The most important part is to execute those tests regularly to get immediate feedback on the quality of the target software.

At the first glance, it might seem like executing these tests is an easy task — you just launch the tests and check the results. But everything is not so simple.

At the minimum, this requires a configured environment for the programming language the tests are written in. As the number of automated tests grows, test suites begin to take longer and longer to execute, requiring some kind of scaling for the execution environment. At the extreme case, this leads to establishing a TestOps team which will constantly maintain the execution environment.

Zebrunner eliminates the need to worry about the execution environment and allows the teams of automation engineers to focus on the test automation.

Zebrunner Launcher is a tool that you can use to execute your test suites stored in a Git repository and get comprehensive reporting at the same time.

In a nutshell#

If your automated tests reside in a Git repository, you can create Launchers for them in a few steps:

  1. Integrate Zebrunner reporting agent into your test project
  2. Connect a Git repository where your tests are located
  3. Create a Launcher for tests from the repository
  4. Launch your tests using the Launcher

You can find more information about steps 2-4 below.

Connecting a Git repository#

In order to add a Git repository into Zebrunner, you first need to navigate to the Launchers page and click the Add Repo button which is placed under the list of already added repositories.

Note

By default, only the Default project has pre-created demo launchers for zebrunner/carina-demo repository. Any other project will not have a pre-created launcher

Add Repo button

You will then need to select your Git provider. Currently, Zebrunner supports only GitHub and Bitbucket. Support for GitLab will be added a bit later.

GitHub#

There are two ways in which you can add your GitHub repository into Zebrunner:

  1. Granting access to your GitHub account. With this approach, you give access to all the repos you have access to
  2. By providing URL, username and password for the repository

Options for GitHub

Note

If access to a GitHub account has already been granted for your Zebrunner user, after clicking the Add Repo button, you will see the list of accessible GitHub repositories instead of the options depicted on the screenshot above. If you intend to add a repository using its URL and credentials, you need to switch the with URL toggle

Option 1: Granting access to GitHub account#

Granting access to your GitHub account is the most convenient way to operate with your repositories. With this approach, you grant access only for the first time and then Zebrunner will use the same permission to add other repositories to Zebrunner project.

In order to grant access to your GitHub account, you need to click on the Login with GitHub button. At this point, you need to be signed in to GitHub. If you don't, GitHub will show you the sign-in form.

Then you will be asked for consent to grant Zebrunner access to your account. Zebrunner requests only the minimal set of permission to execute tests.

Grant access to GitHub account

After granting the access rights, you will be redirected back to Zebrunner. Now you should see the list of repositories accessible to your GitHub account.

Note

The credentials which Zebrunner obtains to access GitHub on your behalf are securely encrypted

List of GitHub repositories

If either you don't have access to any GitHub repository or all the accessible repositories have been added to the Zebrunner project, you will see the corresponding message.

In order to add a repository from the list, you need to click the Add button next to the name of desired repository. At this point, if you want to add a repository using its URL and credentials, you need to switch the with URL toggle.

A repository added by clicking the Add button will appear in the list of repositories. The note under the Login with GitHub button at the center of the screen should state Repo is connected. If it says that there was an error connecting to the repository, you should double-check if your GitHub user still has access to this repository.

Added GitHub repository

If you click the Add Repo button again, you should see the list of repositories available to your user, but without already added ones.

Info

If you would like to revoke access granted to Zebrunner, you can do so directly from GitHub.

Navigate to SettingsApplicationsAuthorized OAuth Apps → click More Options next to the Zebrunner app → RevokeI understand, revoke access.

Zebrunner will then not be able to access GitHub on your behalf, including pulling the repository and executing tests

Option 2: Adding repository by URL#

Adding a repository by URL and credentials is not as convenient as adding via GitHub authentication. This requires specifying an URL and credentials for each repository. On the other hand, this allows you to have separate credentials for every repository, which is good in terms of security and traceability.

The repository URL must be either an HTTPS browser link like https://github.com/zebrunner/carina-demo or .git HTTPS URL like https://github.com/zebrunner/carina-demo.git.

If the repository you intend to add is publicly available, you are free to not specify username and token. Every private repository must have username and token specified, otherwise GitHub will not allow to pull it.

In order to generate access token for repository complete the following steps:

  1. Go to SettingsDeveloper settingsPersonal access tokensGenerate new token
  2. In the Note field, provide a meaningful description and set the desired Expiration
  3. Grant Full control of private repositories permission by checking the repo scopes for the token
  4. Click Generate token button at the bottom of the page

GitHub access token

Token expiration

Note, that after the expiration date, Zebrunner will lose access to all repositories associated with the token, so don't forget to regularly renew the token in repository settings in Zebrunner. As an alternative to rotating tokens manually, you can issue a token without expiration date, but this is not a good security practice.

By clicking on the Add button, Zebrunner will verify that the provided credentials (if any) are valid for the repository. If a connection to the repository can be established, you will be redirected to the repository information page. On this page, you can change the credentials and check if Zebrunner has access to the repository.

Note

Any provided credentials are securely encrypted by Zebrunner

GitHub repository which is added by URL

Info

The generated token can be manually deleted on the Personal access tokens page in GitHub

Bitbucket#

Adding Bitbucket repository requires specifying a URL and credentials for each repository.

The repository URL must be either an HTTPS browser link like https://bitbucket.org/Zebrunner/carina-demo/src/master/ or .git HTTPS URL like https://Zebrunner@bitbucket.org/Zebrunner/carina-demo.git.

If the repository you intend to add is publicly available, you are free to not specify username and token. Every private repository must have username and token specified, otherwise Bitbucket will not allow to pull it.

In order to generate access token for repository complete the following steps:

  1. Сlick on user icon in the top upper corner of the page → Personal settingsApp passwords → click Create app password button
  2. In the Label field, provide a meaningful description for the app password
  3. Grant Repositories - Read permission for the token
  4. Click Create button at the bottom of the page

Bitbucket app password

Bitbucket will show you a pop-up with the new App password. Copy this value and paste it together with your Bitbucket username into corresponding field in Zebrunner.

Bitbucket add repo

By clicking on the Add button, Zebrunner will verify that the provided credentials (if any) are valid for the repository. If a connection to the repository can be established, you will be redirected to the repository information page. On this page, you can change the credentials and check if Zebrunner has access to the repository.

Note

Any provided credentials are securely encrypted by Zebrunner

GitHub access token

Info

The created App password can be manually deleted in Bitbucket settings. To do this, click on user icon in the top upper corner of the page → Personal settingsApp passwords → click Revoke button next to the App password you wand to delete.

Adding a Launcher#

Launcher is a pre-defined configuration for launching tests which are resided in a particular Git repository. Under the hood, Zebrunner uses Docker containers to execute your tests. This allows you to reproduce any desired execution environment and provides an ability to customize it using environment variables.

Info

Launcher can be added only as a child item of a Git repository. To learn more about connecting a Git repository, refer to Connecting a Git repository guide

To add a new launcher for repository, you need to click the Add new launcher button which is placed as a child item of the target repository. A click on this button opens Launcher Constructor. Every launcher must have a name, Git branch to pull tests from, execution environment, testing platform for web or mobile tests, set of web driver capabilities, and notification channels.

After providing all the launcher configs, you need to click the Add button. Zebrunner does not automatically verify if the tests can actually be launched with the given config. You should deliberately choose the Docker image, the launch command, and the rest of configs.

Basic information#

Basic information consists of launcher's name and Git branch. The name must be unique within the Git repository. The branch selector fetches currently available branches for the Git repository.

Launcher basic information

Execution environment#

Execution environment consists of Docker image, launch command to be executed inside the container, and an optional set of environment variables. By default, Zebrunner offers a few Docker images and their corresponding basic launch commands. You are free to choose a Docker image from the list or provide another one. The launch command will be executed inside the Docker container created from the given Docker image, so it must use only processes available inside the container. Also, you can use the ${env_var_name} placeholders in the launch command to refer to the environment variables.

Note

If you want to use a custom Docker image from a private Docker registry, you need to configure integration with the registry. This functionality is not currently available from Zebrunner UI, but you can contact Zebrunner support for assistance

Launcher execution environment

Environment variables#

Environment variables are regular OS environment variables that are available at container runtime. Each variable has a name, type, and default value. Type of variable constrains possible inputs. Currently, we support Strings, Integers, Booleans, and Choice of Strings.

Choice of Strings works as a usual Strings input field, but allows providing a list of suggestions. To change the list of suggestions, you need to click on icon that is placed on the right side of the input for default value. This will open a dialogue with a test area. Each row of the text area specifies a separate suggestion value. The first value is considered a default one.

Value of an environment variable will be passed to container runtime as a string value, regardless of it type.

Choice values

Testing platform#

Testing platform section specifies Selenium Grid and a set of capabilities that will be automatically used when a new instance of RemoteWebDriver gets created. Zebrunner Selenium Grid is available by default for all projects, but you can provide credentials and use any supported Selenium Grid.

After choosing a testing platform, you can select an OS, device, browser, and so on, which will be used by web driver instances. The allowed values are obtained directly from the Selenium Grid provider and are regularly updated. Hence, you can use the newest device or browser as soon as it is available for Selenium Grid.

If OS, device, and browser are not the only capabilities you want to use with web driver instances, you can provide custom ones with the constructor of custom capabilities. The constructor works in the same way as the constructor for environment variables (for more information, refer to Environment variables). The only difference is that the type of a capability also determines its type in the source code of tests.

Launcher testing platform

Notification channels#

Notification channels determine where to send messages when a test suite execution is finished. The notification contains brief information about the results of the suite execution, such as its name, the number of passed and failed tests.

Zebrunner supports sending notifications to Microsoft Teams, Slack and email. For the Teams and Slack notifications to work, you must provide a valid Integration configuration.

The list of target channels or emails must be presented as a comma-separated string with an optional whitespace between items. If the string contains a non-existent channel or email, or a not-configured Teams channel, it will be skipped.

Launcher notification targets

Launching tests#

After connecting a Git repository and creating a launcher for it, you can launch the tests from the repository.

As the first step, choose the desired launcher from the list. Before clicking on the Launch button at the bottom of the page, you can update any of the available settings. The changes you make to the launcher at this point will not be automatically saved and will only be used for this specific launch. This might be useful if you want to do some experiments with the configuration and save it only if the changes work as expected.

Here you can find a setting at the top of the page which is not configurable from the Launcher Constructor — Milestone. This is the existing Zebrunner Milestone that the suite execution should be linked to.

Launch Milestone

Another useful feature isthe ability to edit environment variables and custom capabilities. To do this, you can either click on the icon on the right side of the input or edit the values manually as a plain string. Clicking on the icon opens a modal window with handy controls to change values.

Environment variables modal window

Manual editing of the values provides more possibilities: - quick editing of a value without opening the modal window; - removing of any name=value pair from the set of values; - adding a new name=value pair ot the set; - entering any value regardless of the original value type. This action changes the value type to String. The only constraint to the manual editing is that the result string must have strict format. Name and value pair should be divided by = symbol; pairs should be divided by ; symbol.

Manually edited environment variables

When all the settings have desired values, you can launch the tests. For this, navigate to the bottom of the page and click on the Launch button. The Launch another checkbox to the left of the button controls the redirection behaviour. If the checkbox is unchecked, you will be redirected to the list of test suite executions. Otherwise, you will stay on the launcher's page and can do another launch (probably, with updated configs).

Note

Zebrunner does not automatically verify if the tests can actually be launched with the given config. You should deliberately choose the Docker image, the launch command, and the rest of configs

The just launched tests suite will have the name of the original launcher. Also, initially, it will be in queued status, because it takes some time to pull the source code of the tests and start the execution.

Queued executions

As soon as the tests start running, you will receive real-time updates about the progress without the need to reload the page.

Running tests

Altering an existing Launcher#

If you want to change configuration of an existing launcher, you need to select the target launcher and click on the Change Defaults button at the top of the page. This will open the Launcher Constructor.

Altering an existing Launcher

Here you can change any of the available settings. All the rules applicable for launcher creation are also valid for editing.

When you finish with the update, scroll to the bottom of the page and click the Save button. If you want to discard the changes, click on the Cancel button.

Deleting a Launcher#

If a launcher becomes obsolete, it can be removed. For this, click on the obsolete launcher and scroll to the bottom of the launcher definition. Click on the Delete Launcher button will open a confirmation model window.

<!--

Delete Launcher confirmation window
-→

To confirm the action, click on the Delete button.

Deleting a Launcher