Skip to content

Zebrunner Cypress reporting agent#

The official Zebrunner Cypress reporting agent.

Inclusion into your project#

To include reporting agent into your project complete the following steps:

  1. Install reporting agent package by executing the following command:

    npm install @zebrunner/javascript-agent-cypress

    yarn add @zebrunner/javascript-agent-cypress
    1. Enable reporting plugin in your cypress/plugins/index.js:
    const zbrPlugin = require('@zebrunner/javascript-agent-cypress/lib/plugin');

module.exports = (on, config) => { zbrPlugin(on, config); }

  2. Add reporting commands by including the following in your cypress/plugins/index.js:

    require('@zebrunner/javascript-agent-cypress/lib/commands/commands');

  3. In order to enable browser registration in Zebrunner include the following code in your cypress/support/index.js:

    before(() => {
    cy.registerBrowser(Cypress.browser)
})

Configuring reporting of test executions#

Once the agent package is included and enabled in your test project, the valid configuration must be provided.

It is currently possible to provide the configuration via:

  1. Environment variables
  2. cypress.json

The configuration lookup will be performed in the order listed above. It is also possible to override configuration parameters by passing them through a configuration provider with higher precedence.

via environment variables#

The following configuration parameters are recognized by the agent:

Parameter Description
REPORTING_SERVER_HOSTNAME Mandatory if reporting is enabled. It is Zebrunner server hostname. It can be obtained in Zebrunner on the 'Account & profile' page under the 'Service URL' section
REPORTING_SERVER_ACCESS_TOKEN Mandatory if reporting is enabled. Access token must be used to perform API calls. It can be obtained in Zebrunner on the 'Account & profile' page under the 'Token' section
REPORTING_PROJECT_KEY Optional value. It is the project that the test run belongs to. The default value is DEF. You can manage projects in Zebrunner in the appropriate section
REPORTING_RUN_DISPLAY_NAME Optional value. It is the display name of the test run. The default value is Default Suite
REPORTING_RUN_BUILD Optional value. It is the build number that is associated with the test run. It can depict either the test build number or the application build number
REPORTING_RUN_ENVIRONMENT Optional value. It is the environment where the tests will run
REPORTING_CI_RUN_ID Optional value. Predefined id of test run in Zebrunner.

via Cypress config#

In order to configure agent via Cypress config (cypress.json by default), reporter configuration needs to be provided.

Please see reporter configuration snippet below:

"reporter": "@zebrunner/javascript-agent-cypress",
"reporterOptions": {
  "reportingServerHostname": "<YOUR_ZEBRUNNER_SERVER_URL>",
  "reportingServerAccessToken": "<YOUR_ZEBRUNNER_ACCESS_TOKEN>",
  "reportingProjectKey": "DEF",
  "reportingRunEnvironment": "STAGE",
  "reportingRunBuild": "1.0-alpha",
  "reportingRunDisplayName": "My regression suite",
  "reportingRunLocale": "en_US",
  "reportingCiRunId": "46190073-55db-4411-ac42-fd42b7c96958"
}

The following configuration parameters are recognized by the agent:

Parameter Description
reportingServerHostname Mandatory if reporting is enabled. It is Zebrunner server hostname. It can be obtained in Zebrunner on the 'Account & profile' page under the 'Service URL' section
reportingServerAccessToken Mandatory if reporting is enabled. Access token must be used to perform API calls. It can be obtained in Zebrunner on the 'Account & profile' page under the 'Token' section
reportingProjectKey Optional value. It is the project that the test run belongs to. The default value is DEF. You can manage projects in Zebrunner in the appropriate section
reportingRunDisplayName Optional value. It is the display name of the test run. The default value is Default Suite
reportingRunBuild Optional value. It is the build number that is associated with the test run. It can depict either the test build number or the application build number
reportingRunEnvironment Optional value. It is the environment where the tests will run
reportingCiRunId Optional value. Predefined id of test run in Zebrunner.
reportingRunLocale Optional value. locale, that will be displayed for the run in Zebrunner if specified.

Parallelized runs#

When results are getting tracked in Zebrunner all of them are coming into single test run instance. Such approach is followed for parallel tests execution as well. To set up proper tracking of parallel runs with Zebrunner you need to specify CI run id parameter. See configuration section to figure out how to make this using reportingCiRunId cypress property or REPORTING_CI_RUN_ID environment variable. CI run id must be unique value across Zebrunner project. So simple CI build number will not work as it might intercept with runs on another pipelines within same Jenkins instance. For example to generate new value inside of Jenkins pipeline you could you construction like this: REPORTING_CI_RUN_ID = UUID.randomUUID().toString() Note: Solution was tested with cypress dashboard parallelization approach.

Additional reporting capabilities#

Tracking test maintainer#

You may want to add transparency to the process of automation maintenance by having an engineer responsible for evolution of specific tests or test classes. Zebrunner comes with a concept of a maintainer - a person that can be assigned to maintain tests.

owner attribute should be set in your test implementation. Here is an example:

describe('some spec', () => {
  it('test name', {'owner': '<username>'}, () => {...}
}

The maintainer username should be a valid Zebrunner username, otherwise it will be set to anonymous.

Collecting additional artifacts#

By default agent pushes to Zebrunner server screenshot for every test failure. You may find it in the details for the failed tests at the report page. Also agent automatically sends video of entire spec execution to Zebrunner for every failed test. You may find it attached to appropriate test results page.

Syncing test executions with external TCM systems#

Zebrunner provides an ability to upload test results to external test case management systems (TCMs) on test run finish. For some TCMs it is possible to upload results in real-time during the test run execution.

This functionality is currently supported for TestRail and Xray.

TestRail#

Zebrunner provides built in integration with TestRail test case management tool.

To enable integration with TestRail it's needed to set next properties in reporterOptions of your cypress.json:

    "reportingTestrailEnabled": "",
    "reportingTestrailSuiteId": "",
    "reportingTestrailTestrunName": "",
    "reportingTestrailTestrunID": "",
    "reportingTestrailMilestone": "",
    "reportingTestrailAssignee": "",
    "reportingTestrailIncludeAll": ""

Here's the summary of configuration parameters:

  • reportingTestrailEnabled - true of false to enable or disable integration
  • reportingTestrailSuiteId - ID of suite in TestRail
  • reportingTestrailTestrunName - (optional) name of existent test run in TestRail
  • reportingTestrailTestrunID - (optional) ID of existent test run in TestRail
  • reportingTestrailMilestone - (optional) milestone for the run in TestRail
  • reportingTestrailAssignee - (optional) assignee for the run in TestRail
  • reportingTestrailIncludeAll - (optional)

To map TestRail case ID to test body the following metadata attribute should be added to test implementation:

describe('some spec', () => {
  it('test name', {'testrailTestCaseId': 'case_id'}, () => {...}
}

where case_id is related TestRail test case ID.

If you need to pass multiple case IDs at once please follow the next pattern:

describe('some spec', () => {
  it('test name', {'testrailTestCaseId': ['case_id_1', 'case_id_2']}, () => {...}
}

Xray#

Zebrunner provides built in integration with Xray test management tool.

To enable integration with Xray it's needed to set next properties in reporterOptions of your cypress.json:

    "reportingXrayEnabled": "",
    "reportingXrayTestExecutionKey": ""

where parameters are:

  • reportingXrayEnabled - true of false to enable or disable integration
  • reportingXrayTestExecutionKey - execution key obtained at Xray

To map Xray case to test body the following metadata attribute should be added to test implementation:

describe('some spec', () => {
  it('test name', {'xrayTestKey': 'test_keys'}, () => {...}
}

where test_keys is list of related Xray cases split by a comma.