Helm Chart Deployment Guide#

This guide explains the prerequisites and steps necessary to deploy an on-premise instance of the Zebrunner Testing Platform using a Helm chart.

Dependencies#

Zebrunner is a distributed application that consists of various Java-based backend services. The application relies on the following dependencies:

Docker image pull password. All Zebrunner Docker images are stored in private Docker Hub repositories. To access these images, you need to obtain an access key from the Zebrunner Team. The Helm chart automatically creates a Kubernetes secret required for pulling the images.
Zebrunner License Key. The deployed system will not accept requests without a valid and non-expired License Key, which should be provided in the values.yaml file. The License Key is supplied by the Zebrunner Team.
Credentials for migrating data from hosted version. If you are currently using the hosted version of Zebrunner ( available at the zebrunner.com domain), you can migrate all your data and files to the on-premise instance. Credentials required to access data backups can be requested from the Zebrunner Team.
Ingress-nginx controller. The Helm chart creates an Ingress for most of the backend services. To accommodate custom routing rules specific to Nginx, your Kubernetes cluster must have the ingress-nginx controller installed.
PostgreSQL. Most of the Zebrunner services utilize PostgreSQL as the permanent storage solution. The minimal required version is 13, though Zebrunner is also fully compatible with versions v14 and v15. Depending on your Zebrunner distribution and workload, you may need to provision between one and three database instances. Please note that these instances are not created by the Helm chart and must be set up separately.
RabbitMQ. RabbitMQ is required for internal communication between Zebrunner services. The Helm chart can either utilize an existing RabbitMQ instance/cluster or deploy a three-node cluster itself.
S3-compatible storage. Zebrunner requires S3-compatible storage to store various assets, including those uploaded from the user interface or generated by automated tests.
SMTP server. Zebrunner can send different types of emails, so an SMTP server is necessary.
Elasticsearch. If your Zebrunner distribution includes the Test Case Management module, an Elasticsearch instance/cluster is required. The Helm chart can either use an existing instance/cluster or deploy a single-node cluster itself.
Server instances. The resource requirements for running a particular Zebrunner distribution are described in a subsection below.

Docker Image Pull Password#

The Helm chart automatically creates a k8s Secret used to pull images. Registry server, username and email are populated with correct values automatically. The only parameter you need to specify in values.yaml is the registry access token.

Ingress-nginx controller#

For more information on deploying the ingress-nginx controller, visit the official documentation. To verify which version of the ingress controller is compatible with your Kubernetes cluster, refer to the supported versions table.

Additionally, for the latest versions of the ingress-nginx controller, you also need to explicitly allow snippet annotations. To do so, you can either modify the installation manifest manually or run the following command after installing the controller:

kubectl patch configmap ingress-nginx-controller -n ingress-nginx --patch '{"data":{"allow-snippet-annotations": "true"}}'

PostgreSQL#

We recommend starting with instances having 2 vCPUs and 4 GiB of memory (AWS representative is db.t4g.medium) and changing the instance shape depending on your workloads. From our experience, increasing the number of available resources is very rarely needed.

Depending on the Zebrunner distribution, you will need from 1 to 3 database instances. For specifics, let's call them postgres1, postgres2, and postgres3. These 3 instances should be considered as logical DB instances. Technically, it is possible to run all of them within 1 physical database instance, and in practice it should be enough for most workloads. However, if you are using Automation Reporting module and running dozens of thousands of automated tests every day and store dozens of thousands of test cases, it is recommended to use multiple database instances.

Here are some general recommendations for the number of DB instances based on the number of users:

Number of users	Recommended number of instances	Comment
1 - 100	1	`postgres1`, `postgres2`, and `postgres3` reside on the same physical DB instance
100 - 250	2	`postgres1` and `postgres2` reside on one physical DB instance, and `postgres3` resides on another one
250+	3	`postgres1`, `postgres2`, and `postgres3` are separate physical database instances

Here are some considerations regarding the databases depending on the Zebrunner distribution:

Test Case Management. This distribution requires only 1 database instance (postgres1) with 20 GiB of storage.
Automation reporting. For moderate workloads, 1 database instance (postgres2) with 40 GiB of storage should be enough. If you are running thousands of automated tests every day, it is better to have 2 databases - postgres2 for results of automation (20 GiB of storage), and postgres3 for storing logs (40 GiB of storage).
Testing Platform (Test Case Management + Automation reporting). In this case, we recommend having at least 2 database instances for moderate workloads (postgres1 with 20 GiB of storage and postgres2 with 30 GiB of storage), and 3 instances for large teams (20 GiB of storage for postgres1 and postgres2, and 40 GiB of storage for postgres3).

Since most of the data is not evicted from Zebrunner over time, we also recommend enabling storage autoscaling.

Note

Feel free to contact the Zebrunner Team to get a personal recommendation regarding the resources required for databases.

RabbitMQ#

RabbitMQ can either be deployed externally and provided by you or deployed by the Helm chart itself. Whether the RabbitMQ should be deployed or not is determined by the global.rabbitmq.deploy parameter in values.yaml file. You can find more information in the corresponding section of this guide.

If global.rabbitmq.deploy is set to true, the Helm chart deploys a 3-node cluster, which is based on Bitnami RabbitMQ Helm chart. Additionally, Zebrunner enables Stomp plugin and adds pods requests and limits.

Pricing considerations for managed RabbitMQ clusters

Deployment of an external RabbitMQ instance may not be cost-effective. In particular, AWS's Amazon MQ service charges for a managed RabbitMQ cluster that typically comprises 3 nodes — a cost that can exceed the expense of running all Zebrunner servers.

Since persistent storage of the RabbitMQ cluster is not critical for Zebrunner, we generally recommend using the RabbitMQ cluster deployed by this Helm chart.

S3-compatible storage#

Zebrunner uses S3 to store different kinds of assets. In values.yaml file, you must specify bucket name and region parameters. You can optionally provide an IAM user's access key and secret key to communicate with S3, but we strongly recommend that you use AWS roles attached to instances instead.

The IAM policy for interaction with S3 should look like this:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "ZebrunnerS3",
      "Effect": "Allow",
      "Action": [
        "s3:PutObject",
        "s3:GetObject",
        "s3:DeleteObject",
        "s3:ListBucket"
      ],
      "Resource": [
        "arn:aws:s3:::<bucket-name>",
        "arn:aws:s3:::<bucket-name>/*"
      ]
    }
  ]
}

Communication issue when using IAM role to grant access to S3 in AWS

When running a Kubernetes cluster on AWS that is not managed by EKS, you may encounter issues accessing S3 buckets despite correctly attaching an IAM role with the required permissions to your instances. A typical symptom is that Zebrunner service cannot access the necessary S3 resources.

Long story short, the root cause of this problem is usually related to the EC2 Instance Metadata Service (IMDS) firewall configuration. To overcome the issue, you must adjust the hop limit of HTTP requests for IMDS on each EC2 instance in your cluster. Use the following AWS CLI command to increase the hop limit:

    aws ec2 modify-instance-metadata-options \
        --instance-id <instance-id> \
        --http-put-response-hop-limit 3 \
        --http-endpoint enabled

A hop limit of 3 is typically sufficient for most network configurations. Adjust accordingly if your specific setup requires a different value.

Elasticsearch#

Elasticsearch is required only if your distribution includes Test Case Management module.

It can either be provided by you or deployed by the Helm chart itself. If global.elasticsearch.deploy parameter is set to true, a single-node cluster (based on Bitnami Elasticsearch Helm chart) will be deployed.

Server Instances#

Since Zebrunner is a Java-based distributed system, the primary resource is memory. CPU consumption is pretty low even during working hours. We recommend using either general-purpose instances (AWS representative is m6a family) or instances with an increased amount of memory (AWS representative is r6a family). The memory consumption depends on:

Zebrunner distribution.
Base replica count (brc). This property defines the value of spec.replicas property of k8s deployments. For most deployments, spec.replicas == brc, but for the crucial ones spec.replicas = brc + 1.
Whether you are using Zebrunner Engine - it requires a few extra services to be deployed.

The following table shows the total memory request for running Zebrunner Testing Platform:

Distribution	Total memory request formula	`brc` == 1	`brc` == 2
Test Case Management	`brc` * 5050 MiB + 4400 MiB	9450 MiB	14500 MiB
Automation Reporting (w/t Zebrunner Engine)	`brc` * 5900 MiB + 2550 MiB	8450 MiB	14350 MiB
Automation Reporting (w/ Zebrunner Engine)	`brc` * 7500 MiB + 2550 MiB	10050 MiB	17550 MiB
Testing Platform (w/t Zebrunner Engine)	`brc` * 8600 MiB + 5450 MiB	14050 MiB	22650 MiB
Testing Platform (w/ Zebrunner Engine)	`brc` * 9800 MiB + 5450 MiB	15250 MiB	25050 MiB

Notes

Keep in mind that k8s rollout upgrades require extra memory for spinning up new pods
The Test Case Management and Testing Platform distributions are calculated for global.application.ai.enabled == true
The Test Case Management and Testing Platform distributions are calculated for global.application.testCasesImport.enabled == false. Setting this parameter to true will lead to increased memory consumption by 1300 MiB

`values.yaml` Parameters#

The Helm chart recognizes hundreds of different parameters, most of which are specific to a particular backend service and required only in very special circumstances. The tables below describe the most common parameters. Some of them must be specified before deploying the Zebrunner Testing Platform.

Docker Registry Parameters#

Parameter	Description	Default value
`global.dockerRegistry.createSecret`	Specifies whether the chart must create k8s Secret with pull image credentials	`true`
`global.dockerRegistry.secretName`	Name of the k8s `kubernetes.io/dockerconfigjson` Secret	`zebrunner-docker-registry-credentials`
`global.dockerRegistry.server`	URL of the Docker Registry	`https://index.docker.io/v1/`
`global.dockerRegistry.email`	Email of the Docker user	`support@zebrunner.com`
`global.dockerRegistry.username`	Docker user	`zebrunnercustomer`
`global.dockerRegistry.accessKey`	Required. Customer-specific access key to pull images. It is provided by the Zebrunner Team	``

Deployment Parameters#

Parameter	Description	Default value
`global.baseReplicaCount`	Defines the value of `spec.replicas` property. For most deployments, `spec.replicas` == `brc`, but for the crucial ones `spec.replicas` = `brc` + 1	`1`
`global.customCertificates`	List of objects with `name` and `content` properties, defining custom TLS certificates. The `name` property defines name of the file that will be mounted to each container's `/opt/java/openjdk/lib/security/self-signed` folder	`[]`
`global.hostAliases`	`spec.template.spec.hostAliases` property of each deployment	`[]`
`global.credentialsJson`	A JSON object containing application credentials. This property is required if you are upgrading Zebrunner deployed by ArgoCD. More info can be found here	``

Ingress Parameters#

Parameter	Description	Default value
`global.ingress.host`	Required. Value that will be set into `spec.tls[0].hosts[0]` and `spec.rules[0].host` properties of each Ingress	``
`global.ingress.className`	Name of the Ingress Class associated with ingress-nginx controller	`nginx`
`global.ingress.tls.enabled`	Specifies if a TLS termination occurs on Ingress level	`false`
`global.ingress.tls.secretName`	Required if `global.ingress.tls.enabled` == `true`. Name of the k8s Secret with TLS certificate	``

Zebrunner Engine Parameters#

Note

These parameters are only needed if you also have a Zebrunner Engine deployment

Parameter	Description	Default value
`global.engine.url`	URL of your Zebrunner Engine deployment	``
`global.engine.admin.username`	Username of a Zebrunner Engine user with admin permissions	`admin`
`global.engine.admin.password`	Password of a Zebrunner Engine user with admin permissions	`admin`
`global.engine.hubUser.username`	Username of a Zebrunner Engine Selenium user	``
`global.engine.hubUser.password`	Password of a Zebrunner Engine Selenium user	``
`global.engine.internalUser.username`	Username of the Zebrunner Testing Platform user with permissions to call Engine-specific endpoints	`engine`
`global.engine.internalUser.password`	Password of the Zebrunner Testing Platform user with permissions to call Engine-specific endpoints	`engine`

S3 Parameters#

Warn

We strongly recommend using AWS roles attached to instances instead of an IAM user (when global.s3.accessKey and global.s3.secretKey parameters are specified).

Parameter	Description	Default value
`global.s3.endpoint`	Required if you are not using AWS S3. Endpoint of your S3-compatible storage	``
`global.s3.bucket`	Required. Name of the S3 bucket	``
`global.s3.region`	Required. Region of your S3 bucket	``
`global.s3.accessKey`	Access Key of your IAM user that has access to S3.	``
`global.s3.secretKey`	Secret Key of your IAM user that has access to S3	``

RabbitMQ Parameters#

Parameter	Description	Default value
`global.rabbitmq.deploy`	Specifies whether the Helm should also deploy a RabbitMQ cluster	`true`
`global.rabbitmq.host`	Required if `global.rabbitmq.deploy` == `false`. Host of the RabbitMQ instance\cluster	``
`global.rabbitmq.port`	Required if `global.rabbitmq.deploy` == `false`. AMQP port of the RabbitMQ instance\cluster	`5672`
`global.rabbitmq.username`	Required if `global.rabbitmq.deploy` == `false`. Username to access the RabbitMQ instance\cluster	`zebrunner`
`global.rabbitmq.password`	Required if `global.rabbitmq.deploy` == `false`. Password to access the RabbitMQ instance\cluster	``

Email Parameters#

Parameter	Description	Default value
`global.email.smtp.host`	Host of your SMTP server, e.g. `smtp.gmail.com`	``
`global.email.smtp.port`	Port of your SMTP server	`587`
`global.email.smtp.username`	Username for communication with your SMTP server	``
`global.email.smtp.password`	Password for communication with your SMTP server	``
`global.email.sender.email`	Email address from which all messages will be sent	``
`global.email.sender.name`	Email sender name	``

Elasticsearch Parameters#

Note

These parameters are only needed if your Zebrunner distribution includes Test Case Management module

Parameter	Description	Default value
`global.elasticsearch.deploy`	Specifies whether the Helm should also deploy a single-node Elasticsearch cluster	`false`
`global.elasticsearch.url`	Required if `global.elasticsearch.deploy` == `false`. Full HTTP(S) URL to the Elasticsearch instance\cluster	``
`global.elasticsearch.username`	Username to access the Elasticsearch instance\cluster	``
`global.elasticsearch.password`	Password to access the Elasticsearch instance\cluster	``

PostgreSQL Parameters#

As described in the PostgreSQL section, Zebrunner requires from 1 to 3 databases. For specifics, these databases were named postgres1, postgres2 and postgres3. The Helm chart takes this idea further and forms 3 logical databases. "Logical" means that actually these databases can be the same, but from the chart perspective they are 3 separate databases strictly distributed among services. So, depending on distribution, databases may have the same properties.

`postgres1` database#

Note

These parameters are needed for any Zebrunner distribution. If your distribution doesn't have Test Case Management or you are going to have one database, then values should be the same as for postgres2.

Parameter	Description	Default value
`global.postgres1.host`	Host of the `postgres1` database	``
`global.postgres1.port`	Port of the `postgres1` database	`5432`
`global.postgres1.username`	Username to access the `postgres1` database	`postgres`
`global.postgres1.password`	Password to access the `postgres1` database	``
`global.postgres1.schema`	Schema name for the `postgres1` database	`zebrunner`
`global.postgres1.initJobEnabled`	If set to `false`, the init job that creates databases and schemas for `postgres1` will not run	`true`
`global.postgres1.initDatabase`	Name of the maintenance database used by the init job to connect to PostgreSQL before creating service databases	`postgres`

`postgres2` database#

Note

These parameters are only needed if your Zebrunner distribution includes Automation Reporting module

Parameter	Description	Default value
`global.postgres2.host`	Host of the `postgres2` database	``
`global.postgres2.port`	Port of the `postgres2` database	`5432`
`global.postgres2.username`	Username to access the `postgres2` database	`postgres`
`global.postgres2.password`	Password to access the `postgres2` database	``
`global.postgres2.schema`	Schema name for the `postgres2` database	`zebrunner`
`global.postgres2.initJobEnabled`	If set to `false`, the init job that creates databases and schemas for `postgres2` will not run	`true`
`global.postgres2.initDatabase`	Name of the maintenance database used by the init job to connect to PostgreSQL before creating service databases	`postgres`

`postgres3` database#

Note

These parameters are only needed if your Zebrunner distribution includes Automation Reporting module. If you are going to use one database instance, then values should be the same as for postgres2.

Parameter	Description	Default value
`global.postgres3.host`	Host of the `postgres3` database	``
`global.postgres3.port`	Port of the `postgres3` database	`5432`
`global.postgres3.username`	Username to access the `postgres3` database	`postgres`
`global.postgres3.password`	Password to access the `postgres3` database	``
`global.postgres3.schema`	Schema name for the `postgres3` database	`zebrunner`
`global.postgres3.initJobEnabled`	If set to `false`, the init job that creates databases and schemas for `postgres3` will not run	`true`
`global.postgres3.initDatabase`	Name of the maintenance database used by the init job to connect to PostgreSQL before creating service databases	`postgres`

Migration Parameters#

Parameter	Description	Default value
`global.dataMigration.migrateDatabases`	If set to `true`, all databases will be migrated from your hosted Zebrunner instance to the on-premise instance	`false`
`global.dataMigration.dropDatabases`	If set to `true`, all databases will be removed before applying data backups. Can be useful when something went wrong during data restoration and when the application was not initially configured properly	`false`
`global.dataMigration.migrateFiles`	If set to `true`, all files from the hosted instance will also be migrated	`false`
`global.dataMigration.zebrunner.host`	Required if either `global.dataMigration.migrateDatabases` == `true` or `global.dataMigration.migrateFiles` == `true`. Base host name of your hosted Zebrunner version. E.g. `https://hostname.zebrunner.com`	``
`global.dataMigration.zebrunner.username`	Required if either `global.dataMigration.migrateDatabases` == `true` or `global.dataMigration.migrateFiles` == `true`. Username to access data backups and files. The value is provided by Zebrunner Team	``
`global.dataMigration.zebrunner.password`	Required if either `global.dataMigration.migrateDatabases` == `true` or `global.dataMigration.migrateFiles` == `true`. Password to access data backups and files. The value is provided by Zebrunner Team	``

Application Parameters#

If a parameter description starts with [TCM], then the parameter works only if your Zebrunner distribution includes Test Case Management module.

If a parameter description starts with [AR], then the parameter works only if your Zebrunner distribution includes Automation Reporting module.

Parameter	Description	Default value
`global.application.baseUrl`	Required. HTTP(S) URL to WEB UI of your Zebrunner Testing Platform distribution	``
`global.application.licenseKey`	Required. Your Zebrunner License Key	``
`global.application.administrator.email`	Required. Email of the workspace owner user that will be automatically created in a new deployment	`admin@user.com`
`global.application.administrator.username`	Required. Username of the workspace owner user that will be automatically created in a new deployment	`admin`
`global.application.administrator.password`	Required. Password of the workspace owner user that will be automatically created in a new deployment	`changeit`
`global.application.administrator.enforce`	If set to `true`, the workspace owner user will be updated according to the properties listed above on every deployment of `iam-service`	`true`
`global.application.automationArtifacts.retentionPeriodDays`	[AR] Retention period for screenshots and logs produced by automated tests. A negative value means an infinite storage period	`90`
`global.application.testCasesImport.enabled`	[TCM] If set to `true`, the Importing Test Cases functionality will be available in Test Repository. Since this is a one-time action, we recommend setting this property to `false` after the import is complete. Setting value to `false` also reduces the amount of resources required by Zebrunner backend	`false`
`global.application.ai.enabled`	[TCM] The `true` value enables gen-AI functionality	`false`
`global.application.ai.provider`	[TCM] Name of the gen-AI provider. Should be either `openai` or `anthropic`	`openai`
`global.application.ai.model`	[TCM] Name of the model of the gen-AI provider. Examples of the model name are `gpt-4o`, `gpt-4o-mini`, `gpt-4-turbo`, `gpt-4`, `gpt-3.5-turbo`, `claude-3-5-sonnet-20240620`, `claude-3-opus-20240229`, `claude-3-haiku-20240307`. The most relevant model names may depend on the model deployment, and can be found in API documentation of the gen-AI provider	`gpt-4o`
`global.application.ai.apiKey`	[TCM] The API key that will be used for communicating with the generative AI provider	``
`global.application.systemSchema.name`	Name of the PostgreSQL schema used by Zebrunner for internal infrastructure tables (CDC, response cache). Services that need these tables create the schema automatically on startup unless the init job handles this	`_system`
`global.application.extensionSchema.name`	Name of the PostgreSQL schema where database extensions (e.g., `pgcrypto`) are installed. In most cases, this should be the same as `systemSchema.name`. The extension schema is not auto-created by the application; it must either already exist or be created by the init job. Only iam-service installs the `pgcrypto` extension into this schema	`_system`

Example#

global:
  dockerRegistry:
    accessKey: dckr_some_key

  baseReplicaCount: 2

  ingress:
    host: "onprem.zebrunner.com"
    tls:
      enabled: true
      secretName: zebrunner.com-tls

  customCertificates:
    - name: zebrunner.internal.crt
      content: |
        -----BEGIN CERTIFICATE-----
        certificate+content
        -----END CERTIFICATE-----
  hostAliases:
    - ip: 10.71.15.284
      hostnames:
        - zebrunner.internal

  engine:
    url: "https://engine.zebrunner.com"
    admin:
      username: admin
      password: admin
    hubUser:
      username: hubUser
      password: hubUser
    internalUser:
      username: internalUser
      password: internalUser

  s3:
    region: us-east-1
    bucket: zebrunner-onprem-assets

  elasticsearch:
    deploy: true

  postgres1:
    host: onprem1.zebrunner.us-east-1.rds.amazonaws.com
    password: password1
  postgres2:
    host: onprem2.zebrunner.us-east-1.rds.amazonaws.com
    password: password2
  postgres3:
    host: onprem3.zebrunner.us-east-1.rds.amazonaws.com
    password: password3

  application:
    baseUrl: "https://onprem.zebrunner.com"
    licenseKey: ey...sh-eDQ
    systemSchema:
      name: _system
    extensionSchema:
      name: _system
    administrator:
      email: support@zebrunner.com
      username: admin
      password: password
      enforce: false
    automationArtifacts:
      retentionPeriodDays: 360
    testCasesImport:
      enabled: false
    ai:
      enabled: true
      model: gpt-4o
      apiKey: "some-api-key"

Deployment#

To deploy the Zebrunner Testing Platform on a Kubernetes cluster with the required number of worker nodes and having a prepared values.yaml file, you can use the following command:

helm upgrade \
     --install \
     -f ./values.yaml \
     --timeout 300m \
     zebrunner oci://registry-1.docker.io/zebrunner/testing-platform --version 1.4.0

Please note that you need to replace ./values.yaml with the path to your values.yaml file. Additionally, the command must specify the version of the Zebrunner Testing Platform Helm chart to use (1.4.0 in this example).

--timeout argument

The --timeout argument is required when and only during the initial data migration step when transferring your Zebrunner workspace from hosted version to your private on-premise instance. For brand new installations it can be omitted.

Warn

Please be aware that Zebrunner backend services do not support live-reloading of updated configurations stored in Kubernetes config maps and secrets. Therefore, if you make changes to a configuration value, you may need to run kubectl rollout restart deployment to ensure the changes take effect.

If you encounter any difficulties during the deployment process, you can reach out to Zebrunner Support for further assistance.

Migration from hosted (`zebrunner.com`) instance#

If you currently use a hosted Zebrunner instance (e.g., https://your-company.zebrunner.com) and plan to migrate to a private, on-premise installation, you can transfer all existing data and files during the initial installation.

To perform this migration, first contact Zebrunner Team to request migration credentials. These credentials will provide access to database dumps and enable file transfers from your hosted instance. You will use these credentials in the configuration parameters global.dataMigration.zebrunner.username and global.dataMigration.zebrunner.password, as detailed in Migration Parameters.

Encrypted and hashed data

Sensitive data stored in Zebrunner is either encrypted or hashed. During migration, cryptographic keys are not transferred, meaning you will need to reconfigure all integrations. Additionally, all users utilizing password authentication will need to reset their passwords.

Data migration steps:

Set both the global.dataMigration.migrateDatabases and global.dataMigration.migrateFiles parameters to true
Define global.dataMigration.zebrunner.host with your hosted instance's URL in format https://hostname.zebrunner.com
Enter the migration credentials provided by the Zebrunner Team into the appropriate configuration parameters mentioned above
Run the installation as described here. This step may take significant time because databases will be populated from your hosted instance.
Note: Pay special attention to the --timeout argument. Typically, a 300-minute timeout is sufficient. However, if your network setup might slow down connections between your Kubernetes cluster and Zebrunner servers, consider extending this timeout
Once the installation finishes successfully and all Kubernetes pods reach the "Running" state, you can begin using your on-premise instance. If you enabled file migration (global.dataMigration.migrateFiles set to true), the file transfer may continue even after the application is ready. You can track file migration progress by reviewing logs from the file-management-service pod. Logs will indicate the number of files migrated successfully, failed transfers, and skipped files. The migration completion is indicated by the log entry: Migration completed. {} out of {} files was processed. Skipped: {}, migrated: {}, failed: {}
After successful file migration, remove all global.dataMigration.* parameters from values.yaml file to prevent accidental migration and data loss during future updates

Files migration speed

Typically, approximately 3,000 files are migrated per minute during the file migration process.

CDN rules#

If you are deploying a CDN in front of your Zebrunner distribution, you need to specify several routing rules for Zebrunner endpoints and their caching policies.

The following table provides an ordered list of path patterns that you need to specify, as well as the request parts that must constitute the cache key for those paths. If the cache key is not specified for a path, then caching must be disabled.

Note

For all the specified rules, the CDN must forward all request headers and query parameters.

Path Pattern	Cache Key
`/api/*`
`/artifacts/*`
`/saml2/*`
`/login/saml2/*`
`/webhooks/*`
`/avatars`	path + query parameters
`/files/*`	path
`/*.js`	path
`/*.css`	path
`/*.svg`	path
`/*.jpg`	path
`/*.jpeg`	path
`*` (default)

Upgrade#

In order to upgrade your Zebrunner Testing Platform to a newer version, run the following command:

helm upgrade --install \
    -f ./values.yaml \
    zebrunner oci://registry-1.docker.io/zebrunner/testing-platform --version <version>

where <version> is the version you are going to upgrade to, and ./values.yaml should be replaced with the path to your values.yaml file.

Warning

It is strongly recommended to upgrade one minor version at a time because some versions perform complex one-time jobs during the upgrade.

Some of the minor version releases require the successful deployment of several preparatory releases before the actual minor release. Currently, such version is only 1.2.0 (requires 3 preparatory versions).

For example, if you are going to upgrade Zebrunner from version 1.0.1 to 1.2.0, you should first upgrade to the next minor version: 1.1.0. Then sequentially to 1.2.0-pre1, 1.2.0-pre2, and 1.2.0-pre3. Finally, deploy version 1.2.0. Version 1.1.1 can be skipped.

The current version and release notes for each version can be found here.

Info

If you encounter any problems during upgrade, please contact Zebrunner support.

ArgoCD#

ArgoCD does not use Helm to install or upgrade charts. Under the hood, it utilizes helm template command to obtain raw Kubernetes deployment manifests and applies them one by one to the Kubernetes cluster.

During the initial installation, the Zebrunner Helm chart generates various credentials, including encryption and signing keys, default passwords, and more. Preserving these credentials during an upgrade is essential for the application to function correctly. When you execute the helm upgrade command to upgrade your Zebrunner distribution, this chart fetches the current credentials from the Kubernetes cluster and utilizes them with new version.

However, when syncing a new version of the chart in ArgoCD, the helm template command doesn't establish a connection to the Kubernetes cluster. Instead, it returns empty values for the credentials. Consequently, all application credentials are recreated, which leads to failures during data decryption and application startup.

To prevent such failures when using ArgoCD, you need to export the current credentials by running the following command:

kubectl get secret credentials-storage -o=jsonpath='{.data}' | jq -c 'walk(if type == "string" then @base64d else . end)'

Output of this command is a JSON object containing the raw credentials. You must provide this JSON to the chart using the global.credentialsJson property before syncing the deployment:

global:
  credentialsJson: '{"administrator.email":"support@zebrunner.com","administrator.username":"admin","auth.token.signing-key":"bpwcKCr7g5O3U4A5ads36lCI6llwyvcFKH5mN6apsNHCcucdKhpp7VsLao3FMgYc","encryption-password.integrations-service":"tYIYvSOJJXiFCQmhl1MlnBDLBpAEvVx9",...}'

Failed update of RabbitMQ#

The RabbitMQ cluster deployed by this Helm chart is based on the Bitnami RabbitMQ Helm chart. During an update, the Bitnami chart may print an error message like the following:

Error: UPGRADE FAILED: execution error: 
PASSWORDS ERROR: You must provide your current passwords when upgrading the release.
                 Note that even after reinstallation, old credentials may be needed as they may be kept in persistent volume claims.
                 Further information can be obtained at https://docs.bitnami.com/general/how-to/troubleshoot-helm-chart-issues/#credential-errors-while-upgrading-chart-releases

    'auth.password' must not be empty, please add '--set auth.password=$RABBITMQ_PASSWORD' to the command. To get the current value:

        export RABBITMQ_PASSWORD=$(kubectl get secret --namespace "zebrunner" rabbitmq -o jsonpath="{.data.rabbitmq-password}" | base64 -d)

If the recommendation from the error message does not work, you can try providing the current RabbitMQ password using the rabbitmq.auth.password property instead of the auth.password.

If you have lost access to the password, you can perform rollback to a release state before you applied the corrupting action. To do so, run the following command:

helm rollback zebrunner 14

In this command, zebrunner is the name of the Helm release, and 14 is the version of the release to roll back to.

To see all available release versions, execute:

kubectl get secret

In the output, look for secrets named in the format sh.helm.release.v1.<release>.v<version>.

Helm Chart Deployment Guide#

Dependencies#

Docker Image Pull Password#

Ingress-nginx controller#

PostgreSQL#

RabbitMQ#

S3-compatible storage#

Elasticsearch#

Server Instances#

values.yaml Parameters#

Docker Registry Parameters#

Deployment Parameters#

Ingress Parameters#

Zebrunner Engine Parameters#

S3 Parameters#

RabbitMQ Parameters#

Email Parameters#

Elasticsearch Parameters#

PostgreSQL Parameters#

postgres1 database#

postgres2 database#

postgres3 database#

Migration Parameters#

Application Parameters#

Example#

Deployment#

Migration from hosted (zebrunner.com) instance#

CDN rules#

Upgrade#

ArgoCD#

Failed update of RabbitMQ#

`values.yaml` Parameters#

`postgres1` database#

`postgres2` database#

`postgres3` database#

Migration from hosted (`zebrunner.com`) instance#