GitHub Actions for Yandex Cloud Load Testing service

NOTE: All actions require auth-key-json to authorize requests to Yandex Cloud services. Please check this documentation page for instructions how to create one.

Short reference

test-suite - execute multiple load tests sequentially
agents-create - create and start load testing agents using Yandex Cloud Compute VMs as a hosting (Compute is billed separately)
agents-delete - delete load testing agents
test-single-run - execute a single load test
test-single-check - check results of a single load test

Action: `test-suite`

uses: yandex-cloud/yc-github-loadtesting-ci/test-suite@main

Execute and assess a bunch of tests in a single action step.

Each test is specified by a separate directory containing:

test-config.yaml test configuration file
(optional) meta.json json file specifying test properties
(optional) additional test data files which will be transferred to the agent via data-bucket

A list of directories containing test-config.yaml test configuration files should be provided via test-directories input parameter.

NOTE: Why data-bucket is needed? Test data files often contain sensitive information such as access tokens and customer data. Using data-bucket as a proxy storage guarantees that this sensitive information never gets stored elsewhere.

Inputs

Input	Description	Default	Required
`folder-id`	ID of a folder in Yandex Cloud.	n/a	yes
`auth-key-json-base64`	BASE64 encoded authorized key string in JSON format. This setting is preferred over `auth-key-json`. The action will perform all operations on behalf of a service account for which this authorized key was generated. Given a json file, encode it via command line `base64 <authorized_key.json >authorized_key.pem` and add the content of result file to GitHub secrets.	n/a	no
`auth-key-json`	Use 'auth-key-json-base64'. An authorized key string in JSON format. The use of this parameter is discouraged because it can lead to unwanted logs obfuscation (see naming your secrets).	``	no
`action-log-level`	Action log level.	`NOTICE`	no
`data-bucket`	Bucket used as a proxy storage to send arbitrary test data needed for test execution to agent. WARNING: CI service account (authorized via auth-key-json(-base64)) must be able to upload data to this bucket WARNING2: agent service account must be able to download data from this bucket.	``	no
`artifacts-bucket`	Bucket to store artifacts generated by agent during test execution. WARNING: agent service account must be able to upload data to this bucket.	``	no
`test-directories`	A whitespace-separated list of directory paths containing test configuration files.	n/a	yes
`agent-filter`	A filter expression to select agents to execute tests. Example: - 'name contains github' - agents with 'github' substring in name. - 'labels.workflow=$WORKFLOW_ID' - agents with label 'workflow' equals to $WORKFLOW_ID	`labels.workflow=${{ github.run_id }}`	no
`add-description`	Specified string will be added to description of created tests.	`Run from GitHub Actions`	no
`add-labels`	Specified labels will be added to label set of created tests. Format: 'key1=value1,key2=value2'.	`ci=github`	no

Outputs

Output	Description
`test-ids`	IDs of performed tests.
`test-infos-file`	File containing a JSON array of objects with information about performed tests.
`test-infos`	JSON array of objects with information about performed tests.
`execution-report-file`	Path to generated .md execution report file.
`checks-report-file`	Path to generated .md checks report file.
`artifacts-dir`	Action artifacts directory. If needed, save it using actions/upload_artifacts.

Required roles

loadtesting.loadTester - to create and run the test
storage.uploader - to upload test data to Object Storage (required, if data-bucket is specified)

Example

- uses: actions/checkout@v4
- uses: yandex-cloud/yc-github-loadtesting-ci/test-suite@main
  with:
    folder-id: ${{ vars.YC_FOLDER_ID }}
    auth-key-json-base64: ${{ secrets.YC_KEY_BASE64 }}
    test-directories: |-
      _impl/testdata/http-const-rps
      _impl/testdata/https-const-rps

Action: `agents-create`

uses: yandex-cloud/yc-github-loadtesting-ci/agents-create@main

Create agents alongside with Cloud Compute VMs to host them.

This action is usually used used in pair with 'agents-delete' action.

NOTE: For security reasons you should create new service account with 'loadtesting.generatorClient' role to assign to loadtesting agent (see service-account-id input below)

Inputs

Input	Description	Default	Required
`folder-id`	ID of a folder in Yandex Cloud.	n/a	yes
`auth-key-json-base64`	BASE64 encoded authorized key string in JSON format. This setting is preferred over `auth-key-json`. The action will perform all operations on behalf of a service account for which this authorized key was generated. Given a json file, encode it via command line `base64 <authorized_key.json >authorized_key.pem` and add the content of result file to GitHub secrets.	n/a	no
`auth-key-json`	Use 'auth-key-json-base64'. An authorized key string in JSON format. The use of this parameter is discouraged because it can lead to unwanted logs obfuscation (see naming your secrets).	``	no
`action-log-level`	Action log level.	`NOTICE`	no
`count`	Number of agents to be created.	`1`	no
`service-account-id`	ID of a service account to create agent VM with. The service account must have 'loadtesting.generatorClient' role.	``	no
`vm-zone`	Compute zone to create agent VM in.	`ru-central1-d`	no
`name-prefix`	If count=0, a name of created agent. if count>0, a name prefix of created agents.	`onetime-ci-agent`	no
`description`	Description of created agents.	`Create from Github Actions`	no
`labels`	Labels of created agents. Format: 'key1=value1,key2=value2'.	`workflow=${{ github.run_id }}`	no
`cli-args`	Additional command line arguments to be passed to `yc loadtesting agent create` (see `yc loadtesting agent create -h` for more details). Override rules: * '--zone ARG' overrides 'vm-zone' input parameter * '--service-account-id ARG' overrides 'service-account-id' input parameter * '--network-interface' overrides default network interface settings (which is - 1-to-1 NAT (dynamic public IP) in automatically chosen subnet) * '--name ARG' overrides 'name-prefix' input parameter * '--description ARG' overrides 'description' input parameter * '--labels' overrides 'labels' input parameter	``	no
`timeout`	Time to wait for agents to become READY_FOR_TEST. Usually, reaching this timeout means either missing permissions or invalid agent network settings.	`600`	no

Outputs

Output	Description
`agent-ids`	IDs of created agents.
`artifacts-dir`	Action artifacts directory. If needed, save it using actions/upload_artifacts.

Required roles

loadtesting.loadTester - to create an agent
compute.editor - to create and start agent VM
iam.serviceAccounts.user - to assign service account to agent VM
vpc.user - to configure network interface on agent VM
vpc.publicAdmin - to configure public IP on agent VM (the default way, may be omitted if you know what you are doing)

Examples

Simple:

# - 2 CPU, 2 GB RAM
# - one-to-one NAT in an automatically selected subnet

uses: yandex-cloud/yc-github-loadtesting-ci/agents-create@main
with:
  folder-id: ${{ vars.YC_FOLDER_ID }}
  auth-key-json-base64: ${{ secrets.YC_KEY_BASE64 }}
  service-account-id: ${{ vars.YC_LOADTESTING_SA_ID }}
  labels: "workflow=${{ github.run_id }}"
  # count: 1
  # vm-zone: ru-central1-a

Custom CPU and RAM settings:

uses: yandex-cloud/yc-github-loadtesting-ci/agents-create@main
with:
  folder-id: ${{ vars.YC_FOLDER_ID }}
  auth-key-json-base64: ${{ secrets.YC_KEY_BASE64 }}
  service-account-id: ${{ vars.YC_LOADTESTING_SA_ID }}
  labels: "workflow=${{ github.run_id }}"
  cli-args: |-
    --cores 2
    --memory 2G
  # count: 1
  # vm-zone: ru-central1-a

Custom network settings:

This version is essentially identical to yc loadtesting agent create ${cli-args}.

uses: yandex-cloud/yc-github-loadtesting-ci/agents-create@main
with:
  folder-id: ${{ vars.YC_FOLDER_ID }}
  auth-key-json-base64: ${{ secrets.YC_KEY_BASE64 }}
  cli-args: |-
    --service-account-id "${{ vars.YC_LOADTESTING_SA_ID }}"
    --labels "workflow=${{ github.run_id }}"
    --cores 2
    --memory 2G
    --zone 'ru-central1-a'
    --network-settings "subnet-name=default-a,security-group-ids=${{ vars.YC_LOADTESTING_AGENT_SECURITY_GROUP_ID }}"
  # count: 1

Action: `agents-delete`

uses: yandex-cloud/yc-github-loadtesting-ci/agents-delete@main

Delete agents (and the VMs).

This action is usually used used in pair with 'agents-create' action.

Inputs

Input	Description	Default	Required
`folder-id`	ID of a folder in Yandex Cloud.	n/a	yes
`auth-key-json-base64`	BASE64 encoded authorized key string in JSON format. This setting is preferred over `auth-key-json`. The action will perform all operations on behalf of a service account for which this authorized key was generated. Given a json file, encode it via command line `base64 <authorized_key.json >authorized_key.pem` and add the content of result file to GitHub secrets.	n/a	no
`auth-key-json`	Use 'auth-key-json-base64'. An authorized key string in JSON format. The use of this parameter is discouraged because it can lead to unwanted logs obfuscation (see naming your secrets).	``	no
`action-log-level`	Action log level.	`NOTICE`	no
`agent-ids`	A string containing whitespace-separated list of agents to be deleted. Normally, should be used with conjunction with `agents-create.outputs.agent-ids`.	n/a	yes

Outputs

Output	Description
`artifacts-dir`	Action artifacts directory. If needed, save it using actions/upload_artifacts.

Required roles

loadtesting.loadTester - to delete an agent
compute.editor - to stop and delete agent VM

Example

This action is usually used in pair with agents-create action:

loadtesting:
  name: loadtesting job
  runs-on: ubuntu-latest
  steps:
    - id: create-agents
      uses: yandex-cloud/yc-github-loadtesting-ci/agents-create@main
      with:
        folder-id: ${{ vars.YC_FOLDER_ID }}
        auth-key-json-base64: ${{ secrets.YC_KEY_BASE64 }}
        service-account-id: ${{ vars.YC_LOADTESTING_SA_ID }}
    #
    # here we run some tests on created agents
    # - id: run-test
    #   ...
    #
    - id: delete-agents
      if: always() # make sure delete is called even if previous steps fail
      uses: yandex-cloud/yc-github-loadtesting-ci/agents-delete@main
      with:
        folder-id: ${{ vars.YC_FOLDER_ID }}
        auth-key-json-base64: ${{ secrets.YC_KEY_BASE64 }}

        # pass created agent ids to make sure they are deleted
        agent-ids: ${{ steps.agent-create.outputs.agent-ids }}

Action: `test-single-run`

uses: yandex-cloud/yc-github-loadtesting-ci/test-single-run@main

Execute a single test.

Each test should be specified by a separate directory containing:

test-config.yaml test configuration file
(optional) meta.json json file specifying test properties
(optional) additional test data files which will be transferred to the agent via data-bucket

A directory containing test-config.yaml test configuration file should be provided via test-directory input parameter.

NOTE: Why data-bucket is needed? Test data files often contain sensitive information such as access tokens and customer data. Using data-bucket as a proxy storage guarantees that this sensitive information never gets stored elsewhere.

Inputs

Input	Description	Default	Required
`folder-id`	ID of a folder in Yandex Cloud.	n/a	yes
`auth-key-json-base64`	BASE64 encoded authorized key string in JSON format. This setting is preferred over `auth-key-json`. The action will perform all operations on behalf of a service account for which this authorized key was generated. Given a json file, encode it via command line `base64 <authorized_key.json >authorized_key.pem` and add the content of result file to GitHub secrets.	n/a	no
`auth-key-json`	Use 'auth-key-json-base64'. An authorized key string in JSON format. The use of this parameter is discouraged because it can lead to unwanted logs obfuscation (see naming your secrets).	``	no
`action-log-level`	Action log level.	`NOTICE`	no
`data-bucket`	Bucket used as a proxy storage to send arbitrary test data needed for test execution to agent. WARNING: CI service account (authorized via auth-key-json(-base64)) must be able to upload data to this bucket WARNING2: agent service account must be able to download data from this bucket.	``	no
`artifacts-bucket`	Bucket to store artifacts generated by agent during test execution. WARNING: agent service account must be able to upload data to this bucket.	``	no
`test-directory`	A path to a directory containing test configuration files.	n/a	yes
`agent-filter`	A filter expression to select agents to execute tests. Example: - 'name contains github' - agents with 'github' substring in name. - 'labels.workflow=$WORKFLOW_ID' - agents with label 'workflow' equals to $WORKFLOW_ID	`labels.workflow=${{ github.run_id }}`	no
`add-description`	Specified string will be added to description of created tests.	`Run from GitHub Actions`	no
`add-labels`	Specified labels will be added to label set of created tests. Format: 'key1=value1,key2=value2'.	`ci=github`	no

Outputs

Output	Description
`test-id`	ID of performed test.
`test-info-file`	File containing a JSON object with information about performed test.
`test-info`	JSON object with information about performed test.
`report-file`	Path to generated .md report file.
`artifacts-dir`	Action artifacts directory. If needed, save it using actions/upload_artifacts.

Required roles

loadtesting.loadTester - to create and run the test
storage.uploader - to upload test data to Object Storage (required, if data-bucket is specified)

Example

- uses: actions/checkout@v4
- uses: yandex-cloud/yc-github-loadtesting-ci/test-run-single@main
  with:
    folder-id: ${{ vars.YC_FOLDER_ID }}
    auth-key-json-base64: ${{ secrets.YC_KEY_BASE64 }}
    test-directory: _impl/testdata/https-const-rps

Action: `test-single-check`

uses: yandex-cloud/yc-github-loadtesting-ci/test-single-check@main

Assess results measured in a single test run.

The assessment is performed via custom scripts (check_summary.sh and check_report.sh) which can be added to test directory:

TEST_ID="bb4asdfjljbhh" # some test id
TEST_DIR="_impl/testdata/http-const-rps" # test-directory

yc --format json loadtesting test get $TEST_ID > summary.json
yc --format json loadtesting test get-report-tables $TEST_ID > report.json

set -e
$DIR/check_summary.sh summary.json
$DIR/check_report.sh report.json

If check_report.sh or check_summary.sh are not found in test directory, some basic checks will be performed instead.

Inputs

Input	Description	Default	Required
`folder-id`	ID of a folder in Yandex Cloud.	n/a	yes
`auth-key-json-base64`	BASE64 encoded authorized key string in JSON format. This setting is preferred over `auth-key-json`. The action will perform all operations on behalf of a service account for which this authorized key was generated. Given a json file, encode it via command line `base64 <authorized_key.json >authorized_key.pem` and add the content of result file to GitHub secrets.	n/a	no
`auth-key-json`	Use 'auth-key-json-base64'. An authorized key string in JSON format. The use of this parameter is discouraged because it can lead to unwanted logs obfuscation (see naming your secrets).	``	no
`action-log-level`	Action log level.	`NOTICE`	no
`test-id`	ID of a test to be checked.	n/a	yes
`test-directory`	Directory with test configuration files.	n/a	yes

Outputs

Output	Description
`raw-output`	Raw stdout of a check script.
`report-file`	Path to generated .md report file.
`artifacts-dir`	Action artifacts directory. If needed, save it using actions/upload_artifacts.

Required roles

loadtesting.loadTester - to retrieve information about the test

Example

- uses: actions/checkout@v4
- uses: yandex-cloud/yc-github-loadtesting-ci/test-run-single@main
  id: run-test
  with:
    folder-id: ${{ vars.YC_FOLDER_ID }}
    auth-key-json-base64: ${{ secrets.YC_KEY_BASE64 }}
    test-directory: _impl/testdata/https-const-rps
- uses: yandex-cloud/yc-github-loadtesting-ci/test-check-single@main
  with:
    folder-id: ${{ vars.YC_FOLDER_ID }}
    auth-key-json-base64: ${{ secrets.YC_KEY_BASE64 }}
    test-id: ${{ steps.run-test.outputs.test-id }}
    test-directory: _impl/testdata/https-const-rps

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

README.md

README.md

GitHub Actions for Yandex Cloud Load Testing service

Short reference

Action: `test-suite`

Inputs

Outputs

Required roles

Example

Action: `agents-create`

Inputs

Outputs

Required roles

Examples

Action: `agents-delete`

Inputs

Outputs

Required roles

Example

Action: `test-single-run`

Inputs

Outputs

Required roles

Example

Action: `test-single-check`

Inputs

Outputs

Required roles

Example

Files

README.md

Latest commit

History

README.md

File metadata and controls

GitHub Actions for Yandex Cloud Load Testing service

Short reference

Action: test-suite

Inputs

Outputs

Required roles

Example

Action: agents-create

Inputs

Outputs

Required roles

Examples

Action: agents-delete

Inputs

Outputs

Required roles

Example

Action: test-single-run

Inputs

Outputs

Required roles

Example

Action: test-single-check

Inputs

Outputs

Required roles

Example

Action: `test-suite`

Action: `agents-create`

Action: `agents-delete`

Action: `test-single-run`

Action: `test-single-check`