forked from vitessio/vitess
-
Notifications
You must be signed in to change notification settings - Fork 2
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add MySQL 8 Support to Backup Tests (vitessio#10691)
* Add support for MySQL 8.0 in backup tests Signed-off-by: Matt Lord <[email protected]> * Add 8.0 workflow Signed-off-by: Matt Lord <[email protected]> * whitespace Signed-off-by: Matt Lord <[email protected]> * Use vtctldclient SetKeyspaceDurabilityPolicy to manage semi-sync This needed to be done after the shard was setup in order to satisfy the semantic assumptions related to semi-sync in the tests. Signed-off-by: Matt Lord <[email protected]> * Remove extraneous changes Signed-off-by: Matt Lord <[email protected]> * We need lz4 for TestXtrabackupStreamWithlz4Compression Signed-off-by: Matt Lord <[email protected]> * Try using Percona Repo for MySQL 8 to align mysqld and xtrabackup versions Signed-off-by: Matt Lord <[email protected]> * Specify stream type everywhere Signed-off-by: Matt Lord <[email protected]> * Remove repeated server install Signed-off-by: Matt Lord <[email protected]> * Moar... Signed-off-by: Matt Lord <[email protected]> * Move vtctlbackup test to 8.0 Signed-off-by: Matt Lord <[email protected]> * Rename vtbackup test and move to MySQL 8 Signed-off-by: Matt Lord <[email protected]> * Split the xbstream tests so the workflow doesn't time out Otherwise it was going over the 10min limit and getting killed. Signed-off-by: Matt Lord <[email protected]> * Use MySQL 8 compat method for setting passwords Signed-off-by: Matt Lord <[email protected]> * Test increasing timeout at another level Signed-off-by: Matt Lord <[email protected]> * Don't use the init passwords file with 8.0 mysqlctl doesn't start... This is likely due to the change in initialization behavior with MySQL 8.0 as it goes though two phases and you can't simply start up mysqld and pass it data, it has to initialize and restart first. Signed-off-by: Matt Lord <[email protected]> * Fix incorrect password update statement for vt_repl user Signed-off-by: Rohit Nayak <[email protected]> * Bump timeouts for 8.0 backup tests Signed-off-by: Matt Lord <[email protected]> * Bump it more :( Signed-off-by: Matt Lord <[email protected]> * Increase backup/restore timeout in backup_utils Signed-off-by: Matt Lord <[email protected]> * Apply new 8.0 template everywhere Signed-off-by: Matt Lord <[email protected]> * Fix bugs around how the compression flags were getting passed Signed-off-by: Matt Lord <[email protected]> * Use 45m timeout for the workflow, 30m for the run. Signed-off-by: Matt Lord <[email protected]> * These changes were no longer needed so limiting diff. Signed-off-by: Matt Lord <[email protected]> * Explicitly skip new linter check Signed-off-by: Matt Lord <[email protected]> Co-authored-by: Rohit Nayak <[email protected]>
- Loading branch information
1 parent
c1e0882
commit df7c355
Showing
25 changed files
with
1,504 additions
and
220 deletions.
There are no files selected for viewing
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
116 changes: 116 additions & 0 deletions
116
.github/workflows/cluster_endtoend_onlineddl_revertible_mysql80.yml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,116 @@ | ||
| # DO NOT MODIFY: THIS FILE IS GENERATED USING "make generate_ci_workflows" | ||
|
|
||
| name: Cluster (onlineddl_revertible) mysql80 | ||
| on: [push, pull_request] | ||
| concurrency: | ||
| group: format('{0}-{1}', ${{ github.ref }}, 'Cluster (onlineddl_revertible) mysql80') | ||
| cancel-in-progress: true | ||
|
|
||
| env: | ||
| LAUNCHABLE_ORGANIZATION: "vitess" | ||
| LAUNCHABLE_WORKSPACE: "vitess-app" | ||
| GITHUB_PR_HEAD_SHA: "${{ github.event.pull_request.head.sha }}" | ||
|
|
||
| jobs: | ||
| build: | ||
| name: Run endtoend tests on Cluster (onlineddl_revertible) mysql80 | ||
| runs-on: | ||
| group: vitess-ubuntu20 | ||
| timeout-minutes: 45 | ||
|
|
||
| steps: | ||
| - name: Check out code | ||
| uses: actions/checkout@v2 | ||
|
|
||
| - name: Check for changes in relevant files | ||
| uses: frouioui/paths-filter@main | ||
| id: changes | ||
| with: | ||
| token: '' | ||
| filters: | | ||
| end_to_end: | ||
| - 'go/**/*.go' | ||
| - 'test.go' | ||
| - 'Makefile' | ||
| - 'build.env' | ||
| - 'go.[sumod]' | ||
| - 'proto/*.proto' | ||
| - 'tools/**' | ||
| - 'config/**' | ||
| - 'bootstrap.sh' | ||
| - '.github/workflows/**' | ||
| - name: Set up Go | ||
| if: steps.changes.outputs.end_to_end == 'true' | ||
| uses: actions/setup-go@v2 | ||
| with: | ||
| go-version: 1.18.3 | ||
|
|
||
| - name: Set up python | ||
| if: steps.changes.outputs.end_to_end == 'true' | ||
| uses: actions/setup-python@v2 | ||
|
|
||
| - name: Tune the OS | ||
| if: steps.changes.outputs.end_to_end == 'true' | ||
| run: | | ||
| echo '1024 65535' | sudo tee -a /proc/sys/net/ipv4/ip_local_port_range | ||
| # Increase the asynchronous non-blocking I/O. More information at https://dev.mysql.com/doc/refman/5.7/en/innodb-parameters.html#sysvar_innodb_use_native_aio | ||
| echo "fs.aio-max-nr = 1048576" | sudo tee -a /etc/sysctl.conf | ||
| sudo sysctl -p /etc/sysctl.conf | ||
| - name: Get dependencies | ||
| if: steps.changes.outputs.end_to_end == 'true' | ||
| run: | | ||
| # Setup Percona Server for MySQL 8.0 | ||
| sudo apt-get update | ||
| sudo apt-get install -y lsb-release gnupg2 curl | ||
| wget https://repo.percona.com/apt/percona-release_latest.$(lsb_release -sc)_all.deb | ||
| sudo DEBIAN_FRONTEND="noninteractive" dpkg -i percona-release_latest.$(lsb_release -sc)_all.deb | ||
| sudo percona-release setup ps80 | ||
| sudo apt-get update | ||
| # Install everything else we need, and configure | ||
| sudo apt-get install -y percona-server-server percona-server-client make unzip g++ etcd git wget eatmydata xz-utils | ||
| sudo service mysql stop | ||
| sudo service etcd stop | ||
| sudo ln -s /etc/apparmor.d/usr.sbin.mysqld /etc/apparmor.d/disable/ | ||
| sudo apparmor_parser -R /etc/apparmor.d/usr.sbin.mysqld | ||
| go mod download | ||
| # install JUnit report formatter | ||
| go install github.com/vitessio/go-junit-report@HEAD | ||
| - name: Setup launchable dependencies | ||
| if: steps.changes.outputs.end_to_end == 'true' | ||
| run: | | ||
| # Get Launchable CLI installed. If you can, make it a part of the builder image to speed things up | ||
| pip3 install --user launchable~=1.0 > /dev/null | ||
| # verify that launchable setup is all correct. | ||
| launchable verify || true | ||
| # Tell Launchable about the build you are producing and testing | ||
| launchable record build --name "$GITHUB_RUN_ID" --source . | ||
| - name: Run cluster endtoend test | ||
| if: steps.changes.outputs.end_to_end == 'true' | ||
| timeout-minutes: 30 | ||
| run: | | ||
| # We set the VTDATAROOT to the /tmp folder to reduce the file path of mysql.sock file | ||
| # which musn't be more than 107 characters long. | ||
| export VTDATAROOT="/tmp/" | ||
| source build.env | ||
| set -x | ||
| # run the tests however you normally do, then produce a JUnit XML file | ||
| eatmydata -- go run test.go -docker=false -follow -shard onlineddl_revertible | tee -a output.txt | go-junit-report -set-exit-code > report.xml | ||
| - name: Print test output and Record test result in launchable | ||
| if: steps.changes.outputs.end_to_end == 'true' && always() | ||
| run: | | ||
| # send recorded tests to launchable | ||
| launchable record tests --build "$GITHUB_RUN_ID" go-test . || true | ||
| # print test output | ||
| cat output.txt |
Oops, something went wrong.