Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

slack-vitess-r14.0.5-dsdefense: sync changes from slack-vitess-r14.0.5 #121

Conversation

timvaillancourt
Copy link
Member

@timvaillancourt timvaillancourt commented Aug 31, 2023

Description

This PR syncs the latest changes from slack-vitess-r14.0.5 -> slack-vitess-r14.0.5-dsdefense

Mainly, I'd like to get #119 onto this branch to resolve CI issues

cc @ejortegau

Related Issue(s)

Checklist

  • "Backport to:" labels have been added if this change should be back-ported
  • Tests were added or are not required
  • Did the new or modified tests pass consistently locally and on the CI
  • Documentation was added or is not required

Deployment Notes

Roderick Yao and others added 9 commits July 5, 2023 08:56
Signed-off-by: Priya Bibra <[email protected]>
Signed-off-by: 'Priya Bibra' <[email protected]>
* Empty commit to trigger CI

Signed-off-by: Eduardo J. Ortega U <[email protected]>

* `slack-vitess-r14.0.5-dsdefense`: use larger runner

Signed-off-by: Tim Vaillancourt <[email protected]>

* Use runner group instead

Signed-off-by: Tim Vaillancourt <[email protected]>

* Rename group

Signed-off-by: Tim Vaillancourt <[email protected]>

* Move more jobs to runner group

Signed-off-by: Tim Vaillancourt <[email protected]>

* use vitess-ubuntu20 runner group

Signed-off-by: Tim Vaillancourt <[email protected]>

* Rever change of runner type for e2e ERS PRS new features heavy test

Signed-off-by: Eduardo J. Ortega U <[email protected]>

---------

Signed-off-by: Eduardo J. Ortega U <[email protected]>
Signed-off-by: Tim Vaillancourt <[email protected]>
Co-authored-by: Tim Vaillancourt <[email protected]>
…_heavy. (#116)

We do this because the test is taking a lot longer & failing in some cases with
the larger runner.

Signed-off-by: Eduardo J. Ortega U <[email protected]>
* `slack-vitess-r14.0.5`: use dedicated larger runner

Signed-off-by: Tim Vaillancourt <[email protected]>

* fix fileNameFromPosition

Signed-off-by: Tim Vaillancourt <[email protected]>

* empty commit

Signed-off-by: Tim Vaillancourt <[email protected]>

* empty commit

Signed-off-by: Tim Vaillancourt <[email protected]>

---------

Signed-off-by: Tim Vaillancourt <[email protected]>
* `slack-vitess-r14.0.5`: allow conn overrides in consul topo

Signed-off-by: Tim Vaillancourt <[email protected]>

* fix e2e test

Signed-off-by: Tim Vaillancourt <[email protected]>

---------

Signed-off-by: Tim Vaillancourt <[email protected]>
…error (#117)

* Release-15: Cherry pick vtorc no cgo (vitessio#12223)

* Move vtorc from go-sqlite3 to modernc.org/sqlite (vitessio#12214)

* Move vtorc from go-sqlite3 to modernc.org/sqlite

This moves vtorc from the go-sqlite3 library that uses CGO, to use
modernc.org/sqlite which is a pure Go implementation.

vtorc is the only component we have to build with CGO but it's causing
pain for releases since we need to build it against an old Linux for
linking against glibc.

Using modernc.org/sqlite allows for using Go only again and makes all
Vitess components buildable without CGO.

In
https://datastation.multiprocess.io/blog/2022-05-12-sqlite-in-go-with-and-without-cgo.html
someone ran some basic benchmarks. It shows that the pure Go version can
be twice as slow, but the usage of vtorc is very limited and we operate
on small datasets, so I think the performance impact purely of a
somewhat slower sqlite implementation is negligable.

None of this is in a hot query serving path or anything like that, so I
have little concern performance wise.

Signed-off-by: Dirkjan Bussink <[email protected]>

* Fix error handling in RowToArray

Signed-off-by: Dirkjan Bussink <[email protected]>

---------

Signed-off-by: Dirkjan Bussink <[email protected]>

* empty-commit

Signed-off-by: Manan Gupta <[email protected]>

* Run go mod tidy

Signed-off-by: Dirkjan Bussink <[email protected]>

---------

Signed-off-by: Dirkjan Bussink <[email protected]>
Signed-off-by: Manan Gupta <[email protected]>
Co-authored-by: Manan Gupta <[email protected]>

* Fix Makefile

Signed-off-by: Tim Vaillancourt <[email protected]>

* Undo dir rename

Signed-off-by: Tim Vaillancourt <[email protected]>

---------

Signed-off-by: Dirkjan Bussink <[email protected]>
Signed-off-by: Manan Gupta <[email protected]>
Signed-off-by: Tim Vaillancourt <[email protected]>
Co-authored-by: Dirkjan Bussink <[email protected]>
Co-authored-by: Manan Gupta <[email protected]>
…test + compressors (#119)

* Delete all legacy sharding related code (vitessio#10278)

* Delete all legacy sharding related code

Signed-off-by: Matt Lord <[email protected]>

* Move used until functions from initialsharding to cluster

Signed-off-by: Matt Lord <[email protected]>

* Remove vtctl commands

Signed-off-by: Matt Lord <[email protected]>

* Kill vtworker and SetKeyspaceServedFrom cmd

Signed-off-by: Matt Lord <[email protected]>

* WaitForDrain related stragglers

Signed-off-by: Matt Lord <[email protected]>

* Legacy local straggler workflow

Signed-off-by: Matt Lord <[email protected]>

* Get rid of SetKeyspaceShardingInfo & wait for drain stragglers

Signed-off-by: Matt Lord <[email protected]>

* Remove vtworker stragglers

Signed-off-by: Matt Lord <[email protected]>

* Update throttlerservice protobuf

Signed-off-by: Matt Lord <[email protected]>

* Rename test 24, add hashicorp vault test to it (now mysql_server_vault)

Signed-off-by: Matt Lord <[email protected]>

* Remove last mentiones of legacy sharding in vtctl

Signed-off-by: Matt Lord <[email protected]>

* remove binlog_use_v3_resharding_mode

Signed-off-by: Matt Lord <[email protected]>

* Address review comments

Signed-off-by: Matt Lord <[email protected]>

* Address review comments

Signed-off-by: Matt Lord <[email protected]>

* Correct vtgate help output

Signed-off-by: Matt Lord <[email protected]>

* go fmt

Signed-off-by: Tim Vaillancourt <[email protected]>

* Remove v2 resharding fields (vitessio#10409)

* cleanup: remove sharding_column_name and sharding_column_type

Signed-off-by: Arvind Murty <[email protected]>

* cleanup: remove sharding_column_name and sharding_column_type

Signed-off-by: Arvind Murty <[email protected]>

* cleanup: remove sharding_column_name and sharding_column_type

Signed-off-by: Arvind Murty <[email protected]>

* cleanup: remove sharding_column_name and sharding_column_type

Signed-off-by: Arvind Murty <[email protected]>

* generate vtadmin files

Signed-off-by: Arvind Murty <[email protected]>

* cleanup: remove sharding_column_name and sharding_column_type from vtadmin

Signed-off-by: Arvind Murty <[email protected]>

* Merge from main

Signed-off-by: Arvind Murty <[email protected]>

* Fix bad merge conflict resolution

Signed-off-by: Tim Vaillancourt <[email protected]>

* Fix missing 'sharding' import

Signed-off-by: Tim Vaillancourt <[email protected]>

* Fix bad conflict resolution in go/test/endtoend/cellalias/cell_alias_test.go

Signed-off-by: Tim Vaillancourt <[email protected]>

* Backup/Restore: add support for external compressors and decompressors (vitessio#10558)

* change to support an external decompressor

Signed-off-by: Renan Rangel <[email protected]>
Signed-off-by: Rameez Sajwani <[email protected]>

* add external compressor support + builtin additional compressors

Signed-off-by: Renan Rangel <[email protected]>
Signed-off-by: Rameez Sajwani <[email protected]>

* wrap external compressor/decompressor

Signed-off-by: Renan Rangel <[email protected]>
Signed-off-by: Rameez Sajwani <[email protected]>

* go mod tidy + comments

Signed-off-by: Renan Rangel <[email protected]>
Signed-off-by: Rameez Sajwani <[email protected]>

* add copyright notices

Signed-off-by: Renan Rangel <[email protected]>
Signed-off-by: Rameez Sajwani <[email protected]>

* add support for builtin engine

Signed-off-by: Renan Rangel <[email protected]>
Signed-off-by: Rameez Sajwani <[email protected]>

* Adding test case for buckup compression

Signed-off-by: Rameez Sajwani <[email protected]>

* Fixing unit test and run mod tidy

Signed-off-by: Rameez Sajwani <[email protected]>

* Removing unwanted unit tests

Signed-off-by: Rameez Sajwani <[email protected]>

* Increase timeout of backup tests

Signed-off-by: Rameez Sajwani <[email protected]>

* fixing linter errors

Signed-off-by: Rameez Sajwani <[email protected]>

* Change test logic to accomodate running selective tests

Signed-off-by: Rameez Sajwani <[email protected]>

* removing lint warning

Signed-off-by: Rameez Sajwani <[email protected]>

* fixing test failure

Signed-off-by: Rameez Sajwani <[email protected]>

* Removing un-necessary test

Signed-off-by: Rameez Sajwani <[email protected]>

* Fixing code review feeback

Signed-off-by: Rameez Sajwani <[email protected]>

* Change builtinEngine to consider 'auto' decompressor

Signed-off-by: Rameez Sajwani <[email protected]>

* fixing Upgrade/Downgrade test

Signed-off-by: Rameez Sajwani <[email protected]>

* Fix type & add summary under release notes

Signed-off-by: Rameez Sajwani <[email protected]>

* Fixing typos in summary

Signed-off-by: Rameez Sajwani <[email protected]>

* Fixing flag name typos

Signed-off-by: Rameez Sajwani <[email protected]>

Co-authored-by: Renan Rangel <[email protected]>
Co-authored-by: Renan Rangel <[email protected]>

* Add MySQL 8 Support to Backup Tests (vitessio#10691)

* Add support for MySQL 8.0 in backup tests

Signed-off-by: Matt Lord <[email protected]>

* Add 8.0 workflow

Signed-off-by: Matt Lord <[email protected]>

* whitespace

Signed-off-by: Matt Lord <[email protected]>

* Use vtctldclient SetKeyspaceDurabilityPolicy to manage semi-sync

This needed to be done after the shard was setup in order to satisfy
the semantic assumptions related to semi-sync in the tests.

Signed-off-by: Matt Lord <[email protected]>

* Remove extraneous changes

Signed-off-by: Matt Lord <[email protected]>

* We need lz4 for TestXtrabackupStreamWithlz4Compression

Signed-off-by: Matt Lord <[email protected]>

* Try using Percona Repo for MySQL 8 to align mysqld and xtrabackup versions

Signed-off-by: Matt Lord <[email protected]>

* Specify stream type everywhere

Signed-off-by: Matt Lord <[email protected]>

* Remove repeated server install

Signed-off-by: Matt Lord <[email protected]>

* Moar...

Signed-off-by: Matt Lord <[email protected]>

* Move vtctlbackup test to 8.0

Signed-off-by: Matt Lord <[email protected]>

* Rename vtbackup test and move to MySQL 8

Signed-off-by: Matt Lord <[email protected]>

* Split the xbstream tests so the workflow doesn't time out

Otherwise it was going over the 10min limit and getting
killed.

Signed-off-by: Matt Lord <[email protected]>

* Use MySQL 8 compat method for setting passwords

Signed-off-by: Matt Lord <[email protected]>

* Test increasing timeout at another level

Signed-off-by: Matt Lord <[email protected]>

* Don't use the init passwords file with 8.0

mysqlctl doesn't start...

This is likely due to the change in initialization behavior with
MySQL 8.0 as it goes though two phases and you can't simply
start up mysqld and pass it data, it has to initialize and restart
first.

Signed-off-by: Matt Lord <[email protected]>

* Fix incorrect password update statement for vt_repl user

Signed-off-by: Rohit Nayak <[email protected]>

* Bump timeouts for 8.0 backup tests

Signed-off-by: Matt Lord <[email protected]>

* Bump it more :(

Signed-off-by: Matt Lord <[email protected]>

* Increase backup/restore timeout in backup_utils

Signed-off-by: Matt Lord <[email protected]>

* Apply new 8.0 template everywhere

Signed-off-by: Matt Lord <[email protected]>

* Fix bugs around how the compression flags were getting passed

Signed-off-by: Matt Lord <[email protected]>

* Use 45m timeout for the workflow, 30m for the run.

Signed-off-by: Matt Lord <[email protected]>

* These changes were no longer needed so limiting diff.

Signed-off-by: Matt Lord <[email protected]>

* Explicitly skip new linter check

Signed-off-by: Matt Lord <[email protected]>

Co-authored-by: Rohit Nayak <[email protected]>

* Fix test file merge issues

Signed-off-by: Tim Vaillancourt <[email protected]>

* Go 1.18.7 to fix error

Signed-off-by: Tim Vaillancourt <[email protected]>

* Fix lint

Signed-off-by: Tim Vaillancourt <[email protected]>

* Fix vttablet.txt

Signed-off-by: Tim Vaillancourt <[email protected]>

* Remove sharding_column_name and sharding_column_type from vtctld2 (vitessio#10459)

* Remove sharding_column_name and sharding_column_type

Signed-off-by: notfelineit <[email protected]>

* Run make web_build to compile production files

Signed-off-by: notfelineit <[email protected]>

* Remove vars from test

Signed-off-by: notfelineit <[email protected]>

* Remove sharding column type select box test from web test

Signed-off-by: notfelineit <[email protected]>

---------

Signed-off-by: Matt Lord <[email protected]>
Signed-off-by: Tim Vaillancourt <[email protected]>
Signed-off-by: Arvind Murty <[email protected]>
Signed-off-by: notfelineit <[email protected]>
Co-authored-by: Matt Lord <[email protected]>
Co-authored-by: Arvind Murty <[email protected]>
Co-authored-by: Rameez Sajwani <[email protected]>
Co-authored-by: Renan Rangel <[email protected]>
Co-authored-by: Renan Rangel <[email protected]>
Co-authored-by: Rohit Nayak <[email protected]>
Co-authored-by: Frances Thai <[email protected]>
…e-branch-slack-vitess-r14.0.5-dsdefense

Signed-off-by: Tim Vaillancourt <[email protected]>
@timvaillancourt timvaillancourt requested a review from a team as a code owner August 31, 2023 12:00
Signed-off-by: Tim Vaillancourt <[email protected]>
…into update-branch-slack-vitess-r14.0.5-dsdefense

Signed-off-by: Tim Vaillancourt <[email protected]>
Signed-off-by: Tim Vaillancourt <[email protected]>
Signed-off-by: Tim Vaillancourt <[email protected]>
Signed-off-by: Tim Vaillancourt <[email protected]>
Signed-off-by: Tim Vaillancourt <[email protected]>
Signed-off-by: Tim Vaillancourt <[email protected]>
@github-actions
Copy link

github-actions bot commented Oct 6, 2023

This PR is being marked as stale because it has been open for 30 days with no activity. To rectify, you may do any of the following:

  • Push additional commits to the associated branch.
  • Remove the stale label.
  • Add a comment indicating why it is not stale.

If no action is taken within 7 days, this PR will be closed.

@github-actions github-actions bot added the Stale label Oct 6, 2023
@github-actions
Copy link

This PR was closed because it has been stale for 7 days with no activity.

@github-actions github-actions bot closed this Oct 13, 2023
@timvaillancourt timvaillancourt deleted the update-branch-slack-vitess-r14.0.5-dsdefense branch November 22, 2023 02:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants