Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CHECKLIST] Add more testing to the DKG #614

Open
tbraun96 opened this issue May 19, 2023 · 1 comment
Open

[CHECKLIST] Add more testing to the DKG #614

tbraun96 opened this issue May 19, 2023 · 1 comment

Comments

@tbraun96
Copy link
Contributor

tbraun96 commented May 19, 2023

Overview

Improve the testing in the pipeline, and automate deployment to the testnet.

Checklist

  • Yarn tests: we can stick with the current yarn tests. However, we should also add a t=3,n=5 yarn test to match the testnet, and make these parameters configurable with a command-line flag if possible to run multiple configurations

  • Stress tests: we can take advantage of (stress) testing-harness-v1, and spin up a series of tests to stress test. This has insofar been instrumental in finding bugs, and can help rule out bugs introduced in future PRs.

  • Staging phase: automated launching to the testnet ("staging") after the pipeline is green (upon manual approval). Make the testnet pass before even considering merging to master (unless, of course, the PR does not touch the DKG code). This phase will require a manual approval, but once approved, it will send the PR to the testnet.

@drewstone
Copy link
Contributor

The checklist here @tbraun96 should be more thorough, it should contain amongst other things all todo items for each section. Such as

Yarn tests

  • Add a test for t=3,n=5 to match the testnet
  • Make the yarn tests configurable from a CLI.

Stress Tests

  • Stress test 1
  • Stress test 2
  • ...

Staging Phase

  • Automate the deployment of staging releases to testnet.

etc.

My take is that we don't actually need so much integration tests if we just also hook up things to the runtime as closely as possible too. We can leverage the test externalities environment of Substrate to drive tests that things are working through. This would cover everything but the gossip issues (but to the extent that the harness exposes those we'd be good).

Unit testing improvements

  • Integrate testing harness directly with the runtime environment.
  • Create simulations that drive the testing harness through the TestExternalities structure.
  • Reduce our dependency on integration tests (potentially the yarn tests, etc.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants