Enhancement: Use Sandbox for Testing #360

tzaffi · 2022-08-16T22:48:34Z

+= Sandbox -= Indexer Snapshots

Undertaking the work of removing obsolete indexer steps and testing using the Sandbox.

Defining Issue

algorand/algorand-sdk-testing#221

TODO

Has trim-indexer branch of algorand-sdk-testing been merged?
- Repoint the dependency here to algorand-sdk-testing/master
- Now we can merge

Summary of Changes

.test-env - new test-only environment file
Makefile - more fine grained control over the test process in line with other SDK's
test-harness.sh - replaces test/docker/run_docker.sh and is in line with the new approach of other SDK's
test/:
- applications_integration_test.go - step removal
- applications_unit_test.go - removal of unused helper
- docker/ remove run_docker.sh and sdk.py
helpers.go- removing functions no longer used after step deletion
indexer_integration_test.go - removed as only used by indexer.feature
steps_test.go - step removal
transactions_test.go - step removal

Step Removals

Please refer to the step removal guide in the companion SDK Testing PR.

Additional Steps being removed following Unused Steps Analysis script

See this google sheet

s.Step('^we make a Search For Transactions call with account "([^"]*)" NotePrefix "([^"]*)" TxType "([^"]*)" SigType "([^"]*)" txid "([^"]*)" round (\d+) minRound (\d+) maxRound (\d+) limit (\d+) beforeTime (\d+) afterTime (\d+) currencyGreaterThan (\d+) currencyLessThan (\d+) assetIndex (\d+) addressRole "([^"]*)" ExcluseCloseTo "([^"]*)"$' ...)
s.Step('^the parsed Pending Transactions Information response should have sender "([^"]*)"$' ...)
s.Step('^we make a Pending Transaction Information against txid "([^"]*)" with max (\d+)$' ...)
s.Step('^we make a Pending Transactions By Address call against account "([^"]*)" and max (\d+)$' ...)
s.Step('^we make a Get Block call against block number (\d+)$' ...)
s.Step('^we make a SearchForAssets call with limit (\d+) creator "([^"]*)" name "([^"]*)" unit "([^"]*)" index (\d+) and afterAsset (\d+)$' ...)
s.Step("I create the payment transaction" ...)
s.Step('I read a transaction "([^"]*)" from file "([^"]*)"' ...)
s.Step("I write the transaction to file" ...)
s.Step("the transaction should still be the same" ...)
s.Step("I do my part" ...)
s.Step('key registration transaction parameters (\d+) (\d+) (\d+) "([^"]*)" "([^"]*)" "([^"]*)" (\d+) (\d+) (\d+) "([^"]*)" "' ...)
s.Step("I create the key registration transaction", ...)

Timing Stats

Before: 9m 49s
After: 8m 45s

algochoi

Nice, LGTM - one comment about error handling

test/steps_test.go

…rug the unused result.

test-harness.sh

.test-env

Makefile

tzaffi · 2022-08-18T17:45:36Z

test/integration.tags

+@c2c
+@compile
+@compile.sourcemap
+@dryrun


deleted tags:

@indexer @indexer.231 @indexer.applications

tzaffi · 2022-08-18T17:45:57Z

test/unit.tags

@@ -0,0 +1,26 @@
+@unit.abijson


should be the same tags

michaeldiamant

@tzaffi Nice work here - Thanks for the extra cycles to minimize error when dealing with Cucumber tag configuration.

Aside: I like the spirit of REMOVE_LOCAL_FEATURES - it supports faster local iteration. Though I'm slightly concerned it will lead to difficult to debug states.

Overall, I think it's worth trialing.
In a future iteration, if we think it's worth pursuing, a possible alternative I'm imagining is to supply configuration analogous to sandbox's TYPE="CHANNEL" / TYPE="SOURCE" to conditionally choose the algorand-sdk-testing source. The intent being to find an algorand-sdk-testing installation on the local filesystem.

tzaffi · 2022-08-18T19:32:45Z

@tzaffi Nice work here - Thanks for the extra cycles to minimize error when dealing with Cucumber tag configuration.

Aside: I like the spirit of REMOVE_LOCAL_FEATURES - it supports faster local iteration. Though I'm slightly concerned it will lead to difficult to debug states.

Overall, I think it's worth trialing.

In a future iteration, if we think it's worth pursuing, a possible alternative I'm imagining is to supply configuration analogous to sandbox's TYPE="CHANNEL" / TYPE="SOURCE" to conditionally choose the algorand-sdk-testing source. The intent being to find an algorand-sdk-testing installation on the local filesystem.

I introduced REMOVE_LOCAL_FEATURES when I realized that the previous behavior in the SDK's did not wipe out new features, so I didn't want to spring any surprises on anyone who was expecting the maintenance of this behavior. But I can see the advantages of just wiping clean each time and not needing a switch. I don't feel strongly about this.

tzaffi · 2022-08-22T00:23:30Z

Makefile

@@ -1,6 +1,9 @@
 SRCPATH     := $(shell pwd)
 TEST_SOURCES := $(shell cd $(SRCPATH) && go list ./...)
 TEST_SOURCES_NO_CUCUMBER := $(shell cd $(SRCPATH) && go list ./... | grep -v test)
+UNIT_TAGS :=  "$(shell awk '{print $2}' test/unit.tags | paste -s -d, -)"
+INTEGRATIONS_TAGS := "$(shell awk '{print $2}' test/integration.tags | paste -s -d, -)"


Cucumber tags now reside in:

test/unit.tags

test/integration.tags

tzaffi · 2022-08-22T00:23:59Z

Makefile

+	cd test && go test -timeout 0s --godog.strict=true --godog.format=pretty --godog.tags=$(INTEGRATIONS_TAGS) --test.v .
+
+display-all-go-steps:
+	find test -name "*.go" | xargs grep "github.com/cucumber/godog" 2>/dev/null | cut -d: -f1 | sort | uniq | xargs grep -Eo "Step[(].[^\`]+" | awk '{sub(/:Step\(./,":")} 1' | sed -E 's/", [a-zA-Z0-9]+\)//g' #| grep "with path"


Command that generates text used by Unused Steps Analysis script

.test-env

michaeldiamant

Approval assumes #360 (comment) is addressed prior to merge.

Co-authored-by: Michael Diamant <[email protected]>

Makefile

Co-authored-by: Michael Diamant <[email protected]>

* Github-Actions: Adding pr title and label checks (#336) * Dev Tools: Source map decoder (#335) * Updated CHANGELOG.md * AVM: Consolidate TEAL and AVM versions (#345) * Testing: Use Dev mode network for cucumber tests (#349) * AVM: Use avm-abi repo (#352) * Updated CHANGELOG.md * Enhancement: Use Sandbox for Testing (#360) Co-authored-by: Michael Diamant <[email protected]> * Bug-Fix: passthru verbosity (#371) * Enhancement: Deprecating use of langspec (#366) Co-authored-by: Michael Diamant <[email protected]> * StateProof: State proof support (#374) * Regenerate client. * Update txn and block objects, cucumber tests passing. * Enable cucumber tests. * Update types/stateproof.go * Add transactions-root-256 * Remove unused code. * Remove unused file * Add txn commitments blockheader fields * Rename signature, add allocbound definitions * Add txn commitment to header * PR comments * Remove unused file Co-authored-by: Will Winder <[email protected]> * State Proofs: Use generic type for StateProof txn field. (#378) * Use generic type for StateProof txn field. * Add missing type. * Remove GenericDigest. * fix type assertion on source map version (#370) * let the json package figure out how to decode the elements of the incoming map (#380) * StateProof: State Proof Verification additions (#377) * Regenerate client. * Remove unused file * moved files from stateproof verification repo * now using updated go-stateproof-verification * fixed go mod * moved consts near their usage * added working state proof verifier test * added unit test and resources * now using go 1.15 supported function * changes to avoid dependency of stateproof verification on SDK * added needed types to stateproof.go for message hashing * reverted packages, now using up to date go-stateproof-verification * CR: renamed verification functions * hashing sp message is now in crypto * removed hashing of light block header * CR: now allocating space upfront in hashing state proof message * attempting to use embedded files * CR: using GenericDigest in block.go * CR: Renamed package to match directory * Removed go 1.15 from CI * improved embed usage * CR: removed stateproof verification logic * tidied go mod file Co-authored-by: Will Winder <[email protected]> Co-authored-by: Eric Warehime <[email protected]> * State Proofs: added compute leaf function for light block header to sdk (#382) * added compute leaf function for light block header to sdk * renamed blockheader256 * changed passed type to pointer * reverted back to copying * renamed light block header hash func (#383) * Enable stpf cucumber unit tests (#386) Co-authored-by: Jack <[email protected]> Co-authored-by: Ben Guidarelli <[email protected]> Co-authored-by: Jack Smith <[email protected]> Co-authored-by: Michael Diamant <[email protected]> Co-authored-by: algochoi <[email protected]> Co-authored-by: Jason Paulos <[email protected]> Co-authored-by: Barbara Poon <[email protected]> Co-authored-by: Zeph Grunschlag <[email protected]> Co-authored-by: Eric Warehime <[email protected]> Co-authored-by: Will Winder <[email protected]> Co-authored-by: Almog Tal <[email protected]>

Enhancement: Use Sandbox for Testing

dad6d1e

tzaffi marked this pull request as draft August 16, 2022 22:48

tzaffi added the Enhancement label Aug 16, 2022

Zeph Grunschlag added 2 commits August 16, 2022 23:04

almost passing

ffb2347

dependent 2nd order removals

62ab557

tzaffi marked this pull request as ready for review August 17, 2022 04:47

tzaffi requested a review from algochoi August 17, 2022 05:23

algochoi reviewed Aug 17, 2022

View reviewed changes

test/steps_test.go Outdated Show resolved Hide resolved

Zeph Grunschlag added 3 commits August 17, 2022 08:35

per CR: try to percolate a read error, instead of sweeping under the …

a601bc3

…rug the unused result.

unswallow more err's

2c6b93a

don't default to deleting local features, and introduce verbose mode

e293ed5

tzaffi commented Aug 17, 2022

View reviewed changes

test-harness.sh Outdated Show resolved Hide resolved

tzaffi and others added 2 commits August 17, 2022 11:08

Update test-harness.sh

5fd47ef

variable for destination test directory

46e3138

tzaffi requested review from algochoi and michaeldiamant August 17, 2022 21:56

tzaffi commented Aug 17, 2022

View reviewed changes

.test-env Outdated Show resolved Hide resolved

Update .test-env

4c521a5

michaeldiamant reviewed Aug 18, 2022

View reviewed changes

Makefile Outdated Show resolved Hide resolved

This was referenced Aug 18, 2022

Enhancement: Use Sandbox for Testing algorand/java-algorand-sdk#363

Merged

Enhancement: Use sandbox for SDK Testing and remove Indexer v1 steps algorand/js-algorand-sdk#623

Merged

bite the bullet and seperate out the cucumber tags

9d842a9

tzaffi commented Aug 18, 2022

View reviewed changes

test/unit.tags

@@ -0,0 +1,26 @@

@unit.abijson

Copy link

Contributor Author

tzaffi Aug 18, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should be the same tags

quotes

9514d6a

michaeldiamant approved these changes Aug 18, 2022

View reviewed changes

Zeph Grunschlag added 2 commits August 21, 2022 18:10

display-all-go-steps used by Unused Steps Script

dc02aeb

removing steps after Unused Steps Analysis Script

0d40c4a

tzaffi requested a review from michaeldiamant August 22, 2022 00:22

tzaffi commented Aug 22, 2022

View reviewed changes

Zeph Grunschlag added 2 commits August 22, 2022 09:47

Merge remote-tracking branch 'origin' into sandbox-sdk-testing

58c3684

add note about starting test harness without running all tests

be7c151

michaeldiamant reviewed Aug 22, 2022

View reviewed changes

.test-env Outdated Show resolved Hide resolved

michaeldiamant approved these changes Aug 22, 2022

View reviewed changes

Update .test-env

4cd2d66

Co-authored-by: Michael Diamant <[email protected]>

tzaffi commented Aug 22, 2022

View reviewed changes

Makefile Outdated Show resolved Hide resolved

remove comment from make display-* command

2951489

tzaffi merged commit 0bc0ce7 into develop Aug 22, 2022

tzaffi deleted the sandbox-sdk-testing branch August 22, 2022 16:25

michaeldiamant mentioned this pull request Aug 26, 2022

Enhancement: Remove the dataset indexer images and the indexer.feature + utilize sandbox for algod/indexer/postgres containers algorand/algorand-sdk-testing#220

Merged

8 tasks

almog-t pushed a commit to almog-t/go-algorand-sdk that referenced this pull request Aug 29, 2022

Enhancement: Use Sandbox for Testing (algorand#360)

1cd2f26

Co-authored-by: Michael Diamant <[email protected]>

This was referenced Sep 1, 2022

FOR REVIEW ONLY - go-algorand-sdk v1.20.0 #385

Closed

FOR REVIEW ONLY - go-algorand-sdk v1.20.0 #388

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Enhancement: Use Sandbox for Testing #360

Enhancement: Use Sandbox for Testing #360

tzaffi commented Aug 16, 2022 •

edited

Loading

algochoi left a comment

tzaffi Aug 18, 2022

tzaffi Aug 18, 2022

michaeldiamant left a comment

tzaffi commented Aug 18, 2022

tzaffi Aug 22, 2022

tzaffi Aug 22, 2022

michaeldiamant left a comment

Enhancement: Use Sandbox for Testing #360

Enhancement: Use Sandbox for Testing #360

Conversation

tzaffi commented Aug 16, 2022 • edited Loading

+= Sandbox -= Indexer Snapshots

Defining Issue

TODO

Summary of Changes

Step Removals

Additional Steps being removed following Unused Steps Analysis script

Timing Stats

algochoi left a comment

Choose a reason for hiding this comment

tzaffi Aug 18, 2022

Choose a reason for hiding this comment

tzaffi Aug 18, 2022

Choose a reason for hiding this comment

michaeldiamant left a comment

Choose a reason for hiding this comment

tzaffi commented Aug 18, 2022

tzaffi Aug 22, 2022

Choose a reason for hiding this comment

tzaffi Aug 22, 2022

Choose a reason for hiding this comment

michaeldiamant left a comment

Choose a reason for hiding this comment

tzaffi commented Aug 16, 2022 •

edited

Loading