Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat: Add statistics info for jsontests #258

Merged

Conversation

mrLSD
Copy link
Contributor

@mrLSD mrLSD commented Dec 6, 2023

Description

➡️ At the moment, when tests are run, the output results are uninformative and difficult to analyze. This PR adds minimal statistical information to the tests passed and changes the output to make it more convenient to analyze.

🌇 Changes

  • Added TestCompletionStatus type
  • For test-run functions added completion status result
  • Added statistic info function for:
    • File test
    • Directory test
    • Total tests pass
  • Changed output for tests with format:
[FORK_NAME] SHORT_TEST_FILE_NAME | TEST_NAME/INDEX: COMPLETION_STATUS

✅ Example output for a single test

[Berlin] VMTests/vmTests/calldataload.json | calldataload/2: ok
[Cancun] VMTests/vmTests/calldataload.json | calldataload/0: skipped

✅ Example statistic info output

COMPLETED: 3 tests
SKIPPED: 15 tests

TOTAL tests for: jsontests/res/ethtests/GeneralStateTests/VMTests/vmTests/
	COMPLETED: 136
	SKIPPED: 680

TOTAL: 5240 tests
	COMPLETED: 772
	SKIPPED: 4468

for filename in fs::read_dir(filename)? {
let filepath = filename?.path();
let filename = filepath.to_str().ok_or(Error::NonUtf8Filename)?;
run_file(filename, debug)?;
println!("RUM for: {filename}");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
println!("RUM for: {filename}");
println!("RUN for: {filename}");

@sorpaas sorpaas merged commit ae87a5b into rust-ethereum:master Dec 6, 2023
3 checks passed
// Print total statistics info
pub fn print_total(&self) {
println!(
"\nTOTAL: {:?} tests\n\tCOMPLETED: {:?}\n\tSKIPPED: {:?}",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"\nTOTAL: {:?} tests\n\tCOMPLETED: {:?}\n\tSKIPPED: {:?}",
"\nTOTAL: {} tests\n\tCOMPLETED: {}\n\tSKIPPED: {}",

@mrLSD mrLSD deleted the feat/jsontests-statistics branch December 6, 2023 11:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants