Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Legion built without zlib generates uncompressed logs without warning #1738

Open
Tracked by #1032
elliottslaughter opened this issue Aug 4, 2024 · 2 comments
Open
Tracked by #1032
Milestone

Comments

@elliottslaughter
Copy link
Contributor

If you build Legion with USE_ZLIB=0 or -DLegion_USE_ZLIB=OFF (or the auto-detect fails), Legion will generate Legion Prof binary format logs, but those logs will be uncompressed. Current Legion Prof expects logs to always be compressed and cannot parse them unless you manually gzip the files. This is confusing for users and makes inefficient use of disk space and bandwidth.

Note this has nothing to do with the ASCII log format. These are binary logs, just uncompressed.

There are two things we could do (individually or in combination) to address this:

  1. Teach Legion Prof to attempt parsing the logs even if gzip decode fails.
  2. Issue a warning in Legion when profiling is requested and zlib is not available.
@elliottslaughter elliottslaughter added this to the 24.09 milestone Sep 10, 2024
@elliottslaughter
Copy link
Contributor Author

A fix for (1) is in https://gitlab.com/StanfordLegion/legion/-/merge_requests/1462.

Open question whether we want to warn users in this scenario, as the logs will not use disk space/bandwidth as efficiently.

@elliottslaughter
Copy link
Contributor Author

MR 1462 has been merged, @mariodirenzo please confirm this fixes the immediate issue about parsing uncompressed logs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant