-
Notifications
You must be signed in to change notification settings - Fork 254
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix some nondeterminism when generating binpkgs #1098
Conversation
ismell
commented
Sep 19, 2023
- config: Don't directly modify FEATURES
- bin/phase-functions: Move du stats into subshell
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please review our copyright policy and add Signed-off-by.
Iterating on self.features returns items in a non-deterministic order, it also leaves self.features out of sync. If we instead use the `remove` method, the values get sorted and synchronized correctly. Bug: https://bugs.gentoo.org/914441 Signed-off-by: Raul E Rangel <[email protected]>
These variables are only used inside this subshell. This avoids polluting the environment. Apparently this calculation isn't hermetic. I'm not sure why: @@ -268,10 +268,10 @@ declare -x cros_setup_hooks_run="booya" declare -a exclude_hermetic=([0]="--exclude-non-hermetic") declare -- f -declare -a isz=([0]="264" [1]="/var/tmp/portage/dev-libs/libffi-3.1-r8/image/") -declare -a nsz=([0]="2803" [1]="/var/tmp/portage/dev-libs/libffi-3.1-r8/work") +declare -a isz=([0]="16" [1]="/var/tmp/portage/dev-libs/libffi-3.1-r8/image/") +declare -a nsz=([0]="2599" [1]="/var/tmp/portage/dev-libs/libffi-3.1-r8/work") declare -- phase_func __eapi6_src_install () Bug: https://bugs.gentoo.org/914441 Signed-off-by: Raul E Rangel <[email protected]>
Without the sort, we end up extending the USE_EXPAND variables with a non deterministic ordering. This makes binary package creation non hermetic. i.e., ``` -declare -x PYTHON_TARGETS="python3_8 python3_7 python3_6 python3_10 python3_9" +declare -x PYTHON_TARGETS="python3_8 python3_9 python3_10 python3_7 python3_6" ``` Assuming `PYTHON_TARGETS=python3_8` in make.conf, after this change we get: ``` declare -x PYTHON_TARGETS="python3_8 python3_10 python3_6 python3_7 python3_9" ``` Ideally we would completely sort the USE_EXPAND variables, but the LINGUAS variable appears to need a very specific ordering. Bug: https://bugs.gentoo.org/914441 Signed-off-by: Raul E Rangel <[email protected]>
Oops. I added the signed off line and also added an additional patch for some non-determinism when generating USE_EXPAND variables. |
Thank you! |
The issue is that |
Oh interesting. I guess we need to keep the open coded removal. I can add a sorted(x) instead.
…On Fri, Oct 13, 2023, at 8:43 AM, Zac Medico wrote:
> @ismell <https://github.com/ismell> Please take a look at the issue mentioned in b150419 <b150419>.
>
The issue is that `self.features.remove("test")` creates a persistent setting in `self._features_overrides` which is not reverted by `self.reset()`. Meanwhile, direct modification of the FEATURES variable just changes `self.configdict["env"]` which is reverted by each call to `self.reset()`.
—
Reply to this email directly, view it on GitHub <#1098 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAEJOIUHCL7AK22PFRJASULX7FHQ5ANCNFSM6AAAAAA464JXDM>.
You are receiving this because you were mentioned.Message ID: ***@***.***>
|
Without the sort, we end up extending the USE_EXPAND variables with a non deterministic ordering. This makes binary package creation non hermetic. i.e., ``` -declare -x PYTHON_TARGETS="python3_8 python3_7 python3_6 python3_10 python3_9" +declare -x PYTHON_TARGETS="python3_8 python3_9 python3_10 python3_7 python3_6" ``` Assuming `PYTHON_TARGETS=python3_8` in make.conf, after this change we get: ``` declare -x PYTHON_TARGETS="python3_8 python3_10 python3_6 python3_7 python3_9" ``` Ideally we would completely sort the USE_EXPAND variables, but the LINGUAS variable appears to need a very specific ordering. Bug: https://bugs.gentoo.org/914441 Signed-off-by: Raul E Rangel <[email protected]> Closes: gentoo#1098 Signed-off-by: Sam James <[email protected]>
Thanks! Please do. |
Without the sort, we end up extending the USE_EXPAND variables with a non deterministic ordering. This makes binary package creation non hermetic. i.e., ``` -declare -x PYTHON_TARGETS="python3_8 python3_7 python3_6 python3_10 python3_9" +declare -x PYTHON_TARGETS="python3_8 python3_9 python3_10 python3_7 python3_6" ``` Assuming `PYTHON_TARGETS=python3_8` in make.conf, after this change we get: ``` declare -x PYTHON_TARGETS="python3_8 python3_10 python3_6 python3_7 python3_9" ``` Ideally we would completely sort the USE_EXPAND variables, but the LINGUAS variable appears to need a very specific ordering. Bug: https://bugs.gentoo.org/914441 Signed-off-by: Raul E Rangel <[email protected]> Closes: gentoo#1098 Signed-off-by: Sam James <[email protected]>
Thanks for pushing the new commit! |
i think that's on one of @palao's branches but not sure where... |
This reverts commit b150419. It turns out we need to keep the open coded version to avoid creating a persistent setting. See gentoo#1098 (comment) This change just adds a sorted() so we get deterministic ordering. Bug: https://bugs.gentoo.org/914441 Signed-off-by: Raul E Rangel <[email protected]>
This reverts commit b150419. It turns out we need to keep the open coded version to avoid creating a persistent setting. See gentoo#1098 (comment) This change just adds a sorted() so we get deterministic ordering. Bug: https://bugs.gentoo.org/914441 Signed-off-by: Raul E Rangel <[email protected]>
This reverts commit b150419. It turns out we need to keep the open coded version to avoid creating a persistent setting. See #1098 (comment) This change just adds a sorted() so we get deterministic ordering. Bug: https://bugs.gentoo.org/914441 Signed-off-by: Raul E Rangel <[email protected]> Closes: #1312 Signed-off-by: Zac Medico <[email protected]>