Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

License check in "auto-review" mode: the Gitlab token will soon expire #14127

Open
marcdumais-work opened this issue Sep 4, 2024 · 0 comments
Open

License check in "auto-review" mode: the Gitlab token will soon expire #14127

marcdumais-work opened this issue Sep 4, 2024 · 0 comments

Comments

@marcdumais-work
Copy link
Contributor

marcdumais-work commented Sep 4, 2024 •
edited
Loading

Bug Description:

In this repo, the license check that's run in CI uses dash-license's so-called auto-review mode. In that mode, dash-licenses will try to automatically open IP ticket, on the Eclipse Foundation Gitlab, for all dependencies found to require extra scrutiny from the IP team. This requires an EF Gitlab token to work, currently set as a secret in this repo and exposed during CI as an environment variable.

I set the "secrets.DASH_LICENSES_PAT" secret a few years ago, using a token generated using my own Eclipse Gitlab account. I was recently notified that this token is expiring in 2 days, on September 6. AFAIK this should not cause too much disturbance in Theia's CI, but it will mean that committer's PR, using the main repo as source, will no longer trigger automatic opening of IP tickets for newly added dependencies. If it's desired to keep having automatic IP tickets creation, a new Gitlab token should be set as a secret, overwriting the current soon-to-be-expired one. Maybe webmaster can generate a token using an appropriately named bot account and set it up in this repo? Else a normal user token would also work.

Alternatively, "yarn license:check" could be used in CI to report dependencies that need be analysed, but not attempt to open the IP tickets automatically. Then a committer can run the "review" mode locally on their laptop, when the license check CI fails (I do that in cdt.cloud repos I work-on). This still requires a token, but then it does not need to be setup in the repo as a secret.

Steps to Reproduce:

N/A

Additional Information

I simulated what the log will look-like when the token expires, when there are dependencies to open IP tickets about (after performing a local "yarn upgrade") - there is a runtime exception "org.gitlab4j.api.GitLabApiException: 401 Unauthorized" but the results still list the packages that need to be analysed by the IP team:

[main] INFO License information could not be automatically verified for the following content:
[main] INFO 
[main] INFO npm/npmjs/-/eslint-plugin-no-unsanitized/4.1.0
[main] INFO npm/npmjs/@azure/msal-browser/3.23.0
[main] INFO npm/npmjs/@vscode/vsce-sign-alpine-arm64/2.0.2
[main] INFO npm/npmjs/@vscode/vsce-sign-alpine-x64/2.0.2
[main] INFO npm/npmjs/@vscode/vsce-sign-darwin-arm64/2.0.2
[main] INFO npm/npmjs/@vscode/vsce-sign-darwin-x64/2.0.2
[main] INFO npm/npmjs/@vscode/vsce-sign-linux-arm/2.0.2
[main] INFO npm/npmjs/@vscode/vsce-sign-linux-arm64/2.0.2
[main] INFO npm/npmjs/@vscode/vsce-sign-linux-x64/2.0.2
[main] INFO npm/npmjs/@vscode/vsce-sign-win32-arm64/2.0.2
[main] INFO npm/npmjs/@vscode/vsce-sign-win32-x64/2.0.2
[main] INFO npm/npmjs/@vscode/vsce-sign/2.0.4
[main] INFO 
[main] INFO This content is either not correctly mapped by the system, or requires review.
[main] INFO A review is required for npm/npmjs/@vscode/vsce-sign-linux-arm64/2.0.2.
Exception in thread "main" java.lang.RuntimeException: org.gitlab4j.api.GitLabApiException: 401 Unauthorized
	at org.eclipse.dash.licenses.review.GitLabSupport.lambda$createReviews$0(GitLabSupport.java:90)
	at org.eclipse.dash.licenses.review.GitLabSupport.execute(GitLabSupport.java:119)
	at org.eclipse.dash.licenses.review.GitLabSupport.createReviews(GitLabSupport.java:46)
	at org.eclipse.dash.licenses.review.CreateReviewRequestCollector.close(CreateReviewRequestCollector.java:49)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1596)
	at org.eclipse.dash.licenses.cli.Main.doit(Main.java:144)
	at org.eclipse.dash.licenses.cli.Main.main(Main.java:68)
Caused by: org.gitlab4j.api.GitLabApiException: 401 Unauthorized
	at org.gitlab4j.api.AbstractApi.validate(AbstractApi.java:678)
	at org.gitlab4j.api.AbstractApi.get(AbstractApi.java:214)
	at org.gitlab4j.api.Pager.<init>(Pager.java:92)
	at org.gitlab4j.api.IssuesApi.getIssues(IssuesApi.java:190)
	at org.gitlab4j.api.IssuesApi.getIssuesStream(IssuesApi.java:204)
	at org.eclipse.dash.licenses.review.GitLabConnection.lambda$findIssue$1(GitLabConnection.java:40)
	at org.eclipse.dash.licenses.review.GitLabConnection.rateLimit(GitLabConnection.java:80)
	at org.eclipse.dash.licenses.review.GitLabConnection.findIssue(GitLabConnection.java:32)
	at org.eclipse.dash.licenses.review.GitLabSupport.lambda$createReviews$0(GitLabSupport.java:73)
	... 6 more
WARN: Command [
  "java",
  "-jar",
  "/home/<user>/theia/scripts/download/dash-licenses.jar",
  "yarn.lock",
  "-batch",
  "50",
  "-timeout",
  "240",
  "-project",
  "ecd.theia",
  "-summary",
  "/home/<user>/theia/dependency-check-summary.txt",
  "-review",
  "-token",
  "***"
] exited with code: 1
INFO: Checking results against the baseline...
WARN: Some entries in the baseline did not match anything from dash-licenses output:
> npm/npmjs/-/advanced-mark.js/2.6.0
npm/npmjs/-/advanced-mark.js/2.6.0: Manually approved
ERROR: Found results that aren't part of the baseline!
X npm/npmjs/-/eslint-plugin-no-unsanitized/4.1.0, 
X npm/npmjs/@azure/msal-browser/3.23.0, 
X npm/npmjs/@vscode/vsce-sign-alpine-arm64/2.0.2, LicenseRef-scancode-ms-xamarin-uitest3.2.0
X npm/npmjs/@vscode/vsce-sign-alpine-x64/2.0.2, LicenseRef-scancode-ms-xamarin-uitest3.2.0
X npm/npmjs/@vscode/vsce-sign-darwin-arm64/2.0.2, LicenseRef-scancode-ms-xamarin-uitest3.2.0
X npm/npmjs/@vscode/vsce-sign-darwin-x64/2.0.2, LicenseRef-scancode-ms-xamarin-uitest3.2.0
X npm/npmjs/@vscode/vsce-sign-linux-arm/2.0.2, LicenseRef-scancode-ms-xamarin-uitest3.2.0
X npm/npmjs/@vscode/vsce-sign-linux-arm64/2.0.2, LicenseRef-scancode-ms-xamarin-uitest3.2.0
X npm/npmjs/@vscode/vsce-sign-linux-x64/2.0.2, LicenseRef-scancode-ms-xamarin-uitest3.2.0
X npm/npmjs/@vscode/vsce-sign-win32-arm64/2.0.2, LicenseRef-scancode-ms-xamarin-uitest3.2.0
X npm/npmjs/@vscode/vsce-sign-win32-x64/2.0.2, LicenseRef-scancode-ms-xamarin-uitest3.2.0
X npm/npmjs/@vscode/vsce-sign/2.0.4, LicenseRef-scancode-ms-xamarin-uitest3.2.0 AND MIT
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
marcdumais-work added a commit that referenced this issue Sep 6, 2024
@marcdumais-work
[CI] Disable dash-licenses auto-review mode
06dc17f 
Related issue: #14127

The Eclipse Foundation Gitlab token, required for dash-licenses to
automatically open IP ticket for suspicious license in dependencies.
is about to expire. Until it's replaced, we can have the workflow use
the basic mode, where dependecies with suspicious licenses are only
listed, and have to be handled offline [1].

[1]: To have dash-licenses help with opening IP tickets automatically
     e.g. after a PR license check workflow failure. Any committer can
     generate a token from EF Gitlab at the link below and set it in
     an environment variable, and then use it when running dash-licenses
     from their laptop.

e.g.

theia$ git checkout <PR branch> && yarn
theia$ export DASH_LICENSES_PAT="<token>"
theia$ yarn license:check:review

https://gitlab.eclipse.org/-/user_settings/personal_access_tokens

Signed-off-by: Marc Dumais <[email protected]>
@marcdumais-work marcdumais-work mentioned this issue Sep 6, 2024
Open
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@marcdumais-work