Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The 'auto build' should not use Maven's target folder #1381

Open
anthonyvdotbe opened this issue Apr 12, 2020 · 23 comments
Open

The 'auto build' should not use Maven's target folder #1381

anthonyvdotbe opened this issue Apr 12, 2020 · 23 comments

Comments

@anthonyvdotbe
Copy link

anthonyvdotbe commented Apr 12, 2020

The fact that the 'auto build' uses Maven's target folder is problematic. It causes independent Maven builds to fail in mysterious ways. Or even worse: the build might succeed, but the build artifacts may differ from the ones that are expected (e.g. ecj-compiled classes may end up in build artifacts, even though the Maven build itself uses javac).

Environment
  • Operating System: Windows 10
  • JDK version: JDK 14
  • Visual Studio Code version: 1.44.0
  • Java extension version: 0.59.1
Steps To Reproduce

As an example of the build that might fail:

  1. git clone https://github.com/validator/htmlparser.git
  2. Add Folder to Workspace... -> htmlparser
  3. open a Terminal (inside VS Code) & set the JAVA_HOME environment variable to a JDK 8 installation, e.g. $Env:JAVA_HOME='C:\Program Files\Java\jdk1.8.0_202'
  4. in the pom.xml, add the following under build.plugins & ignore any messages about build synchronization
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-javadoc-plugin</artifactId>
        <version>3.2.0</version>
        <configuration>
          <doclint>none</doclint>
        </configuration>
      </plugin>
  1. run mvn clean source:jar javadoc:jar package repository:bundle-create in the terminal

Note: this will not fail consistently, and quite often it will also just work. But that's simply because it's about uncontrollable interference between 2 processes.

Current Result

I've seen the build fail with 3 different kinds of errors already (note that the first one explicitly mentions that the failure is possibly caused by ecj-generated classes):

[INFO] --- maven-bundle-plugin:2.3.7:bundle (default-bundle) @ htmlparser ---
[ERROR] Bundle nu.validator.htmlparser:htmlparser:bundle:1.4 : The default package '.' is not permitted by the Import-Package syntax.
 This can be caused by compile errors in Eclipse because Eclipse creates
valid class files regardless of compile errors.
The following package(s) import from the default package [nu.validator.htmlparser.io]
[ERROR] Error(s) found in bundle configuration
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] --- maven-compiler-plugin:3.8.1:testCompile (default-testCompile) @ htmlparser ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 25 source files to C:\Users\me\Desktop\htmlparser\target\test-classes
[INFO] Some messages have been simplified; recompile with -Xdiags:verbose to get full output
[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] /C:/Users/me/Desktop/htmlparser/test-src/nu/validator/htmlparser/test/TokenPrinter.java:[44,8] nu.validator.htmlparser.test.TokenPrinter is not abstract and does not override abstract method endTag(ElementName) in nu.validator.htmlparser.common.TokenHandler
[ERROR] /C:/Users/me/Desktop/htmlparser/test-src/nu/validator/htmlparser/test/TokenPrinter.java:[140,28] cannot access Tokenizer
  class file for Tokenizer not found
[ERROR] /C:/Users/me/Desktop/htmlparser/test-src/nu/validator/htmlparser/test/TokenizerTester.java:[101,29] incompatible types: nu.validator.htmlparser.impl.ErrorReportingTokenizer cannot be converted 
to Tokenizer
[ERROR] /C:/Users/me/Desktop/htmlparser/test-src/nu/validator/htmlparser/test/JSONArrayTokenHandler.java:[41,8] nu.validator.htmlparser.test.JSONArrayTokenHandler is not abstract and does not override 
abstract method endTag(ElementName) in nu.validator.htmlparser.common.TokenHandler
[INFO] 4 errors
[INFO] -------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] --- maven-bundle-plugin:2.3.7:bundle (default-bundle) @ htmlparser ---
[ERROR] An internal error occurred
java.io.FileNotFoundException: C:\Users\me\Desktop\htmlparser\target\classes\nu\validator\htmlparser\annotation\package.html (The system cannot find the file specified)
    at java.io.FileInputStream.open0 (Native Method)
    at java.io.FileInputStream.open (FileInputStream.java:195)
    at java.io.FileInputStream.<init> (FileInputStream.java:138)
    at aQute.lib.osgi.FileResource.openInputStream (FileResource.java:15)
    at aQute.lib.osgi.FileResource.copy (FileResource.java:36)
    at aQute.lib.osgi.FileResource.write (FileResource.java:31)
    at aQute.lib.osgi.Jar.writeResource (Jar.java:445)
    at aQute.lib.osgi.Jar.write (Jar.java:224)
    at aQute.lib.osgi.Jar.write (Jar.java:192)
    at org.apache.felix.bundleplugin.BundlePlugin.execute (BundlePlugin.java:362)
    at org.apache.felix.bundleplugin.BundlePlugin.execute (BundlePlugin.java:264)
    at org.apache.felix.bundleplugin.BundlePlugin.execute (BundlePlugin.java:255)
    at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
    at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
    at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
    at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
    at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
    at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
    at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
    at org.apache.maven.cli.MavenCli.execute (MavenCli.java:957)
    at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:289)
    at org.apache.maven.cli.MavenCli.main (MavenCli.java:193)
    at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke (Method.java:498)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
    at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
    at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
    at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
Expected Result

The build always succeeds and none of the build artifacts contain any ecj-generated classes.

@anthonyvdotbe anthonyvdotbe changed the title Extension should not touch Maven's target folder The 'auto build' should not use Maven's target folder Dec 13, 2020
@wheezil
Copy link

wheezil commented May 20, 2021

+1

We hit this all the time on Windows, and after blaming our anti-virus, backup software and everything else we could think of, we finally stopped using VS.code for java and everything was better. This plugin in VS.code seems to interfere with maven builds on the command-line, leading to all sorts of weird errors, especially where there are code-generation steps involved. It often manifests as "generated class does not exist" or "generated class doesn't implement an abstract method" or "generated package doesn't exist", but when we go look at the offending generated item, it is always fine after the fact. I've also seen errors reported in target class files, as in "the file XXX.class contains class YYY".

@DeegC
Copy link

DeegC commented May 20, 2021

I have the same problem as wheezil on Linux. It seems worse in projects with code generation. On vanilla projects vscode is ok but in workspaces with code generation it is near unusable.

@testforstephen
Copy link
Collaborator

@wheezil and @DeegC thank you for reaching out. Would you mind sharing more about the specific problems you have with code generation in VS Code? Is that your 3rd party maven code generation plugin is not well recognized by VS Code Java extension?

@wheezil
Copy link

wheezil commented May 21, 2021

Hi @testforstephen. The issue doesn't really manifest in VS.code itself. It is more like VS code is continuously building and/or/analyzing the java code, and that activity (at least on Windows) seems to occur simultaneously with a "mvn install" performed on the command-line, and interferes with the maven build. I suspect that our code-generation causes updated files to be seen by VS code, and it responds immediately with its building/scanning operation to keep up to date. However, this operation (apparently) opens files in such a way that it prevents or delays visibility of files created by our code-generator or javac.

We use this maven plugin to integrate the generated code:

<plugin>
  <groupId>org.codehaus.mojo</groupId>
  <artifactId>build-helper-maven-plugin</artifactId>
  <version>3.2.0</version>
  <executions>
    <execution>
      <id>add-source</id>
      <phase>generate-sources</phase>
      <goals>
        <goal>add-source</goal>
      </goals>
      <configuration>
        <sources>
          <source>${project.build.directory}/generated-sources/java/</source>
          <source>${project.build.directory}/generated-sources/resources/</source>
        </sources>
      </configuration>
    </execution>
  </executions>
</plugin>

Our code-generation step is custom, basically ingesting a service IDL and generating java.

<plugin>
  <groupId>org.codehaus.mojo</groupId>
  <artifactId>exec-maven-plugin</artifactId>
  <executions>
    <execution>
      <id>ipc-generator</id>
      <phase>generate-sources</phase>
      <configuration>
      ...
      </configuration>
      <goals>
        <goal>exec</goal>
      </goals>
    </execution>
  </executions>
</plugin>

This tool tries to do a "minimal update" of the target generated code, by first writing to a temporary file, and comparing to the target. If the target is missing or different, it then deletes the target and renames the temp to the target. I have seen before on Windows, that if someone has a file open, you are allowed to delete it and rename another file onto it, but global visibility of the change is somehow delayed until the file handle is closed. This may influence the issue.

However, while generated code is most often seen as being involved in the issue, this is not always the case. I've also seen that .class files created during the build are reported as corrupt by later stages that attempt to use them.

In all cases, when we investigate the error reported by the build, the files in question are perfectly fine. It is as if the build is getting a stale or transient snapshot of the build-product files, seeing incorrect content during a very brief window.

Finally, we don't have this issue on NetBeans, although it has other problems keeping up to date with generated code -- seems like nobody really has got this right.

@DeegC
Copy link

DeegC commented May 23, 2021

I don't have too much to add; wheezil covered most of it. I use Java Immutables. If I run mvn install (or test, etc) while vscode is open then the build never works. The errors can be different but it's usually the compiler complaining about a missing generated file. I have to turn off autobuild in vscode and run clean to get it maven to finish without errors.

The reverse is also a problem; running maven will screw up vscode so it can't find generated/compiled files. Any time I want to run maven commands I'm forced to go through a shuffle of:

  1. Turn off vscode's autobuild.
  2. mvn clean install
  3. When I'm done, turn on autobuild back on.
  4. Force clean and delete of env in vscode.

You guys would know better than me but it seems this could be avoided by having vscode use a different target directory.

@fbricon
Copy link
Collaborator

fbricon commented May 23, 2021

As a possible workaround, add the following bits to your project('s parent) pom.xml:

<properties>
   ...
  <build.output>target</build.output>
</properties>
....
<build>
  <directory>${build.output}</directory>
  ...
</build>
...
<profiles>
  <profile>
    <activation>
      <property><name>m2e.version</name></property>
    </activation>
    <properties>
      <build.output>target-ide</build.output>
    </properties>
  </profile>
...
<profiles>

this allows the build output directory to be overridden to target-ide when running in m2e (vscode-java, Eclipse IDE, vim...), while maintaining the default target during CLI builds. Don't forget to add target-ide to your .gitignore

@wheezil
Copy link

wheezil commented May 23, 2021

@fbricon thanks, that is promising. I have to admit I do not know what this is doing:

    <activation>
         <property><name>m2e.version</name></property>
    </activation>

Do the IDEs automatically inject this property or do I need to configure them to define the property?
Do you know if this also works with Netbeans? We have related but different problems there.
john

@fbricon
Copy link
Collaborator

fbricon commented May 23, 2021

It'll only work for IDEs based on Eclipse m2e. m2e injects the m2e.version property to the Maven reactor during incremental builds. So it's possible to activate certain profiles by checking for the presence of the m2e.version property in the build. In our case, that profile, when running in the context of m2e, overrides the build directory.
I don't know/use Netbeans, so I'm not aware of a similar mechanism in that IDE.

@testforstephen
Copy link
Collaborator

@wheezil @DeegC Thank you for sharing the specific details about this issue. Yes, the property m2e.version is automatically injected by VS Code Java tooling when importing your project into VS Code, no need to configure it by yourself. Could you have a try on @fbricon's suggestion? thanks.

@wheezil
Copy link

wheezil commented May 24, 2021

@testforstephen thanks for the update. I'm not the VS.code user in our development group, but I will relay the information. Since it involves a change to the pom.xml files and not just user settings.xml it might take some negotiation to make the change.

@wheezil
Copy link

wheezil commented May 24, 2021

Good news! The workaround by @fbricon seems to work for us.

@MarcelMerkle
Copy link

I have the exact same problem. It would be really nice to have a workaround, or better solution, that does not involve changing the actual build files.

@CarlPer
Copy link

CarlPer commented Oct 12, 2021

I do not have the possibility to use the workaround, will we get a better solution to this ?

@nehaev
Copy link

nehaev commented Mar 30, 2022

Is anyone looking into this bug? Any progress/updates?

@appleseedexm
Copy link

This bug has been an issue for me for way too long, on a project with maven, lombok and mapstruct, and I always blamed it on some of these tools, but turns out that vscode-java is the culprit, suddenly all builds without the extension running work.
For me it mostly manifested with "missing files" during maven builds that exist just fine.
I am honestly surprised that issue doesn't get more attention, the workaround is not really a solution for me and without that vscode is basically useless to work with on java.

@bearArthurAog
Copy link

For me adding this settings in settings.json helped

"java.autobuild.enabled": false,
"java.project.resourceFilters": [".git", "*\target"],

@fopnet
Copy link

fopnet commented Aug 1, 2022

Vscode should be how to configure to the automatic build to ignore a list of folder / files

@appleseedexm
Copy link

appleseedexm commented Sep 23, 2022

I've used the workaround to override the m2e build path which worked for quite a while.
But now depending on other features / plugins this still seems to be an issue, i.e. installing CodeTogether lead to newly created target folders, making vscode-java loop forever again.

Guess the only option is to disable autobuild, which is definitely annoying, but at least I can use other features / plugins.

@szurilo
Copy link

szurilo commented Mar 5, 2024

I might have found a workaround if your only problem is that the build goes wrong in case of a build command from a terminal. Forget the terminal and build your project with the Maven for Java extension. You can set the profiles and execute a goal on any maven project there. This way the vscode auto build will not be triggered.

@zoily
Copy link

zoily commented Mar 13, 2024

Hello ! @szurilo, I re-tryed your solution but didn't work for me. Maybe, you have a specific config that work that way, may you share your settings.json ?
My main solution is still to avoid the issue with a workaround that disable then enable the autobuild. That's sad to still be there in 2024. My script to disable the autobuild, in case anyone need it :
(Get-Content -Path ./.vscode/settings.json -Raw) | ForEach-Object { $_ -replace '"java.autobuild.enabled":\s+(true|false)', '"java.autobuild.enabled": false' } | Set-Content -Path ./.vscode/settings.json

@szurilo
Copy link

szurilo commented Mar 18, 2024

Hello ! @szurilo, I re-tryed your solution but didn't work for me. Maybe, you have a specific config that work that way, may you share your settings.json ? My main solution is still to avoid the issue with a workaround that disable then enable the autobuild. That's sad to still be there in 2024. My script to disable the autobuild, in case anyone need it : (Get-Content -Path ./.vscode/settings.json -Raw) | ForEach-Object { $_ -replace '"java.autobuild.enabled":\s+(true|false)', '"java.autobuild.enabled": false' } | Set-Content -Path ./.vscode/settings.json

I don't think my settings are specific but here you go:
{
"java.configuration.updateBuildConfiguration": "interactive",
"java.compile.nullAnalysis.mode": "disabled",
"java.jdt.ls.vmargs": "-XX:+UseParallelGC -XX:GCTimeRatio=4 -XX:AdaptiveSizePolicyWeight=90 -Dsun.zip.disableMemoryMapping=true -Xmx2G -Xms100m -Xlog:disable",
"sonarlint.connectedMode.project": {
"connectionId": "https-quality-bank-hu-",
"projectKey": "com.something:else"
}
}

@zoily
Copy link

zoily commented Mar 18, 2024

Okay thanks, I retried with "java.configuration.updateBuildConfiguration": "interactive", but it didn't change anything.
The rebuild may come from the fact that I am using a root maven project with some maven projects inside. Once maven built the first project, the java server throws issues while maven is still building other projects.
It may also be related to mapstruct... such a complex project :(

@zoily
Copy link

zoily commented Mar 21, 2024

Okay ! I think I have an answer. For those that it can help :
Java server is building using profiles as maven. So the issue is that java server was building with a random profile instead of the same profile as maven.
I changed the "activeProfiles=myProfile" (myProfile is the one in my maven settings) in ".settings/org.eclipse.m2e.core.prefs" file then reload vscode.

No more random build aside maven one's :)
I definitly think that the extension should implement something that tell users how to set it up

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests