Using gradlew build inside Docker fails

GENERAL
1

Hi,

I am trying to perform the ./gradlew build of a bnd workspace in Docker for build automation purposes, but it fails. Locally, the gradle build works fine and produces a working JAR, just in Docker is giving me problems. When I add --debug, I find it has some trouble when downloading and unpacking a particular JAR, which comes from a p2 repository configured in the bnd project.

This is the log:

2023-12-01T17:28:19.261+0000 [DEBUG] [sun.net.www.protocol.http.HttpURLConnection] sun.net.www.MessageHeader@7919cdcb6 pairs: {GET /modeling/emf/emf/builds/release/2.26/plugins/org.eclipse.emf.edit_2.16.0.v20190920-0401.jar.pack.gz HTTP/1.1: null}{Accept-Encoding: deflate, gzip}{User-Agent: Java/17.0.9}{Host: download.eclipse.org}{Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2}{Connection: keep-alive}
2023-12-01T17:28:19.622+0000 [DEBUG] [sun.net.www.protocol.http.HttpURLConnection] sun.net.www.MessageHeader@7a5f27ad18 pairs: {null: HTTP/1.1 200 OK}{Server: nginx}{Date: Fri, 01 Dec 2023 17:27:39 GMT}{Content-Type: application/x-gzip}{Content-Length: 98976}{Connection: keep-alive}{Keep-Alive: timeout=50}{Last-Modified: Mon, 31 May 2021 05:26:57 GMT}{ETag: "182a0-5c3997966e004"}{X-NodeID: download1}{Content-Security-Policy: frame-ancestors 'self'}{X-Frame-Options: SAMEORIGIN}{strict-transport-security: max-age=63072000; includeSubDomains; preload}{Cache-Control: private, max-age=8m, no-transform}{X-Content-Type-Options: nosniff}{X-XSS-protection: 1; mode=block}{X-Proxy-Cache: MISS}{Accept-Ranges: bytes}
2023-12-01T17:28:19.622+0000 [DEBUG] [aQute.bnd.http.HttpClient] result TaggedData [tag="182a0-5c3997966e004", code=200, modified=Mon May 31 05:26:57 UTC 2021, url=https://download.eclipse.org/modeling/emf/emf/builds/release/2.26/plugins/org.eclipse.emf.edit_2.16.0.v20190920-0401.jar.pack.gz, state=UPDATED]
2023-12-01T17:28:20.088+0000 [DEBUG] [aQute.bnd.build.LoggingProgressPlugin] Finished
2023-12-01T17:28:20.089+0000 [DEBUG] [aQute.bnd.http.URLCache] Unlocking url cache https://download.eclipse.org/modeling/emf/emf/builds/release/2.26/plugins/org.eclipse.emf.edit_2.16.0.v20190920-0401.jar.pack.gz
2023-12-01T17:28:20.090+0000 [DEBUG] [aQute.p2.packed.Unpack200] Calling: null /root/.bnd/urlcache/shas/C713B1C0192942E9BDABA9F85BDA25F79DB01310.content.original /root/.bnd/urlcache/shas/C713B1C0192942E9BDABA9F85BDA25F79DB01310.content
2023-12-01T17:28:20.090+0000 [DEBUG] [aQute.libg.command.Command] executing cmd: [null, /root/.bnd/urlcache/shas/C713B1C0192942E9BDABA9F85BDA25F79DB01310.content.original, /root/.bnd/urlcache/shas/C713B1C0192942E9BDABA9F85BDA25F79DB01310.content]
2023-12-01T17:28:20.091+0000 [DEBUG] [aQute.bnd.util.repository.DownloadListenerPromise] Xtext-4: get org.eclipse.emf.edit;2.16.0.v20190920-0401 https://download.eclipse.org/modeling/emf/emf/builds/release/2.26/plugins/org.eclipse.emf.edit_2.16.0.v20190920-0401.jar.pack.gz: failure
java.lang.RuntimeException: java.lang.NullPointerException
	at aQute.p2.packed.Unpack200.unpack(Unpack200.java:92)
	at aQute.p2.packed.Unpack200.unpackAndLinkIfNeeded(Unpack200.java:104)
	at aQute.bnd.repository.p2.provider.P2Indexer.lambda$get$0(P2Indexer.java:117)
	at org.osgi.util.promise.DeferredPromiseImpl$Map.accept(DeferredPromiseImpl.java:500)
	at org.osgi.util.promise.DeferredPromiseImpl.result(DeferredPromiseImpl.java:151)
	at org.osgi.util.promise.DeferredPromiseImpl$Map.run(DeferredPromiseImpl.java:492)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:840)
Caused by: java.lang.NullPointerException
	at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1092)
	at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1073)
	at org.gradle.internal.classpath.Instrumented.start(Instrumented.java:330)
	at aQute.libg.command.Command.execute(Command.java:113)
	at aQute.libg.command.Command.execute(Command.java:52)
	at aQute.p2.packed.Unpack200.unpack(Unpack200.java:84)
	... 8 more

Any idea why this happens and how to solve it?

2

Yes :frowning:

This is a problem that is fixed in 7.0.0.

Oracle in their infinite wisdom decided to deprecate pack200 file format. Our code had not taken that into account since backward compatibility used to be the primary benefit of Java. We fixed it in may so it should be in bnd 7.0.0.

So I guess on Docker you got a newer Java VM than on your desktop.

3

Both Desktop and Docker are Java (Eclipse Temurin) 17. That error is still there even on bnd 7.0.0.

Anyway, reading the debug log more in detail (15k lines) there is another error line where Task:myproject:compileJava failed. It is weird, because the workspace compiles successfully in Eclipse. In Eclipse there are no errors, just a few warnings, and the resulting JAR behaves perfectly correct in runtime.

Back in Docker, I run again the gradlew build command, with --stacktrace, and the output is:

org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':myBundleProject:compileJava'.
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.lambda$executeIfValid$1(ExecuteActionsTaskExecuter.java:142)
        at org.gradle.internal.Try$Failure.ifSuccessfulOrElse(Try.java:282)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeIfValid(ExecuteActionsTaskExecuter.java:140)
        ...
Caused by: org.gradle.api.GradleException: myBundleProject has errors, one error was reported
        at aQute.bnd.gradle.BndPlugin.checkProjectErrors(BndPlugin.java:1004)
        at aQute.bnd.gradle.BndPlugin.checkErrors(BndPlugin.java:978)
        at aQute.bnd.gradle.BndPlugin$2.execute(BndPlugin.java:470)
        at aQute.bnd.gradle.BndPlugin$2.execute(BndPlugin.java:466)
        at org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:820)
        ...

So it is clear that the gradle plugin sees exactly one error that doesn’t appear in eclipse. Is there a way to skip this check? Or a way to find out which error it is?

4

So, this unpacking is blocking me. Given that the Pack200 Tools and API are deprected, I guess they shouldn’t rely on them in Java >= 14. The repository – all p2 repos, I guess – includes 2 versions of the all the JARs: 1 packed and 1 unpacked
org.eclipse.emf.edit_2.16.0.v20190920-0401.jar 294.6k 2021-05-31 01:26
org.eclipse.emf.edit_2.16.0.v20190920-0401.jar.pack.gz 96.7k 2021-05-31 01:26

Wouldn’t make sense to ignore the .pack.gz and just download the unpacked version? Or at least add a configuration option to control this behavior?

5

I am pretty sure there is a message from bnd somewhere in your log explaining what goes on. The stack trace shows that bnd is inspecting its internal log.

Regardless, it might be a good idea to exclude pack200 and avoid it like the plague. This is the dark magic of P2 and I would have no idea how to exclude this.

Did you try to run Java 21 on your desktop? Might be easier to diagnose?

6

Finally. I got it working. The solution summarizes in the following point:

  • Make sure to delete the cnf/cache directory inside docker. This can be done either by adding it to .dockerignore or by executing rm -rf cnf/cache as a docker build step.

It seems there is some platform dependent or local path caching done somewhere in cnf/cache. This also explains why my local (windows) Eclipse and local gradle wrapper work, but the docker build fails until the cache is deleted.

What I tried regarding the java versions may be also helpful:

  • Java 17 + Gradle 7.6 + bndtools 6.4.0 → some unpack200 errors appear in the normal output, but the build is successful.
  • Java 17 + Gradle 7.6 + bndtools 7.0.0 → no errors are shown in the normal output (unpack200 errors still seen when using --debug flag). The build is successful.
  • Java 21 + Gradle 8.5 + bndtools 7.0.0 → no errors are shown in the normal output (same as previous). The build is successful.

So, bndtools 7.0.0 seems to be compatible with java 21 and gradle 8.5 (first gradle version compatible with java 21).

3 Likes
7

Thanks for keeping us posted on this, this might become useful to others.

1 Like