Improve build speed in bndtools by using -compression: STORE

As you know bnd always produces a .jar file per bundle as its build output, and .jar files are actually zip files. And zip files can be used with and without compression.

I just found a simple way to speed up performance of builds, by using the -compression instruction to disable compression.

This can be used to control whether the .jar files bnd produces are compressed or not.

  • DEFLATE: (default) compression is enabled
  • STORE: no compression

Using STORE for no compression saves CPU cycles needed to compress the files.

I have now put the following into our cnf/build.bnd:

-compression: ${if;${driver;eclipse};STORE;DEFLATE}

This means that in Eclipse IDE when you are developing you will use STORE (no compression = faster, but bigger files). In all other builds (e.g. Maven, Gradle, Tycho etc. you will use DEFLATE (compressed, slower, smaller files). It uses the ${if} macro and the ${driver} macro.

The idea behind this tuning is my assumption , that locally in Eclipse you do not care about compression and thus larger files are acceptable, but instead you want things to be fast.

Impact

I did some poor mans measurements to give you an impression:

Full Workspace refresh

Doing a full workspace refresh rebuilds everything in your workspace:

-compression: DEFLATE: 70 seconds (avg)
-compression: STORE: 57s seconds (avg)

STORE was around 16-19% faster.

web-templates bundle

This bundle contains little java code, but lots of files (web templates, images etc.) which are copied to the .jar file.
I did 3 runs.

DEFLATE:

filesize: 9.459.121 bytes
BUILD FULL web-templates 1 file was built in 1.257 sec
BUILD FULL web-templates 1 file was built in 1.304 sec
BUILD FULL web-templates 1 file was built in 1.252 sec

STORE:

filesize: 28.791.742 bytes
BUILD FULL web-templates 1 file was built in 0.576 sec
BUILD FULL web-templates 1 file was built in 0.611 sec
BUILD FULL web-templates 1 file was built in 0.611 sec

STORE is > 50% faster.

core-services bundle

This bundle contains a lot of java files, but little other files.

DEFLATE

BUILD FULL core-services 1 file was built in 5.317 sec
BUILD FULL core-services 1 file was built in 5.446 sec
BUILD FULL core-services 1 file was built in 5.610 sec

STORE

BUILD FULL core-services 1 file was built in 4.687 sec
BUILD FULL core-services 1 file was built in 4.687 sec
BUILD FULL core-services 1 file was built in 4.736 sec

STORE is about 12% faster.

Conclusion

I think this is a reasonable speed up in build performance during developing OSGi bundles in the IDE with bndtools / bnd, where waiting time matters a lot, while filesize often is not that important.

Of course, it depends a little bit on the size of your workspace and the contents of your bundle. But I think this is a reasonable little trick to feel better pressing Cmd + s and drink less coffee :slight_smile:

3 Likes

Nice, thanks heaps for that nugget! I’ll definitely give it a try.