About wrapping of dependencies / jars and how to tame their transitive dependencies

[chrisrueger]

Let’s take Micrometer: https://mvnrepository.com/artifact/io.micrometer/micrometer-core/1.12.0

• Micrometer specifies lots of DynamicImport-Packages because it supports all kinds of systems in the world

• unfortunately no all bundles of Micrometer are valid OSGI bundles so I decided to wrap them in an own bundle via -includeresource with lib option:

-includeresource: \
	${repo;io.micrometer:micrometer-core;latest}; lib:=true,\
	${repo;io.micrometer:micrometer-commons;latest}; lib:=true,\
	${repo;io.micrometer:micrometer-observation;latest}; lib:=true,\
	${repo;io.micrometer:context-propagation;latest}; lib:=true,\

This creates mybundle.jar with mandatory Import-Packages like this, basically requiring the whole world.
The Resolver then says “Nooooo”.

To make the resolver happy and start my app I ended up with excluding everything we don’t use:

Version A

Import-Package: \
	com.codahale.*;resolution:=optional,\
	com.mongodb.*;resolution:=optional,\
	com.netflix.*;resolution:=optional,\
	io.grpc.*;resolution:=optional,\
	javax.annotation.meta;resolution:=optional,\
	javax.cache.*;resolution:=optional,\
	kotlinx.*;resolution:=optional,\
	net.sf.*;resolution:=optional,\
	org.HdrHistogram.*;resolution:=optional,\
	org.LatencyUtils.*;resolution:=optional,\
	org.apache.catalina.*;resolution:=optional,\
	org.apache.hc.*;resolution:=optional,\
	org.apache.kafka.*;resolution:=optional,\
	org.apache.logging.log4j.*;resolution:=optional,\
	org.aspectj.*;resolution:=optional,\
	org.bson.*;resolution:=optional,\
	org.glassfish.*;resolution:=optional,\
	org.hibernate.*;resolution:=optional,\
	org.jooq.*;resolution:=optional,\
	*

Alternatively I could have made everything optional:

Version B

Import-Package: \
	*;resolution:=optional,\

Questions:

Is there another way?

Option A took me lots of trial and error.
Option B is a wildcard with less trial and error, but I guess at the cost that the resolver has more work todo.

Would it (theoretically) make sense that ${repo;io.micrometer:micrometer-core;latest}; lib:=true would treat all packages as if they were;resolution:=optional which are a DynamicImport-Package in micrometer:micrometer-core?

Or is there already some macro?
Or am I doing stupid stuff?

[chrisrueger]

After thinking some more about it. I guess any “automatic” solution is probably not a good idea too. I read that DynamicImport-Package is also not a good practise.

So I will give this some more thought.

[pkriens]

The best way to handle this “shit” (excusé le mot) is to run away from these crippled bundles. Unless they have some code that is really not found anywhere else, and too much to write yourself, you’d better run fast.

Sadly, this is not always an option. So you have to wrap.

The issue that is important is called cohesion. How much do you package unrelated packages in your Jar?

The general structure is that there is core code that over time was expanded with bridges. Where a bridge coupled the functionality useful in some other context. Done properly, the bridges strictly point to the core. More commonly, the core couples to the bridges and the internet will be transitively drawn in by using any class in the core.

What I do in this case is first analyze what my application actually needs really are. In numerous cases the application only needs a very small part of the overall functionality. I then design a proper API that reflects the needs of the application.

The trick here is to not got overboard. It is always tempting to make more API then you need. It is on your build, do the absolute minimum of what you absolutely need. You can always add later. The wrapped Jar is only for you.

That said, as with all APIs, absolutely minimize the dependencies on other API except the Java classes. The best API’s only use base java and no types of other APIs.

If possible I abstract the underlying code according to my needs so I can use services. Alas, also this is not always possible but even doing this for a small part helps. If I can keep the shitty underlying code out of the API, I will create a separate API jar.

I then implement this API in a JAR. Since the details are handled by the shitty jar (which I may hope gets the functionality right) this is usually surprisingly little work. Since this is just a facade, coding the API in a single class often works, using inner classes. Inner class can reduce the amount of code significantly since they can see the methods and fields from their ancestors.

Then the bnd magic comes in … There is an instruction -conditionalpackage. This instruction looks at the code you included in your Jar and then drags in the referred packages by copying the classes to the private area. It is basically how in the C world static linking works. This is repeated until there are no longer any imported packages that match the clauses of the -conditionalpackage.

For example, in your case I would start with:

-conditionalpackage    io.micrometer.*

In the bnd editor or the Resolution view, you can see the imports. It is usually a good idea to look what classes caused the import.

There are now two cases with the imports:

• They are crucial for your functionality
  • good bundle available : let them be good imports
  • another shitty bundle: add package pattern to -conditionalpackage
• You are sure they will never be used (more about this later)
  • you’re really sure: remove then from the Import-Package, like Import-Package !com.netflix.*, *
  • well, eh, maybe: make them optional, like Import-Package com.netflix.*;resolution:=optional, *

This is of course a recursive process until the imports do not hurt so badly anymore that you can’t live with it.

If it all works out, you got a very useful bundle for your application that encapsulates the inner mess while providing a well behaved service interface with proper API.

Then remains testing. The danger there is to start testing the implementation of the wrapped jar. That is something I would rely on. (If not, you really have a shitty specimen, run.) The test should be about the wrapping and the constructs you added for your own API.

I’ve been dabbling in automating this since the dawn of OSGi. However, it really requires to know what you want. There is an interesting class in aQute.lib, see Tarjan. We even once had a Chinese students for a Google Summer of Code that turned out to have less interest in software engineering than money.

Last thing, maybe change the title?

[chrisrueger]

Thanks a lot.

@pkriens

It is usually a good idea to look what classes caused the import.

My current work in Resolution view: allow reqs/caps filtering on the tooltip text by chrisrueger · Pull Request #5928 · bndtools/bnd · GitHub I try to address the “what caused” with the “FROM:” search filter on the requirments in the Resolution view.

@pkriens

recursive process until the imports do not hurt so badly anymore that you can’t live with it.

Yeah, I think this is also what drives me to make this recursive process a bit shorter or let the tool help a bit more… if possible.

@pkriens

I’ve been dabbling in automating this since the dawn of OSGi. However, it really requires to know what you want.

Not sure if that scares or motivates me But let’s see what happens with the PR. This maybe a tiny improvement in this recursive process as you have more ways to filter. Maybe it is possible in the future to build upon that.

@pkriens

Last thing, maybe change the title?

Ok.

EDIT: and thanks for the tip with -conditionalpackage. I haven’t used this and start looking into it now.