MultiRelease jar and resolution

I am using a couple of jars (not developed or maintained by me) that have MultiRelease: true in them. I’m also using a locally-built copy of bndtools on the 7.0.0 stream.

When I try to get the resolver to resolve a bndrun file that includes these bundles I get the following error:

Resolution failed. Capabilities satisfying the following requirements could not be found:
[<>]
⇒ osgi.identity: (osgi.identity=org.eclipse.ecf.provider.jersey.server)
⇒ [org.eclipse.ecf.provider.jersey.server version=1.11.1.202202112253]
⇒ osgi.wiring.package: (&(osgi.wiring.package=org.glassfish.jersey)(version>=2.30.1))
⇒ [org.glassfish.jersey.core.jersey-common version=2.30.1]
⇒ bnd.multirelease: (&(bnd.multirelease=org.glassfish.jersey.core.jersey-common)(&(version>=2.30.1)(version<=2.30.1)))
[jakarta.xml.bind-api version=2.3.2]
⇒ bnd.multirelease: (&(bnd.multirelease=jakarta.xml.bind-api)(&(version>=2.3.2)(version<=2.3.2)))

I’m not clear on what these failures mean…nor how to fix. It appears that the jersey and jakarta bundles are MultiRelease (and they in fact do have that manifest entry in both of them), but there is no capability provided for multirelease (?). How is such a capability provided for using these third-party jars?

Thanksinadvance for any info.

Scott

The bnd.multirelease is provided by the indexer in bnd. We encode a multi-release JAR with a head resource (a bundle) that has the capability but requires a bnd.multirelease capability. This capability is provided in a synthetic resource generated on the fly for each supported VM in the JAR. This synthetic resource provides this capability, requires anything that that would be required when run on the VM version, and it requires the VM exactly.

It sucks, multi release are utterly brain damaged ideas. However, our test cases work. You might have a problem with the bnd version in Gradle/bndtools?

If not, could you make a small repo and share it so I can debug it?

I’m using a recent bndtools version 7 snapshot, so I would expect not.

This is a problem, because the resolution fails on a bndrun file that I’m generating via a jaxrs remote service wizard (using the bndtools 7 service wizard additions I contributed).

I can tell you that the problematic multirelease bundle is the jersey-common bundle (from eclipse jersey project) which is in this repo:

-plugin.13.ECFJAXRS:
aQute.bnd.repository.p2.provider.P2Repository;
url = https://raw.githubusercontent.com/ECF/JaxRSProviders/master/build/;
pretty = true;
name = ECF JAX-RS Distribution Provider

This repo one of several repos present in the ECF workspace template:

I expect that any bndtools project and bndrun file that uses jersey-common version in the above repo should show the same problem. I would be happy to provide a zip of the generated projects (all of which compile fine…but resolution of the generated bndrun file fails as described in original message…always wrt the jersey-common bundle…which is a multi-release jar produced by the jersey folks).

I’ve took a deep dive but the jersey bundle seems ok. However, there are lots of dependencies missing in the repository you pointed to.

I can test this further if you can give me the list of G:A:V for the Maven Bnd Repository.

Jersey is not ok. To repeat here’s the resolve error with bndtools 7.0.0

Resolution failed. Capabilities satisfying the following requirements could not be found:
[<>]
⇒ osgi.identity: (osgi.identity=org.eclipse.ecf.provider.jersey.server)
⇒ [org.eclipse.ecf.provider.jersey.server version=1.11.1.202202112253]
⇒ osgi.wiring.package: (&(osgi.wiring.package=org.glassfish.jersey)(version>=2.30.1))
⇒ [org.glassfish.jersey.core.jersey-common version=2.30.1]
⇒ bnd.multirelease: (&(bnd.multirelease=org.glassfish.jersey.core.jersey-common)(&(version>=2.30.1)

Here is a screenshot of the bndtools jar file viewer for jersey-common 2.30.1

If you want to initiate the resolve yourself, I suggest that you just use the ECF remote services workspace template (master branch).

This workspace template has a JaxRS api, impl, and consumer projects, and in the impl and consumer projects there is a bndrun file which when resolve/run is attempted will provide the bnd.multirelease error as above.

The steps are:

  1. Create new workspace with remote services workspace template (url above)
  2. Create JaxRS API project from Bndtools OSGi project templates
  3. Create JaxRS Impl project from Bndtools OSGi project templates (have to give name of JaxRS API project from 1) in wizard.
  4. Open bndrun file in JaxRS Impl project
  5. Attempt to resolve

This is what I’m doing to show the bndtools 7 resolve error for jersey-common multirelease jar (picture above).

Could you make the the workspace with the projects as you describe in the instructions and place it on github to save me some time? I am quite busy at the moment.

Ok Peter. I’ve created the three jaxrs projects (api, impl, consumer) in this workspace

on the pkriens branch

In the workspace are three projects:

org.pkriens.jaxrsservices
org.pkriens.jaxrsservices.impl
org.pkriens.jaxrsservices.consumer

In the impl project and consumer project are a bndrun file. If you open that file, and click resolve you will get two bnd.multirelease errors

Resolution failed. Capabilities satisfying the following requirements could not be found:
[<>]
⇒ osgi.identity: (osgi.identity=org.eclipse.ecf.provider.jersey.server)
⇒ [org.eclipse.ecf.provider.jersey.server version=1.11.1.202202112253]
⇒ osgi.wiring.package: (&(osgi.wiring.package=org.glassfish.jersey)(version>=2.30.1))
⇒ [org.glassfish.jersey.core.jersey-common version=2.30.1]
⇒ bnd.multirelease: (&(bnd.multirelease=org.glassfish.jersey.core.jersey-common)(&(version>=2.30.1)(version<=2.30.1)))
[jakarta.xml.bind-api version=2.3.2]
⇒ bnd.multirelease: (&(bnd.multirelease=jakarta.xml.bind-api)(&(version>=2.3.2)(version<=2.3.2)))
[osgi.cmpn version=7.0.0.201802012110]
⇒ osgi.unresolvable: (&(must.not.resolve=)(!(must.not.resolve=)))

BTW. These errors do not occur in bndtools 6 and below and they didn’t have the bnd.multirelease support.

Hmm. It works on the first try.

I made a very small update this week in how the Multi Release Jars (MRJ) are encoded. However, as far as I can see this only affected the size but should not make a functional difference.

It puzzles me and I’d like to know what the heck is going on. Looking at your output it is clear that the MRJs are using the modern encoding, and that encoding looks as intended to me. Curiouser and curiouser …

Can you try again with yesterday’s snapshot of bndtools?

With Eclipse 4.28RC2 windows

Version: 2023-06 (4.28)
Build id: I20230601-1220
\jdk-17.0.2\

I installed

Bndtools https://bndtools.org/ biz.aQute.bnd.embedded-repo 7.0.0.202306011458-SNAPSHOT biz.aQute.bnd.embedded-repo

and using impl project (and consumer project) bndrun file I’m still getting:

Resolution failed. Capabilities satisfying the following requirements could not be found:
[<>]
⇒ osgi.identity: (osgi.identity=org.eclipse.ecf.provider.jersey.server)
⇒ [org.eclipse.ecf.provider.jersey.server version=1.11.1.202202112253]
⇒ osgi.wiring.package: (&(osgi.wiring.package=org.glassfish.jersey)(version>=2.30.1))
⇒ [org.glassfish.jersey.core.jersey-common version=2.30.1]
⇒ bnd.multirelease: (&(bnd.multirelease=org.glassfish.jersey.core.jersey-common)(&(version>=2.30.1)(version<=2.30.1)))
[jakarta.xml.bind-api version=2.3.2]
⇒ bnd.multirelease: (&(bnd.multirelease=jakarta.xml.bind-api)(&(version>=2.3.2)(version<=2.3.2)))
[osgi.cmpn version=7.0.0.201802012110]
⇒ osgi.unresolvable: (&(must.not.resolve=)(!(must.not.resolve=)))

More testing:

I’ve also been able to recreate the same resolve problem by building/debugging the latest bndtools 7.0.0 source (on my fork) and the same workspace at GitHub - ECF/bndtools.workspace: Bndtools Workspace Template for ECF Remote Services Development on pkriens branch.

Perhaps you could tell me where in the bndtools source would be relevant to finding why this fails in my environment and apparently succeeds in your environment. I’ll then debug into it.

Some more debugging:

It appears to me that the resolve fails for the jersey-common jar because when the resolver gets around to looking for the bnd.multirelease requirement, it calls:

                        // Next find out if the requirement is satisfied by a capability on
			// the same resource
			processMandatoryResource(requirement, firstStageResult, requirement.getResource());

of AbstractResolveContext class. When the requirement is (namespace) bnd.multirelease, and the resource is the jersey-common jar the processMandatoryResource does not find the bnd.multiresource capability in the resource.capabilityMap. That is when ResourceImpl.getCapabilities(“bnd.multirelease”) is called on the jersey-common resourceimpl, the ResourceImpl line 55 returns an empty list:

	return (List) capabilityMap.getOrDefault(namespace, Collections.emptyList());

And the whole resolve fails because the bnd.multirelease requirement cannot be found on the jersey-common (or the jakarta.xml.bind-api, which is also multi-release).

It seems to me that the processMandatoryRequirement call above on the same jar as above should succeed in finding the bnd.multirelease in the resource capabilityMap, but it’s not there.

Is it possible that the reading of these 3rd party library jars from repos (jersey-common, jakarta.xml.bind-api) is somehow not adding the bnd.multrelease to the capability map when being created as a resource impl?

Interestingly on the jersey-common resource it does have this in the capabilityMap

bnd.p2=[bnd.p2;md5=dabb40ba58199304c640b7bd8bb2fbac]

but the bnd.multirelease is not in the capabilityMap for either the jersey-common or jakarta.xml.bind-api 3rd-party resourceimpls.

Also kind of surprising: my attempts to debug into the call to aQute.bnd.osgi.resource.RequirementImpl.getResource() actually doesn’'t find the getResource() method at all…and shows this as the source in debugger:

Hi Peter. I think I’ve figured out why we are seeing different things. In short, it’s the cnf/cache directory (the repo resources cache).

If I manually delete the cnf/cache directory and all subdirectories, and then refresh the workspace, the resolve will succeed (your first experience succeeded).

However, if I stop eclipse and restart…and then try to resolve either of the impl or consumer project bndrun files, I get the bnd.multirelease errors as above.

If I then delete the cnf/cache directory and refresh workspace I can then resolve successfully again until I exit eclipse and restart…where it begins failing again.

It appears that there is either something wrong with the writing (to cache) and/or reading from the cache logic wrt bnd.multirelease handling.

I don’t think I’m actually supposed to manually delete the cnf/cache directory right? That’s why I wasn’t initially seeing what you were seeing, because for some reason my cache (and probably yours) wasn’t saving/loading the requirements/capabilities correctly.

I’m pretty sure you will see the same thing (i.e. exiting/restarting eclipse will then disallow resolve, until cache directory deleted and workspace refreshed). Please let me know.

It is a problem in the P2 repository :frowning:

It works fine until there is a refresh. On refresh, it rereads the P2 repo but it does some seemingly hare brained optimization scheme based on the current set. In the MRJ jar, we have synthetic resources that get orphaned.

I am removing this refresh function. This is a workspace refresh and should just reread the XML file if present. I am assuming these repos are static?

Yes these repos are generally static, but can anything be done to refresh them in case they are not static (e.g. new release)? It seems to me that it would be a pretty large limitation to require that the p2 repos being used always be static.

If this is a problem with p2 itself and we can isolate it as such, I’m pretty sure that I can get it prioritized (after 4.28 release later this month) for fix.

I saw some discussion elsewhere about management of the cnf/cache but did not watch the discussion closely. Would bndtools cache mgmt provide a more flexible solution?

I added a menu entry to the repo view. You can now just refresh an entry. I will delete the XML file and then reinitialize, this will parse the url again. The way it was handled wasn’t correct anyway.

If you think this is not sufficient, I then need someone to tell me how p2 would signal a change. The way it was currently handled was basically reparsing the p2 tree and looking at the md5s.

I merged the change, would be nice if you could test it.