Invalid content checksum when using bnd-maven-plugin

DEVELOPMENT
1

Hi,

I’m using a maven project with the bnd-maven-plugin, and in my test project, I’m then using from my integration-test.bndrun file the aQute.bnd.repository.osgi.OSGiRepository plugin: it allows me to reuse an existing remote OBR.

here is my integration-test.bndrun file:

-plugin.1.OBR: aQute.bnd.repository.osgi.OSGiRepository; \
    name=OBR; \
    poll.time=-1; \
    max.stale=-1; \
    cache=${.}/target/cache; \
    locations=https://.../repository/obr.xml

-runtrace: true
-standalone: true
-resolve.effective: resolve

Test-Cases: ${classes;CONCRETE;PUBLIC;NAMED;*Test}

Import-Package: org.osgi.framework.*;version="[1.8,2)",*

-runsystempackages: sun.reflect,\
    com.sun.nio.sctp,sun.net.util,sun.security.action,com.sun.net.httpserver,sun.misc,\
    com.sun.org.apache.xerces.internal.jaxp,sun.management,com.sun.management,\
    sun.net.spi.nameservice,javax.xml.bind;version=2.3,javax.xml.bind.annotation;version=2.3,javax.xml.bind.annotation.adapters;version=2.3

-runfw: org.apache.felix.framework
-runee: JavaSE-1.8

-runsystemcapabilities: ${native_capability}

-runbundles: \
     org.apache.servicemix.bundles.junit;version='[4.12.0,4.12.1)',\
     ...

now, when the OSGiRepository is using a local obr URL, like file://…/ , then the mvn clean install command works fine.

but when I configure the OSGiRepository with a remote OBR like:

-plugin.1.OBR: aQute.bnd.repository.osgi.OSGiRepository;
name=OBR;
poll.time=-1;
max.stale=-1;
cache=${.}/target/cache;
locations=https://…/repository/obr.xml

then when running my tests all downloads are failing with for example the following errors:

[ERROR] Error   : Bundle file "java.io.IOException: **Invalid content checksum** 9B316CBEBAB5E9CE3AB9DB32B86E9E56306E59E4763944468AE1B9BFC356A040 for https:/.../repository/com/nokia/casr/com.nokia.as.util.gogoclient/1.0.2/com.nokia.as.util.gogoclient-1.0.2.jar; expected d6e1f97d43a3f31df67b445231b05946e3cf6f8aff9ee5d5197ab5a65698c445: null" does no
t exist, given error is java.io.IOException: **Invalid content checksum** 9B316CBEBAB5E9CE3AB9DB32B86E9E56306E59E4763944468AE1B9BFC356A040 for https://repo.lab.pl.alcatel-lucent.com/csf-mvn-deli
vered/com/nokia/casr/com.nokia.as.util.gogoclient/1.0.2/com.nokia.as.util.gogoclient-1.0.2.jar; expected d6e1f97d43a3f31df67b445231b05946e3cf6f8aff9ee5d5197ab5a65698c445

I tried to use http instead of https, and I’m still getting the “invalid content checksum” download exceptions. Is it really caused by a download network issue ? note that I could manually download the urls using the curl command,

any suggestions about this download checksum issue ?

thanks & regards
pierre

2

What checksum do you get on the files when you manually download them? The index.xml will have the expected checksum

<capability namespace='osgi.content'>
  <attribute name='osgi.content' value='e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855'

osgi.content Namespace

which is expected to match the downloaded artifact.

So you should manually download the index.xml (or obr.xml as you are calling it) and the artifacts and compare the computed checksum (SHA-256) to the expected checksum.

You also do no mention what version the bnd-maven-plugin you are using.

3

I’m using biz.aQute.bnd:bnd-maven-plugin:5.3.0

so, I have downloaded the obr.xml

 <resource>
    <capability namespace="osgi.identity">
      <attribute name="osgi.identity" value="com.nokia.as.util.gogoclient"/>
      <attribute name="type" value="osgi.bundle"/>
      <attribute name="version" type="Version" value="1.0.2"/>
    </capability>
    <capability namespace="osgi.content">
      <attribute name="osgi.content" value="d6e1f97d43a3f31df67b445231b05946e3cf6f8aff9ee5d5197ab5a65698c445"/>
      <attribute name="url" value="../../../../../com/nokia/casr/com.nokia.as.util.gogoclient/1.0.2/com.nokia.as.util.gogoclient-1.0.2.jar"/>
      <attribute name="size" type="Long" value="404939"/>
      <attribute name="mime" value="application/vnd.osgi.bundle"/>
    </capability>

and I downloaded the artifact, then computed the sha256:

sha256sum com.nokia.as.util.gogoclient-1.0.2.jar
9b316cbebab5e9ce3ab9db32b86e9e56306e59e4763944468ae1b9bfc356a040  com.nokia.as.util.gogoclient-1.0.2.jar

and, indeed, the exception reports that the expected checksum is the one displayed by my sha256sum,

[INFO] Retrying invalid download: Invalid content checksum 9B316CBEBAB5E9CE3AB9DB32B86E9E56306E59E4763944468AE1B9BFC356A040 for https://repo.lab.pl.alcatel-lucent.com/csf-mvn-delivered/com/nokia/casr/com.nokia.as.util.gogoclient/1.0.2/com.nokia.as.util.gogoclient-1.0.2.jar; expected d6e1f97d43a3f31df67b445231b05946e3cf6f8aff9ee5d5197ab5a65698c445. delay=1000, retries=2

the obr has been manually created using an old repoindex.jar which comes from biz.aQute.bnd:org.osgi.impl.bundle.repoindex.cli:3.2.0 like this:

java -cp $RUNTIME/repoindex.jar org.osgi.impl.bundle.bindex.cli.Index --pretty -r obr.xml -t ${ROOTPATH}"%p%f" -n “$obrversion” <list of bundles …>

so, it’s likely that the issue is that the old biz.aQute.bnd:org.osgi.impl.bundle.repoindex.cli:3.2.0 index generator is not using sha256 ?

what is odd is that when I use a local OBR generated using the same tool, it seems to work fine. I will investigate the old biz.aQute.bnd:org.osgi.impl.bundle.repoindex.cli:3.2.0 in oder to see how it generates the checksum

4

I checked the old biz.aQute.bnd:org.osgi.impl.bundle.repoindex.cli:3.2.0 tool, and the checksum seems to be generated using sha256:

        private String calculateSHA(Resource resource) throws IOException, NoSuchAlgorithmException {
                MessageDigest digest = MessageDigest.getInstance(SHA_256);
                byte[] buf = new byte[1024];

                InputStream stream = null;
                try {
                        stream = resource.getStream();
                        while (true) {
                                int bytesRead = stream.read(buf, 0, 1024);
                                if (bytesRead < 0)
                                        break;

                                digest.update(buf, 0, bytesRead);
                        }
                } finally {
                        if (stream != null)
                                stream.close();
                }

                return Hex.toHexString(digest.digest());
        }

so, I’m wondering how it’s possible to have such a checksum missmatch. I’m investigating.

5

You do not want to use org.osgi.impl.bundle.repoindex.cli as that is ancient and way out of date. Use the bnd index command which is the proper way using the command line.

➜  ~ bnd help index

NAME
  index                       - Index bundles from the local file system

SYNOPSIS
   index [options] <bundles...>

OPTIONS

   [ -b, --base <uri> ]       - URI from which to make paths in the index file
                                relative (default: relative to the output file
                                directory). The specified value must be a prefix
                                of the absolute output file directory in order
                                to have any effect
   [ -d, --directory <file> ] - The directory to write the repository index file
                                (default: the current directory)
   [ -n, --name <string> ]    - The name of the index (default: name of the
                                output file directory)
   [ -r, --repositoryIndex <file> ] - The name of the repository index file
                                (default: 'index.xml'). To enable GZIP
                                compression use the file extension '.gz' (e.g.
                                'index.xml.gz')
   [ -v, --verbose ]          - prints more processing information
6

See Command Line section of https://bnd.bndtools.org/chapters/120-install.html for info about the bnd command.

7

ok, thank you. I will update the tool we are using.

thank you !
/pierre

8

Hey Pierre,

You are using maven, so why do you need an index at all?

Bnd maven plugins for years at this point have been able to function without any pre-indexing of any kind (create a BOM for shared collections of artifacts instead it’s more widely usable).

Did you know that bnd maven plugins will index dependencies (including if you want BOMs) on the fly and is very fast?

You don’t even need to specify any repos (outside of the maven ones of course), the indexing of dependencies is implied by any plugins that require it.

I frequently use a company bom (available publicly here [1]) with more than a thousand dependencies with no noticeable impact on performance (once all the deps are downloaded of course.)

There’s a slew of open source projects I can point to which are evidence of this both in maven and in (plain) gradle.

Anyhow, I just thought you might like to know and perhaps simplify your life :wink:

Sincerely,
Ray

[1] https://search.maven.org/artifact/com.liferay.portal/release.dxp.bom

9

Hello Ray ! :slight_smile:

The issue is actually related to an old question I posted some time ago, and you replied to it, see [1].

So, I see what you mean, but in my case, I’m not using any BOMS, because I’m already using a giant OBR where all my OSGI dependencies are available, and the OBR bundles are generated using BndTools, not using any poms or boms, some we have no boms for the moment.

thank you
/pierre

[1] https://groups.google.com/g/bndtools-users/c/X7qR8OSFI9w