The resolve cache mode

In 6.4.0 I added a new resolve mode called cache. If you select this mode then it is no longer necessary to update the -runbundles. Whenever the run bundles are requested, the resolver will run if the bndrun/bnd file and/or workspace has changed since the last time, otherwise it returns the last result.

This really takes away a lot of pain having to merge the -runbundles with git if you work with a team.

The cache algorithm does not detect changes in the repositories, and/or changes in the bundles that you include. In some cases this requires forcing a manual resolve.

1 Like

Does that help preventing bndrun files from getting dirty when running a Maven build? I guess the resolver-plugin will still need to be run as part of a Maven build and will still make the run files dirty, right?

It would be nice if the bndrun file could be added to the scm maybe without any runbundles at all and the resolver/testing maven plugins (as well as the eclipse tooling) could just make a copy of that during the build, update the copy and voilá?

The maven build will not (or should not, haven’t verified) write the bndrun files when you use cache mode.

I am running unit tests using bnd’s Maven plugins. For this, I have a bndrun file that defines the environment. During a build, I first need to run the resolver plugin to update the runbundles in that file, and afterwards the test plugin is called to run the tests.

If I use the cache mode and it does not update the bndrun files, (how) will the resolved information arrive at the test plugin?

Ok, it appears as if the -runbundles section is written to the bndrun file even if -resolve: cache is set in the bndrun file. Probably I am misunderstanding something on how this is supposed to work.

I guess this could happen if you use the Gradle or maven plugins. I am not too familiar with these but I guess they have some option to not write the file?

What is your setup?

The setup is roughly like this:

  • bnd-resolver-maven-plugin → resolve the bndrun file and save it
  • bnd-testing-maven-plugin → run the tests using the resolved bndrun file

It seems like the bnd-testing-maven-plugin might be able to resolve the bndrun file itself (in my setup the resolve option is currently set to false here) - so maybe I don’t need the bnd-resolver-maven-plugin at all… I’ll try that :wink:

I think you can just remove the resolving. When the test plugin runs, it will automatically resolve.

Again, I am out of my league here but I’d expect this to work.

Wonderful!

Removing the resolver plugin, turning on resolving in the test plugin, setting the resolve mode to cache in the bndrun files and removing the runbundles from the bndrun file seems to work nicely. The OSGI unit tests in Maven still work and when I use the bndrun file to run unit tests in Eclipse it also works (although it seems to take a bit longer now to fire up the tests, even the second time when the cache has already been set up).

No more dirty bndrun files :partying_face:

1 Like

Ok, correction… it works, but also something very strange is happening.

When I right-click on a bndrun file in Eclipse to run the JUnit tests in OSGI, it now not only runs the tests in the current module - it also seems to run all tests in of the Maven modules on which the current module has a dependency…?! How is that happening…?

…

It seems that the stringAttribute key="org.bndtools.testnames" is now no longer added to the run configuration created by Eclipse. Also, source_locator information for debugging is no longer added to the run configuration.

…

Ok, looks like the Eclipse workspace was somehow in a weird state and I guess the BND plugins were unable to walk the Java model to find tests…

So it works now?

More or less. It seems a bit fragile overall. Every once in a while, Eclipse runs into a situation where there seems to be again some kind of loop between Setting classpath and the BND Decorator to the point where trying to cancel stuff does not help and I need to shut down or even force-kill Eclipse. After that, of course, the workspace is a mess. Then it takes various iterations of cleaning and rebuilding until everything is reasonably in sync again… and then it works… (on a blue moon…).

I don’t know why, but recently Eclipse’s incremental build and Maven integraton (still using m2e 1.x here atm, but colleagues using m2e have reported issues as well) is not working well in my environment…

It seems I can tell if things are not right if the option to run BND OSGI tests with JUnit does not appear on a project root node but only on the bndrun file…

It looks like the BND OSGI Test Launcher (JUnit) only appears on a project node after I have opened a bndrun file in the corresponding project…

Sorry, m2e & maven are above my pay grade.

If you want to move to software heaven with bndtools and gradle on the CI, let me know :slight_smile:

Every time I tried working with gradle, I ended up running away screaming. Terrible experiences. Something as simple as trying to debug deep into some code by importing a dependency of a project as another project in Eclipse did not work because the Gradle support in Eclipse does not allow workspace project resolution… I wasn’t happy :face_with_spiral_eyes:

What I like about bndtools is that I never have to work with the command line. The gradle supportis so good and so close the Eclipse/bndtools behavior that the CI virtually always works.

There are cases where you need the flexibility of a build tool but I find that that the breadth of supported functions in bnd is sufficiently wide for common development. In many shops where they can hack what they want I see that the build systems tend to get unstable quickly do to the massive amount of hacking. Or in the case of maven, gigantic dependency trees.

However, it is true that I am willing to sacrifice some comfort and functionality to reduce complexity. That is something a lot of people are not willing to do.

Sorry @rec, I realise I’m late to the party but hopefully this helps:

It looks like the BND OSGI Test Launcher (JUnit) only appears on a project node after I have opened a bndrun file in the corresponding project…

If you have a bit more specific detail on how to reproduce this and submit an issue, I can have a look.
One issue that I know is that if you have a project that has only bndrun files in it, there is a chance you might not have configured it with the Bnd nature. In this case, the Bnd OSGi Test Launcher context menu item might not show.

When I right-click on a bndrun file in Eclipse to run the JUnit tests in OSGI, it now not only runs the tests in the current module - it also seems to run all tests in of the Maven modules on which the current module has a dependency…?! How is that happening…?

There are two possible causes that I can think of:

  • The runbundles for your test framework includes those bundle dependencies that have their own Test-Cases header; or
  • The macro that is calculating the Test-Cases header for your own test bundles is finding and including the test cases from the dependent modules. Have a look at the Test-Cases header in your built bundle. If this is the cause, then you should be able to tweak it by tweaking the macro that calculates your Test-Cases header.

The other explanation that you offered (ie, that the Bnd macro could not calculate the correct Test-Cases header because of a corrupt workspace) is also possible.

It seems that the stringAttribute key="org.bndtools.testnames" is now no longer added to the run configuration created by Eclipse.

I think I was the last person to touch this code. The last major overhaul was in [bndtools] Counterintuitive JUnit launch delegate behaviour · Issue #3320 · bndtools/bnd · GitHub

tester.names property only gets added to the launch (new or existing) when you launch JUnit from right-clicking on a specific test class or test method. If you are not launching from a specific class or method, it will be cleared at launch.

Also, source_locator information for debugging is no longer added to the run configuration.

The source locator shouldn’t be necessary, as Bnd launches install a custom SourceLookupDirector that should properly handle source lookup during debugging. Please let me know if your source lookup isn’t working - I know that I have occasionally seen issues where it fails under m2e.

Thanks for the detailed reply. So here is what I meanwhile found.

In order to make the BND OSGi Test Launcher context menu consistently appear, I need to open a .bndrun file and click then e.g. on a project while the .bndrun file is open in the focussed editor.

So if I open a .bndrun file and then right-click on a project, I can always use the BND OSGi Test Launcher context menu to run the tests.

I can also run tests of a single Java file containing JUnit tests by right-clicking on them - however, that only works if no editor for this file is open. Remember, the .bndrun file must be the focussed editor in order for the BND OSGi Test Launcher context menu to appear. If the Java file with the tests is open in an editor, right-clicking on the file to bring up the context menu also puts the file’s editor into focus - and that happens before the popup menu appears, so the BND OSGi Test Launcher won’t be in the context menu anymore.

None of the projects in my workspace has the BND nature. I am using Maven / m2e and obviously have the BND plugins installed. It seems that the BND plugins are aware of Maven as they e.g. interpolate variables like ${project.groupId} in the .bndrun files. But when importing/updating a project, they do not add the BND nature to bnd-enabled projects.

I am pretty sure that the rare cases where the launcher runs not only the tests in the selected project but also in dependencies is due to a corrupt workspace state. I have seen it now 2-3 times and I believe that in all cases, I had to interfere with the automatic builds either by cancel-spamming or force-shutting-down Eclipse due to some type of infinite build loop.

The issue that I have to manually add projects to debug sessions in order for Eclipse to display the sources has been there from since the beginning when I started using bnd tooling some time last year. If I can help you figuring out why this happens, please let me know how.

Sorry @rec, looks like I missed this response.

These seem like definite bugs in the context-sensitive menu display logic. It should not be necessary to open the .bndrun file in order to make that menu item appear, nor to close it in order for you to be able to run single JUnit tests.

Out of curiosity, what happens if you and the BND nature to the project containing the .bndrun?

I had a solid attempt at trying to fix the debug source lookup some time ago but it always seemed a bit more flaky under m2e for reasons I didn’t understand (never got my head too deep into the m2e space). The problem you’re describing looks like it might be a manifestation of this issue: [m2e] MavenWorkspaceRepository does not implement source lookup · Issue #4249 · bndtools/bnd · GitHub I will see if I can have a look at it at some point.

It tells me “The Bnd cnf directory is not part of the Eclipse workspace”.

I had assumed that I needed to open the .bndrun file first because there could theoretically be more than one .bndrun file per project and the context menu might then not know which one to use.

Possibly related: [launch] M2E launch shortcut being removed · Issue #4956 · bndtools/bnd · GitHub