Testing - outdated documentation - current best practices

jkoerkel · April 15, 2024, 2:52pm

Hello everyone,

ChatGPT is making up inaccurate info, RTFMed but it’s outdated and confusing, I’m trying to add testing configuration to an existing codebase with no prior tests and the overall experience is frustrating, so I hope it is OK to ask in this “I did try to work with the docs, they’re not helping too much, how do you guys who actually know the bnd codebase and all the tricks do it in practice?” kinda way.

Assume I am starting to integrate tests into an existing codebase with zero prior tests ( ), can therefore start with JUnit5 ( ) and I have

a Seperate maven repo for test dependencies for better overview
cnf/tests.maven as its index
a start bundle that only contains .bndrun files that shall, after adding the test configuration, also contain tests.bndrun which is supposed to execute all tests of the workspace projects added to whatever property they need to be added to

How would I go about setting up the projects under test first for only plain JUnit5 tests?

Do the projects only need configuration in their bnd.bnd or also each their own .bndrun for the tests?
With -testpath: \ junit-platform-launcher,\ junit-jupiter-api (courtesy ChatGPT) set in one of my projects’ bnd.bnd, by right-clicking on a Test-Cases-included class in a testsrc-included class in the same package as the class under test, under Run As, there is no option “JUnit Test”, but this should be possible for our devs developing in eclipse
Once I have this option in my right-click context menu, how do I then go about adding this to the aforementioned tests.bndrun? Which property holds the bundles under test for a test run? There seems to not be a -testrequires and -testbundles instruction

How would I then go about setting up the projects under test for integration testing with launchpad so we can test services and stuff in a running OSGi framework?
I have seen there seems to not be a way to add additional test source folders, so do integration tests always need to be integrated in the same test source as plain JUnit tests? How do you guys differentiate so you find your way around still? Not having really thought about this, I’m thinking maybe postfixing plain JUnit tests with Test and prefixing integration tests additionally with Integration, but surely you have tried and tested ideas for this that have already stood the test of time.

I know this is a lot to ask, but maybe in the end this thread could be used to (help) update the docs.

Cheers
Julian

jkoerkel · April 16, 2024, 11:34am

For a basic configuration, doing it all in the bnd.bnd seems to be enough. As in there, -runrequires and -runbundles need to be set for the test runs, splitting it up into .bndruns is probably a good idea, because this way it is possible to define seperate bndruns for plain junit-, and integration tests. These then can be imported into the respective .bndrun in the startbundle that runs all junit- or integration tests.

The “smallest” possible -testpath (drag the bundles in from the repositories view) seems to be:

# project under test's bnd.bnd
-testpath: \
	junit-jupiter-api;version=latest,\
	junit-jupiter-engine;version=latest,\
	junit-platform-commons;version=latest,\
	junit-platform-engine;version=latest,\
	junit-platform-launcher;version=latest,\
	org.opentest4j;version=latest

As has been well documented, so that plain JUnit5 tests work, it is necessary to omit biz.aQute.tester.junit-platform from the -testpath, but add it to the -runrequires and -tester instructions:

# project under test's bnd.bnd
-tester: biz.aQute.tester.junit-platform
-runrequires: \
	bnd.identity;id='biz.aQute.tester.junit-platform',\
	bnd.identity;id=junit-jupiter-engine,\
	bnd.identity;id='project.under.test'

Add the Test-Cases Header like so:

# project under test's bnd.bnd
Test-Cases: ${classes;CONCRETE;NAMED;*Test*}

Then, resolve and update the runbundles from the “Run” tab of the project under test’s bnd.bnd.
This sequence of well documented steps has given me the context menu entry “Run as → JUnit Test”.

chrisrueger · April 17, 2024, 7:49am

Thanks. Based on your reply I added links to the bottom of the Tutorial section you mentioned, which link to the Testing pages of the manual.

If you would like to add there, feel free. Contributions are always welcome to improve the docs. Also if you would like to put your response into a dedicated Tutorial about testing, just let me know. Maybe here would be a good place.

jkoerkel · April 17, 2024, 10:29am

I am now trying to add OSGI tests with Launchpad.
The old tutorial -as I’ve seen in the current bnd sources- is outdated with regard to needing a dedicated test bundle which contains its tests in its src, so it seems it should be possible running OSGI tests with Launchpad from an implementation bundles’ testsrc to test its services.

Yet with a config that seemingly should work, when right-clicking on my project under test → Run As → Bnd OSGi Test Launcher (JUnit), I get a console output for all the started bundles in my -runbundles, but my tests don’t get started, so the JUnit tab in eclipse shows no test results and nothing about test results is logged to the console. It does neither execute the normal JUnit tests, nor my minimal Launchpad OSGi test.
Surely I have just configured some gotcha wrongly, the following listings are the infos I can provide at first glance, of course if more info is needed, I’ll provide that.

# project under tests' bnd.bnd
-testpath: \
	junit-jupiter-api;version=latest,\
	junit-jupiter-engine;version=latest,\
	junit-platform-commons;version=latest,\
	junit-platform-engine;version=latest,\
	junit-platform-launcher;version=latest,\
	org.opentest4j;version=latest,\
	biz.aQute.launchpad,\
	aQute.libg;version=latest
Conditional-Package: aQute.lib.*, aQute.libg.*
-tester: biz.aQute.tester.junit-platform
-noparallel: launchpad;task="test"
-runrequires: \
	bnd.identity;id='biz.aQute.tester.junit-platform',\
	bnd.identity;id=junit-jupiter-engine,\
	// snip: bundle under test and its optional dependencies
	bnd.identity;id='biz.aQute.launchpad'
-runbundles: \
	biz.aQute.tester.junit-platform;version='[7.0.0,7.0.1)',\
	junit-platform-commons;version='[1.10.2,1.10.3)',\
	junit-platform-engine;version='[1.10.2,1.10.3)',\
	junit-platform-launcher;version='[1.10.2,1.10.3)',\
	org.opentest4j;version='[1.3.0,1.3.1)',\
	junit-jupiter-api;version='[5.10.2,5.10.3)',\
	junit-jupiter-engine;version='[5.10.2,5.10.3)',\
	// snip: bundle under test and its dependencies
	aQute.libg;version='[7.0.0,7.0.1)',\
	biz.aQute.junit;version='[7.0.0,7.0.1)',\
	biz.aQute.launchpad;version='[7.0.0,7.0.1)'
Test-Cases: ${classes;CONCRETE;NAMED;*Test*}

// Minimal Launchpad test to check if testing with Launchpad works in the first place
import static org.junit.jupiter.api.Assertions.assertNotNull;

import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.osgi.framework.BundleContext;

import aQute.launchpad.Launchpad;
import aQute.launchpad.LaunchpadBuilder;
import aQute.launchpad.Service;

class LaunchpadInClasspathModeTest {
	static final String EQUINOX_FRAMEWORK = "org.eclipse.osgi;version=[3.18.300,3.19)";
	LaunchpadBuilder builder;
	
	@Service
	BundleContext context;
	
	@BeforeEach
	void before() {
		builder = new LaunchpadBuilder();
	}
	
	@Test
	void launchpadWorksInClasspathMode() throws Exception {
		try (Launchpad launchpad = builder.runfw(EQUINOX_FRAMEWORK)
				.create()
				.inject(this)) {
			assertNotNull(context);
		}
	}
}

jkoerkel · April 17, 2024, 10:31am

As it didn’t fit in the previous post, here’s the log output:

// Console Output after right click on project -> Run As -> Bnd OSGi Test Launcher (JUnit), comments omitted
------------------------------- REPORT --------------------------

Framework                                class org.eclipse.osgi.launch.Equinox
Framework type                           META-INF/services
Storage                                  null
Keep                                     false
Security                                 true
Has StartLevels                          false
Startlevel                               1
Run bundles                              C:/Users/julian/.m2/repository/biz/aQute/bnd/biz.aQute.tester.junit-platform/7.0.0/biz.aQute.tester.junit-platform-7.0.0.jar
                                         C:/Users/julian/.m2/repository/org/junit/platform/junit-platform-commons/1.10.2/junit-platform-commons-1.10.2.jar
                                         C:/Users/julian/.m2/repository/org/junit/platform/junit-platform-engine/1.10.2/junit-platform-engine-1.10.2.jar
                                         C:/Users/julian/.m2/repository/org/junit/platform/junit-platform-launcher/1.10.2/junit-platform-launcher-1.10.2.jar
                                         C:/Users/julian/.m2/repository/org/opentest4j/opentest4j/1.3.0/opentest4j-1.3.0.jar
                                         C:/Users/julian/.m2/repository/org/junit/jupiter/junit-jupiter-api/5.10.2/junit-jupiter-api-5.10.2.jar
                                         C:/Users/julian/.m2/repository/org/junit/jupiter/junit-jupiter-engine/5.10.2/junit-jupiter-engine-5.10.2.jar
                                         // snip: bundle under test and its dependencies
                                         C:/Users/julian/.m2/repository/biz/aQute/bnd/aQute.libg/7.0.0/aQute.libg-7.0.0.jar
                                         C:/Users/julian/.m2/repository/biz/aQute/bnd/biz.aQute.junit/7.0.0/biz.aQute.junit-7.0.0.jar
                                         C:/Users/julian/.m2/repository/biz/aQute/bnd/biz.aQute.launchpad/7.0.0/biz.aQute.launchpad-7.0.0.jar
Java Home                                C:\Users\julian\.p2\pool\plugins\org.eclipse.justj.openjdk.hotspot.jre.full.win32.x86_64_17.0.8.v20230831-1047\jre
Classpath                                C:/Users/julian/bnd_workspaces/*REDACTED*/cnf/cache/7.0.0/bnd-cache/biz.aQute.launcher/biz.aQute.launcher.pre.jar
System Packages                          org.eclipse.core.runtime.adaptor;x-friends:="org.eclipse.core.runtime"
                                         org.eclipse.core.runtime.internal.adaptor;x-internal:=true
                                         org.eclipse.equinox.log;version="1.1";uses:="org.osgi.framework
                                         org.osgi.service.log"
                                         org.eclipse.osgi.container;version="1.6";uses:="org.eclipse.osgi.report.resolution
                                           org.osgi.framework.wiring
                                           org.eclipse.osgi.framework.eventmgr
                                           org.osgi.framework.startlevel
                                           org.osgi.framework
                                           org.osgi.framework.hooks.resolver
                                           org.osgi.service.resolver
                                           org.osgi.resource
                                           org.eclipse.osgi.service.debug"
                                         org.eclipse.osgi.container.builders;version="1.0";uses:="org.eclipse.osgi.util
                                         org.eclipse.osgi.container"
                                         org.eclipse.osgi.container.namespaces;version="1.0";uses:="org.osgi.resource"
                                         org.eclipse.osgi.framework.console;version="1.1";uses:="org.osgi.framework"
                                         org.eclipse.osgi.framework.eventmgr;version="1.2"
                                         org.eclipse.osgi.framework.internal.reliablefile;x-internal:=true
                                         org.eclipse.osgi.framework.log;version="1.1";uses:="org.osgi.framework"
                                         org.eclipse.osgi.framework.util;x-internal:=true
                                         org.eclipse.osgi.internal.debug;x-internal:=true
                                         org.eclipse.osgi.internal.framework;x-internal:=true
                                         org.eclipse.osgi.internal.hookregistry;x-friends:="org.eclipse.osgi.tests"
                                         org.eclipse.osgi.internal.loader;x-internal:=true
                                         org.eclipse.osgi.internal.loader.buddy;x-internal:=true
                                         org.eclipse.osgi.internal.loader.classpath;x-internal:=true
                                         org.eclipse.osgi.internal.loader.sources;x-internal:=true
                                         org.eclipse.osgi.internal.location;x-internal:=true
                                         org.eclipse.osgi.internal.messages;x-internal:=true
                                         org.eclipse.osgi.internal.provisional.service.security;version="1.0.0";x-friends:="org.eclipse.equinox.security.ui"
                                         org.eclipse.osgi.internal.provisional.verifier;x-friends:="org.eclipse.ui.workbench
                                         org.eclipse.equinox.p2.artifact.repository"
                                         org.eclipse.osgi.internal.service.security;x-friends:="org.eclipse.equinox.security.ui"
                                         org.eclipse.osgi.internal.serviceregistry;x-internal:=true
                                         org.eclipse.osgi.internal.signedcontent;x-internal:=true
                                         org.eclipse.osgi.internal.url;x-internal:=true
                                         org.eclipse.osgi.launch;version="1.1";uses:="org.osgi.framework
                                         org.osgi.framework.launch
                                         org.osgi.framework.connect"
                                         org.eclipse.osgi.report.resolution;version="1.0";uses:="org.osgi.service.resolver
                                         org.osgi.resource"
                                         org.eclipse.osgi.service.datalocation;version="1.4.0"
                                         org.eclipse.osgi.service.debug;version="1.2"
                                         org.eclipse.osgi.service.environment;version="1.4"
                                         org.eclipse.osgi.service.localization;version="1.1";uses:="org.osgi.framework"
                                         org.eclipse.osgi.service.pluginconversion;version="1.0"
                                         org.eclipse.osgi.service.resolver;version="1.6";uses:="org.osgi.framework
                                         org.osgi.framework.hooks.resolver
                                         org.osgi.framework.wiring"
                                         org.eclipse.osgi.service.runnable;version="1.1"
                                         org.eclipse.osgi.service.security;version="1.0"
                                         org.eclipse.osgi.service.urlconversion;version="1.0"
                                         org.eclipse.osgi.signedcontent;version="1.1";uses:="org.osgi.framework"
                                         org.eclipse.osgi.storage;x-friends:="org.eclipse.osgi.tests"
                                         org.eclipse.osgi.storage.bundlefile;x-internal:=true
                                         org.eclipse.osgi.storage.url.reference;x-internal:=true
                                         org.eclipse.osgi.storagemanager;version="1.0"
                                         org.eclipse.osgi.util;version="1.1"
                                         org.osgi.dto;version="1.1.1"
                                         org.osgi.framework;version="1.10"
                                         org.osgi.framework.connect;version="1.0";uses:="org.osgi.framework.launch"
                                         org.osgi.framework.dto;version="1.8";uses:="org.osgi.dto"
                                         org.osgi.framework.hooks.bundle;version="1.1";uses:="org.osgi.framework"
                                         org.osgi.framework.hooks.resolver;version="1.0";uses:="org.osgi.framework.wiring"
                                         org.osgi.framework.hooks.service;version="1.1";uses:="org.osgi.framework"
                                         org.osgi.framework.hooks.weaving;version="1.1";uses:="org.osgi.framework.wiring"
                                         org.osgi.framework.launch;version="1.2";uses:="org.osgi.framework"
                                         org.osgi.framework.namespace;version="1.2";uses:="org.osgi.resource"
                                         org.osgi.framework.startlevel;version="1.0";uses:="org.osgi.framework"
                                         org.osgi.framework.startlevel.dto;version="1.0";uses:="org.osgi.dto"
                                         org.osgi.framework.wiring;version="1.2";uses:="org.osgi.framework
                                         org.osgi.resource"
                                         org.osgi.framework.wiring.dto;version="1.3";uses:="org.osgi.dto
                                         org.osgi.resource.dto"
                                         org.osgi.resource;version="1.0.1"
                                         org.osgi.resource.dto;version="1.0.1";uses:="org.osgi.dto"
                                         org.osgi.service.condition;version="1.0"
                                         org.osgi.service.condpermadmin;version="1.1.2";uses:="org.osgi.framework"
                                         org.osgi.service.log;version="1.5";uses:="org.osgi.framework"
                                         org.osgi.service.log.admin;version="1.0";uses:="org.osgi.service.log"
                                         org.osgi.service.packageadmin;version="1.2.1";uses:="org.osgi.framework"
                                         org.osgi.service.permissionadmin;version="1.2.1"
                                         org.osgi.service.resolver;version="1.1.1";uses:="org.osgi.resource"
                                         org.osgi.service.startlevel;version="1.1.1";uses:="org.osgi.framework"
                                         org.osgi.service.url;version="1.0.1"
                                         org.osgi.util.tracker;version="1.5.3";uses:="org.osgi.framework"
System Capabilities                      osgi.native;osgi.native.osname:List<String>="Windows10
                                         Windows 10
                                         Win32";osgi.native.osversion:Version="10.0.0";osgi.native.processor:List<String>="x86-64
                                         amd64
                                         em64t
                                         x86_64";osgi.native.language=de_DE
Properties                               
launch.activators                        
launch.framework.restart                 false
tester.controlport                       59099
tester.names                             *REDACTED*.LaunchpadInClasspathModeTest,*REDACTED*
tester.continuous                        true
launch.trace                             true
jvmarg -DtestEnvironment                 true
framework.elasticsearch.home             C:\Users\julian\*REDACTED*\bndtools/data/elasticsearch
user.timezone                            Europe/Berlin
framework.webserver.superuser            C:\Users\julian\*REDACTED*\bndtools/config/realm.properties
framework.logging.folder                 C:\Users\julian\*REDACTED*\bndtools/logs
launch.embedded                          false
launch.services                          true
osgi.clean                               true
osgi.console                             
java.io.tmpdir                           C:\Users\julian\AppData\Local\Temp\
launch.notificationPort                  -1
launch.timeout                           0
launch.noreferences                      false
jsdev                                    true
framework.database.config                C:\Users\julian\*REDACTED*\bndtools/config/db.properties
tester.dir                               C:\Users\julian\bnd_workspaces\*REDACTED*\*REDACTED*\target\test-reports
launch.bundles                           C:/Users/julian/.m2/repository/biz/aQute/bnd/biz.aQute.tester.junit-platform/7.0.0/biz.aQute.tester.junit-platform-7.0.0.jar,C:/Users/julian/.m2/repository/org/junit/platform/junit-platform-commons/1.10.2/junit-platform-commons-1.10.2.jar,C:/Users/julian/.m2/repository/org/junit/platform/junit-platform-engine/1.10.2/junit-platform-engine-1.10.2.jar,C:/Users/julian/.m2/repository/org/junit/platform/junit-platform-launcher/1.10.2/junit-platform-launcher-1.10.2.jar,C:/Users/julian/.m2/repository/org/opentest4j/opentest4j/1.3.0/opentest4j-1.3.0.jar,C:/Users/julian/.m2/repository/org/junit/jupiter/junit-jupiter-api/5.10.2/junit-jupiter-api-5.10.2.jar,C:/Users/julian/.m2/repository/org/junit/jupiter/junit-jupiter-engine/5.10.2/junit-jupiter-engine-5.10.2.jar,// snip: bundle under test and its dependencies,C:/Users/julian/.m2/repository/biz/aQute/bnd/aQute.libg/7.0.0/aQute.libg-7.0.0.jar,C:/Users/julian/.m2/repository/biz/aQute/bnd/biz.aQute.junit/7.0.0/biz.aQute.junit-7.0.0.jar,C:/Users/julian/.m2/repository/biz/aQute/bnd/biz.aQute.launchpad/7.0.0/biz.aQute.launchpad-7.0.0.jar
launch.system.capabilities               osgi.native;osgi.native.osname:List<String>="Windows10,Windows 10,Win32";osgi.native.osversion:Version="10.0.0";osgi.native.processor:List<String>="x86-64,amd64,em64t,x86_64";osgi.native.language=de_DE
launch.system.packages                   org.eclipse.core.runtime.adaptor;x-friends:="org.eclipse.core.runtime",org.eclipse.core.runtime.internal.adaptor;x-internal:=true,org.eclipse.equinox.log;version="1.1";uses:="org.osgi.framework,org.osgi.service.log",org.eclipse.osgi.container;version="1.6";uses:="org.eclipse.osgi.report.resolution,  org.osgi.framework.wiring,  org.eclipse.osgi.framework.eventmgr,  org.osgi.framework.startlevel,  org.osgi.framework,  org.osgi.framework.hooks.resolver,  org.osgi.service.resolver,  org.osgi.resource,  org.eclipse.osgi.service.debug",org.eclipse.osgi.container.builders;version="1.0";uses:="org.eclipse.osgi.util,org.eclipse.osgi.container",org.eclipse.osgi.container.namespaces;version="1.0";uses:="org.osgi.resource",org.eclipse.osgi.framework.console;version="1.1";uses:="org.osgi.framework",org.eclipse.osgi.framework.eventmgr;version="1.2",org.eclipse.osgi.framework.internal.reliablefile;x-internal:=true,org.eclipse.osgi.framework.log;version="1.1";uses:="org.osgi.framework",org.eclipse.osgi.framework.util;x-internal:=true,org.eclipse.osgi.internal.debug;x-internal:=true,org.eclipse.osgi.internal.framework;x-internal:=true,org.eclipse.osgi.internal.hookregistry;x-friends:="org.eclipse.osgi.tests",org.eclipse.osgi.internal.loader;x-internal:=true,org.eclipse.osgi.internal.loader.buddy;x-internal:=true,org.eclipse.osgi.internal.loader.classpath;x-internal:=true,org.eclipse.osgi.internal.loader.sources;x-internal:=true,org.eclipse.osgi.internal.location;x-internal:=true,org.eclipse.osgi.internal.messages;x-internal:=true,org.eclipse.osgi.internal.provisional.service.security;version="1.0.0";x-friends:="org.eclipse.equinox.security.ui",org.eclipse.osgi.internal.provisional.verifier;x-friends:="org.eclipse.ui.workbench,org.eclipse.equinox.p2.artifact.repository",org.eclipse.osgi.internal.service.security;x-friends:="org.eclipse.equinox.security.ui",org.eclipse.osgi.internal.serviceregistry;x-internal:=true,org.eclipse.osgi.internal.signedcontent;x-internal:=true,org.eclipse.osgi.internal.url;x-internal:=true,org.eclipse.osgi.launch;version="1.1";uses:="org.osgi.framework,org.osgi.framework.launch,org.osgi.framework.connect",org.eclipse.osgi.report.resolution;version="1.0";uses:="org.osgi.service.resolver,org.osgi.resource",org.eclipse.osgi.service.datalocation;version="1.4.0",org.eclipse.osgi.service.debug;version="1.2",org.eclipse.osgi.service.environment;version="1.4",org.eclipse.osgi.service.localization;version="1.1";uses:="org.osgi.framework",org.eclipse.osgi.service.pluginconversion;version="1.0",org.eclipse.osgi.service.resolver;version="1.6";uses:="org.osgi.framework,org.osgi.framework.hooks.resolver,org.osgi.framework.wiring",org.eclipse.osgi.service.runnable;version="1.1",org.eclipse.osgi.service.security;version="1.0",org.eclipse.osgi.service.urlconversion;version="1.0",org.eclipse.osgi.signedcontent;version="1.1";uses:="org.osgi.framework",org.eclipse.osgi.storage;x-friends:="org.eclipse.osgi.tests",org.eclipse.osgi.storage.bundlefile;x-internal:=true,org.eclipse.osgi.storage.url.reference;x-internal:=true,org.eclipse.osgi.storagemanager;version="1.0",org.eclipse.osgi.util;version="1.1",org.osgi.dto;version="1.1.1",org.osgi.framework;version="1.10",org.osgi.framework.connect;version="1.0";uses:="org.osgi.framework.launch",org.osgi.framework.dto;version="1.8";uses:="org.osgi.dto",org.osgi.framework.hooks.bundle;version="1.1";uses:="org.osgi.framework",org.osgi.framework.hooks.resolver;version="1.0";uses:="org.osgi.framework.wiring",org.osgi.framework.hooks.service;version="1.1";uses:="org.osgi.framework",org.osgi.framework.hooks.weaving;version="1.1";uses:="org.osgi.framework.wiring",org.osgi.framework.launch;version="1.2";uses:="org.osgi.framework",org.osgi.framework.namespace;version="1.2";uses:="org.osgi.resource",org.osgi.framework.startlevel;version="1.0";uses:="org.osgi.framework",org.osgi.framework.startlevel.dto;version="1.0";uses:="org.osgi.dto",org.osgi.framework.wiring;version="1.2";uses:="org.osgi.framework,org.osgi.resource",org.osgi.framework.wiring.dto;version="1.3";uses:="org.osgi.dto,org.osgi.resource.dto",org.osgi.resource;version="1.0.1",org.osgi.resource.dto;version="1.0.1";uses:="org.osgi.dto",org.osgi.service.condition;version="1.0",org.osgi.service.condpermadmin;version="1.1.2";uses:="org.osgi.framework",org.osgi.service.log;version="1.5";uses:="org.osgi.framework",org.osgi.service.log.admin;version="1.0";uses:="org.osgi.service.log",org.osgi.service.packageadmin;version="1.2.1";uses:="org.osgi.framework",org.osgi.service.permissionadmin;version="1.2.1",org.osgi.service.resolver;version="1.1.1";uses:="org.osgi.resource",org.osgi.service.startlevel;version="1.1.1";uses:="org.osgi.framework",org.osgi.service.url;version="1.0.1",org.osgi.util.tracker;version="1.5.3";uses:="org.osgi.framework"
osgi.noShutdown                          true
eclipse.ignoreApp                        true
launch.keep                              false
testEnvironment                          true
osgi.console.enable.builtin              false
launch.name                              bnd.bnd
developmentEnvironment                   true
framework.webserver.webbundles.workdir   C:\Users\julian\*REDACTED*\bndtools/data/webbundles
launch.activation.eager                  false
tester.unresolved                        true

Id    Levl State Modified     Location
0     0    ACTIV <>           System Bundle
1     1    ACTIV 202310060956 file:/C:/Users/julian/.m2/repository/biz/aQute/bnd/biz.aQute.tester.junit-platform/7.0.0/biz.aQute.tester.junit-platform-7.0.0.jar
2     1    ACTIV 202402040835 file:/C:/Users/julian/.m2/repository/org/junit/platform/junit-platform-commons/1.10.2/junit-platform-commons-1.10.2.jar
3     1    ACTIV 202402040835 file:/C:/Users/julian/.m2/repository/org/junit/platform/junit-platform-engine/1.10.2/junit-platform-engine-1.10.2.jar
4     1    ACTIV 202402040835 file:/C:/Users/julian/.m2/repository/org/junit/platform/junit-platform-launcher/1.10.2/junit-platform-launcher-1.10.2.jar
5     1    ACTIV 202307061225 file:/C:/Users/julian/.m2/repository/org/opentest4j/opentest4j/1.3.0/opentest4j-1.3.0.jar
6     1    ACTIV 202402040835 file:/C:/Users/julian/.m2/repository/org/junit/jupiter/junit-jupiter-api/5.10.2/junit-jupiter-api-5.10.2.jar
7     1    ACTIV 202402040835 file:/C:/Users/julian/.m2/repository/org/junit/jupiter/junit-jupiter-engine/5.10.2/junit-jupiter-engine-5.10.2.jar
// snip: bundle under test and its dependencies
114   1    ACTIV 202310060954 file:/C:/Users/julian/.m2/repository/biz/aQute/bnd/aQute.libg/7.0.0/aQute.libg-7.0.0.jar
115   1    ACTIV 202310060955 file:/C:/Users/julian/.m2/repository/biz/aQute/bnd/biz.aQute.junit/7.0.0/biz.aQute.junit-7.0.0.jar
116   1    ACTIV 202310060956 file:/C:/Users/julian/.m2/repository/biz/aQute/bnd/biz.aQute.launchpad/7.0.0/biz.aQute.launchpad-7.0.0.jar
# snip: bundle under test and its dependencies
11:37:50,397 INFO  [aQute.libg] - BundleEvent STARTING
11:37:50,397 INFO  [aQute.libg] - BundleEvent STARTED
11:37:50,397 INFO  [biz.aQute.junit] - BundleEvent STARTING
11:37:50,403 INFO  [biz.aQute.junit] - ServiceEvent REGISTERED {java.lang.Runnable}={service.description=JUnit tester, service.id=75, service.bundleid=115, service.scope=singleton, main.thread=true}
11:37:50,403 INFO  [biz.aQute.junit] - BundleEvent STARTED
11:37:50,404 INFO  [biz.aQute.launchpad] - BundleEvent STARTING
11:37:50,404 INFO  [biz.aQute.launchpad] - BundleEvent STARTED
11:37:50,513 INFO  [biz.aQute.tester.junit-platform] - ServiceEvent REGISTERED {java.lang.Object}={osgi.command.scope=tester, service.id=76, service.bundleid=1, service.scope=singleton, osgi.command.function=[runTests,getTesterNames,setTesterNames]}

jkoerkel · April 17, 2024, 1:53pm

BUT = bundle under test

Ok ok, I just found out that with the correct configuration, just Run As → JUnit Test will be necessary, also to run Launchpad tests.

Now, not strictly related to only testing:
I have a bundle in my -buildpath that is signed and exports org.slf4j, that is a dependency of my BUT. But slf4j.simple, which I’m trying to use to provide an SLF4JServiceProvider, which isn’t provided by the logging bundle in my BUT, is not signed and (I guess, but don’t know) therefore I get the following exception:

java.lang.ExceptionInInitializerError
	at aQute.bnd.remoteworkspace.client.RemoteWorkspaceClientFactory.<clinit>(RemoteWorkspaceClientFactory.java:28)
	at aQute.launchpad.LaunchpadBuilder.<clinit>(LaunchpadBuilder.java:51)
	at *REDACTED*.LaunchpadInClasspathModeTest.before(LaunchpadInClasspathModeTest.java:22)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
Caused by: java.lang.SecurityException: class "org.slf4j.IMarkerFactory"'s signer information does not match signer information of other classes in the same package
	at java.base/java.lang.ClassLoader.checkCerts(ClassLoader.java:1163)
	at java.base/java.lang.ClassLoader.preDefineClass(ClassLoader.java:907)
	at java.base/java.lang.ClassLoader.defineClass(ClassLoader.java:1015)
	at java.base/java.security.SecureClassLoader.defineClass(SecureClassLoader.java:150)
	at java.base/jdk.internal.loader.BuiltinClassLoader.defineClass(BuiltinClassLoader.java:862)
	at java.base/jdk.internal.loader.BuiltinClassLoader.findClassOnClassPathOrNull(BuiltinClassLoader.java:760)
	at java.base/jdk.internal.loader.BuiltinClassLoader.loadClassOrNull(BuiltinClassLoader.java:681)
	at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:639)
	at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:525)
	at org.slf4j.LoggerFactory.<clinit>(LoggerFactory.java:95)
	... 6 more

Is there a way to hide packages from -buildpath bundles once they are in the -testpath?
I’ve tried this with no success:

-testpath: \
	junit-jupiter-api;version=latest,\
	junit-jupiter-engine;version=latest,\
	junit-platform-commons;version=latest,\
	junit-platform-engine;version=latest,\
	junit-platform-launcher;version=latest,\
	org.opentest4j;version=latest,\
	biz.aQute.launchpad,\
	slf4j.api;version=latest,\
	slf4j.simple;version=latest

-testpath+:\
	"signed.slf4j-wrapping.bundle";packages="!org.slf4j.*,*"

pkriens · April 17, 2024, 4:40pm

Just a note. Launchpad is a fabulous when you use API based programming where API is strictly separated from the implementation. It really doesn’t like hacks. If you have a hackish code base it is better to use the OSGi launching. It is a bit slower and less Junitish but it works very reliably in all circumstances.

That said. In my experience it really is worth the effort to write 85% of you tests on component level with plain JUnit without OSGi. The beauty of the components API is that you rarely need OSGi API and with a bit of careful design you can make components that can be well tested without the Framework running.

I am moving more and more code to Launchpad but I generally use the OSGi testing for integration tests since there are so many bad bundles out there.

jkoerkel · April 22, 2024, 9:47am

I haven’t found a way to make that work. The runbundles get started, the console output gets shown for all that, but no tests get executed before the launch terminates again. How would I make that work?

Unfortunately, pretty much all our public Service methods have a parameter object that we get out of a Service. We could mock that object, but then directly testing in-framework seems like it really is what we want to be doing, it would take a while to explain why.

I was so far under the impression what you call OSGi testing is Launchpad testing. Could you clear up my confusion?

pkriens · April 22, 2024, 12:25pm

Did you set the Test-Cases header?

That is a pity. In general you want to make your service design to never use other services in the API. This is generally not necessary because the implementation can easily depend on them. The OSGi services APIs are very good in this, there are very few cross references between the standard services. The more the parameters are basic Java VM objects, the less complexity you will have. Today’s Java records are almost perfect for this.

But I understand the complexity of an existing code base. You go to war with the code base you have, not the one you’d like to have

Standard OSGi testing is the model where you create test bundles. That is, you encode your tests in a separate project and the test code is in the normal source tree, not the test src tree. Each of these bundles has a header Test-Cases that point out which classes are the JUnit test cases. You then build up a bndrun file (although you can also use the bnd file) with a run configuration. When you launch it with the test Bnd OSGi Test menu (or the Test button in the bnd editor for the Run tab), bnd will launch the framework and will execute all Test-Cases. I.e. when you test case is called, OSGi is completely setup. This type of testing is completely integrated with the Eclipse JUnit support. I.e. you can select a JUnit test method and call up the Run menu which will show Bnd OSGi Testing.

Launchpad works differently. It is a plain old JUnit test in the test source tree. Its perfect if you need one or more frameworks running in your test. During testing, it communicates with Eclipse/Gradle to calculate what code comes from the classpath (-testpath) and what bundles should be run. Since Java makes it impossible to restrict the classes that are on the classpath, this puts some severe limits on how you setup the project. It works like a charm when you follow the OSGi rules: strict separation of API and implementation.

The advantage is that you can easily setup multiple frameworks and you can make tests where the components are an inner class in your test code. It is a lot easier to manipulate the OSGi environment and test multiple configurations. For example, in the distributed OSGi I recently worked on, I create 50 frameworks for testing.

Launchpad is best used in testing a single service. In that case the supporting code are normal bundles and you fetch the services from there but the code under test (API+implementation) is on the -testpath. OSGi testing is best used for large integration tests.

You indicated earlier, rather scathingly if I may say, that you were not impressed with the documentation. You are aware of the bnd.bndtools.org site? I am the last one to say it is perfect but there is a decent amount of information available. We also recently updated bndtools.org with the documents that were developed in OSGi enRoute.

I am not fond of using this discourse to advertise but looking at your story so far, I think I could save you untold headaches and myriad of costs if you hire me for a few day to help you with setting up your workspace.

jkoerkel · April 22, 2024, 3:38pm

Aaah , that wasn’t clear to me. I had assumed something along the lines of the OSGI Test launcher picking up all Test-Cases-included testsrc-contained classes with a @aQute.launchpad.Service on one of their fields.

Yeah we’re in one of these ecosystems that take this whole collaborative software thing and at the same time step it up a notch but also make nearly everything you wanna do relatively complicated by having to do lots of policy and politics (need a dependency? File a JIRA ticket, it’s gonna take as long as… it takes).
Wrt OSGi-philosphy; we’re only one of a plethora of partners writing bundles for a large platform with some other partners at similar structural complexity, where we’re in with 95 bundles, most of which XL, becoming more and each smaller in close future as I had already gotten your point about cleanly doing OSGi. I’ve fought tooth and nails to be allowed to switch our working paradigm and build system to bndtools with all that entails cause of CI/CD and actually won, so no hacky shortcuts in the future, but we still need to get there.

pkriens:

Standard OSGi testing is the model where you create test bundles. That is, you encode your tests in a separate project and the test code is in the normal source tree, not the test src tree. Each of these bundles has a header Test-Cases that point out which classes are the JUnit test cases. You then build up a bndrun file (although you can also use the bnd file) with a run configuration. When you launch it with the test Bnd OSGi Test menu (or the Test button in the bnd editor for the Run tab), bnd will launch the framework and will execute all Test-Cases. I.e. when you test case is called, OSGi is completely setup. This type of testing is completely integrated with the Eclipse JUnit support. I.e. you can select a JUnit test method and call up the Run menu which will show Bnd OSGi Testing.

Launchpad works differently. It is a plain old JUnit test in the test source tree. Its perfect if you need one or more frameworks running in your test. During testing, it communicates with Eclipse/Gradle to calculate what code comes from the classpath (-testpath) and what bundles should be run. Since Java makes it impossible to restrict the classes that are on the classpath, this puts some severe limits on how you setup the project. It works like a charm when you follow the OSGi rules: strict separation of API and implementation.

The advantage is that you can easily setup multiple frameworks and you can make tests where the components are an inner class in your test code. It is a lot easier to manipulate the OSGi environment and test multiple configurations. For example, in the distributed OSGi I recently worked on, I create 50 frameworks for testing.

Launchpad is best used in testing a single service. In that case the supporting code are normal bundles and you fetch the services from there but the code under test (API+implementation) is on the -testpath. OSGi testing is best used for large integration tests.

That is a wonderful Introduction for bnd.bndtools.org - Testing and I’ll refer on to that in my future attempts, now I first need to wisen up whether I can alleviate the problem with our services needing a service method returned object from platform, cause it looks a lot like I can

I indicated my dispiritedness with bndtools’ documentation in an unnecessarily provocative way -not having had my usual breakfast thesaurus I had to google scathingly- while neither wanting to come off as angry, bitter, cruel or shrill, nor wanting to overstay my welcome here, having seen posts from over ten years ago already apologetically adressing the incomplete state of bndtools’ documentation, I found the hurdles to get started with just testing very high. If I were to find a technology for a yet-to-be-started project, bndtools would be off the list after only very shallow research, because quality of documentation is my #1 metric for assumed quality of technology.
And yes, I knew the bnd.bndtools.org site, I would have been 100% lost if not at least that documentation had been there.

I get you, it’s cool.
Economically, I guess you’re completely right. But possibly only about the very company I work for directly. Headache-wise, that’s unfortunately the kind of pain I live for.
Now in all seriousness, seeing as I am the only one here really working on our bndtools conversion, while I’m a junior-like paid junior (confused ), I think it’s gonna be hard to reason that through with management, as they might want my learnings to at some point be committed back somehow into bndtools, so not only we here internally, but also platform (our parent company) and partners can profit and maybe use that whole open source shebang even a little bit as marketing for potential new hires. But also I am personally of the opinion that improved documentation can help many projects that should transition from PDE to bndtools and the fewer PDE projects are still in use, the faster this whole digitization in this weird country 1000 km north-east of you goes.

pkriens · April 24, 2024, 12:23pm

We handle those automations with macros, in this case the classes macro.

My life long experience. Still worth fighting for.

Its the common response and my life long experience tells me it would be the highest return money the company ever spent Builds and workspace tend to live very long lives, even a slight inefficiency will hurt productivity for decades.

In the nineties, when OO was introduced, companies had no problem doling out thousands of dollars for courses because education was deemed important. Now we have the internet so they assume you can figure it out yourself … You can, but it will take a lot longer and you’ll miss out on a lot of handy tricks.

But keep posting your problems here and I’ll try to help.

pkriens · April 24, 2024, 12:26pm

BTW, did you read: OSGi Starter - Bndtools

Topic		Replies	Views
Can't run junit integration test from eclipse GENERAL	2	468	November 12, 2021
Creating a minimal stand alone maven integration test TIPS	9	299	April 16, 2022
UNit test failed (java.lang.NullPointerException) when running with Gradle GENERAL	4	2100	March 29, 2021
Invalid content checksum when using bnd-maven-plugin DEVELOPMENT	8	2396	May 21, 2021
Eclipse, Java 17, and JUnit testen TIPS	3	131	November 17, 2023

Testing - outdated documentation - current best practices

Related Topics