Does anybody know of a possibility to add additional bundles when releasing a project? On e.g. gradle release we release all bundles to the specified release repositories. I search for a method to add additional bundles and release them as well.
Something like:
-release: org.apache.felix.http.jetty
Look at the biz.aQute.remote project in the bndtools project which does exactly this.
https://github.com/bndtools/bnd/blob/master/biz.aQute.remote/bnd.bnd
You want to copy bundles from one repository to another? Sounds like a bad idea since this bundle is on Maven Central?
Can you explain what you want to achieve so I understand?
There is a -conduit function for the purpose that you want an external bundle treated as a project.
We have a lot of projects will or do provide own workspace templates. They in turn are no full workspaces but āadditionsā to an existing one. They simply adding a bnd file to the ext, that defines all necessary dependencies you need to use it.
In currently create a Workspace, that will should build after each release of any of our projects (project here are separate workspaces and git repos). In this I have referenced all our project repositories (wew release to maven and as OSGiRepo). In there is a bndrun, that has a requirement to collect all our workspace templates. I resolve the bnd run in the build and want to add all -rundbundles to one OSGiRepo.
Do we have any documentation for -conduit?
I think youāre falling for the (pipe dream of) transitive dependencies ā¦ If you think that is a better model, then Maven seems more suitable? It sounds like youāre fundamentally starting to fight the workspace model. In my experience, that is not a good idea, go with the flow.
No I do not think there is any specific documentation for -conduit except the code. I think you could specify a repo dependency.
No, I donāt want to fight the model. I want to automatically create a OSGiRepo, that I can reference in the bndtools repository preferences, without manually act, if some of our projects do a release build.
This could basically do the trick:
providers: ${findproviders;org.bndtools.template;(&(org.bndtools.template=workspace)(type=extension))}
-conduit: ${providers}
This unfortunately fails, because conduit expects files and nothing like org.gecko.rest.bnd.template.workspace. Do we have something like a toFile macro? I couldnāt find something. While writing this, I realized that Iām not really sure if conduit could handle multiple files.
BTW: we have documentation for it: -conduit | bnd
I want to automatically create a OSGiRepo, that I can reference in the bndtools repository preferences, without manually act, if some of our projects do a release build.
I am still not understanding what you try to achieve 
Literally, the bnd code says that you want to release all the workspace template JARs with a release?
That does smell like you want to create an aggregate. The primary purpose of a release is to place each artifact in a public place, definitely not to create an aggregate. Aggregates are clearly in the realm of the bndrun exporters.
You could -export a ZIP file with all the released artifacts from you workspace and any additional providers. The Maven Bnd Repository has a multi option that parses zip files and finds OSGi artifacts inside. I.e. you release this zip file from workspace A, then the users with workspace B only have to include one file in their repository but they get the whole enchillada.
That said, in the case I misunderstand you.
You can use the ${repo} macro to turn the provider into a file. And -conduit is likely to be for one file. Should be not too hard to fix this in bnd, you probably need to emulate multiple sub builders. However, in the case of workspace templates, of which you have one or two, Iād consider if this is worth the work.
First off: You are basically right. I want an aggregate and this is in the realm of an exporter. For development purposes an aggregate might be more comfortable in some places, but Iām with you, that I should get my dependencies from where they originate. If I want to provide e.g. an SDK for my Product a Repository that contain everything a developer might need also makes sense though. I will create an exporter, that results in an index repository if we donāt already have one.
To explain where Iām coming from in this Particular case:
Part of our job is to convince people to use bnd and OSGi in general. While bnd is a great tool, the first steps are quite hard if you are new to it. The same goes for OSGi. Last year we stumbled upon https://start.spring.io and thought that it would be great if we would have something close to this for us as well. I always saw the templates we have as a good starting point to establish something close in bndtools.
I realized that we already have most of the things required to provide something like this for bndtools. Wokspace and project templates in a jar provide most of the functionality. A workspace template can be something like an extension to an existing workspace by just adding a repo and e.g. maven index file to the ext folder. This in turn can bring all necessary bundles for a certain functionality and e.g. fitting project templates. Additionally bndtools allows as to to set a template repo in its preferences.
The goal would be to start a new workspace and select your base workspace. When this is applied we can automatically scan the repos for workspace extensions so the user can easily mark what he wants. The end would be to say, I want the latest bnd workspace + our gecko JaxRs Whiteboard + ECF + what ever is available. After this, we can march on and ask if the users want to use some of the newly detected project templates the extensions provide.
Most of the things need some polishing. At the moment all of this can be achieved by clicking new bndworkspace over and over, coupled with a lot of WS reloads and sometimes restarts.
Iām currently trying to collect all the templates provided by all of our workspaces in a central workspace build and create and Aggregated index, we can use as a preset bnd workspace repository.
Forgot to mention: We would call it bndbootstrap.
If my aggregate workspace finally does its job, we might be able to provide people a central platform, where they can publish their workspace extensions. E.g. Scott Lewis eagerly provides nice workspace templates. Unfortunately, you have to use his template or the bnd workspace template.
I am not sure I agree that we do not already have this. You can create such a workspace extension in the ext/ directory. A bnd file in there can define a repository (and using the source property of the Maven Bnd Repository, you can even include the repo contents). The merged properties allow you to influence lots of instructions. You could even make this conditional, only when a project opts in will the extension function for that project take place. (This is only for project functions of course, not for workspace instructions like repos.) Since most new functions use the repos, adding a repo will also allow you to add External Plugins & templates.
In the classic enRoute, this was the route I took with the distro being defined in an ext file. All setup to use enRoute was in there. The build.bnd was empty.
So I am not sure what youāre missing? You just want to automate it more? Dragging a text file to your ext directory is pretty good isnāt it? I guess the hardest part is discovery. We could create a repository on Github where people could store & update their ext files?
Another solution is to use the -include. The -include accepts URIs so it is easy then to include an extension directly from Github. If this is a good idea, we could use a new instruction in build.bnd that would read a setup file for a workspace extension directly from Github?
-workspace-extension \
pkriens/awesome, \
eclipse/ecf;version=1.0
I think we can more or less do what you want today. Of course the devil is in the details. However, if you have the motivation, then adding the missing parts in bnd would not be too painful.
As a side note. I started to think about creating a tool to manage the workspace itself. I really dislike the myriad of gradle files that litter the root. I was thinking to define the important gradle information in the build.bnd file and then let this tool create the files just before the build so they do not have to be stored in git. The tool could also understand workflows and maybe Java VMs. Just an idea that might be relevant here.
I am not sure I agree that we do not already have this. You can create such a workspace extension in the
ext/directory. A bnd file in there can define a repository (and using thesourceproperty of the Maven Bnd Repository, you can even include the repo contents). The merged properties allow you to influence lots of instructions.
This is what I realized last week. I thought Iāve said so.
You could even make this conditional, only when a project opts in will the extension function for that project take place. (This is only for project functions of course, not for workspace instructions like repos.)
You mean by adding e.g. a repo plugin in a projects bnd file?`
So I am not sure what youāre missing? You just want to automate it more? Dragging a text file to your ext directory is pretty good isnāt it? I guess the hardest part is discovery. We could create a repository on Github where people could store & update their ext files?
Annotations like e.g. @RequireHttpWhiteboard are great and easy to use. In order to use them, I need to know they exist and need the bundle and maybe some dependencies in order to use it. Iād hope we can find a way to establish something like the eclipse marketplace for to provide a comfortable way to add new functionality and help keep deps up to date.
Yes draging in a file is easy if you know what you need and where to get it from. I hope to find a way to make it as comfortable and started friendly as possible.
Another solution is to use the -include. The -include accepts URIs so it is easy then to include an extension directly from Github. If this is a good idea, we could use a new instruction in build.bnd that would read a setup file for a workspace extension directly from Github?
I havenāt thought about the fact that include can grep content from a URI. This makes things much easier. The way with the -workspace-extension instruction, I like even more. It should be relatively straight forward.
The question would be how to design it as flexible as possible while keeping it comfortable and what conventions the repo should adhere to. Iād like to maintain such a extension file in my projects workspace. We could e.g. look for a workspaceExtension.bnd in the projects root or another predefined location.
As a side note. I started to think about creating a tool to manage the workspace itself. I really dislike the myriad of gradle files that litter the root. I was thinking to define the important gradle information in the build.bnd file and then let this tool create the files just before the build so they do not have to be stored in git. The tool could also understand workflows and maybe Java VMs. Just an idea that might be relevant here.
I see your point. This is basically how some of my current work started. I restructured our workspaces and the bnd workspace template (approval pending) so it will become easier to update all the gradle things to the latests version.
Iām not sure how your idea would work though. Right now gradlew or a native gradle installation is the starting point. You would need to call something else, if you want the gradle files to be generated on build.
Yes, but I meant that in the project bnd file you could say:
-enable-ecf
And then in the cnf/ext/ecf..bnd you can use macros to selectively add things to the -buildpath, -testpath, etc.
-buildpath.ecf = ${if;${enable-ecf};ecf.lib}
A repo is a workspace instruction so that can not be selected on project level.
Yes, that is not well known. However, there is currently no caching in place.
Yes. Something like bnd preparebuild. For people that do a lot in Gradle that wonāt work but my lifeās goal is to do everything in bnd!
Hi Peter,
Iāve given it a shot and it works like a charm.
My bnd file looks as follows:
-plugin.geckoEMF: \
aQute.bnd.repository.maven.pom.provider.BndPomRepository;\
releaseUrl = "https://repo.maven.apache.org/maven2/,https://devel.data-in-motion.biz/nexus/repository/dim-release/"; \
snapshotUrl = "https://devel.data-in-motion.biz/nexus/repository/dim-snapshot/";\
revision = "${bomDeps}" ;\
transitive = false;\
name=GeckoEMF Workspace Extension
-buildpath.emf = ${if;${-enable-gecko-emf};${emf.lib}}
bomDeps: \
org.osgi:org.osgi.util.function:1.1.0,\
org.osgi:org.osgi.util.promise:1.1.0,\
org.apache.felix:org.apache.felix.configadmin:1.9.16,\
org.apache.felix:org.apache.felix.scr:2.1.16,\
org.eclipse.emf:org.eclipse.emf.common:2.20.0,\
org.eclipse.emf:org.eclipse.emf.ecore:2.23.0,\
org.eclipse.emf:org.eclipse.emf.ecore.xmi:2.16.0,\
org.geckoprojects.emf:org.gecko.emf.osgi.api:3.3.8-SNAPSHOT,\
org.geckoprojects.emf:org.gecko.emf.osgi.component:3.3.8-SNAPSHOT,\
org.geckoprojects.emf:org.gecko.emf.osgi.ecore:3.3.8-SNAPSHOT,\
org.geckoprojects.emf:org.gecko.emf.osgi.model.info.api:3.3.8-SNAPSHOT,\
org.geckoprojects.emf:org.gecko.emf.osgi.model.info.impl:3.3.8-SNAPSHOT,\
org.geckoprojects.emf:org.gecko.emf.osgi.bnd.templates.project:3.3.8-SNAPSHOT,\
org.geckoprojects.emf:org.gecko.emf.osgi.annotation:3.3.8-SNAPSHOT
emf.lib: \
org.gecko.emf.osgi.annotation;version=latest,\
org.gecko.emf.osgi.api;version=latest,\
org.eclipse.emf.common,\
org.eclipse.emf.ecore,\
org.eclipse.emf.ecore.xmi
In a workspace that wants to use it, simply do the following:
-include: https://gitlab.com/gecko.io/geckoEMF/-/raw/develop/cnf/geckoEMF/geckoEMF.bnd
This works as of yesterday evening after after BJ was so kind to handle this issue.
This brings me back to your other idea.
-workspace-extension:\
gecko.io/geckoEMF;type=gitlab;branch=develop,\
pkriens/awesome, \
eclipse/ecf;tag=1.0
With this we would know, in what repository to look. The question would be where to look there. I would opt for e.g. cnf/workspaceextension/extension.bnd or should we handle it more like the workspace template via git and include everything in the repo? As I would like to maintain sich extensions in the workspaces of e.g. geckoEMF I would like to not apply the whole repo.
It looks like youāve been busy! Nice work.
I would suggest that you checkout the branch in ācache/ext/ā, using a directory with the extension name (org/repo). In the repo, in the root directory there should be a workspace.bnd file that does all the setup work for the workspace. There might be a project.bnd that would be read with a generic opt-in instruction in the projectās bnd.bnd:
-use pkriens/awesome
Since we checkout the branch, you can include any number of files and directories for help files since not everything is in a repository.
The issue that was solved is kind of interesting. We might first want to read ext/*, then build.bnd and then the extensions. However, extensions should not be able to override the things set in the build.bnd.
Hmm. This does of course starts to resemble sub modules in git ā¦
Not sure if we should think about security?
@juergen-albert I am very much warming up to this idea. Any progress?
Me too. Iām currently in the process of updating a couple of our projects with the -include Version to get a feel for the land. I actually need to stop shaving the yak for a because I need to get an actual progress in the project I want to use this for.
This will become my weekend project, if the kids allow.
I like your idea, but I need to look what I would need to do in order to do the checkout.
Regarding your -use instruction: would this be similar to what I have in my example with -enable-gecko-emf: true?
Iāve done a bit more thinking ā¦ Maybe we should not use Git repos but good old bundles? An extension could easily be stored in a JAR and a Capability describing it.
Provide-Capability \
bnd.workspace.extension; \
bnd.workspace.extension=biz.aQute.workspace.awesome; \
where=we/awesome; \
version = 1.2.3, \
bnd.workspace.extension; \
bnd.workspace.extension=biz.aQute.workspace.javagen; \
where=we/javagen; \
version = 4.5.6
The humongous advantage is then that search, discovery, caching, naming, versioning, version ranges, distribution, security, building, annotations, releasing, snapshots, authentication, format, etc. are all already in place and done pretty professionally.
Key problem is of course that they then come from repositories and they should be able to create repositories. However, the plus side is so large that I think it is worth considering this avenue.
-workspace-extension \
biz.aQute.workspace.awesome;version=1.2, \
...
We could make a context menu entry that lists all workspace extensions in the repositories and automatically inserts it in the instruction, like I did for the -plugins.
Close but not exactly. I think we need a workspace.bnd that defines global things and a per project project.bnd that is only included when you use -use. So the workspace.bnd could still do global things and the project.bnd could be empty. However, it allows the extension to delay some stuff until a project really uses it.
I had a similar thought, but wasnāt sure about the implications for the loading sequence as you have mentioned as well.
All in all I like this idea more then the git version. First of, it is close to my original intention and see the other advantages you named. In addition to that, it would also follow the concept we use in most other parts and fits nicely with e.g. the templates, external plugins etc.
If we want to add the possibility to have the project specific extension as well, I would suggest the following:
Provide-Capability \
bnd.workspace.extension; \
bnd.workspace.extension=biz.aQute.workspace.awesome; \
where=we/awesome; \
type=workspace;\
version = 1.2.3, \
bnd.workspace.extension; \
bnd.workspace.extension=biz.aQute.workspace.javagen; \
where=we/javagen; \
type=project;\
version = 4.5.6, \
bnd.workspace.extension; \
bnd.workspace.extension=biz.aQute.workspace.bndrun; \
where=we/bndrun; \
type=bndrun;\
version = 4.5.6,,
so, we have can have extensions for each workspaces, projects and bndruns. Iām not quite sure what your idea behind the where was however.
We should only have one type: workspace extension. However, just like the templates, we should be able to have any number of these things in one JAR. So the where attribute is to indicate where the extension lives in the bundle. The workspace will then extract the content on that location to cnf/cache/ext/<name>/.
Iāve put some more thought into this.
I like the idea with the -use. Iād like to avoid however the concept of requiring certain files in the where location. This requires magical knowledge and is limiting at the same time. limiting in the sense that an extension can only provide one set of additional properties properties to a project. Thus I would suggest a declarative approach.
Example:
Provide-Capability \
bnd.workspace.extension; \
bnd.workspace.extension=biz.aQute.workspace.awesome; \
description="Something we can later use in the UI";\
where=we/awesome, \
bnd.use.extension; \
bnd.use.extension=fancyFeatureForProject; \
where=we/javagen; \
description="Will blow your mind",\
bnd.use.extension; \
bnd.use.extension=bndrunBoost; \
where=we/bndrun; \
description="Your bndrun will be set for life!"
With this, an extension can provide multiple use declarations and provide more then one feature.
Iām not sure if we would need a Version in the capability, because a version would be provided by the declaring bundle.
Another thing that crossed my mind, while I finished the generator for our EMF Project would be a parameterized use declaration.
-use:\
geckoEMFCodeGen;output=src-gen;genmodel=model/test.genmodel
The bnd file for the use for this could then look this:
-generate:\
${geckoEMFCodeGen.genmodel}/; \
output=${geckoEMFCodeGen.output}/; \
generate="geckoEMF -g ${geckoEMFCodeGen.genmodel}"; \
Not sure if this takes it too far, but I could be quite helpful. For UI support we could even add a metadata attribute to the use capability similar to the project templates to provide a hint for the UI.