The End of an Era

GENERAL
1

Last Friday we had a bnd calls where BJ Hargrave indicated that he would have no more time to work on bnd professionally.

BJ and I have been working on OSGi since 1998 and now 25 years later it does feel that for the first time (well for me) we are now reaching uncharted territory. I think nobody that had any view of what was happening in bnd could have any doubts how crucial BJ was for the quality of the code-base and the smooth operation of the project.

Personally it will be a bit of a shock to not have him around to rely on. However, I think most readers also realize how important his function was and how much we owe him.

However, that leaves the question how to continue. There are still numerous products out there that are based on OSGi and bnd tends to play an important role in getting the necessary headers in open source bundles. Like many software projects, it will require that someone provides the loving care that BJ did for so many years. For example, new VMs now are produced at a staggering rate and each VM will require some attention. Many libraries change and sometimes this requires some adaption. We have numerous PR’s that need to be reviewed and accepted. Also the issues reported need to be discussed and a decision made on. If you, or better your company, is willing to provide some support, then let me know.

Anyway, lets not hijack this topic with the practical issues. From the bottom of my heart, BJ, thank you for the immense amount of work you did!

3 Likes
2

Thanks for the kind words. I wish OSGi and bnd the best and, while I wont be working on bnd on a day-to-day basis anymore, I look forward to bnd continuing to be important and relevant.

4 Likes
3

Hear, hear! It’s been a pleasure to be able to work with BJ and I would like to thank him for all the professional support that he has given to the project and to me personally. May God bless you in your new endeavours!

1 Like
4

Thanks BJ for the good work. I was always amazed by your foresighted and clear explanations on various topics. I was always a bit like: wow , how does he know that x or z can have this or that effect / disadvantage / drawback / impact. I wish you the best.

1 Like
5

I concur with all the sentiments here! Thanks BJ for all the hard work you’ve put in to get BND this far.

Take care and best wishes.

6

@bjhargrave Thanks for the great work and the kind support!

@pkriens When reading your post, I stumbled over the “still” here and wondered what it implies. Do you consider OSGi an outgoing technology? If so, what would you consider being the replacement (if any)? Java 9 modules?

7

Do you consider OSGi an outgoing technology? If so, what would you consider being the replacement (if any)? Java 9 modules?

That is a difficult question :slight_smile:

It is true that I am disappointed where OSGi is today. I think the architecture with services, modules, the resolver, and baselining is software engineering reified. I see how incredibly effective it can be when companies apply a few simple rules:

  • Strictly separate API from implementation
  • Make the modules cohesive
  • Never not lie in your metadata

When they’re my customer, I can use brute force to let hem adhere to these rules and it works then amazingly well. However, without a purist like me the metadata deteriates and OSGi becomes a terrible bitch.

I am very proud of what we achieved but its hard to deny it lacks wide spread adoption.

OSGi is far superior to JPMS, it only has the pesky problem that Sun was against it. The idea that you can make a module system without versions and that requires a single namespace is in my book close to malpractice. A private class name in a module should never be able to conflict with a private class name in another module but amazingly JPMS achieved this feat. (And if you have a single namespace, well, things are quite easy and do not require the impressive complexity of JPMS.) The fact that they have no clue is clear with the multi release jars. If you have a module path with a multi release jar, it might work on one VM and fail on the next version since the requirements can change depending on the VM version you run it on but their modulepath cannot express this. This is a build problem in OSGi but in JPMS it will be discovered in runtime. And don’t get me started on the comparison between OSGi services and ServiceLoader :slight_smile:

Then again, probably the most important reason I am depressed about the future is that Java itself seems to have little future left. Since Java is not the primary language in universities anymore, I see all the excitement going to Javascript and Python. Probably the languages that ignore software engineering the most.

But the biggest bomb is of course AI that is a game changer. We probably still need some software engineering in the very high performance backends that provide the APIs but virtually all software is going to be throw away software, generated on the fly. A type of software that does not need software engineering aficionados like me. And in that area Java does not seem to fare well. Already the quality of the generated code in Python and Javascript is much better than in Java. (Try concurrency.)

So I am gloomy. It is extremely frustrating to see how well the OSGi technology works when you apply the simple rules but then see how its values are not, or no longer, appreciated. I’d pick the OSGi as basis for a serious product that needs long term maintenance every day though.

2 Likes
8

Do you consider OSGi an outgoing technology?

Let’s face it, yes. OSGi is a great technology with a traditionally very bad “marketing” (here is how to do it differently) and a community that has never welcomed newbies (I know what I’m talking about, I have experienced it myself).

The feeling of using a great technology comes after a tremendous learning effort (not really supported by the outdated-before-finished “official” tutorials that you can only understand after having understood most things anyway). And, I’m afraid, the advantages are only apparent for people building and maintaining large (enterprise scale) projects (although my experience is that mid-scale or even small projects can benefit from the patterns that OSGi forces you to use).

The major problem for OSGi’s future will be the diminishing support in libraries. Not only great new libraries that appear on github and that you’d really like to use but have to “adapt” first (because the maintainer(s) doesn’t know anything about OSGi). It’s the Java “foundation” that turns its back to OSGi on the long term.

I found last year that the latest versions of JAF and Jakarta Mail have problems with OSGi. I could achieve a partial success (here and here) with respect to “proper” OSGi support, but the remaining issues are stalled and I don’t have the feeling that somebody cares.

Without the possibility to use something as fundamental as e.g. the mail api in your OSGi project without workarounds, it will be hard to convince anybody about OSGi’s advantages.

9

To my, and the others, defense, I also think the learning curve is partly so steep because some of the concepts:

  • Innovative, concepts like the require capability model tends to glaze people’s eyes over.
  • Problem it solves so well is often not well understand
  • Payoff is long term
  • Solving second and third order problems
  • All or nothing, not a magic sauce

Further, I am convinced that many/most developers are lazy and prefer a quick hack or magic sauce over true thorough software engineering.

Last, there was a lot of push back from the Sun people at the time and am not sure all that push back was based on rational arguments. A lot imho had to do with the relation they had with IBM.

Anyway, we’ve got ChatGPT now! In my plugin envy (still on the waiting list) I am building my own client! And yes, it is using Java & OSGi, and most of all the amazing Apache gogo & web console.

10

I agree. I first heard and learned about OSGI in 2009 by Gunnar and later got convinced about bndtools by @juergen-albert. Not sure I got that far without external help here and there. So kudos to them :slight_smile:
But we definitely do not regret having used OSGI so far. But of course the content out there by you guys.

We a scratching the surface of OSGI and bndtools, but for us, the biggest advantages are:

  • separate classloaders allowing to use different versions of the same lib
    • this is needed since we have a plugin system, and OSGI bundles map pretty great to that what we call “plugin”. and since our plugins can do various different things it is very likely that we need different kinds of sometimes conflicting dependecies. I wouldn’t know how we should achieve the same without it
  • easy way of defining dependencies and versions in our central.maven file and .bndrun
  • having multiple .bndrun files for different scenarios (we have a main.bndrun, main-gc-stats.bndrun, main-jitdebug.bndrun. testcases.bndrun, testswithcodecoverage.bndrun, selenium-uitest-chrome.bndrun, selenium-uitest-firefox.bndrun all inheriting from the main file but use different parameters. )
  • having easy setup in Eclipse and also a equally functioning gradle build
  • the “deploy bundles at runtime”, although we still do not make use of that

Kind of true. I don’t know how to do it better.

Maybe when look next door to the JS guys, one thing I noticed is that those frameworks often provide “Starter Themes”.

Those are viewable even in those new fency sandbox environments like this Astro-js - Astro Theme Starter - StackBlitz

Maybe this is one idea to increase adoption, since you see something working almost immediately infront of your eyes and fingertips. I admit the comparison lags a bit: the stuff OSGi solves is more invisible than the stuff you can build with those JS frameworks above. But they are also somewhat comparable because those JS frameworks also promote the notion of “Components” a lot. When I had discussions with webdevs about OSGi, the comparison to “VueJS components” helped me to explain stuff.

But I also know that this needs someone to do it. Unfortunately time is limited for everybody.

Anyway, I hope there will be more enthusiasts like the everybody here, driving bnd / bndtools forward :slight_smile:

1 Like
11

To my, and the others, defense, I also think the learning curve is partly so steep because some of the concepts:
* …

I don’t know why I’m still trying to convince people to do something about the documentation. Maybe because after years of struggle I found out how useful OSGI concepts are and I still would like to see OSGi succeed in some way, though I know I won’t. (And I still feel that I know too little to write something beyond some “tutorial” myself, I keep being surprised about new information in discussions too often.)

OSGi dates back to 1999 (at least) and sometimes I have the impression that so do its protagonists. Look at all the nice and shiny things that you get today with all the documentation for free (because they have understood that this is the key to success).

I just come from learning kubernetes. It took me barely three month to write my first operator (and only maybe half of that time was spent with kubernetes, the other half was about concepts of the domain that it is operating). And you know what? No consultancy required. It’s all in the docs. Of course, sometimes goolgle led me to some blog or to stackoverflow. But in all cases I could find the information in the original documentation as well (and the blogs didn’t really add anything to it, they are mostly self-marketing).

Now compare this experience with this great official documentation of the bndtools resolution view. Makes you immediately want to work with this tool, right? And there’s not even somebody “blogging” about it, try this. (Not to speak of the many bnd tricks that have never been properly documented.) And please don’t tell me that everything is in the specifications. First, this does not apply to tooling and second, we all know that these specifications are not written with somebody wanting to learn about a technology in mind. The average internet RFC in its “informal parts” provides more information about the background of and the motivation for what it specifies than most of the OSGi specs do. An example of how to do documentation and specification together is Java. Did you ever need more than the JavaDoc to use its standard library? Okay, sometimes the tutorials are helpful to get the general idea of some concept. (Yes, I did need the tutorial to grasp generics.)

I understand that OSGi is a “company thing” and not a “community thing” and that every expert involved wants to make money with his knowledge. However, considering this and complaining about OSGi not being the success that it should have been is nothing but shedding crocodile tears.

12

I think you have a rather wrong impression what OSGi was.

It started in 1999 by large companies and there was lots of money with 81 members paying $20k each. However, the 2001 dot com crisis decimated the alliance; the industry change to open source crippled any of the remaining business cases. I.e. we were created by big business but got caught into the open source maelstream. Leaving us in limbo. If it had not been for some people inside IBM, we’d have died around 2005. And these people had a lot of internal resistance. At the same time, During all this time, Sun worked hard to get rid of us.

Around 2014 the I tried, by being paid by the OSGi, to make OSGi attractive to open source but this never worked out due to many reasons but mostly because I failed to get people engaged. On the inside some people did not like the direction and on the outside people had the impression it was too hard. People want a simple quick fix, not a solid foundation that requires effort.

Comparing it to a library of a language that resembles most other languages does not seem a fair comparison. OSGi has many concepts that do not exist with the same strength anywhere else: the resource model, the service model, the versioning with baselining, extender pattern, etc. This is of a very different dimension than figuring out how to open a database or get the length of a string. I find that almost no developer uses reflection while it is for me one of the most important aspects of Java. which is partly because the concept does not really exist in other languages. Just like OSGi, the Javadoc is there but without a deep understanding of the concepts and its utility you’re going to be lost.

And I do think that the information is in the specs, but the general feedback is that 1500 pages is too much. In my bookshelf I have 11 OSGi books not counting the spec and there were more. I’d expect that none of the authors made more than $1 per hour of their impressive efforts. As you’ve found out, OSGi requires a certain mode of thinking that does not seem to come natural for a lot of people.

About bnd. It was started by me in 1999 to solve manifest chores and later Neil Bartlett used it in bndtools around 2005. There never has been a company or significant community behind those tools. Neil and I mostly did it by consulting for companies and spending copious amounts of our own time. Over time we got more people and BJ Hargrave was paramount because he actually had IBM time, which greatly increased the quality.

I think our bad luck was that we started as a corporate based effort, got crippled by the economy, and then were left behind by the open source world. Yes, it feels like a failure. And yes, I’d be willing to put effort in it if I thought I could convince a significant number of people to use it. Yes, I still need some income though.

I know I am biased, but I still think it is the greatest software architecture around though. I am no longer shedding (crocodile) tears but changed to thinking it is their loss. And my customers agree.

I appreciate you take the time to provide feedback though.

13

Hello OSGi buddies,

I’m working with Eclipse since version 2.1 and called myself an expert after several years.
After participating a OSGi master class with Neil in 2012 I degraded myself to OSGi beginner.
Working with OSGi and bndtools was always hard but very satisfying when understood and applied correctly.
I had the luck to convince my customer to switch from PDE to bndtools. So I was working for more than 10 years with OSGi and bndtools and still continue in my non-paid time :wink:

My general problem in promoting OSGi was exactly the OSS vs company driven interest topic, which I faced when I trying to contribute or make it more consumeable. It felt like some OSGi parties are more interested in hiding, than sharing knowledge.

On the other side convincing developers of OSGi is most of the times easy. The developer has an actual problem and OSGi is most of the time offering a solution for that (baselining, api/impl separation, component architecture, …)
So the point I would like to make is, that OSGi is solving many, many problems which appear during long term support projects. Most of the time my impression is, that I try to solve issues, which are not yet faced or noticed inside the projects and their members.

The old enRoute was/is a wonderful example and showcase, what could be achieved.
Having such a catalog of solutions as wizards/template inside bnd tooling would make it consumable for beginner developers - which are always the largest crowd inside projects, due to our unlimited industry growth.

“resistance is futile” - I don’t think so!

Even developer beginners can be attracted, if we create a solutions catalog describing the concepts e.g. api design, baselining, javas development, debugging and runtime capabilities in parallel to concrete code examples like in “old” enRoute.

Maybe the new roof of OSGi “eclipse foundation” is a chance for changing this :partying_face: @juergen-albert

I know that all of us (including me) must earn money, but our technology OSGi is worth to be continued!

2 Likes
14

OSGi is not just solving problems that only show for larger projects that stay around a long time, it is also about collaborative software.

In the nineties I read a book called Super Distribution from Brad Cox. Brad had this idea of developing software like hardware with IC like components. Reusability had been the grail for some time but his idea of well defined interfaces (TTL in hardware) made sense to me as hardware engineer. However, sharing, and thus abstracting, is hard. The saddest moment in my (professional) life was when I single stepped through a JPA implementation and saw that every transaction manager, for which an excellent API existed, was checked for and treated differently. As community, we seem to never let a hack go unused if it saves us a few hours.

At the same time, the OSGi swam against the stream. The bundle was an excellent delivery model for code and resources, the service model is an unsurpassed model for collaborative and adaptive software, the requirement-capability model provided a far superior dependency model than the primitive module-to-module model so popular in the rest of the world, and last but not least, the baselining that ensures your API are backward compatible. Clearly I am biased but even after all this time I still love it.

I’ve noticed with all my customers that OSGi just works extremely well when you know how to use it. One developer remarked that he had been using OSGi for another employer and thought it sucked. However, now he saw how it was used in his new company, he’d fallen in love with it.

Peter, I think you put the finger on where the pain is. Although we could have done lots of things better, the key thing is that most people just don’t get what we’re trying to do and are too impatient to find out. With that perspective, OSGi is overly complex and a pain. Maybe better documentation or focusing on maven would’ve helped but I doubt it. The spec is excellent but I frequently hear it is too big.

I still think the original goal of enRoute was right, make Java/OSGi attractive for the Ruby on Rails crowd. However, on one side we had the enterprise crowd that got stuck with some awful APIs and were unwilling to let that go, on the RoR side we never got any interest.

Interestingly, I showed Christer Rueger the old enRoute site, he had never seen it and it does not show up on Google. There is lots of cool stuff there, maybe we can integrate this in the bndtools.org site.

Anyway, thanks for the post. Makes my old heart warm to see this energy :slight_smile:

15

Yes indeed, I was surprised to see this.
Currently the reason it is not indexed in google is this.

image
image988Ă—144 31.3 KB

But anyway it would be good to move stuff to bndtools.org
We probably should move the discussion about it to another topic.

1 Like