Wondering what magic bndtools uses to get get it’s reolve of bndrun files so fast. In bndtools the resolve takes 15 seconds max on complex resolves, On the command line 2+ minutes. Is there some caching or flags I am missing?
Hard to tell from the outside. Could you share which bnd resolve CLI command you use?
The docs mentions “bndrun files” (plural) at the top… so maybe CLI is processing multiple?
Are Eclipse bndtools version and bnd CLI version the same? (what’s the output of bnd version on CLI?)
I am just using a little program. Versions are the same
var run = Bndrun.createBndrun(null,runFile);
run.getModel().setRunBundles(new java.util.ArrayList())
String res = null;
try
{
res = run.resolve(true,false);
}
10-15 seconds in eclipse
2 minutes standalone
Root Cause Analysis: Eclipse (10-15s) vs. Standalone (2 minutes)
The performance gap is not caused by a bug or algorithmic difference in the resolver itself. It comes from two compounding differences in how repositories are initialized and how their indexes are loaded between the two environments.
1. Repository Index Already Loaded In-Memory (Eclipse)
In Eclipse/Bndtools, the Workspace is a long-lived singleton that is created once when the IDE starts. All configured OSGiRepository (or other Repository) plugins are already initialized and their indexes are already parsed and cached in-memory as live Java objects.
When you call run.resolve(...) inside Eclipse, BndrunResolveContext.init() calls loadRepositories() → getAllRepos(), which retrieves already-initialized, in-memory repository objects from the shared workspace. The index data (the full OSGi resource/capability/requirement graph) is already in RAM and the BridgeRepository Promise is already resolved.
In standalone mode (Bndrun.createBndrun(null, runFile)), a brand-new Workspace is created via Workspace.createStandaloneWorkspace(...) and ws.open(). Every OSGiRepository plugin starts with inited = false. The very first call to findProviders() (during resolver.resolve(rc)) triggers OSGiRepository.prepare(), which:
- Creates a new
OSGiIndex - Calls
readIndexes(false), which downloads all index XMLs viaHttpClient - Parses potentially large OBR/R5 index XML files
- Builds the full
ResourcesRepositoryandBridgeRepository
All this happens synchronously at resolve time (the Promise is awaited via .getValue() inside getBridge()), taking most of the 2 minutes.
2. HTTP-Level Index Cache Is Not Warm (Standalone)
HttpClient defaults its URL cache to Home.getUserHomeBnd("urlcache") — a shared disk cache. However:
- The
OSGiIndex.download()call usesclient.build().useCache(staleTime)wherestaleTimedefaults toYEAR(365 days). So once the HTTP cache is warm, subsequent standalone runs should be fast too. - The first run (or any run after the cache expires or is cleared) must fetch index files from the network. For a remote OBR repository with a large XML index, this can easily add tens of seconds to minutes depending on the size and network latency.
- In Eclipse, the same
HttpClientinstance has been running for hours/days. Its URL cache is always warm for the index files, so it serves them from the local disk cache with an HTTP 304/UNMODIFIED response.
3. Plugin/Workspace Initialization Overhead (Standalone)
Workspace.createStandaloneWorkspace() + ws.open() initializes the full plugin infrastructure from scratch:
- Loads and instantiates all
-pluginentries from properties - Initializes
HttpClient, reads connection settings - Creates
ResourceRepositoryImpl,CachedFileRepo, etc. - If
-standalonepoints to remote repository URIs,OSGiRepository.prepare()is called for each one duringgetPlugins(Repository.class)(triggered by thePrepareinterface andinitRepositories())
In Eclipse, the workspace is initialized once at IDE startup, amortized over all operations done during the session.
Summary Table
| Factor | Eclipse | Standalone |
|---|---|---|
| Workspace already initialized |  Yes (live since IDE start) |
 No (created fresh per call) |
| Repositories initialized in memory | Yes |
No (initialized on demand at resolve time) |
| Repository index XML parsed | Already done |
Done during resolve() call |
| HTTP index cache state | Warm (Eclipse ran earlier) |
Cold (first run), or warm after first run |
| OSGi index in-memory | BridgeRepository Promise already resolved |
Promise resolved during resolve, blocking |
Mitigation
If you need the standalone call to be consistently fast, you can:
- Reuse the
Workspaceacross multipleBndrun.resolve()calls rather than recreating it each time — this amortizes the initialization cost. - Ensure the HTTP URL cache is warm before resolving (the
~/.bnd/urlcache/directory). After the first standalone run the index files will be cached on disk and subsequent runs will be much faster. - Set
-offline: trueor-gestalt: offlinein the bndrun file to prevent re-checking remote indexes (relies entirely on disk cache). - Use the
cache:config attribute in-standaloneor-plugin.repoto point theOSGiRepositorycache to a fixed local directory that persists across JVM invocations. - Consider setting
max_staleexplicitly to a large value so the cached index is used without re-validation for longer.
Very nice explanation, thank you @peterkir !
It sound like it would be nice to have a kind of bnd-server optional start mode for bnd CLI, allowing us to save time for those (probably rare) people who do not use bndtools (Eclipse) and only rely on a IntelliJ based IDE (like Community or OpenIDE ) and bnd-CLI for work with OSGi-bnd projects.
By the way, I prefer to avoid Eclipse, in favor of IntelliJ productivity, and I think it could be quite possible if bnd-CLI add this kind of improvements.
I remember reading some notes written by @pkriens remarking important design considerations taken into account when designing bnd-CLI interacting with any IDE.
It would be very nice to review such considerations (probably requirements) stated just from the beginning of the design of bnd-CLI and, at the same time take in considerations LSP and also what has been learning in Eclipse Bndtools in order to take such benefits extrapolated to Emacs and IntelliJ based IDEs or just bnd-CLI.
By the way, I am willing to work in LSP for .bnd files. Mi initial idea is to develop the LSP using OCaml programming language, but weather it should be write using Java-based framework in order to get relevant people invest time, improving it, for me it is not a problem. But if I am “solo”, I prefer to make the bnd-LSP in OCaml.
I few options come to mind which you could try:
-
shell [options] | bnd (“The shell also supports bnd commands…”): I haven’t used it myself but I think it was mentioned to me in the past somewhere for similar things
-
bnd CLI commands:
bnd devandbnd build -watch- Experimental: new Commands:
bnd dev&bnd build -watchfor continuous build run workflow (see dev | bnd). This allows IDE-independent live coding workflow known from the javascript / npm world e.g. “vue-cli serve” (see) which starts a dev-webserver and allows for hot code reloading when changes are detected. It is an attempt to make the Eclipse experience of “Build automatically” available, but without Eclipse - so theoretically you can use any text editor VSCode, vim / neovim etc. ([CLI] new Command: bnd dev & bnd build -watch for continuous build run workflow by chrisrueger · Pull Request #6768 · bndtools/bnd · GitHub)
- Experimental: new Commands:
Those things above start a long running process which could fit your use case of doing expensive initialization work once.