Caching mvn source jars/multiple downloads on Eclipse start

Gordon.Hutchison · May 25, 2022, 12:48pm

When I restart Eclipse, the bndtools plugin downloads the src jar for bnd referenced mvn libraries multiple times (guessing once every time it is referenced). I have a big project (GitHub - OpenLiberty/open-liberty: Open Liberty is a highly composable, fast to start, dynamic application server runtime environment) and this means eclipse an Eclipse (re)start (for example when installing a new bit of software or when my laptop has been off or every now and then when it’s getting a bit grumpy) takes a loooooong time.

Is there any way to cache these or reduce it to one download per referenced library rather than 1 per reference? Thanks for the great tooling which keeps getting better.
Gordon

bjhargrave · May 25, 2022, 7:49pm

@kriegfrj Do you know where in Bndtools sources jars are downloaded? I don’t think MavenBndRepository is the requestor.

kriegfrj · May 26, 2022, 12:55am

The code directly responsible for downloading the sources is in MvnBndRepository:

github.com

bndtools/bnd/blob/c621f12861b9d3398b8fba013dfba16d0ee2532d/biz.aQute.repository/src/aQute/bnd/repository/maven/provider/MavenBndRepository.java#L501-L504

      
        
            		Archive archive = index.find(bsn, version);
            		if (archive == null) {
            			return trySources(bsn, version, listeners);
            		}

github.com

bndtools/bnd/blob/c621f12861b9d3398b8fba013dfba16d0ee2532d/biz.aQute.repository/src/aQute/bnd/repository/maven/provider/MavenBndRepository.java#L935-L957

      
        
            	private File trySources(String sourceBsn, Version version, DownloadListener... listeners) throws Exception {
            		if (!sourceBsn.endsWith(BSN_SOURCE_SUFFIX))
            			return null;
            
            
		String originalBsn = sourceBsn.substring(0, sourceBsn.length() - BSN_SOURCE_SUFFIX.length());
            
            
		Archive primaryArchive = index.find(originalBsn, version);
            		if (primaryArchive == null)
            			return null; // don't get sources when not in index
            
            
		Archive sourcesArchive = primaryArchive.getOther(Archive.JAR_EXTENSION, Archive.SOURCES_CLASSIFIER);
            		if (sourcesArchive == null)
            			return null;
            
            
		Promise<File> promise = storage.get(sourcesArchive);
            		if (listeners.length != 0) {
            			Map<String, String> attrs = sourcesArchive.attributes();
            			new DownloadListenerPromise(reporter, "Get sources " + sourceBsn + "-" + version + " for " + getName(),
            				promise, attrs, listeners);
            			return storage.toLocalFile(sourcesArchive);

This file has been truncated. show original

However, that doesn’t explain why this code is being triggered for @Gordon.Hutchison. I was under the impression that sources are never downloaded automatically during builds (hence the reason why . It is only invoked on-demand when you try to do a source lookup. So I’m not sure what’s triggering the source downloads for @Gordon.Hutchison .

A bnd launch will trigger a download of sources during source lookup:

github.com

bndtools/bnd/blob/c621f12861b9d3398b8fba013dfba16d0ee2532d/org.bndtools.launch/src/bndtools/launch/sourcelookup/containers/BundleSourceContainer.java#L75-L114

      
        
            	public void init(ISourceLookupDirector director) {
            		super.init(director);
            		// Look for possible source bundle.
            		final String sourceBSN = bundle.getBundleSymbolicName() + BSN_SOURCE_SUFFIX;
            		final Version version = new Version(bundle.getVersion());
            		for (RepositoryPlugin repo : RepositoryUtils.listRepositories(true)) {
            			if (repo == null || repo instanceof WorkspaceRepository) {
            				continue;
            			}
            			try {
            				File sourceBundle = repo.get(sourceBSN, version, Collections.emptyMap(), new DownloadListener() {
            					@Override
            					public void success(File file) {
            						delegate.compareAndSet(null, new ExternalArchiveSourceContainer(file.toString(), false));
            						flag.countDown();
            					}
            
            
					@Override
            					public void failure(File file, String reason) {
            						logger.logError("Error trying to download " + file + ": " + reason, null);

This file has been truncated. show original

Also, the MavenBndRepository should only try and download the source if it’s not already present in the local Maven cache.

So we probably need a little bit more context from @Gordon.Hutchison to try and debug this one. @Gordon.Hutchison, if you could do a small video capture of the behaviour when it is happening and post it here, that would help a lot. Also please confirm the Bndtools version that you are using. Thanks!

Topic		Replies	Views
How to generate a source bundle with the BND Maven plugin GENERAL	8	335	December 22, 2022
Maven dependencies not resolved GENERAL	17	36	September 22, 2024
Bndtools Release 7.0.0-RC1 GENERAL	26	496	November 17, 2023
Bnd/Bndtools 6.0.0 release candidate build available GENERAL	6	397	September 28, 2023
Getting started with bnd/bndtools and Eclipse as developer GENERAL	3	204	October 3, 2023

Caching mvn source jars/multiple downloads on Eclipse start

Related Topics