Product plugins ends up as directory instead of as jar files

[mdaloia]

Hi team!

We are seeing an strange issue that I'm not able to replicate it in a test (yet).

We are upgrading our product from Eclipse 4.28 to 4.31 and the plugins folder for our product ends up having a lot of bundles (even the ones from Eclipse itself) as directory instead of a .jar file as used to be.

See for example.... the launcher and others end up being directories although they don't have a Eclipse-BundleShape: dir set in their MANIFEST.MF:

We were using Tycho 4.0.4 but updated it to 4.0.7 and nothing changed.

I saw this commit from p2 eclipse-equinox/p2@89f12f7 and I think it could be related. This change seems to be part of the version 1.6.0 of org.eclipse.equinox.p2.publisher.eclipse (which as per the Tycho 4.0.7 release notes it was bumped to this version).
This fix seems to be related to eclipse-pde/eclipse.pde#730 introduced in Eclipse 4.30 (https://eclipse.dev/eclipse/news/4.30/pde.php#unnecessary-attributes-removal).
I also tried applying the same cleanup as it was done here eclipse-equinox/p2@df5d137 but still I'm not able to get them as .jar files

One important thing (I guess) is to mention that we are using <directorRuntime>standalone</directorRuntime> for tycho-p2-director:materialize-products... this seems to use a pre-bundled artifact (org.eclipse.tycho:tycho-bundles-external) with hardcoded version 2.7.5... this bundle seems to have the version 1.4.2 of org.eclipse.equinox.p2.publisher.eclipse.
I think this could be the source of the issue but not sure.

Do you have any other pointers?

Thanks in advance

[laeubi]

One important thing (I guess) is to mention that we are using standalone

Yes this is somewhat "outdated", so can you explain why this option is used?

[mdaloia]

We have some custom touchpoints, so I guess that it is because of this:

This option is needed if the product to be installed includes artifacts with meta-requirements (e.g. to a non-standard touchpoint action)

Source: https://tycho.eclipseprojects.io/doc/4.0.7/tycho-p2-director-plugin/materialize-products-mojo.html#directorRuntime

[laeubi]

I always wondered how one uses custom touchpoints can you provide an integration-test to demonstrate the usage?

With recent changes in Tycho I think it could be done but its a quite esoteric use-case so I can't promise to handle this with highest priority, if it is crucial to your business you can find dedicated support options for Tycho here:
https://github.com/eclipse-tycho/tycho#getting-support

[mdaloia]

That is what I was trying to do... but, do you think it could be related to having a touchpoint instead of the standalone director?

[laeubi]

@mdaloia I'm not completely sure what the question is, but of course you need to use <directorRuntime>standalone</directorRuntime> for your testcase as well.

[mdaloia]

UPDATE:

This seems to have been related with the fact that we were using Tycho 2.7.5 with the org.jboss.tools.tycho-plugins:target-platform-utils:2.7.5 plugin to mirror the repository and somehow the metadata produced (we didn't compared it) was not compatible with new versions of p2 director.

Now we are using the tycho-p2-extras:mirror goal specifying the list of <iu> to mirror directly in the pom.xml file [1].

The disadvantages of doing it this way are:

• No way to control from which p2 Update Site an IU is mirrored (because a list of repositories is given separated from the list of IUs, different to a .target file).
• When you update the US URL (for example of the Eclipse Simultaneous Release) there is no automated way (at least that we know) to update the version of the same IU, so you need to update it manually or create an app/script using p2 libs to automate it.
• If you depend from SNAPSHOT builds that changes daily, the specification of the IU version must be somewhat dynamic, so we ended up using a <query> like this to avoid updating the version with the version available at the time of mirroring the IUs from that US:

   <iu>
   	<query>
   		<expression>id == $0 &amp;&amp; version >= $1</expression>
   		<parameters>org.eclipse.rcptt.tesla.feature.group,2.5.5.20240501</parameters>
   	</query>
   </iu>

[1]: Even though that the mirror goal apparently supports using a .target file (see <targetPlatformAsSource>), via a target-platform-configuration, it seems to not be working in 4.0.7. See this.

[mdaloia]

See also this other alternative that I found later: #1554 (comment)