Goal publish-features-and-bundles inherits metadata available in reactor

[mdaloia]

The goal publish-features-and-bundles reuses the metadata available in the reactor when source and destination repository are the same location and append is used, even if you, before calling this goal, delete the artifacts.* and content.* files in repository. If you do a modification that changes some metadata it keeps the old value.

This seems to be working as the Eclipse standalone application for 4.31 using the append with the same destination as the source. The difference is that it keeps old metadata only if you DON'T delete the artifacts.* and content.* files before calling it.

Previously, in Tycho 2.7.5, this goal recreated the metadata only with the size and checksums properties.

In Tycho 4.0.7, it seems that the metadata information present somewhere in the reactor context is being "leaked" to the new metadata.

Example:

1. Mirror some IUs from a repository
2. (Re)sign (or modify) some IUs .jar
3. Delete artifacts.* and content.* files
4. Enforce file don't exists
5. Execute publish-features-and-bundles goal

If you run those steps as part of the same Maven execution the issue surfaces.
If 1,2,3 and 4 are run in one Maven execution and 5 in another (could be implemented via Maven profiles) then we get an output like 2.7.5.

Example pom.xml

Changing just the tycho.version property to 2.7.5 it works.

<?xml version="1.0" encoding="UTF-8" ?>

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
	<modelVersion>4.0.0</modelVersion>

	<groupId>com.martindaloia.reports.tycho-issue-publish-features-and-bundles</groupId>
	<artifactId>parent</artifactId>
	<version>1.0.0-SNAPSHOT</version>
	<packaging>pom</packaging>

	<name>Tycho Issue publish-features-and-bundles</name>

	<properties>
		<tycho.version>4.0.7</tycho.version>

		<repo.source>${project.build.directory}/repository</repo.source>
		<repo.destination>${project.build.directory}/repository</repo.destination>
		<test.file>${repo.source}/plugins/com.google.guava.failureaccess_1.0.2.jar</test.file>

		<!-- Tycho: Disable usage of p2 mirror repositories. Improves download speed. -->
		<eclipse.p2.mirrors>true</eclipse.p2.mirrors>
		<tycho.disableP2Mirrors>true</tycho.disableP2Mirrors>
	</properties>

	<build>
		<plugins>
			<plugin>
				<groupId>org.eclipse.tycho</groupId>
				<artifactId>tycho-maven-plugin</artifactId>
				<version>${tycho.version}</version>
				<extensions>true</extensions>
			</plugin>

			<plugin>
				<groupId>org.eclipse.tycho.extras</groupId>
				<artifactId>tycho-p2-extras-plugin</artifactId>
				<version>${tycho.version}</version>
				<executions>
					<execution>
						<id>mirror-plugin</id>
						<phase>compile</phase>
						<goals>
							<goal>mirror</goal>
						</goals>
						<configuration>
							<source>
								<repository>
									<url>https://download.eclipse.org/releases/2024-03/202403131000/</url>
									<layout>p2</layout>
								</repository>
							</source>
							<destination>${repo.source}</destination>
							<ius>
								<iu>
									<id>com.google.guava.failureaccess</id>
									<version>1.0.2</version>
								</iu>
							</ius>
						</configuration>
					</execution>
					<execution>
						<id>create-p2-metadata</id>
						<phase>prepare-package</phase>
						<goals>
							<goal>publish-features-and-bundles</goal>
						</goals>
						<configuration>
							<sourceLocation>${repo.source}</sourceLocation>
							<artifactRepositoryLocation>${repo.destination}</artifactRepositoryLocation>
							<metadataRepositoryLocation>${repo.destination}</metadataRepositoryLocation>
							<append>true</append>
							<compress>false</compress>
						</configuration>
					</execution>
				</executions>
			</plugin>

			<plugin>
				<groupId>org.apache.maven.plugins</groupId>
				<artifactId>maven-antrun-plugin</artifactId>
				<version>3.1.0</version>
				<executions>
					<execution>
						<id>touch-jar</id>
						<phase>generate-test-resources</phase>
						<goals>
							<goal>run</goal>
						</goals>
						<configuration>
							<target>
								<checksum algorithm="SHA-256" file="${test.file}" property="originalSHA256"/>
								<echo message="File original SHA-256 = ${originalSHA256}"/>

								<unzip dest="${project.build.directory}/tmp" src="${test.file}"/>
								<echo append="false" file="${project.build.directory}/tmp/META-INF/test.txt">simulate change</echo>
								<zip basedir="${project.build.directory}/tmp" destfile="${test.file}"/>

								<checksum algorithm="SHA-256" file="${test.file}" property="newSHA256"/>
								<echo message="File new SHA-256 = ${newSHA256}"/>
							</target>
						</configuration>
					</execution>
					<execution>
						<id>remove-incomplete-metadata</id>
						<phase>test</phase>
						<goals>
							<goal>run</goal>
						</goals>
						<configuration>
							<target>
								<delete>
									<fileset dir="${repo.source}" includes="artifacts.*"/>
								</delete>
								<delete>
									<fileset dir="${repo.source}" includes="content.*"/>
								</delete>
							</target>
						</configuration>
					</execution>
				</executions>
			</plugin>

			<plugin>
				<groupId>org.apache.maven.plugins</groupId>
				<artifactId>maven-enforcer-plugin</artifactId>
				<version>3.4.1</version>
				<executions>
					<execution>
						<id>enforce-dont-exist</id>
						<phase>test</phase>
						<goals>
							<goal>enforce</goal>
						</goals>
						<configuration>
							<rules>
								<requireFilesDontExist>
									<files>
										<file>${repo.destination}/artifacts.jar</file>
										<file>${repo.destination}/artifacts.xml</file>
										<file>${repo.destination}/artifacts.xml.xz</file>

										<file>${repo.destination}/content.jar</file>
										<file>${repo.destination}/content.xml</file>
										<file>${repo.destination}/content.xml.xz</file>
									</files>
								</requireFilesDontExist>
							</rules>
							<fail>true</fail>
						</configuration>
					</execution>
				</executions>
			</plugin>
		</plugins>
	</build>
</project>

Example run

Run the build

Compare the SHA-256

[laeubi]

Can you provide an integration-test to demonstrate the issue from your example with proper assertions? Then it is easy to verify it worked (differently) in 2.7.x and we can make sure if there is a fix that it does not break in the future again.

[laeubi]

By the way is there a reason you are not using fix-artifacts-metadata instead of this quite complicated steps?

[mdaloia]

Can you provide an integration-test to demonstrate the issue from your example with proper assertions?

Sure, I will try to add an IT later.

By the way is there a reason you are not using fix-artifacts-metadata instead of this quite complicated steps?

We tried to use that goal, but we are impacted by this other issue #2875 because we are using an standalone director which uses and old p2 director and it compares the MD5 and as we need to re-sign some .jar files it fails.
Why we are using the standalone instead of the internal? because we have custom touchpoints and using the internal director fails when a product is being materialized as it seems it doesn't found it.