Verify error when license exceptions are used in an expression in LicenseConcluded but not in LicenseInfoInFile

[alpianon]

I ran into this issue while trying to convert an SPDX file generated with the REUSE tool. I'm not sure if this an issue of SPDX tools or of REUSE tool (I will explain why below), so I'm opening the same issue in both projects 🙂

link to the issue in REUSE tool's repo: fsfe/reuse-tool#890

In a nutshell, it seems that SPDX tools refuse to validate (and to convert) an SPDX file when both:

1. a license exception tag is stored as a separate LicenseInfoInFile field
2. the same tag is found (as a "WITH" exception) in the LicenseConcluded field

but it validates it when only (1) happens (and LicenseConcluded is NOASSERTION), or when only (2) happens and the license exception is stored in a license expression ("some-license WITH the-exception") in LicenseInfoInFile

I ran into this issue while I was trying to convert an SPDX file generated by REUSE tool from the qtbase source package, and I ran into an error due to (1)+(2), but I don't know whether SPDX tools is too picky here, or REUSE tool is not following an (implied) requirement of SPDX specs. In other words, I don't know if the expected behavior is that SPDX tools validates such files generated by REUSE tool, or if the expected behavior is that REUSE tool produces SPDX files without doing (1).

Steps to reproduce:

• install REUSE tool and SPDX tools (java)
• download https://download.qt.io/archive/qt/6.5/6.5.3/submodules/qtbase-everywhere-src-6.5.3.tar.xz and extract it
• within the dir where you extracted source directory, do:

cd qtbase-everywhere-src-6.5.3/
reuse spdx --add-license-concluded --creator-person "John Doe" --creator-organization ACME \ 
  > ../qtbase-everywhere-src-6.5.3.spdx.tag
cd ..
java -jar <path-to>/tools-java-1.1.8-jar-with-dependencies.jar Verify qtbase-everywhere-src-6.5.3.spdx.tag

you get the following error:

Analysis exception processing SPDX file: Unexpected Error: org.spdx.library.model.SpdxIdInUseException: 
Can not create Qt-GPL-exception-1.0.  It is already in use with type ListedLicenseException which is 
incompatible with type ExtractedLicensingInfo

The problem seems to be the SPDX file generated by REUSE tool, where you find file entries like:

FileName: ./cmake/qbatchedtestrunner.in.cpp
SPDXID: SPDXRef-7b211d4592fde6cb29719920ef203c52
FileChecksum: SHA1: 91d0b31b6e60054b87c9fa5d36b9b86f57ef1a7a
LicenseConcluded: GPL-3.0-only WITH Qt-GPL-exception-1.0 OR LicenseRef-Qt-Commercial
LicenseInfoInFile: GPL-3.0-only
LicenseInfoInFile: LicenseRef-Qt-Commercial
LicenseInfoInFile: Qt-GPL-exception-1.0
FileCopyrightText: <text>Copyright (C) 2022 The Qt Company Ltd.</text>

(note that Qt-GPL-exception-1.0 is stored as a separate LicenseInfoInFile entry, and not as an exception)

If you manually change all the entries like the above to:

FileName: ./cmake/qbatchedtestrunner.in.cpp
SPDXID: SPDXRef-7b211d4592fde6cb29719920ef203c52
FileChecksum: SHA1: 91d0b31b6e60054b87c9fa5d36b9b86f57ef1a7a
LicenseConcluded: GPL-3.0-only WITH Qt-GPL-exception-1.0 OR LicenseRef-Qt-Commercial
LicenseInfoInFile: GPL-3.0-only WITH Qt-GPL-exception-1.0
LicenseInfoInFile: LicenseRef-Qt-Commercial
FileCopyrightText: <text>Copyright (C) 2022 The Qt Company Ltd.</text>

the Verify command returns no error and says that "This SPDX Document is valid".

The thing that makes me wonder whether this is an issue of SPDX tools or of REUSE tool, is that I cannot find anything in the current SPDX specs that mandates that LicenseInfoInFile fields are coherent with LicenseConcluded fields when using license exceptions; moreover, if I try to generate an SPDX file without LicenseConcluded data with REUSE tool, SPDX tool validates it even if Qt-GPL-exception-1.0 is stored in a separate LicenseInfoInFile field:

cd qtbase-everywhere-src-6.5.3/
reuse spdx > ../qtbase-everywhere-src-6.5.3_no_LicenseConcluded.spdx.tag
cd ..
java -jar <path-to>/tools-java-1.1.8-jar-with-dependencies.jar Verify qtbase-everywhere-src-6.5.3_no_LicenseConcluded.spdx.tag

This SPDX Document is valid.

..even if the entries like the one above are like this:

FileName: ./cmake/qbatchedtestrunner.in.cpp
SPDXID: SPDXRef-7b211d4592fde6cb29719920ef203c52
FileChecksum: SHA1: 91d0b31b6e60054b87c9fa5d36b9b86f57ef1a7a
LicenseConcluded: NOASSERTION
LicenseInfoInFile: GPL-3.0-only
LicenseInfoInFile: LicenseRef-Qt-Commercial
LicenseInfoInFile: Qt-GPL-exception-1.0
FileCopyrightText: <text>Copyright (C) 2022 The Qt Company Ltd.</text>

(note the field LicenseInfoInFile: Qt-GPL-exception-1.0)

Summing up, as anticipated at the beginning, it seems that SPDX tools refuse to validate an SPDX file when both:

1. a license exception tag is stored as a separate LicenseInfoInFile field
2. the same tag is found (as a "WITH" exception) in the LicenseConcluded field

but it validates it when only 1) happens.

Is SPDX tools too picky here, or is REUSE tool not following an (implied) requirement of SPDX specs?

[goneall]

Since LicenseInfoInFile: Qt-GPL-exception-1.0 is an exception and not a license, it should only be used on the right side of a WITH expression.

If LicenseInfoInFile: LicenseRef-Qt-Commercial passes validation, then there is something wrong with the library.

From the error message:

Analysis exception processing SPDX file: Unexpected Error: org.spdx.library.model.SpdxIdInUseException: 
Can not create Qt-GPL-exception-1.0.  It is already in use with type ListedLicenseException which is 
incompatible with type ExtractedLicensingInfo

it looks like the license expression parser is creating a license with the same ID as the exception - which it should not do.

The code should check for a license exception before creating a local license.

BTW - the reuse tools should not be generating a LicenseInfoInFile with just the exception. It should probably generate a WITH expression similar to the LicenseConcluded.

[goneall]

Transferring this to the library repo where the parsing code is implemented.