git-lfs-clean deletes objects still being referenced

[rico-chet]

Describe the bug

Running jf git-lfs-clean removes objects which are still referenced from reachable git history.

When a git-LFS-stored file is committed to git and then removed in a subsequent commit and the commits are reachable from a git ref, the referenced git-LFS object must not be deleted.

Current behavior

When running jf git-lfs-clean, the historic file is cleaned from the repository leading to dangling git-LFS pointer(s) in the git repository, with git lfs fetch --all failing subsequently.

Reproduction steps

I use a script on our corporate network with the corporate service instance:

#!/usr/bin/env bash

set -o errexit
set -o nounset
set -o xtrace

# create "remote" git repo
mkdir --parents git-repo
(cd git-repo
        git init --bare
)

# create "local" git repo
mkdir --parents code
cd code
git init

# create a tracked local/remote branch with a root commit
git commit --allow-empty --message='Add empty commit'
git remote add origin ../git-repo
git push --set-upstream origin master

# set up git LFS
git lfs install --local
git config \
        --file .lfsconfig \
        --add lfs.url \
        'ssh://werner.rtf.siemens.net:1339/artifactory/hmi_fw_bootlair-release-lfs-egll'
git add .lfsconfig
git commit --message='Configure git LFS on JFrog Artifactory'

git lfs track '*.bin'
git add .gitattributes
git commit --message='Track binaries in git LFS'

# use a local branch which is temporary for us
git checkout -b removed-file

# add a file which is created and then removed
echo jfrog > 5.bin
git add 5.bin
git commit --message='Add 5.bin'

# push the file to both git and git LFS
git push --set-upstream origin removed-file

# remove the file locally
git rm 5.bin
git commit --message='Remove 5.bin'
git push

# delete the local branch
git checkout master
git branch -d removed-file

# get JFrog-CLI
curl \
        --location \
        --output jfrog-cli \
        'https://releases.jfrog.io/artifactory/jfrog-cli/v2-jf/\[RELEASE\]/jfrog-cli-linux-amd64/jf'
chmod +x jfrog-cli

export CI=true
export JFROG_CLI_LOG_LEVEL=DEBUG
export JFROG_CLI_REPORT_USAGE=false

./jfrog-cli rt git-lfs-clean \
        --refs 'refs/remotes/*,refs/tags/*' \
        --repo hmi_fw_bootlair-release-lfs-egll \
        --url ssh://werner.rtf.siemens.net:1339

rm --force --recursive .git/lfs/objects/*
git lfs fetch --all

The output:

❯ ./create-git-repo
+ mkdir --parents git-repo
+ cd git-repo
+ git init --bare
Initialized empty Git repository in /home/user/.local/tmp/git-lfs-issue/git-repo/
+ mkdir --parents code
+ cd code
+ git init
Initialized empty Git repository in /home/user/.local/tmp/git-lfs-issue/code/.git/
+ git commit --allow-empty '--message=Add empty commit'
[master (root-commit) 5d67c76] Add empty commit
+ git remote add origin ../git-repo
+ git push --set-upstream origin master
Enumerating objects: 2, done.
Counting objects: 100% (2/2), done.
Writing objects: 100% (2/2), 170 bytes | 170.00 KiB/s, done.
Total 2 (delta 0), reused 0 (delta 0), pack-reused 0
To ../git-repo
 * [new branch]      master -> master
Branch 'master' set up to track remote branch 'master' from 'origin'.
+ git lfs install --local
Updated Git hooks.
Git LFS initialized.
+ git config --file .lfsconfig --add lfs.url ssh://werner.rtf.siemens.net:1339/artifactory/hmi_fw_bootlair-release-lfs-egll
+ git add .lfsconfig
+ git commit '--message=Configure git LFS on JFrog Artifactory'
[master eceade3] Configure git LFS on JFrog Artifactory
 1 file changed, 2 insertions(+)
 create mode 100644 .lfsconfig
+ git lfs track '*.bin'
Tracking "*.bin"
+ git add .gitattributes
+ git commit '--message=Track binaries in git LFS'
[master 2faff37] Track binaries in git LFS
 1 file changed, 1 insertion(+)
 create mode 100644 .gitattributes
+ git checkout -b removed-file
Switched to a new branch 'removed-file'
+ echo jfrog
+ git add 5.bin
+ git commit '--message=Add 5.bin'
[removed-file 4918ea8] Add 5.bin
 1 file changed, 3 insertions(+)
 create mode 100644 5.bin
+ git push --set-upstream origin removed-file
Uploading LFS objects: 100% (1/1), 6 B | 0 B/s, done.
Enumerating objects: 10, done.
Counting objects: 100% (10/10), done.
Delta compression using up to 12 threads
Compressing objects: 100% (7/7), done.
Writing objects: 100% (9/9), 977 bytes | 325.00 KiB/s, done.
Total 9 (delta 1), reused 0 (delta 0), pack-reused 0
To ../git-repo
 * [new branch]      removed-file -> removed-file
Branch 'removed-file' set up to track remote branch 'removed-file' from 'origin'.
+ git rm 5.bin
rm '5.bin'
+ git commit '--message=Remove 5.bin'
[removed-file 76bc443] Remove 5.bin
 1 file changed, 3 deletions(-)
 delete mode 100644 5.bin
+ git push
Enumerating objects: 3, done.
Counting objects: 100% (3/3), done.
Delta compression using up to 12 threads
Compressing objects: 100% (2/2), done.
Writing objects: 100% (2/2), 222 bytes | 222.00 KiB/s, done.
Total 2 (delta 1), reused 0 (delta 0), pack-reused 0
To ../git-repo
   4918ea8..76bc443  removed-file -> removed-file
+ git checkout master
Switched to branch 'master'
Your branch is ahead of 'origin/master' by 2 commits.
  (use "git push" to publish your local commits)
+ git branch -d removed-file
warning: deleting branch 'removed-file' that has been merged to
         'refs/remotes/origin/removed-file', but not yet merged to HEAD.
Deleted branch removed-file (was 76bc443).
+ curl --location --output jfrog-cli 'https://releases.jfrog.io/artifactory/jfrog-cli/v2-jf/\[RELEASE\]/jfrog-cli-linux-amd64/jf'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 23.2M  100 23.2M    0     0  16.7M      0  0:00:01  0:00:01 --:--:-- 78.1M
+ chmod +x jfrog-cli
+ export CI=true
+ CI=true
+ export JFROG_CLI_LOG_LEVEL=DEBUG
+ JFROG_CLI_LOG_LEVEL=DEBUG
+ export JFROG_CLI_REPORT_USAGE=false
+ JFROG_CLI_REPORT_USAGE=false
+ ./jfrog-cli rt git-lfs-clean --refs 'refs/remotes/*,refs/tags/*' --repo hmi_fw_bootlair-release-lfs-egll --url ssh://werner.rtf.siemens.net:1339
09:35:42 [Debug] JFrog CLI version: 2.28.1
09:35:42 [Debug] OS/Arch: linux/amd64
09:35:42 [Debug] Locking config file to run config Clear command.
09:35:42 [Debug] Creating lock in:  /home/user/.jfrog/locks/config
09:35:42 [Debug] Releasing lock:  /home/user/.jfrog/locks/config/jfrog-cli.conf.lck.2082515.1666683342835698050
09:35:42 [Debug] Config Clear command completed successfully. config file is released.
09:35:42 [Debug] Performing SSH authentication...
09:35:42 [Debug] Trying to authenticate via SSH-Agent...
09:35:42 [Debug] Usage info is disabled.
09:35:43 [Debug] SSH authentication successful.
09:35:43 [��Info] Searching files from Artifactory repository hmi_fw_bootlair-release-lfs-egll ...
09:35:43 [Debug] Searching Artifactory using AQL query:
 items.find({"$or":[{"$and":[{"repo":"hmi_fw_bootlair-release-lfs-egll","path":{"$match":"*"},"name":{"$match":"*"}}]}]}).include("name","repo","path","actual_md5","actual_sha1","sha256","size","type","modified","created")
09:35:43 [Debug] Sending HTTP POST request to: https://werner.rtf.siemens.net/artifactory/api/search/aql
09:35:43 [Debug] Artifactory response:  200 OK
09:35:43 [Debug] Streaming data to file...
09:35:43 [Debug] Finished streaming data successfully.
09:35:43 [��Info] Collecting files to preserve from Git references matching the pattern refs/remotes/*,refs/tags/* ...
09:35:43 [Debug] Opened Git repo at /home/user/.local/tmp/git-lfs-issue/code for reading
09:35:43 [Debug] Checking ref refs/heads/master
09:35:43 [Debug] Checking ref refs/remotes/origin/master
09:35:43 [Debug] Checking ref refs/remotes/origin/removed-file
09:35:43 [��Info] Found 0 files to keep, and 1 to clean
09:35:43 [��Info] Deleting 1 files from hmi_fw_bootlair-release-lfs-egll ...
09:35:43 [Debug] Performing SSH authentication...
09:35:43 [Debug] Trying to authenticate via SSH-Agent...
09:35:43 [Debug] SSH authentication successful.
09:35:43 [��Info] [Thread 2] Deleting hmi_fw_bootlair-release-lfs-egll/objects/15/1c/151c50753fceb8d2a8a6943c1cb382ff185fb91e88e5bc3f31f00ddf9f82c153
09:35:43 [Debug] Sending HTTP DELETE request to: https://werner.rtf.siemens.net/artifactory/hmi_fw_bootlair-release-lfs-egll/objects/15/1c/151c50753fceb8d2a8a6943c1cb382ff185fb91e88e5bc3f31f00ddf9f82c153
09:35:43 [Debug] Deleted 1 artifacts.
+ rm --force --recursive .git/lfs/objects/15
+ git lfs fetch --all
fetch: 1 object found, done.
fetch: Fetching all references...
[151c50753fceb8d2a8a6943c1cb382ff185fb91e88e5bc3f31f00ddf9f82c153] Object does not exist: [404] Object does not exist
error: failed to fetch some objects from 'https://werner.rtf.siemens.net/artifactory/hmi_fw_bootlair-release-lfs-egll'

Expected behavior

Historic objects still referenced are kept during cleanup. In the example above, the object is kept.

JFrog CLI-Core version

2.23.1

JFrog CLI version (if applicable)

2.28.1

Operating system type and version

Debian Linux 11

JFrog Artifactory version

No response

JFrog Xray version

No response