luci-app-package-manager: apk update: fetch of package.adb index is shown in Errors output

[ynezz]

Is there an existing issue for this?

• I have searched the existing issues

screenshots or captures

Actual behaviour

1. Go to https://192.168.1.1/cgi-bin/luci/admin/system/package-manager
2. Click on Update lists... action button
3. Once the apk update finishes it shows the output in the Errors section, which is misleading as its not an error

Errors

fetch https://downloads.openwrt.org/snapshots/targets/x86/64/packages/packages.adb
fetch https://downloads.openwrt.org/snapshots/packages/x86_64/base/packages.adb
fetch https://downloads.openwrt.org/snapshots/targets/x86/64/kmods/6.6.60-1-2e46b6878177e1dea330c0dbbbcfb6be/packages.adb
fetch https://downloads.openwrt.org/snapshots/packages/x86_64/luci/packages.adb
fetch https://downloads.openwrt.org/snapshots/packages/x86_64/packages/packages.adb
fetch https://downloads.openwrt.org/snapshots/packages/x86_64/routing/packages.adb
fetch https://downloads.openwrt.org/snapshots/packages/x86_64/telephony/packages.adb

Expected behaviour

Only errors/issues are shown in the Errors section.

Steps to reproduce

1. Go to https://192.168.1.1/cgi-bin/luci/admin/system/package-manager
2. Click on Update lists... action button

Additional Information

{
	"kernel": "6.6.60",
	"hostname": "OpenWrt",
	"system": "13th Gen Intel(R) Core(TM) i9-13900HX",
	"model": "QEMU Standard PC (Q35 + ICH9, 2009)",
	"board_name": "qemu-standard-pc-q35-ich9-2009",
	"rootfs_type": "squashfs",
	"release": {
		"distribution": "OpenWrt",
		"version": "SNAPSHOT",
		"revision": "r28056-40b8fbaa97",
		"target": "x86/64",
		"description": "OpenWrt SNAPSHOT r28056-40b8fbaa97",
		"builddate": "1731280190"
	}
}

What browsers do you see the problem on?

Firefox

Relevant log output

No response

[systemcrash]

Hi @ynezz

At a cursory glance at the source, it appears that a script captures stderr and stdout. So anything landing in the error output is what is, in fact, output by apk to stderr? ( I've not yet verified or run any new apk builds )

luci/applications/luci-app-package-manager/root/usr/libexec/package-manager-call

Lines 90 to 99 in d650688

	if flock -x 200; then
	pkmcmd="$cmd $action $@"
	$cmd $action "$@" </dev/null >/tmp/ipkg.out 2>/tmp/ipkg.err
	code=$?
	stdout=$(cat /tmp/ipkg.out)
	stderr=$(cat /tmp/ipkg.err)
	else
	code=255
	stderr="Failed to acquire lock"
	fi

Could you test on your installation, comment out the later line which does

luci/applications/luci-app-package-manager/root/usr/libexec/package-manager-call

Line 109 in d650688

rm -f /tmp/ipkg.lock /tmp/ipkg.err /tmp/ipkg.out

And double-check the contents of those files?

[ynezz]

So anything landing in the error output is what is, in fact, output by apk to stderr?

Yes, its the case

{
	"code": 0,
	"pkmcmd": "apk update ",
	"stdout": " [https://downloads.openwrt.org/snapshots/targets/x86/64/packages/packages.adb]\n [https://downloads.openwrt.org/snapshots/packages/x86_64/base/packages.adb]\n [https://downloads.openwrt.org/snapshots/targets/x86/64/kmods/6.6.60-1-2e46b6878177e1dea330c0dbbbcfb6be/packages.adb]\n [https://downloads.openwrt.org/snapshots/packages/x86_64/luci/packages.adb]\n [https://downloads.openwrt.org/snapshots/packages/x86_64/packages/packages.adb]\n [https://downloads.openwrt.org/snapshots/packages/x86_64/routing/packages.adb]\n [https://downloads.openwrt.org/snapshots/packages/x86_64/telephony/packages.adb]\nOK: 10505 distinct packages available",
	"stderr": "fetch https://downloads.openwrt.org/snapshots/targets/x86/64/packages/packages.adb\nfetch https://downloads.openwrt.org/snapshots/packages/x86_64/base/packages.adb\nfetch https://downloads.openwrt.org/snapshots/targets/x86/64/kmods/6.6.60-1-2e46b6878177e1dea330c0dbbbcfb6be/packages.adb\nfetch https://downloads.openwrt.org/snapshots/packages/x86_64/luci/packages.adb\nfetch https://downloads.openwrt.org/snapshots/packages/x86_64/packages/packages.adb\nfetch https://downloads.openwrt.org/snapshots/packages/x86_64/routing/packages.adb\nfetch https://downloads.openwrt.org/snapshots/packages/x86_64/telephony/packages.adb"
}

[Ansuel]

@systemcrash magical way of handling apk... But it's probably not intended that those fetch command goes in stderr... I will propose a fix upstream

[systemcrash]

Just to be sure we aren't on a wild goose chase, @ynezz can you test an equivalent apk command and try redirecting the relevant outputs? To see if it really is apk to blame?

[ynezz]

Just to be sure we aren't on a wild goose chase, @ynezz can you test an equivalent apk command and try redirecting the relevant outputs? To see if it really is apk to blame?

https://gitlab.alpinelinux.org/alpine/apk-tools/-/blob/master/libfetch/common.c?ref_type=heads#L176

root@OpenWrt:~# apk update 2>/dev/null
 [https://downloads.openwrt.org/snapshots/targets/x86/64/packages/packages.adb]
 [https://downloads.openwrt.org/snapshots/packages/x86_64/base/packages.adb]
 [https://downloads.openwrt.org/snapshots/targets/x86/64/kmods/6.6.60-1-2e46b6878177e1dea330c0dbbbcfb6be/packages.adb]
 [https://downloads.openwrt.org/snapshots/packages/x86_64/luci/packages.adb]
 [https://downloads.openwrt.org/snapshots/packages/x86_64/packages/packages.adb]
 [https://downloads.openwrt.org/snapshots/packages/x86_64/routing/packages.adb]
 [https://downloads.openwrt.org/snapshots/packages/x86_64/telephony/packages.adb]
OK: 10505 distinct packages available

root@OpenWrt:~# apk update 1>/dev/null
fetch https://downloads.openwrt.org/snapshots/targets/x86/64/packages/packages.adb
fetch https://downloads.openwrt.org/snapshots/packages/x86_64/base/packages.adb
fetch https://downloads.openwrt.org/snapshots/targets/x86/64/kmods/6.6.60-1-2e46b6878177e1dea330c0dbbbcfb6be/packages.adb
fetch https://downloads.openwrt.org/snapshots/packages/x86_64/luci/packages.adb
fetch https://downloads.openwrt.org/snapshots/packages/x86_64/packages/packages.adb
fetch https://downloads.openwrt.org/snapshots/packages/x86_64/routing/packages.adb
fetch https://downloads.openwrt.org/snapshots/packages/x86_64/telephony/packages.adb

[systemcrash]

Wow.

[Ansuel]

Well i checked the usage of fetch_info and it's really mixed for both error condition and real info so it just need some love :D

[Ansuel]

BTW it's not even there... fetch comes from database.c

[systemcrash]

Heh. "Ah, we just been usin' apk for like a decade, never noticed output bug. 🤷 "

[Ansuel]

https://gitlab.alpinelinux.org/alpine/apk-tools/-/merge_requests/243

lets see how they get this... Eventually we can experiment with the -q option and see if stdout is printed anyway....

[sertonix]

The reason for the duplicate output is that apk update shows a list of the repository description followed by the url/path. This is how it is meant to look:

~$ doas apk update
fetch https://mirror.postmarketos.org/postmarketos/master/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/edge/main/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/edge/community/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/edge/testing/x86_64/APKINDEX.tar.gz
main v20240923-4045-g4edeca90fc3 [/home/sertonix/.local/share/abuild/main]
community v20240923-4094-gb9c0be0959f [/home/sertonix/.local/share/abuild/community]
testing v20240923-4156-g4a3a35b9c4f [/home/sertonix/.local/share/abuild/testing]
2024-11-13 08:01:54.161666+00:00 [https://mirror.postmarketos.org/postmarketos/master]
v20240923-4215-gd1699ddc803 [https://dl-cdn.alpinelinux.org/alpine/edge/main]
v20240923-4235-gdf4f1f21465 [https://dl-cdn.alpinelinux.org/alpine/edge/community]
v20240923-4234-gf00c9daaa6c [https://dl-cdn.alpinelinux.org/alpine/edge/testing]
OK: 33164 distinct packages available

It looks like your repositories ether don't have a description or apk is not handling descriptions of adb repositories correctly.

[Ansuel]

@sertonix are you sure that output only comes from stdout and it's not mixed stderr and stdout?

[sertonix]

It is mixed stderr and stdout. I agree that this doesn't make sense. I am uncertain how to fix this. Just wanted to mention that the proposed fix introduces a regression.