International (non-ASCII) characters in filenames - Mac can't read

[claud9999]

How do I get Netatalk to support International characters, such as Déjà Vu.m4a? It shows up on the Mac in the Finder but the Finder is unable to read it. From the shell it reports "file not found" even though auto-complete completes the filename. My Mac is running Monterey (12.7.6) and I'm trying with the latest Docker hub image.

[claud9999]

Ok here's another test. Déjà Vu-1.m4a is the file that can be read on Ubuntu but not on Mac. I scp'd it to the Mac then moved it to Déjà Vu-2.m4a and it can be read both on Mac and Ubuntu. Here's the hexdump of the two filenames on my Ubuntu system:

cdknight@raid:/raid$ ls -1 D*-1.m4a | hexdump -C
00000000  44 65 cc 81 6a 61 cc 80  20 56 75 2d 31 2e 6d 34  |De..ja.. Vu-1.m4|
00000010  61 0a                                             |a.|
00000012
cdknight@raid:/raid$ ls -1 D*-2.m4a | hexdump -C
00000000  44 c3 a9 6a c3 a0 20 56  75 2d 32 2e 6d 34 61 0a  |D..j.. Vu-2.m4a.|
00000010

[claud9999]

(For now I'm rsync-ing the folder from my Ubuntu system to my Mac and then copying back to shared fs to fix this. All non-ascii characters have this issue.)

[rdmark]

This is an odd one. I have tried to reproduce in my environment, but it works from both ends – when creating the file on the Mac and move to the AFP volume, and when creating the file from within the running Docker container's volume (e.g. touch Déjà\ Vu.m4a) on an Ubuntu 22.04 host. I'm using the macOS 15.1.1 AFP client and the latest 4.0.7 Docker image.

We had a similar-looking issue not long ago in #1667 which turned out to be a build system mishap, that has since been firmly fixed.

Would you be able to turn on debug logging and share what it says when you create and copy the affected files?

[APCCV]

Not a solution but I remembering reading something about this behaviour somewhere.
Here: https://eclecticlight.co/2017/04/06/apfs-is-currently-unusable-with-most-non-english-languages/
And follow up: https://eclecticlight.co/2021/05/08/explainer-unicode-normalization-and-apfs/

So "copying back to shared fs to fix" might be the best workaround available as then MacOS is fixing it the way it should be.

rsync has a --iconv option to convert between UTF-8 NFC & NFD, which may allow you to avoid the extra hop, but that defeats the AFPS purpose.

I'm playing with iconv and netatalk as I type this... maybe if you use a netatalk build -Dwith-libiconv=true some of the issues will be addressed. Not sure how to get a docker image to use libiconv or if it already does.