[lld-link] Spourious duplicate symbol error

[llvmbot]

Bugzilla Link	32603
Version	unspecified
OS	Windows NT
Attachments	Reproducer and possible fix.
Reporter	LLVM Bugzilla Contributor
CC	@andreyv,@CyberShadow,@RIscRIpt,@rui314

Extended Description

We have noticed an inconsistency between the link.exe and lld-link.exe

Consider this example:

foo1.ll defines a function foo
foo2.ll defines a function foo
main.ll contains main that calls foo
foo1.ll is assembled into an object file and then into a library
foo2.ll is assembled into an object file

The following command currently fails:

lld-link.exe foo1.lib main.obj foo2.obj

The error is: duplicate symbol: foo

While the same invocation works with link.exe

This is because link processes all .obj files in input first and resorts to .lib inputs only if there are unresolved symbols.
Since the symbol foo is found in the .obj the library is ignored and no error is reported.

A reproducer for this bug is attached in the format of an LLVM test.
The same diff file contains a possible fix.

[rui314]

I was aware that the LLD's symbol resolution order might be different from the MSVC linker, but in most cases it just works fine. What program are you trying to link?

[llvmbot]

This problem showed up when compiling the spec2k6/namd benchmark.
Are you suggesting to change the build command line order as a workaround ?

[rui314]

Yes, if possible.

[llvmbot]

Changing the build system is not always possible for us.
We would like to have a stable solution.

This commit https://reviews.llvm.org/D32317 solves a similar problem.
Taking inspiration from that I can think of reordering in a stable way the input arguments so that .obj files come before .lib file.

Would this be ok for you ?
Do you have other suggestions ?

[rui314]

Sorting input files is probably OK, as I don't think of any failure scenario. Do you want to write a patch?

[llvmbot]

Yes, I can do that.

[CyberShadow]

Seeing this when trying to link with a Microsoft C library (libcmt) with lld-link 8.0.0:

lld-link: error: duplicate symbol: __aenvptr in libcmt.lib(crt0.obj) and in libcmt.lib(wincrt0.obj)
lld-link: error: duplicate symbol: __wenvptr in libcmt.lib(crt0.obj) and in libcmt.lib(wincrt0.obj)
lld-link: error: duplicate symbol: ___error_mode in libcmt.lib(crt0.obj) and in libcmt.lib(wincrt0.obj)
lld-link: error: duplicate symbol: ___app_type in libcmt.lib(crt0.obj) and in libcmt.lib(wincrt0.obj)

In this case, the object files are all within the same .lib file, therefore there is no workaround available such as reordering object files.

Of course, MS link.exe does not exhibit this problem.

[CyberShadow]

"/force:multiple" allows working around the above situation.

[RIscRIpt]

Duplicate symbol error test case for lld-link
Consider this minimal test case.
It's possible to compile and link it using gcc/clang/msvc but not with clangcl:

gcc:

$ rm -rf build && CC=gcc CXX=g++ cmake -B build && cmake --build build

clang:

$ rm -rf build && CC=clang CXX=clang++ cmake -B build && cmake --build build

msvc (on Windows):

$ cmake -B build && cmake --build build --config Release

clangcl (on Windows):

$ cmake -T clangcl -B build && cmake --build build --config Release
...
lld-link : error : duplicate symbol: public: __cdecl CommonClass::CommonClass(void) [D:\projects\dupsym\build\exe.vcxpr

oj]

[poliorcetics]

I'm encountering this problem when using https://github.com/https://github.com/nabijaczleweli/rust-embed-resource to have two different resources, one for tests and one for release.

I can somewhat work around it by using rustc-link-arg-tests and such but I'm still having troubles in some cases, see nabijaczleweli/rust-embed-resource#45

In my case, I can't reorder the arguments since it's cargo and rustc that orders them

[poliorcetics]

Using @CyberShadow's response and adding /force:multipleres half-fixed the problem for me: it compiles but since its the first resource lib that is taken and not the second, I can't override my test resources with the release ones