libcurl static linking instructions and/or pkg-config file

[avih]

Hi,

I'm trying to build txiki.js on debian 11 using the mingw distro toolchain, linked with https://curl.se/windows/latest.cgi?p=win64-mingw.zip (currently curl-7.86.0_2-win64-mingw.zip), and couldn't quite figure out what I should link with.

First, is there some README file at the zip or elsewhere which instructs how to use libcurl at the zip? If there is one, hopefully it answers at least some of the subjects below.

I first tried linking with libcurl.dll.a, which worked, and the resulting binary requires the curl dll (as expected). So that's a success.

I then wanted to instead link statically with libcurl.a, and encountered several missing libcurl symbols. After some searches I realized I needed CURL_STATICLIB defined, and I then used it successfuly.

Then it ended up with many (non-libcurl) missing symbols, but couldn't find which libs it depends on. I tried this list -lcurl -lz -lcrypt32 -lbcrypt -lwldap32 -lnghttp2 -lssh2 -lgsasl -lssl -lcrypto -lbrotlidec -lbrotlicommon -lzstd -lnghttp3 -lngtcp2 which appears at this repo README for the "Default build with OpenSSL and Schannel", but it failed with undefined reference to ngtcp2_crypto_openssl_configure_client_context and few more.

I added -lngtcp2_crypto_openssl which resolved this, but now I had a missing reference to zlibVersion and inflate and few more, so I moved -lz at the list to the end:

-lcurl -lcrypt32 -lbcrypt -lwldap32 -lnghttp2 -lssh2 -lgsasl -lssl -lcrypto -lbrotlidec -lbrotlicommon -lzstd -lnghttp3 -lngtcp2 -lngtcp2_crypto_openssl -lz

It then failed with undefined references to __imp___sys_nerr and __imp___sys_errlist, so (after some more googling) I added -lucrtbase to the list.

This seem to resolve all the undefined references, however, it then failed due to SHA1 defined multiple times, first at the txtki.js sources, and then at libcrypto.a (libcrypto.a(libcrypto-lib-sha1_one.obj):sha1_one.c:(.text+0x70): multiple definition of 'SHA1'; <the original definition location>).

I'll need to think what to do about it, maybe rename it at the txiki.js sources.

Seeing that the question of static linking has been asked more than once, and that the generic FAQ doesn't say much about it (docs/FAQ "2.1 configure fails when using static libraries"), I think it could help to have some info specific to this zip distribution about static linking. Specifically:

• pkg-config file or some other up-to-date list of libraries to link with, especially for static linking with libcurl.a inside this zip (shared is relatively simple).
• Mention CFLAGS=-DCURL_STATICLIB or some such which is needed for static linking (not sure if this is windows only?).
• I think it's possible to create a libcurl.a which embeds all the other static dependencies (similar to how the dll is linked statically with its deps), which could solve many issues (deps list, symbols which the project might not be expecting, etc).
• Consider mentioning (regardless of static/shared) that unlike a typical linux/*nix setup, https WILL fail because it defaults to OpenSSL but without a default ca bundle (embedded/at some path), unless one of the following happens (I think):
  • Curl is initialized with a path to ca bundle, e.g. using curl_easy_setopt(curl_h, CURLOPT_CAINFO, "curl-ca-bundle.crt"); and then put this file at the same dir as the binary (seems to work, but I couldn't find docs for the search dirs of CURLOPT_CAINFO if it's not an absolute path, and also not what to do about unicode paths - does it expect UTF-8?).
  • Or switch to schannel at runtime (one liner could help, if it exists).
• Consider a default configuration where https works out of the box without specific runtime setup? I've read several discussions about it, and I think OpenSSL is recommended over schannel, but maybe try some fixed ca bundle file name (curl-ca-bundle.crt is used by curl.exe at the zip, so I thought it could be a default name for libcurl at the zip as well), and fallback to schannel if it's missing?

[vszakats]

@avih: Thanks for your post. I understand your pains with static linking. You summed up very well what issues can be expected and how to fix them.

However, besides the informal Libs: lines in the README, I don't plan to maintain comprehensive documentation or special support for static linking (such as pkg-config or a "fat" libcurl.a). Simply for lack of time and interest. Such need to be kept in sync (and tested) with the full range of ever-changing bits of curl builds, possibly also considering the combinations this repo supports and varying toolchain needs (e.g. regarding UCRT).

Two suggestions that may help jumping through this:

• Use the lib list as-is from our build log when it's linking curl.exe.
• Look into lib/Makefile in the curl repo which should pick the correct LIBS in the correct order for static linking. (at least for common configurations)

I've also added the ngtcp2 crypto lib to the Libs: lines in the README following your suggestion.

Some points are not specific to these binaries, so may have a better place in the main curl repo. Some of them are already there, e.g. CURL_STATICLIB in docs/INSTALL.md.

As for what supports UTF-8, this is an open issue on Windows, and "it depends". It's not officially documented in curl. See also: https://github.com/curl/curl/wiki/libcurl-and-expected-string-encodings.

OpenSSL recently got support for built-in Windows certs. I haven't looked into it, and it might not made it to a release yet. It will resolve this faster then waiting for Schannel to catch up with OpenSSL features. Having said that, the mini and smaller builds all use Schannel by default. You can use them if you're ready to build your own curl-for-win.

[avih]

Thanks for the reply.

While it would be great if all configurations would be supported and documented equally, my focus when filing this issue was the official build of curl and libcurl for windows, which is available via https://curl.se/windows/ .

Am I correct to assume that, among others, this repo is used to build the official windows binary at https://curl.se/windows/ ?

I presime this zip and the libs/binaries within it are supposed to be used practically and not only as a proof of concept, right? And linking statically with this libcurl is hard to automate in practice.

I think that other than the CURL_STATICLIB which is perhaps generic (but still would be useful to mention at the zip), the rest are specific to this shipped configuration:

• The list of libs to link with (which I was sure would be inside the zip).
• The fact that https doesn't work unless some extra config happens (compared to linux or BSD), and that this extra config has more gotchas (undocumented search dirs), and mention that CURLOPT_CAINFO paths should be UTF-8.

The latter (https doesn't work) is actually not trivial at all to realize (other than the fact that it just doesn't work), and to me is quite surprising.

So I think that a readme specific to the official zip (or to the configuration from which it's built) could be useful.

If you agree, I can try to write such readme file with the issues and solutions mentioned in my first post, but a bit nicer ;)

Is this the correct repo for such readme? Where should be be placed?

[vszakats]

Sure, these builds are the "official" curl builds for Windows. We even link there from here.

By all means you're welcome to create such document.

My concern is keeping it up to date. It feels like a huge effort, over the already huge effort curl-for-win takes.

But, if the document is useful, we can link to it from here, or from other pages. It may even fit curl's INSTALL/FAQ docs, which already mentions static linking. (There is also winbuild/README.md for MSVC.)

[vszakats]

Ref: curl/curl#9944

[avih]

My concern is keeping it up to date. It feels like a huge effort, over the already huge effort curl-for-win takes.

Agreed.

It may even fit curl's INSTALL/FAQ docs, which already mentions static linking

The FAQ maybe, but as far as I can tell INSALL explicitly does not refer to using the curl binary distribution, and does refer to building curl from source - same as curl/curl#9944

What I had in mind is exactly these two parts:

1. Mention https doesn't work unless FOO. I don't think this part can be automated, and it will be in effect, if I understand correctly, as long as the configuration defaults to OpenSSL (or some other backend which depends on a ca bundle), but the default CURLOPT_CAINFO is NULL, correct? If so, maybe some comment can be added to the Makefile/whatever of this configuration to update this document if it changes?
2. How to link statically with libcurl. I feel the main tool for that should be a pkg-config file, and this document should only have notes about it, if any. The reason I think pkg-config is because the info exists when buildng the zip (probably almost exactly the same list of libs used for linking curl.exe inside this zip), and (i'm guessing) almost all distros create one when they build libcurl, so a useful procedure to create a pkg-config file surely exists someplace.

[avih]

(I've not built libcurl myself) does building libcurl generate a pkg-config file in general? It looks like it does - the main repo has libcurl.pc.in, and CMakeLists.txt does seem to use it to generate a pc file.

Does it not get generated when building libcurl-for-windows using this repo? If it is generated, why is it not packaged at the zip?

[vszakats]

These builds use bare GNU Make and don't generate pkg-config.

Alternate CMake and autotools builds do, though at least the latter both generate them with multiple issues (missing CFLAGS, hardcoded absolute paths, duplicate libs, missing libs (pthread), nonsense hardcoded prefix, missing features (HTTP3)). I've found pkg-config generally unusable on Windows, introducing more issue than solving (in fact I have to purge them while doing these builds to make it work). So no, no pkg-config support. But, if you do a local build with CMake (or autotools) there will be one generated.

[vszakats]

Regarding the CA: Perhaps this information would fit best inside curl somewhere, the FAQ maybe, or libcurl docs? It's a general issue when using llibcurl + any TLS backend that doesn't read the OS CA store. Maintaining a list of these TLS backends / versions would also be useful.

[avih]

the FAQ maybe, or libcurl docs?

Probably, but regardless, I do feel strongly that the zip should include some notes specific to this distribution. Like listing the build options, maybe build logs, and the two subjects I mentioned above.

I can think of at least one doc which could be updated - https://curl.se/libcurl/c/CURLOPT_CAINFO.html . It wasn't obvious to me that it can be empty by default and so initially I tried placing curl-ca-bundle.crt in various places - which never worked. It is stated that it depends on the build config, but hard to guess that an empty default is actually a viable distributable config.

This:

Default
  Built-in system specific.

Does not imply that it can be empty... (other than schannel or secure transport where it's unused).

[avih]

re 871a25b , as far as I can tell -lz should be after -lssh2, or else it fails for me.

Also, why -lucrt (or ucrtbase) not added too?

[vszakats]

Because -lucrt is not true when using for example mingw-w64 (which has a convoluted method to override msvcrt, last time I checked). As for -lz, I prefer to keep the current layout for editability and human readability, and just cannot guarantee any particular order there. In fact it might be best to delete those Libs: lines as it's already a hassle to keep up-to-date. And even failing at that, so it only causes confusion even in its current form.

IMO the best indication of the necessary libs is the actual list of .a files shipped inside the distro package. All of those are required. The generic solution to make them link regardless of order is to enclose them between -Wl,--start-group and -Wl,--end-group linker options. This is non-fragile and maintenance free.

Including logs (or even links to logs) break reproducibility, making every build producing a different hash, so that's not workable sadly. So instead we link to the log from the download page.

I agree that CA bootstrapping details would be nice to have on the CURLOPT_CAINFO page.

[avih]

I see. I think this means that no readme should be created/updated, and that there are two todo items outside the scope of this issue: improve CURLOPT_CAINFO docs (default can be empty, non-absolute paths search dirs, UTF-8), and maybe improve the CA bootstrap implementation/configuration so that https works out of the box for libcurl without additional setup compared to *nix?

I this case, I'll just summarize the info here for posterify (and then you can close it and/or help me improve it, or both, or edit it yourself, or whatever).

---

These notes refer to using the mingw libcurl as distributed at https://curl.se/windows/ as of version 7.86.

This libcurl supports both OpenSSL and Schannel TLS backends, and defaults to OpenSSL - which needs a CA bundle file. However, a default CA bundle path is not configured at build-time, and so by default https will fail. Possible solutions:

• Configure a CA bundle path at runtime, for instance curl_easy_setopt(curl_h, CURLOPT_CAINFO, "curl-ca-bundle.crt");. This is the same bundle name which curl.exe inside this zip uses by default, and this file is available at the distribution package. Without an absolute path - the file is searched (at least?) at the executable directory. If an absolute path is used instead, and this path might contain non-ASCII7 chars, then the char * value for CURLOPT_CAINFO path should be UTF-8 (see also string encodings).
• Switch to Schannel TLS backend. (TODO: example one liner?)
• The default behavior might change in the future so that https does work by default. Check the release notes of future versions.

It's recommended to link with libcurl dynamically - with libcurl.dll.a, and place the curl dll from the zip at the same dir as the binary.

Static linking is also possible, but less simple. Some notes:

• CURL_STATICLIB should be defined before including curl.h, for instance, by adding -DCURL_STATICLIB to CFLAGS (assuming the build system applies CFLAGS).
• You'd need to link with several (static) libraries in addition to libcurl.a. At this time libcurl.pc pkg-conig file is not provided. The list of libraries are all the lib/*.a files inside the zip, except libcurl.dll.a. Their order can be important - depending on other linker options:
  • When plainly linking with the libraries using -lcurl -lfoo -lbar..., at the time of writing the list and order is -lcurl -lcrypt32 -lbcrypt -lwldap32 -lnghttp2 -lssh2 -lz -lgsasl -lssl -lcrypto -lbrotlidec -lbrotlicommon -lzstd -lnghttp3 -lngtcp2 -lngtcp2_crypto_openssl, and you may need to add -lucrt depending on the mingw configuraion.
  • One way to automate it so that the order doesn't matter is to extract the list of libs by parsing the lib/*.a files at the zip, and then place this list between -Wl,--start-group and -Wl,--end-group linker options (see sample script below).

Example (posix) sh script to automate the static link flags (starting at the zip root directory):

LIBS="-lcurl -Wl,--start-group"
for f in ./lib/*.a; do
    case $f in *curl*) continue; esac
    f=${f##*/lib}; f=${f%.a}
    LIBS="$LIBS -l$f"
done
printf %s\\n "$LIBS -Wl,--end-group"

[vszakats]

Few more, potentialy interesting points:

• Regardless of static/dynamic linking, it's important that the host application also must be compiled with UCRT enabled.
• Pointers to the build log and lib/Makefile.mk in curl.
• List of necessary Windows systems libraries. This can be read from lib/Makefile.mk for example.
• pthread lib is not currently needed, but may be necessary once we switch to BoringSSL. Also pending a pthread vs UCRT bug in most/certain mingw-w64 toolchain builds.
• Not a one-liner but curl_global_sslset() is the libcurl function to switch TLS backends at runtime.
• llvm'd lld linker "should" accept the lib list in any order.
• It's perfectly normal and expected that a library with dependencies will always require more than a single library when linking statically. I think the wording should not be alarmist about this.

Applies to current default official builds. curl-for-win allows to toggle certain features (e.g. UCRT), toolchains and dependencies.

UPDATE: Added lld bit.

[avih]

Few more, potentialy interesting points:

Thanks. except as noted below, I'll leave them as additions in your own post, because you know about them (log links, etc) more than me.

• It's perfectly normal and expected that a library with dependencies will always require more than a single library when linking statically. I think the wording should not be alarmist about this.

I've edited it. better?

[vszakats]

@avih: Yes, thank you!

[avih]

Thanks for helping me understand the constraints better.

[vszakats]

@avih: Can you describe how would you use a libcurl.pc pkg-config file if there would be one?

FWIW they look like this (copied from local BoringSSL + mbedTLS builds):
cmake:

prefix=/usr
exec_prefix=${prefix}
libdir=/usr/lib
includedir=${prefix}/include
supported_protocols="DICT FILE FTP FTPS GOPHER GOPHERS HTTP HTTPS IMAP IMAPS LDAP MQTT POP3 POP3S RTSP SCP SFTP SMB SMBS SMTP SMTPS TELNET TFTP WS WSS"
supported_features="SSL IPv6 unixsockets libz brotli zstd AsynchDNS IDN Largefile SSPI alt-svc HSTS SPNEGO Kerberos NTLM HTTP2 HTTP3 MultiSSL HTTPS-proxy threadsafe"

Name: libcurl
URL: https://curl.se/
Description: Library to transfer files with ftp, http, etc.
Version: 7.86.0
Libs: -L${libdir} -lcurl  -lucrt -lwldap32 -lpthread -lbcrypt -lssh2 -lws2_32 -lgsasl -lmingw32 -lmoldname -lmingwex -ladvapi32 -lshell32 -luser32 -lkernel32 -lmingw32 -lmoldname -lmingwex -lkernel32 -lwldap32 -lwinmm -lws2_32 /opt/curl-for-win/boringssl/x64-ucrt/usr/lib/libssl.a /opt/curl-for-win/boringssl/x64-ucrt/usr/lib/libcrypto.a /opt/curl-for-win/mbedtls/x64-ucrt/usr/lib/libmbedtls.a /opt/curl-for-win/mbedtls/x64-ucrt/usr/lib/libmbedx509.a /opt/curl-for-win/mbedtls/x64-ucrt/usr/lib/libmbedcrypto.a /opt/curl-for-win/nghttp2/x64-ucrt/usr/lib/libnghttp2.a /opt/curl-for-win/ngtcp2/x64-ucrt/usr/lib/libngtcp2.a /opt/curl-for-win/ngtcp2/x64-ucrt/usr/lib/libngtcp2_crypto_boringssl.a /opt/curl-for-win/nghttp3/x64-ucrt/usr/lib/libnghttp3.a -lnormaliz /opt/curl-for-win/zlib/x64-ucrt/usr/lib/libz.a /opt/curl-for-win/brotli/x64-ucrt/usr/lib/libbrotlicommon.a /opt/curl-for-win/brotli/x64-ucrt/usr/lib/libbrotlidec.a /opt/curl-for-win/zstd/x64-ucrt/usr/lib/libzstd.a /opt/curl-for-win/libssh2/x64-ucrt/usr/lib/libssh2.a -ladvapi32 -lcrypt32 -lbcrypt
Libs.private:  -lucrt -lwldap32 -lpthread -lbcrypt -lssh2 -lws2_32 -lgsasl -lmingw32 -lmoldname -lmingwex -ladvapi32 -lshell32 -luser32 -lkernel32 -lmingw32 -lmoldname -lmingwex -lkernel32 -lwldap32 -lwinmm -lws2_32 /opt/curl-for-win/boringssl/x64-ucrt/usr/lib/libssl.a /opt/curl-for-win/boringssl/x64-ucrt/usr/lib/libcrypto.a /opt/curl-for-win/mbedtls/x64-ucrt/usr/lib/libmbedtls.a /opt/curl-for-win/mbedtls/x64-ucrt/usr/lib/libmbedx509.a /opt/curl-for-win/mbedtls/x64-ucrt/usr/lib/libmbedcrypto.a /opt/curl-for-win/nghttp2/x64-ucrt/usr/lib/libnghttp2.a /opt/curl-for-win/ngtcp2/x64-ucrt/usr/lib/libngtcp2.a /opt/curl-for-win/ngtcp2/x64-ucrt/usr/lib/libngtcp2_crypto_boringssl.a /opt/curl-for-win/nghttp3/x64-ucrt/usr/lib/libnghttp3.a -lnormaliz /opt/curl-for-win/zlib/x64-ucrt/usr/lib/libz.a /opt/curl-for-win/brotli/x64-ucrt/usr/lib/libbrotlicommon.a /opt/curl-for-win/brotli/x64-ucrt/usr/lib/libbrotlidec.a /opt/curl-for-win/zstd/x64-ucrt/usr/lib/libzstd.a /opt/curl-for-win/libssh2/x64-ucrt/usr/lib/libssh2.a -ladvapi32 -lcrypt32 -lbcrypt
Cflags: -I${includedir} 

autotools:

prefix=/usr
exec_prefix=${prefix}
libdir=${exec_prefix}/lib
includedir=${prefix}/include
supported_protocols="DICT FILE FTP FTPS GOPHER GOPHERS HTTP HTTPS IMAP IMAPS LDAP LDAPS MQTT POP3 POP3S RTSP SCP SFTP SMB SMBS SMTP SMTPS TELNET TFTP WS WSS"
supported_features="AsynchDNS GSASL HSTS HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile MultiSSL NTLM SPNEGO SSL SSPI UnixSockets alt-svc brotli libz threadsafe zstd"

Name: libcurl
URL: https://curl.se/
Description: Library to transfer files with ftp, http, etc.
Version: 7.86.0
Libs: -L${libdir} -lcurl -lnghttp2 -lnormaliz -lssh2 -lssh2 -lgsasl -lbcrypt -ladvapi32 -lcrypt32 -lmbedtls -lmbedx509 -lmbedcrypto -lssl -lcrypto -lgdi32 -lwldap32 -lzstd -lzstd -lbrotlidec -lbrotlidec -lz -lws2_32  -lucrt -lbrotlicommon -lzstd -lbcrypt -lnghttp3 -lngtcp2 -lngtcp2_crypto_boringssl
Libs.private: -lnghttp2 -lnormaliz -lssh2 -lssh2 -lgsasl -lbcrypt -ladvapi32 -lcrypt32 -lmbedtls -lmbedx509 -lmbedcrypto -lssl -lcrypto -lgdi32 -lwldap32 -lzstd -lzstd -lbrotlidec -lbrotlidec -lz -lws2_32 -lucrt -lbrotlicommon -lzstd -lbcrypt -lnghttp3 -lngtcp2 -lngtcp2_crypto_boringssl
Cflags: -I${includedir} -DCURL_STATICLIB

[avih]

how would you use a libcurl.pc pkg-config file if there would be one?

Well, I've not linked with libcurl before so it's not something I do frequently, and my first attempt was while building txiki.js which uses CMake. I'd hope that the CMake recipe would be able to pick the libs automatically, but I did not try it in practice.

FWIW they look like this ...

Yeah, I've seen the templates.

I don't think the Libs: part (shared against libcurl.dll.a in this zip) needs all the additional libs (-lssh2 etc) becaue the dll is linked statically, but they are needed at Libs.private: because it's not a 'fat" lib. Also, shouldn't the -DCURL_STATICLIB thing be at Cflags.private: ? (rather than at Cflags:).

[vszakats]

I understand.

Well, yeah, these have many problems:

• Missing -DCURL_STATICLIB. (cmake)
• .a files with absolute paths. (cmake)
• Toolchain-specific low-level libs. (cmake)
• Hardcoded "prefix", where prefix has no meaning on Windows. (both)
• Duplicate libs. (both)
• Missing libs (e.g. pthread, due to the way I had to force feed it to libtool) (autotools)
• Incomplete feature list (due to broken HTTP3 lib detection and the manual method of force-enabling it) (autotools)
• Possibly confused Libs: vs Libs.private:. (both)
• Possibly more.
• No libcurl.pc generated at all. (gnumake aka Makefile.mk)

Some of these are fallouts from build tricks I need to use, some others are cmake/autotools "things", and some others are pkg-config-specific design choices, like the insistence on well-known/hardcoded paths, which do not exist on Windows (outside MSYS2 and similar sub-envs).

Anyhow, if you manage to settle on a well-working libcurl.pc (preferably not just with CMake, but a bare linker command-line) we might use this to generate a usable one in the future. For sure it'd be useful, but I'm not confident that this is a achievable goal.

[avih]

preferably not just with CMake, but a bare linker command-line

Not sure what this means, because I don't think I've seen .pc files used "manually". But I think it does bring up an important question: who is this libcurl for? The curl.exe binary is obvious - a user can get the latest binary and just use it.

But a library, and tools related to it, are designed to be usable in some non-arbitrary environments.

Judging by "mingw" at the zip name, is it intended to be used only by mingw compiler? or also by msvc? Either way, the rest below refers to mingw only, but msvc might need some addressing too.

I think by far the most common environments for mingw are a posix-ish. That includes cross-compiling on *nix or cygwin, and MSYS2 (either as cross or native-ish via the mingw shortcut). However, there exist also less posix-y envs, like w64devkit.

(I'm ignoring msys(1), as it's not used these days I think, and either way it's similar to MSYS2, and also based on (old) cygwin).

The .pc file is then used by whatever build system the "client" project uses. I'd think that typically it can be (alphabetically/historically) autotoos, cmake, or meson.

So I'd say that as a high level goal, we'd want these to be able to link/build using this .pc file in a posix-y envisonment.

It's also important to keep in mind that sometimes these mingw envs already have their own libcurl recipe/binary (e.g. mingw curl as MSYS2 or cygwin or MXE package), however, it's still useful to be able to use the official library as well, but it could be tricky if a libcurl is already installed at this mingw env.

I'd think the most important places where it should work would be relatively barebones mingw envs, without their own libcurl recipe, like the debian mingw-w64 package, possibly fedora's (didn't check), and maybe w64devkit (though it's unclear what would use the .pc file other than manual build).

[vszakats]

Your summary is on point. The target for these libs is indeed "limited" considering the almost infinite variations of lib formats, ABIs and other variations these can come on Windows. (I find this a quite sad toolchain landscape and not a single bit better than say 15 years ago.)

In short, this might be useful for someone using mingw-w64 or llvm-mingw, either gcc or llvm/clang, who might want to link to a libcurl that isn't bound to any specific unixy environment, such as MSYS2, either dynamically, statically or via any other assumptions of these envs. Perfect to ship a standalone Windows .exe, also from a non-Windows env via a cross-build, if that's someone's thing.

MSVC never was a goal for these builds. Nor was Cygwin. And MSYS2/friends, as you said, already offer their own curl packages, though those I've found not really suitable for shipping your own app binaries.

But, yes, nowadays the main goal is to provide curl.exe. I have no stats, not even download numbers, but that's probably what most people use. Then possibly the DLL, which is somewhat more easily pluggable into other envs. Then probably just a few who actually use the static libs.

Having said that, the original aim of these builds was in fact to provide the static libs! Now it's the byproduct. [ I still have a somewhat idle open source project that expects these libs. For my own purposes I switched to pico builds to keep it simplest. ]

[avih]

Then possibly the DLL, which is somewhat more easily pluggable into other envs. Then probably just a few who actually use the static libs.

Right, FWIW, the txiki.js project vendors few sub-projects (git submodules) which it builds and links statically with, and expects libcurl to exist someplace. It does not depend on further libraries.

I was trying to create a semi reproducible mingw build env (in the sense of getting from zero to a working binary, not necessarily bit-identical), and I used a debian live dvd with the mingw toolchain installed from the distro packages after boot, and then needed only libcurl to complete the build deps, where the one from https://curl.se/windows/ was just what the doctor ordered ;) Even with the shared dll I consider it a good success, and to be honest, I did not expect it to work.

I didn't even realize initially that it includes development files, as in the past I only used it for the curl.exe binary (and libcurl recipe/package always existed at my mingw envs).

So it can have real-world use cases, and also serves as a reference to $something. But yeah, OSS/posix build envs for windows are such a pain...

the original aim of these builds was in fact to provide the static libs!

Maybe revive that with a "fat" libcurl.a static lib for mini/pico config? :)

[vszakats]

I'm glad this could fit into your project, and it's something nice to hear.

[ I find "mingw-w64" working rather well, and llvm-mingw, which is fantastic. Also the fact that these toolchains are now ubiquitous on popular platforms. A definite improvment over the years. And LLVM/clang. The binutils tools need some getting used to and had/have to deal with Windows-specific bugs. Most of them obscure. Fixes take years to propagate to releases. But, the knowledge is in large part portable, which I find useful.

What's lacking on Windows in general is standards or conventions even. No ABI, filename, dirname, lib format, or anything has any standard. Especially with MSVC which doesn't even have proper C99 support (or by now it has?). Making a cross-vendor DLL is a pain (it used to be with mingw-w64 at least). Thus, everyone keeps guessing and nothing is reusable. Microsoft could change that, but why would they? ]

Building for Windows is in effect like building for 4-8-16-64 distinct platforms, depending on settings, compiler vendor, version, make tool and whatnot, where the only thing in common is the Win32 API and the PE .exe format.

Though "fat" static lib seems doable technically, it's a step back for reproducibility and transparency IMO. Which is a question of convention, because for DLL and EXE a huge binary is okay, but for static libs, it seems unusual.

Maybe an opportunity to create such script and publish it as a Gist for those interested.

[avih]

I find "mingw-w64" working rather well, and llvm-mingw, which is fantastic

I presume that by "mingw-w64' you refer to the toolchain sources here https://www.mingw-w64.org/ which most mingw setups use these days, and not to a specific set of binaries, right? (agreed)

And that by "llvm-mingw" you refer to the project and a specific set of binaries here https://github.com/mstorsjo/llvm-mingw ? how do you use it? from an MSYS2 setup with the path modified to include the bin/ dir, and then cross compile using its various prefixes?

[rant]...

Agreed :)

[vszakats]

Yes, that's what I meant by mingw-w64 and llvm-mingw.

I've found llvm-mingw the easiest and it also works indentically on all platforms. Unpack, point PATH to the bin dir, select a prefix (or use the full path + triplet as a prefix probably works too), and it's good to go. [ If you don't insist on clang and UCRT, a pure mingw-w64 toolchain should be similarly easy. ]

If you're interested, the toolchain setup happens in _build.sh. Look for the word llvm-mingw.

In fact this is the only toolchain that doesn't bump into the static winpthread + UCRT bug, so the only one able to create BoringSSL builds.

llvm strip tool has some (niche) bugs, so for proper sripping the binuntils strip is necessary. You can ignore that if reproducibility across platforms isn't your goal.

[avih]

• Regardless of static/dynamic linking, it's important that the host application also must be compiled with UCRT enabled.

Actually, I think this mostly applies to static linking.

Judging from my own anecdotical experience, linking with the shared dll passed all txiki.js tests, but linking statically (after working around the roadbumps) failed some tests. I didn't mention it here because I thought I'm still doing something incorrect with the linking.

But then I saw https://www.msys2.org/docs/environments/#msvcrt-vs-ucrt which mentions:

Binaries linked with MSVCRT should not be mixed with UCRT ones because the internal structures and data types are different. (More strictly, object files or static libraries built for different targets shouldn't be mixed. DLLs built for different CRTs can be mixed as long as they don't share CRT objects, e.g. FILE*, across DLL boundaries.) Same rule is applied for MSVC compiled binaries because MSVC uses UCRT by default (if not changed).

I'm not familiar with the libcurl API, but I'm guessing it's not passing FILE* pointers or other CRT objects across library boundary?

[vszakats]

There are a couple of libcurl functions that can exchange FILE * pointers (e.g. CURLOPT_WRITEDATA and co, curl_mfprintf(), etc.). It's fairly easy, but you need to take care to avoid those.

Question what else may be affected, because the above quote cites FILE * as just one example. But sure, if you're certain to not be affected, the DLL can be mixed with non-UCRT.

[avih]

There are a couple of libcurl functions that can exchange...

Right, then I guess txiki.js either does not use them, or does not use them in tests, or by luck no issues were observed.

Generally speaking though, it seems that a UCRT libcurl should not be used with msvcrt application (which is how I compiled txiki.js for now), because even if today it doesn't use such APIs, tomorrow it might.

Question what else may be affected

I'd imagine any libc objects which are created with one crt and are manipulated/used at the other, like FILE* objects, malloc in one and free/realloc at the other, etc.

[vszakats]

Yeah, the sure way is not to mix CRTs.

One thing for consideration is using UCRT instead of msvcrt. I might have missed stuff here, but to me it seemed that unless you need to target XP and especially older Windows versions, UCRT is fine. As for open-source tooling: mingw-w64 hopefully will make it easier to enable it and remaining bugs will also be ironed out, such as the static winphtread one.

[avih]

I agree that if one doesn't care about XP then UCRT is probably the way to go. I don't know how common are UCRT mingw setups, and I think at leastr until very recently the common mingw setups were for MSVCRT.

So far I was using the debian mingw-w64 package, which as far as I can tell uses MSVCRT. As far as I can tell debian does not have UCRT mingw setup. I did not yet look into setting up other UCRT mingw setup

[vszakats]

The default is MSVCRT in the distros I'm aware of. Switching to UCRT isn't much straightforward though: https://stackoverflow.com/questions/57528555/how-do-i-build-against-the-ucrt-with-mingw-w64

The mentioned -mcrtdll=ucrt is not recognized by my mingw-w64. Patching a built-in specs file is rather unexpected (though it works).

Presumably in a few years this will become smoother or the default even.