Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

emscripten: use upstream recommended llvm revision #63183

Closed
wants to merge 1 commit into from

Conversation

alebcay
Copy link
Member

@alebcay alebcay commented Oct 21, 2020

  • Have you followed the guidelines for contributing?
  • Have you checked that there aren't other open pull requests for the same formula update/change?
  • Have you built your formula locally with brew install --build-from-source <formula>, where <formula> is the name of the formula you're submitting?
  • Is your test running fine brew test <formula>, where <formula> is the name of the formula you're submitting?
  • Does your build pass brew audit --strict <formula> (after doing brew install <formula>)?

See #63176, emscripten-ports/SDL2_mixer#2, emscripten-core/emscripten#12551. This should hopefully fix all of them.

@BrewTestBot BrewTestBot added nodejs Node or npm use is a significant feature of the PR or issue python Python use is a significant feature of the PR or issue labels Oct 21, 2020
@alebcay
Copy link
Member Author

alebcay commented Oct 21, 2020

Anyone have any ideas on this High Sierra failure during the llvm build?

[ 60%] Copying CXX header support/ibm/support.h
In file included from /tmp/emscripten--llvm-20201021-71374-z6k23r/compiler-rt/lib/sanitizer_common/sanitizer_stoptheworld_mac.cpp:18:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS12.1.sdk/usr/include/mach/mach.h:65:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS12.1.sdk/usr/include/mach/std_types.h:66:
/tmp/emscripten--llvm-20201021-71374-z6k23r/llvm/build/./bin/../include/c++/v1/stdint.h:106:10: fatal error: '__config' file not found
#include <__config>
         ^~~~~~~~~~

@alebcay alebcay force-pushed the emscripten-track-llvm branch 2 times, most recently from 3463732 to f4421b4 Compare November 5, 2020 19:27
@alebcay
Copy link
Member Author

alebcay commented Nov 26, 2020

Current failure on Big Sur:

[ 62%] Copying CXX header stdexcept
In file included from /tmp/emscripten--llvm-20201115-24307-rmveyg/compiler-rt/lib/lsan/lsan_malloc_mac.cpp:57:
In file included from /tmp/emscripten--llvm-20201115-24307-rmveyg/compiler-rt/lib/lsan/../sanitizer_common/sanitizer_malloc_mac.inc:20:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/CoreFoundation.framework/Headers/CFBase.h:69:
/tmp/emscripten--llvm-20201115-24307-rmveyg/llvm/build/./bin/../include/c++/v1/stdbool.h:22:10: fatal error: '__config' file not found
#include <__config>
         ^~~~~~~~~~
[ 62%] Copying CXX header stdint.h
[ 62%] Copying CXX header stdio.h
1 error generated.
make[5]: *** [compiler-rt/lib/lsan/CMakeFiles/clang_rt.lsan_osx_dynamic.dir/lsan_malloc_mac.cpp.o] Error 1

@carlocab
Copy link
Member

carlocab commented Nov 27, 2020

Quick question, in case you would know (no worries if not, I'll just have a look myself later on instead):

Does emscripten really need such a fully-featured installation of llvm to work?

https://github.com/Homebrew/homebrew-core/blob/168f18b0694692ed85fc0413c0313a8632016f6d/Formula/emscripten.rb#L53-L69

https://github.com/Homebrew/homebrew-core/blob/168f18b0694692ed85fc0413c0313a8632016f6d/Formula/emscripten.rb#L100

The build docs only recommend building the following targets

-DLLVM_ENABLE_PROJECTS='lld;clang'
-DLLVM_TARGETS_TO_BUILD="host;WebAssembly"

Their docs on packaging require the following binaries:

  • clang
  • clang++ (note: this is a symlink to clang)
  • wasm-ld
  • llc
  • llvm-nm
  • llvm-ar
  • llvm-as
  • llvm-dis
  • llvm-dwarfdump

I don't think you need clang-tools-extra, lldb, or polly for these?

As it is the emscripten package takes up 2GB of space, and I imagine a lot of that is unused. Building more targets than you need to also increases the probability of build failures. (Not sure if that's the culprit here, but could be worth a look.)

Ok, I guess that wasn't as quick as I thought. Sorry.

@alebcay
Copy link
Member Author

alebcay commented Nov 27, 2020

I think you raise a fair point. Seeing as the error encountered is while compiling the compiler-rt runtime, I don't think this is necessarily going to fix things, but it should definitely reduce the package size. I copied over a lot of the LLVM cruft/boilerplate over from the main LLVM formula because I was just nervous about touching stuff and breaking something.

@carlocab
Copy link
Member

carlocab commented Nov 27, 2020

No worries. I know how difficult it can be trying to get llvm to build (I must've done so nearly a dozen times in the past week trying to implement an improvement to the llvm formula).

Do you want more input on other things I think you can cut out? I understand if you'd rather not -- I'll just pursue cutting down the installation size in another PR when this one is merged. But I do think there's still some low-hanging fruit.

Also, are you sure any of those runtimes are even necessary? My impression was they're not, and it seems dropping compiler-rt would help the tests. The build docs I linked to don't enable any runtimes when building llvm.

@alebcay
Copy link
Member Author

alebcay commented Nov 27, 2020

Well, I'll be darned. Looks like everything's working now. This one has been sitting around for a while, so I'd kindly ask that we address further improvements in a new PR.

@BrewTestBot
Copy link
Member

🤖 A scheduled task has triggered a merge.

@carlocab
Copy link
Member

Sure, no problem. Leave it with me.

carlocab added a commit to carlocab/homebrew-core that referenced this pull request Nov 27, 2020
The emscripten depends on a non-stable build of llvm, and is therefore
packaged with with the required llvm build.

This commit does the following:
- removes unnecessary build targets in the llvm project
- adds a test that the llvm dependencies were in fact built
- declares a dependency on libffi [*]
- makes llvm build commands consistent with llvm formula

Until Homebrew#63183, the
emscripten installation used nearly 2GB of storage space. Some
improvements were made in that PR, and this commit is meant to follow
through on that.

[*] The arguments to cmake already previously referred to the libffi
formula. It wasn't declared as a dependency, so the llvm install linked
with the built-in libffi.
BrewTestBot pushed a commit that referenced this pull request Dec 3, 2020
* emscripten: cleanup formula
  The emscripten depends on a non-stable build of llvm, and is therefore
  packaged with with the required llvm build.

  This commit does the following:
  - removes unnecessary build targets in the llvm project
  - adds a test that the llvm dependencies were in fact built
  - declares a dependency on libffi [*]
  - makes llvm build commands consistent with llvm formula

  Until #63183, the
  emscripten installation used nearly 2GB of storage space. Some
  improvements were made in that PR, and this commit is meant to follow
  through on that.

  [*] The arguments to cmake already previously referred to the libffi
  formula. It wasn't declared as a dependency, so the llvm install linked
  with the built-in libffi.
* Fix build
* Minimise llvm build size
  The added cmake flags are documented here:

  https://llvm.org/docs/BuildingADistribution.html#options-for-reducing-size
* Remove llvm tests
  They keep failing for some reason, even if it appears llvm built
  properly. Not really sure what's going on.

Closes #65799.

Signed-off-by: Sean Molenaar <[email protected]>
Signed-off-by: BrewTestBot <[email protected]>
@carlocab carlocab mentioned this pull request Dec 4, 2020
5 tasks
carlocab added a commit to carlocab/homebrew-core that referenced this pull request Dec 5, 2020
- bumped emscripten version
- changed llvm revision according to (see formula for details):
  - https://github.com/emscripten-core/emsdk/blob/master/emscripten-releases-tags.txt
  - https://chromium.googlesource.com/emscripten-releases/+/37fc7647c754ac9a28ad588c143b82286de0ef71/DEPS
- use recommended binaryen revision
    - The same has been done with llvm. See: Homebrew#63183
- updated license to reflect bundled llvm and binaryen
BrewTestBot pushed a commit that referenced this pull request Dec 6, 2020
- bumped emscripten version
- changed llvm revision according to (see formula for details):
  - https://github.com/emscripten-core/emsdk/blob/master/emscripten-releases-tags.txt
  - https://chromium.googlesource.com/emscripten-releases/+/37fc7647c754ac9a28ad588c143b82286de0ef71/DEPS
- use recommended binaryen revision
    - The same has been done with llvm. See: #63183
- updated license to reflect bundled llvm and binaryen

Closes #66150.

Signed-off-by: FX Coudert <[email protected]>
Signed-off-by: BrewTestBot <[email protected]>
@BrewTestBot BrewTestBot added the outdated PR was locked due to age label Dec 30, 2020
@Homebrew Homebrew locked as resolved and limited conversation to collaborators Dec 30, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
nodejs Node or npm use is a significant feature of the PR or issue outdated PR was locked due to age python Python use is a significant feature of the PR or issue
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants