Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RJS-2101: "Build optimizations" for iOS and Android #6650

Merged
merged 108 commits into from
Jun 17, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
108 commits
Select commit Hold shift + click to select a range
31ce1e1
Squashed commit of the following:
kraenhansen Apr 18, 2024
8791ddf
Adding a readme to the RNTA environment
kraenhansen Apr 4, 2024
e13483b
The dependencies.list changed into a dependencies.yml
kraenhansen Apr 5, 2024
288410d
WIP rewriting shell scripts in JS
kraenhansen Apr 19, 2024
6dd96ab
Adding a dummy project
kraenhansen Apr 24, 2024
e379cf7
Moved ccache wrapping script into the SDK package
kraenhansen May 3, 2024
252d3dd
Moved scripts to TypeScript src
kraenhansen May 3, 2024
0e16de7
Reworked PR into an xcframework
kraenhansen May 3, 2024
ff2cca9
Deleting unused dummy files
kraenhansen May 3, 2024
b54e851
Committing changes to Podfile.lock
kraenhansen May 3, 2024
0857409
Reverted Xcode downgrade
kraenhansen May 3, 2024
c702ff9
Assert Realm Core source files are available before building from source
kraenhansen May 6, 2024
bdd92b6
Splitting building frameworks from creating Xcframework
kraenhansen May 8, 2024
4b753bc
Removed "podspec-prepare" and driving build from source from the podspec
kraenhansen May 8, 2024
c765e4d
Adding building xcframework to GHA workflow
kraenhansen May 8, 2024
a71fc2e
Ensure submit analytics doesn't fail if TS isn't built
kraenhansen May 8, 2024
8faeb50
Fixed apple framework job name
kraenhansen May 8, 2024
bc15584
Using "macosx" as platform
kraenhansen May 8, 2024
9f2f1a6
Simplified the build archive name
kraenhansen May 8, 2024
832a41d
Download the xcframework when testing ios
kraenhansen May 8, 2024
fbfe618
Fix podspec
kraenhansen May 8, 2024
423dd6c
Adding more specific pod-install:ci script
kraenhansen May 8, 2024
f18e86b
Adding a script to build just for the simulator when testing
kraenhansen May 8, 2024
5aa1d19
Fixed header search paths
kraenhansen May 8, 2024
6f16f99
Declaring SKIP_RUNNER as external env to test scripts
kraenhansen May 8, 2024
5d9b7d3
No need to build:ts on pod-install
kraenhansen May 13, 2024
6c9ec28
Renamed realm-js.xcframework to realm-core.xcframework
kraenhansen May 13, 2024
84ea590
Deleting the bash build script
kraenhansen May 13, 2024
b59d3a8
First attempt at a react-native-android build script
kraenhansen May 13, 2024
2c3dfe3
Manually assembling prebuilds for Android
kraenhansen May 16, 2024
da3a244
Moving headers into xcframework
kraenhansen May 19, 2024
fbfd153
Using the "install" cpack target to build for Android
kraenhansen May 19, 2024
15a31b8
Moved the xcframework location
kraenhansen May 19, 2024
29f1ed3
Merged react-native/ into binding/
kraenhansen May 22, 2024
a4e5222
Fixed building Android and iOS apps
kraenhansen May 22, 2024
b6ad2df
Simplified NPM scripts
kraenhansen May 22, 2024
4155daa
Prebuilding for Android on CI
kraenhansen May 22, 2024
3deb0d1
Moved node prebuild to binding/
kraenhansen May 22, 2024
7d21944
Clarified JSI and TS steps
kraenhansen May 22, 2024
4e77834
Installing NDK before prebuilding for Android
kraenhansen May 22, 2024
216ea57
Using include instead of a matrix.variant
kraenhansen May 22, 2024
dcb7bdb
Installing Java and Android SDK before using it
kraenhansen May 22, 2024
daf24c4
Cleaning up job names
kraenhansen May 22, 2024
4521bc3
Using Ubuntu to build for Android
kraenhansen May 22, 2024
4b51ffb
Fixed xcframework upload to include headers
kraenhansen May 23, 2024
2facd22
Use a patterns to download android prebuilds
kraenhansen May 23, 2024
674e389
Small simplifications
kraenhansen May 23, 2024
cd7d2e6
Update compile definitions for ObjectStore unconditionally
kraenhansen May 23, 2024
8fa5c7c
Removed react-native- prefix from android and apple names
kraenhansen May 23, 2024
97ddcc0
Migrating to use upload-artifact v4 for Node prebuilds
kraenhansen May 23, 2024
56693eb
Fixed android cmake config
kraenhansen May 23, 2024
7b7bdd0
Attempt at fixing the node prebuild path pattern
kraenhansen May 23, 2024
3a74b43
Merging all Node prebuilds when downloading
kraenhansen May 23, 2024
08d7b58
Using "apple-" instead of "react-native-"
kraenhansen May 23, 2024
044122b
Fixed downloading generated jsi source
kraenhansen May 24, 2024
7d6f883
Adding --clean option to build cli
kraenhansen May 24, 2024
cf5ac9e
Trying to use ccache on Android if available
kraenhansen May 24, 2024
7d8724c
Use ccache when prebuilding for Android
kraenhansen May 24, 2024
8b8f6d4
Adding xcworkspace as output from running pod-install
kraenhansen May 24, 2024
d2f821f
Getting rid of the build-realm-cli
kraenhansen May 24, 2024
9275e8a
Renamed prebuild scripts
kraenhansen May 24, 2024
20548a6
Fixed collecting headers on Apple / iOS
kraenhansen May 24, 2024
98e27a6
Trying to disable wireit cache on iOS
kraenhansen May 24, 2024
ade11e9
Make wireit log more explicitly
kraenhansen May 24, 2024
b859334
No need for wireit in the integration tests as all prebuilds are alre…
kraenhansen May 24, 2024
af637d7
Renamed build script files
kraenhansen May 27, 2024
92d7877
Updated per platform docs on building native modules
kraenhansen May 27, 2024
e9ce566
Updated README.md
kraenhansen May 27, 2024
1f87a55
No need to pass REALM_ENABLE_SYNC as we're no longer compiling Core
kraenhansen May 27, 2024
ea1d9c8
Deleted unused header search paths
kraenhansen May 27, 2024
b125c36
Deleting an unused CMakeLists.txt file
kraenhansen May 27, 2024
6d5ecef
Clarified the need for GCC_SYMBOLS_PRIVATE_EXTERN
kraenhansen May 27, 2024
818541c
Excluding libBid and libs2geometry from realm-
kraenhansen May 27, 2024
087a910
Using allowlist for library names
kraenhansen May 27, 2024
c3012d2
Reverted change to docs
kraenhansen May 27, 2024
73a65b6
Deleting the debugging RN entirely
kraenhansen May 27, 2024
a89e64a
Deleted the RNTA readme which now has stale info
kraenhansen May 27, 2024
ecb9e2b
Simplified and documented build.gradle
kraenhansen May 27, 2024
ff9028d
Adding a note on a potential optimization to pr workflow
kraenhansen May 27, 2024
7ba7c2d
Adding a note on the use of REACT_NATIVE_ROOT_DIR
kraenhansen May 27, 2024
333f5b9
Ignoring a warning from the match on type_List and type_Dictionary
kraenhansen May 27, 2024
c34471d
Re-enabling sync headers
kraenhansen May 27, 2024
a10da9a
Making it more explicit that we bring jsi_init.h
kraenhansen May 27, 2024
6a5689e
Mentioning self when referencing waitingForUiFlush to silence warning
kraenhansen May 27, 2024
5a56cf9
Improving prebuild wireit caching
kraenhansen May 27, 2024
b03ad27
Asserting status codes on spawnSync calls
kraenhansen May 27, 2024
dcf1c2b
Moved -dbg right after their release library name.
kraenhansen May 27, 2024
2482c4f
Moving deps to dev-deps
kraenhansen May 27, 2024
c286515
Fixed issue with assertion of status
kraenhansen May 28, 2024
2daa9ae
Adding an assert to ensure the build script is ran through NPM script…
kraenhansen May 28, 2024
68768be
Tuning the contents of the NPM package
kraenhansen May 28, 2024
b407e78
Adding an assert that the build CLI is invoked through NPM script
kraenhansen May 28, 2024
38a0912
Adding a RNTA script to build the Realm native module
kraenhansen May 28, 2024
4b02b39
Added an explicit return false
kraenhansen May 28, 2024
3adf571
Apply suggestions from code review
kraenhansen May 31, 2024
0cb4a47
Removing the `npm_package_name` check from the CLI again
kraenhansen May 31, 2024
981ec40
Removing hack.cpp and all references to it
kraenhansen Jun 3, 2024
4da3eae
Setting visibility preset to hidden
kraenhansen Jun 3, 2024
2521769
Passing "Release" build-type in release mode
kraenhansen Jun 3, 2024
45624ae
Asserting REACT_NATIVE_ROOT_DIR in CmakeLists.txt
kraenhansen Jun 3, 2024
df70e32
Adding realm:build-android:release script
kraenhansen Jun 3, 2024
2aa76e3
Apply suggestions from code review
kraenhansen Jun 10, 2024
989f17c
Updated the Podfile.lock after release + rebase
kraenhansen Jun 14, 2024
83a7d77
Adding RCT_NEW_ARCH_ENABLED to the CI workflow and disabling this by …
kraenhansen Jun 14, 2024
48e8eaa
Comitting changes to locks
kraenhansen Jun 14, 2024
fdef20a
Adding files to the package "files"
kraenhansen Jun 15, 2024
0d3d38d
Incorporated feedback
kraenhansen Jun 17, 2024
e6cff1d
"validate" → "pick" in the Apple script too
kraenhansen Jun 17, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
363 changes: 261 additions & 102 deletions .github/workflows/pr-realm-js.yml

Large diffs are not rendered by default.

23 changes: 17 additions & 6 deletions .github/workflows/publish-release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -53,25 +53,36 @@ jobs:
- name: Install node modules
run: npm ci

- name: Download prebuild artifacts from PR
uses: dawidd6/action-download-artifact@d0f291cf39bd21965ea9c4c6e210fc355c3844ed
- name: Download JSI binding source-code from PR
uses: dawidd6/action-download-artifact@09f2f74827fd3a8607589e5ad7f9398816f540fe
with:
workflow: pr-realm-js.yml
commit: ${{ inputs.commit || github.sha }}
path: ${{ github.workspace }}
workflow_conclusion: "" # Ignores workflow conclusion
github_token: ${{ secrets.REALM_CI_PAT }}
name: realm-js-prebuilds
name: jsi-binding-source

- name: Download bundle artifacts from PR
uses: dawidd6/action-download-artifact@d0f291cf39bd21965ea9c4c6e210fc355c3844ed
- name: Download TypeScript build from PR
uses: dawidd6/action-download-artifact@09f2f74827fd3a8607589e5ad7f9398816f540fe
with:
workflow: pr-realm-js.yml
commit: ${{ inputs.commit || github.sha }}
path: ${{ github.workspace }}
workflow_conclusion: "" # Ignores workflow conclusion
github_token: ${{ secrets.REALM_CI_PAT }}
name: realm-js-bundles
name: ts-build

- name: Download prebuilds from PR
uses: dawidd6/action-download-artifact@09f2f74827fd3a8607589e5ad7f9398816f540fe
with:
workflow: pr-realm-js.yml
commit: ${{ inputs.commit || github.sha }}
path: ${{ github.workspace }}
workflow_conclusion: "" # Ignores workflow conclusion
github_token: ${{ secrets.REALM_CI_PAT }}
name_is_regexp: true
name: +*-prebuild

- name: Read version
id: get-version
Expand Down
21 changes: 7 additions & 14 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -40,18 +40,14 @@ DerivedData
# node.js
node_modules/
npm-debug.log
build/
/prebuilds/
yarn.lock

# Emacs
*~

# Android/IJ
/android
/packages/realm/react-native/android/src/main/jni/core
/packages/realm/react-native/android/.cxx/
/packages/realm/build-android/
/packages/realm/binding/android/.cxx/
.idea
.gradle
local.properties
Expand Down Expand Up @@ -96,15 +92,9 @@ coverage/
**/imported-apps/
**/realm-config

#Ignore symlinked directories (otherwise on Windows they are shown as untracked files)
/react-native/android/src/main/jni/vendor/
/react-native/android/src/main/jni/src/
/build-tmp*/
/cmakebuild/
/react-native/android/src/main/java/io/realm/react/Version.java
/react-native/android/src/main/jniLibs/
/realm*.tgz
/react-native/ios/realm-js-ios.xcframework/

# Ignore template package-lock.json files
/templates/*/package-lock.json
Expand All @@ -114,15 +104,18 @@ coverage/
/packages/realm/bindgen/vendor/bindgen-lib/generated/
/packages/realm/generated/
/packages/realm/binding/generated
/packages/realm/binding/jsi/jsi_init.cpp

# Build artifacts
/packages/realm/dist/
/packages/realm/binding/dist/
/packages/realm/binding/build/
/packages/realm/binding/android/build/
/packages/realm/binding/node/build/
/packages/realm/prebuilds/
/packages/realm/react-native/android/src/main/java/io/realm/react/Version.java
/packages/realm/react-native/android/src/main/jniLibs/
/packages/realm/react-native/ios/realm-js-ios.xcframework/
/packages/realm/binding/android/src/main/java/io/realm/react/Version.java
/packages/realm/binding/android/src/main/jniLibs/
/packages/realm/binding/apple/*.xcframework

# Wireit repo caches
.wireit
Expand Down
2 changes: 1 addition & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
* None

### Enhancements
* None
* Building for iOS and Android has been optimized for compatibility with future React Native versions, by deferring compilation of JSI dependent code to the dependent app's build.
kneth marked this conversation as resolved.
Show resolved Hide resolved

### Fixed
* <How to hit and notice issue? what was the impact?> ([#????](https://github.com/realm/realm-js/issues/????), since v?.?.?)
Expand Down
43 changes: 34 additions & 9 deletions contrib/building.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,8 @@
- [Building for Node.js](#building-for-nodejs)
- [Additional steps for Windows](#additional-steps-for-windows)
- [Building for ARM/Linux](#building-for-armlinux)
- [Building the documentation](#building-the-documentation)
- [Cleaning up build files](#cleaning-up-build-files)
- [Building the documentation](#building-the-documentation)
- [Running the tests](#running-the-tests)
- [Linting the source code](#linting-the-source-code)
- [JS/TS](#jsts)
Expand Down Expand Up @@ -197,35 +197,60 @@ Other editors should also be able to be configured to use the `compile_commands.

## Building Realm JS

In most cases, it's not required to build the SDK explicitly. You can either simply download the package from `npm` or run one of the test scripts in either of the `integration-tests/environments` (which will drive the dependent build-scripts automatically). If you want to invoke these scripts manually, see the individual sections below:

### Building the SDK

Most of Realm JS is platform independent code (commonly referred to as the SDK), which is built explicitly by running:

```sh
npm run build:ts --workspace realm
```

### Building for iOS

You can build and bundle for iOS by running the following command from the root directory:

```sh
npm run build:ios --workspace realm
npm run bundle --workspace realm
# Pre-build Realm Core into an XCFramework
npm run prebuild-apple --workspace realm
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Super nitpicky, but curious why we use a - for one separator and : for the other.

Copy link
Member Author

@kraenhansen kraenhansen Jun 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The honest reason is because I don't want prebuild-node to act as an NPM pre-script for the existing build:node and it would if it was prebuild:node 🙈 We could revisit this later, but this was a fast and (only slightly dirty) workaround.

# Generate the JSI binding code (to be compiled when building the consuming app)
npm run bindgen:jsi --workspace realm
```

The resulting prebuilt binary is stored in `packages/realm/prebuilds/apple`.

### Building for Android

You can build and bundle for Android by running the following command from the root directory:

```sh
npm run build:android --workspace realm
npm run bundle --workspace realm
# Pre-build Realm Core into a CPack install directory
npm run prebuild-android --workspace realm
# Generate the JSI binding code (to be compiled when building the consuming app)
npm run bindgen:jsi --workspace realm
```

The compiled version of the Android module is output to `<project-root>/android`.
The resulting prebuilt binary is stored in `packages/realm/prebuilds/android`.

### Building for Node.js

You can build and bundle for Node.js by running the following command from the root directory:
You can build the native prebuilt binary for Node.js by running the following command from the root directory:

```sh
npm run build:node --workspace realm
npm run bundle --workspace realm
```

The resulting prebuilt binary is the `packages/realm/prebuilds/node/realm.node` file.

If you want to produce a prebuild (a OS +arch specific archive meant for distribution alongside the NPM archive):

```sh
npm run prebuild-node --workspace realm
```

The resulting prebuilt binary is stored in a `packages/realm/prebuilds/realm-*.tar.gz` file.

#### Additional steps for Windows

On Windows you will need to set up the environment for node-gyp:
Expand Down Expand Up @@ -281,7 +306,7 @@ cd realm-js
git submodule update —-init —-recursive
npm install --ignore-scripts
npm run build:node --workspace realm
npm run bundle --workspace realm
npm run build:ts --workspace realm
```

Finally, you can use Realm JS in your example project `MyProject`:
Expand Down
12 changes: 5 additions & 7 deletions contrib/debug-rn-examples.md
Original file line number Diff line number Diff line change
Expand Up @@ -103,13 +103,11 @@ Since we want to keep the simplicity for our users, changes made in order to deb

Make sure to build binaries for Android and iOS in packages/realm by running:

```
npm run build:ios --workspace=realm
```
or

```
npm run build:android --workspace=realm
```sh
# Android
npm run prebuild-android --workspace realm
# Apple / iOS
npm run prebuild-apple --workspace realm
```

### Building failed
Expand Down
111 changes: 0 additions & 111 deletions contrib/debugging-react-native.md

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -1193,7 +1193,7 @@ PODS:
- React-Core
- React-jsi
- ReactTestApp-Resources (1.0.0-dev)
- RealmJS (12.9.0):
- RealmJS (12.10.0):
- React
- RNFS (2.20.0):
- React-Core
Expand Down Expand Up @@ -1449,11 +1449,11 @@ SPEC CHECKSUMS:
ReactNativeHost: 35df5fb9d51dc99eaec21e993642f1232ec8b380
ReactTestApp-DevSupport: 9052e9a0ba3a96a3cc574ee66c7b6089ee76b341
ReactTestApp-Resources: d200e68756fa45c648f369210bd7ee0c14759f5a
RealmJS: e02ad90549318cff0a9c4b76e0b605e43424e8d1
RealmJS: 49d581b10bc4362a12818988657fff29fabe68b7
RNFS: 4ac0f0ea233904cb798630b3c077808c06931688
SocketRocket: abac6f5de4d4d62d24e11868d7a2f427e0ef940d
Yoga: 348f8b538c3ed4423eb58a8e5730feec50bce372

PODFILE CHECKSUM: ba37156b0bddb64052f2f6885355287c2356ac29
PODFILE CHECKSUM: 31a428908a285ec58ff867a2a56fd19780e5be51

COCOAPODS: 1.15.2
Loading
Loading