Hot reload Rust libraries when changed

[SichangHe]

Describe the bug

When the Rust library is modified and the bindings are regenerated, the Flutter app does not load the new version of the Rust library when using hot reload (r) or hot restart (R). A full restart of the run is needed.

Steps to reproduce

1. Create a new Flutter-Rust project:

   flutter_rust_bridge_codegen create rapid_dart_rs
   cd rapid_dart_rs

2. On a separate terminal T1, watch and generate the bindings:

   flutter_rust_bridge_codegen generate --watch

3. Start the app in terminal T2, targeting macOS:

   flutter run

   Output:

   Connected devices:
   macOS (desktop)                 • macos                 • darwin-arm64   • macOS 12.6.3 21G419 darwin-arm64
   Mac Designed for iPad (desktop) • mac-designed-for-ipad • darwin         • macOS 12.6.3 21G419 darwin-arm64
   Chrome (web)                    • chrome                • web-javascript • Google Chrome 125.0.6422.76
   
   Checking for wireless devices...
   
   [1]: macOS (macos)
   [2]: Mac Designed for iPad (mac-designed-for-ipad)
   [3]: Chrome (chrome)
   Please choose one (or "q" to quit): 1
   Launching lib/main.dart on macOS in debug mode...
   --- xcodebuild: WARNING: Using the first of multiple matching destinations:
   { platform:macOS, arch:arm64, id:00008103-0011186234DA001E }
   { platform:macOS, arch:x86_64, id:00008103-0011186234DA001E }
   Building macOS application...                                           
   ✓ Built build/macos/Build/Products/Debug/rapid_dart_rs.app
   [IMPORTANT:flutter/shell/platform/darwin/graphics/FlutterDarwinContextMetalSkia.mm(66)] Using the Skia rendering backend (Metal).
   Syncing files to device macOS...                                    56ms
   
   Flutter run key commands.
   r Hot reload. 🔥🔥🔥
   R Hot restart.
   h List all available interactive commands.
   d Detach (terminate "flutter run" but leave application running).
   c Clear the screen
   q Quit (terminate the application on the device).
   
   A Dart VM Service on macOS is available at: http://127.0.0.1:64585/svdx0LE5lfo=/
   The Flutter DevTools debugger and profiler on macOS is available at: http://127.0.0.1:9100?uri=http://127.0.0.1:64585/svdx0LE5lfo=/

4. Change the greet function in rust/src/api/simple.rs to return something different, wait until the watching generator in T1 finishes (Done!), and press r in T2.

Expected behavior

The UI changes and reflect the behavior of the new Rust library.

Versions

• OS: macOS
• Version of flutter_rust_bridge_codegen: flutter_rust_bridge_codegen 2.0.0-dev.35

Flutter info

[✓] Flutter (Channel stable, 3.22.1, on macOS 12.6.3 21G419 darwin-arm64, locale en-US)
    • Flutter version 3.22.1 on channel stable at /opt/homebrew/Caskroom/flutter/3.16.0/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision a14f74ff3a (17 hours ago), 2024-05-22 11:08:21 -0500
    • Engine revision 55eae6864b
    • Dart version 3.4.1
    • DevTools version 2.34.3

[✓] Android toolchain - develop for Android devices (Android SDK version 32.0.0)
    • Android SDK at /Users/sichanghe/Library/Android/sdk
    • Platform android-34, build-tools 32.0.0
    • Java binary at: /Applications/Android Studio.app/Contents/jbr/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 17.0.6+0-17.0.6b829.9-10027231)
    • All Android licenses accepted.

[!] Xcode - develop for iOS and macOS (Xcode 14.2)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Build 14C18
    ! Flutter recommends a minimum Xcode version of 15.
      Download the latest version or update via the Mac App Store.
    • CocoaPods version 1.15.2

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2022.3)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 17.0.6+0-17.0.6b829.9-10027231)

[✓] VS Code (version 1.89.1)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension can be installed from:
      🔨 https://marketplace.visualstudio.com/items?itemName=Dart-Code.flutter

[✓] Connected device (3 available)
    • macOS (desktop)                 • macos                 • darwin-arm64   • macOS 12.6.3 21G419 darwin-arm64
    • Mac Designed for iPad (desktop) • mac-designed-for-ipad • darwin         • macOS 12.6.3 21G419 darwin-arm64
    • Chrome (web)                    • chrome                • web-javascript • Google Chrome 125.0.6422.76

[✓] Network resources
    • All expected network resources are available.

! Doctor found issues in 1 category.

• Version of clang++: Apple clang version 14.0.0 (clang-1400.0.29.202)

Additional context

Please tell me if there are additional setup I needed to go through, or there is anything I was doing wrong here. Thanks!

[welcome]

Hi! Thanks for opening your first issue here! 😄

[SichangHe]

Shit, this is probably impossible: flutter/flutter#75528

Then, this is not a bug… Sorry for that.

[fzyzcjy]

Yes, I think so...

[SichangHe]

This may be possible if we don't mind creating tons of garbage.

Dart does not support hot reloading FFI libraries, but does it support loading new FFI libraries? If so, in theory we can change the name of the whole Rust library, point the Dart code to this new library, then hot reload the Dart code. How does that sound?

In this scenario, every time we update the Rust code, we create a new dylib and load it into the Dart VM. The old dylib is not freed from the Dart VM, piling up as garbage. But, this may be fine because Rust dylibs are typically just a few MiBs (in my impression), and this garbage process is only done during development.

(I was messing with Python and it turned out it cannot reload C extension modules neither, but it works if I rename the extension module. This is where this idea comes from.)

[fzyzcjy]

Looks interesting! I guess there may be multiple subquestions. A quick brainstorm:

• About reloading FFI libraries: On some platforms (exclude ios/macos I guess?), we use something like DynamicLibrary.open('path/to/your/file.so'). Therefore, I guess it is possible to simply change the content of that .so file, and then re-open it, without renaming.
• When reloading Rust, do we want to keep Rust states (i.e. hot "reload" instead of hot "restart" if using Flutter terminology)? If so, we may need to use hacks such as https://github.com/rksm/hot-lib-reloader-rs.

(Title and tag changed since this looks like a feature request; feel free to re-change if needed)

[SichangHe]

• About reloading FFI libraries: […]

I really don't know about this. Loading an FFI library should be just reading and parsing something into memory and pointing stuff to it, but the issue linked above seems to suggest there are some limitations in the Dart VM by design.

• When reloading Rust, do we want to keep Rust states ([…])? […]

It would be nice to not forcefully delete them, but it is tricky as Rust does not have hot-reload semantics. Hot-changing a struct definition is a good way to get a SegFault. Evcxr (REPL and Jupyter kernel) tries to avoid this by invalidating variables whose types changed, but this requires static analysis of code changes.

If I understand correctly, The only relevant things in a dylib are the functions (and maybe constants)? Then, by loading another dylib, the effect would be similar to changing a bunch of function pointers? New functions would be applied to old structures; this is undefined behavior.

One thing we could consider, is to tell people that using Rust values after changing their structure definitions is undefined behavior, and then just don't do anything about it. People would have the choice to persist changing structures in Dart, or always restart the program when they change Rust types (or shoot themselves in the foot and get a SegFault if they so choose).

[fzyzcjy]

Loading an FFI library should be just reading and parsing something into memory and pointing stuff to it, but the issue linked above seems to suggest there are some limitations in the Dart VM by design.

Not checked the details yet, a quick glances and it seems that, maybe that issue is talking about there is not yet a hook when hot restart.
But this can be solved on our side: For example, when doing RustLib.init(), we can check whether there is already Rust state, and if so, destroy it.

but it is tricky as Rust does not have hot-reload semantics

I think so, thus that package is somehow hacky.

Then, by loading another dylib, the effect would be similar to changing a bunch of function pointers? New functions would be applied to old structures; this is undefined behavior.

Not very sure about your question, but it seems that we do generate Dart code to open dylib and call functions across ffi boundary, so seems no problem: The new Dart code uses new function pointers to new Rust code.

One thing we could consider, is to tell people that using Rust values after changing their structure definitions is undefined behavior, and then just don't do anything about it. People would have the choice to persist changing structures in Dart, or always restart the program when they change Rust types (or shoot themselves in the foot and get a SegFault if they so choose).

Agree, that looks like the approach hot-lib-reloader is taking.

[SichangHe]

I think you got the idea.

[fzyzcjy]

Discussions on Dart:

• [native_toolchain_rust] Introduce a package:native_toolchain_rust dart-lang/native#883 (comment)
• [native_assets] Hot restart native libs dart-lang/sdk#55850

[fzyzcjy]

Mark as awaiting, since native_assets will be the preferred approach in future Dart, thus it would be great if we could build our infra based on native_assets-based things like dart-lang/sdk#55850.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[github-actions]

This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new issue.