xtensa esp32 target

[jan-br]

Hi :)
I am currently trying to access the AWS IoT SDK in Rust from an XTensa ESP32 which uses the ESP-IDF.

So I stumbled across this neat project, rustot, which seems to be able to do exactly what I am trying to do.
There are some other similar libraries, like the official one from AWS. But they all seem to use Tokio, which is - as far as I know - not working with xtensa yet.

So rustot is the only promising library I could find which might work for my usecase.
But mqttrust - which is used by the project - does not compile for me with the ESP Rust toolchain and its xtensa-esp32-espidf target.

My test project is just a simple instance of the template, generated like described in the Rust on ESP Book with the mqttrust_core 0.4.1 dependency.

This also occurs outside of my project, when I checkout a fresh version of mqttrust and try to compile it standalone with the esp toolchain, so this is not a problem in my project setup.

Using the default nightly linux toolchain everything is working fine.
I have no idea why this error is occurring. I'd appreciate any input on this topic.

❯ cargo build

   Compiling mqttrust_core v0.4.1 (/run/media/jan/e061bc91-5c45-46ec-a182-e8145a1c0faa/projects/mqttrust/mqttrust_core)
error[E0432]: unresolved import `state::BOXED_PUBLISH`
  --> mqttrust_core/src/lib.rs:21:25
   |
21 | use state::{StateError, BOXED_PUBLISH};
   |                         ^^^^^^^^^^^^^ no `BOXED_PUBLISH` in `state`

error: cannot find macro `pool` in this scope
  --> mqttrust_core/src/state.rs:43:1
   |
43 | pool!(BOXED_PUBLISH: PublishNotification);
   | ^^^^
   |
note: `pool` is imported here, but it is a module, not a macro
  --> mqttrust_core/src/state.rs:8:16
   |
8  | use heapless::{pool, pool::singleton::Pool};
   |                ^^^^

error[E0433]: failed to resolve: use of undeclared type `BOXED_PUBLISH`
  --> mqttrust_core/src/state.rs:87:9
   |
87 |         BOXED_PUBLISH::grow(unsafe { &mut PUBLISH_MEM });
   |         ^^^^^^^^^^^^^ use of undeclared type `BOXED_PUBLISH`

error[E0433]: failed to resolve: use of undeclared type `BOXED_PUBLISH`
   --> mqttrust_core/src/state.rs:256:29
    |
256 |         let boxed_publish = BOXED_PUBLISH::alloc().unwrap();
    |                             ^^^^^^^^^^^^^ use of undeclared type `BOXED_PUBLISH`

warning: unused import: `pool::singleton::Pool`
 --> mqttrust_core/src/state.rs:8:22
  |
8 | use heapless::{pool, pool::singleton::Pool};
  |                      ^^^^^^^^^^^^^^^^^^^^^
  |
  = note: `#[warn(unused_imports)]` on by default

Some errors have detailed explanations: E0432, E0433.
For more information about an error, try `rustc --explain E0432`.
warning: `mqttrust_core` (lib) generated 1 warning
error: could not compile `mqttrust_core` due to 4 previous errors; 1 warning emitted

[MathiasKoch]

Hi.

Could you try enabling the x86 feature? The issue you are seeing is actually an issue with the heapless library, but I think the x86 feature in mqttrust should fix it.

Edit: explanation. https://github.com/japaric/heapless/blob/e33646c189a8b039fed7475e0d0e0f5c736c855f/src/pool/singleton.rs#L17-L47

[jan-br]

thanks for the quick response. :)
Yes I've already tried that. But this does not change anything.
But I think that makes sense, since my esp is neither an x86 nor x64 processor like it would be required in this line:

https://github.com/japaric/heapless/blob/e33646c189a8b039fed7475e0d0e0f5c736c855f/src/pool/singleton.rs#L26

❯ cargo build --features "x86"                                                                                                                                             mqttrust_core -> master + ? !
   Compiling mqttrust_core v0.4.1 (/run/media/jan/e061bc91-5c45-46ec-a182-e8145a1c0faa/projects/mqttrust/mqttrust_core)
error[E0432]: unresolved import `state::BoxedPublish`
  --> mqttrust_core/src/lib.rs:21:13
   |
21 | use state::{BoxedPublish, StateError};
   |             ^^^^^^^^^^^^ no `BoxedPublish` in `state`

error: cannot find macro `pool` in this scope
  --> mqttrust_core/src/state.rs:43:1
   |
43 | pool!(
   | ^^^^
   |
note: `pool` is imported here, but it is a module, not a macro
  --> mqttrust_core/src/state.rs:8:16
   |
8  | use heapless::{pool, pool::singleton::Pool};
   |                ^^^^

error[E0433]: failed to resolve: use of undeclared type `BoxedPublish`
  --> mqttrust_core/src/state.rs:90:9
   |
90 |         BoxedPublish::grow(unsafe { &mut PUBLISH_MEM });
   |         ^^^^^^^^^^^^ use of undeclared type `BoxedPublish`

error[E0433]: failed to resolve: use of undeclared type `BoxedPublish`
   --> mqttrust_core/src/state.rs:275:29
    |
275 |         let boxed_publish = BoxedPublish::alloc().unwrap();
    |                             ^^^^^^^^^^^^ use of undeclared type `BoxedPublish`

warning: unused import: `pool::singleton::Pool`
 --> mqttrust_core/src/state.rs:8:22
  |
8 | use heapless::{pool, pool::singleton::Pool};
  |                      ^^^^^^^^^^^^^^^^^^^^^
  |
  = note: `#[warn(unused_imports)]` on by default

Some errors have detailed explanations: E0432, E0433.
For more information about an error, try `rustc --explain E0432`.
warning: `mqttrust_core` (lib) generated 1 warning
error: could not compile `mqttrust_core` due to 4 previous errors; 1 warning emitted

[MathiasKoch]

That makes sense. But in that case I guess the first step is to figure out what it takes to get heapless pools working with xtensa. After that I would happily update the heapless dependency version and help with a fix if anything more is needed 👍

[jan-br]

Hm well, I am not really sure about what is going on there.
I didn't use heapless that much, can you explain to me what this pool! macro is doing and for what reason it might not be available under xtensa?

[MathiasKoch]

The pool allocates a piece of fixed memory, from which we can create Boxes, just like std Box, but in a no_std environment.

Why it would not be available in xtensa, I have no idea. Might be as simple as the target arch needs to be added in the feature gating. I would try to open an issue in heapless, or ask in the embedded-rust matrix channel.

[jan-br]

Ah I see, thanks for the clarification.
So yes that probably would be a possible solution. I'll try to talk to the heapless maintainers about that.
But in my usecase, I have the full std environment.
So I can use the heap. Do you think a little feature flag, to not use the heapless pools, but the normal heap instead, might be a good idea?
That would be a quick fix for the problem and also a solution for other weird architectures that are not yet supported by heapless.

[MathiasKoch]

I would have no issue in a std feature flag 👍

[jan-br]

That sounds great. :) Would you mind taking a look at that when you have some time? Shouldn't be that much to implement if I see that right.

[jan-br]

Okay, so I tried to look into the heapless source.
And it seems like the xtensa processors should be able to execute the used CAS operations in the heapless pool.
But the heapless issue section does not seem to be that active and since I cant really mange do that myself, I dont expect the xtensa target to be implemented by anyone else soon.

So I guess, for now, my best bet to get rustot running under xtensa would be to implement the std feature for mqttrust.

[MathiasKoch]

I'd be happy to make the std feature, when I get the time.. might be I get half an hour tomorrow, otherwise it will be Tuesday.

[MathiasKoch]

@jan-br Could you give #44 a spin and see if that fixes your issue?

[jan-br]

Thanks alot. :) This actually compiles for xtensa. Didn't do any runtime tests, but I assume if that implementation works under your architecture, it should also work under the xtensa rust std library.
I cloned rustot and tried to compile it with mqttrust #44 but unfortunately there seem to be other compilation issues.
But this is out of scope for this issue. I'll create another one in the rustot repository. :)

[jan-br]

Okay sooo there is the same problem in mqttrust and rustot and therefore it might be in scope of this issue.

After #44 is merged and published, the mqttrust_core dependency should be updated with the std feature in order to enable compilation for the xtensa architecture.
This by itself would be enough to compile rustot for xtensa.

But it will not really be usable when used with the ESP-IDF, which is absolutely crucial to use features like Wifi/Bluetooth etc. on the ESP32 due to Espressifs proprietary source code.

Unfortunately the ESP-IDF Rust wrapper libraries such as esp-idf-hal which are basically mandatory to develop rust esp applications, use a newer version - 1.0.0-alpha.6 - of embedded-hal which is not compatible with mqttrust and rustot which both use 1.0.0-alpha.4.
So in order for this change to be really usable, both would have to use a compatible version.

[MathiasKoch]

Yeah, i am aware of that, but probably wont be able to address it until i can find the time to upgrade the embedded-hal dependency in our entire crate stack.

[jan-br]

Fair, I guess then I'll try to create a fork of both projects and try to make it work somehow for my tech stack and use that until your stack will be upgraded in the future. :)
Thanks for your help.

[MathiasKoch]

That sound great. You are also more than welcome to open a PR on both when you get it working.. that should expedite the process a bit for me as well

[jan-br]

Sure, I don't know if that fork will meet your code standards, but I'll let you know, when the projects work with version alpha.6 :)

[MathiasKoch]

@jan-br Just wanted to give you a heads up that i have run through my entire stack and updated dependencies..

So this should fix your issue as well, together with FactbirdHQ/rustot#37, that i will merge tomorrow after a release train.

[jan-br]

Oh thanks alot @MathiasKoch . :) I got it to compile with ESP-IDF too, with the updated dependencies.
I didn't create a PR yet, because I ran into further issues regarding the tls implementation for mqttrust and could not really test rustot. There are basically no tls crates that run under xtensa (yet).

So the only way I could see this working until this issue is resolved and deployed to use the experimental mqtt client of the rust esp-idf and create a custom implementation of the mqttrust traits.

[jan-br]

Also to confirm, your changes seem to be working for xtensa. :)

[MathiasKoch]

Ahh, yeah. TLS is a hard requirement for AWS.

Personally i am using an offloaded TCP-IP+TLS stack on my cellular + wifi devices. I would think xtensa has something similar?

An alternative i have seen in the community is FFI with mbed-tls

[jan-br]

yes I think esp-idf uses a modified version of mbed-tls.
But the rust wrapper does not have support for it standalone, so the idea was to instead implement that, try to use the existing experimental mqtt client which ships with the esp-idf rust wrapper and uses mbedtls in the C implementation of esp-idf. Thats what I was talking about earlier.
But sadly I couldn't figure out how to use that thing since there is basically zero documentation for it.
So maybe I'll have to take a look at that tls thing

[MathiasKoch]

Ahh, sad to hear that.

If you do figure it out, it should be fairly easy to implement the MQTT client trait here ontop of it in some version.

Let me know if you figure it out :) It's always nice to follow along on other users use of ones libraries :)