Using signs outside ASCII in response body #468
Comments
|
Yeah that sounds like interop problems would be the first thing to check.
We should definitely have some tests using non-ASCII characters and we
don't, so that's the first thing to add.
…On Sat, 15 Jul 2023, 17:44 Dominik Jeske, ***@***.***> wrote:
I'm testing pact-net and I spotted weird behavior when using signs outside
ASCII table. I was using my native letters from my country Poland (like
ł,ó,ń) - they are UTF-8 compatible but when I'm using them in WithJsonBody
mock that is used under the cover is returning empty body. I cannot find in
documentation is this by design or is it bug. I suspect some but when
interop with native library is used.
—
Reply to this email directly, view it on GitHub
<#468>, or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAD4FKQTKYXME3AX6SIG3B3XQLCINANCNFSM6AAAAAA2LMLMWI>
.
You are receiving this because you are subscribed to this thread.Message
ID: ***@***.***>
|
|
I've tested this out locally and it breaks pretty badly with non-ASCII characters in things like consumer/provider names, interaction names, provider states, etc. It generally seems to fail everywhere because the FFI reports invalid UTF-8. That's a bit of a problem given C# strings are UTF-16 and the P/Invoke options are just "Unicode". We may have to marshall strings manually or something. You also get completely garbled responses back when you try to use the Unicode option to retrieve things like server logs:
The difference is that I get errors whereas you report that the test passes but then silently contains no data. I've not managed to reproduce that, although I could definitely see that being a possibility. I can only apologise for that, and recommend that in the meantime you only use ASCII characters until the issue can be resolved. |
Instead of trying to marshal strings, which don't marshal nicely over the FFI boundary because C# uses UTF-16 but Rust wants UTF-8, instead explicitly convert strings to a UTF-8 `byte[]` and marshal those. Some places don't need to allow non-ASCII, such as the scheme in URLs, whereas others are very tricky, such as consumer filters. This would change the API to `byte[][]` and those can't be marshalled, so some parts still support non-ASCII for now. If that's a problem in the future then some custom marshalling could be implemented, but currently that seems overkill.
|
The linked PR fixes the issues on Windows, but is writing unreadable chars on Linux, so not sure what's happening there. |
See: https://learn.microsoft.com/en-us/dotnet/standard/native-interop/pinvoke-source-generation This is only supported on .Net 7+ and so older versions will still use the old style `extern` support, which is much more difficult to use with non-ASCII character sets when interacting with Rust via FFI. This means that older .Net versions will still not support non-ASCII properly. New style source generation handles marshalling strings as UTF-8 properly and efficiently so that non-ASCII characters can be used. This fixes #468.
I'm testing pact-net and I spotted weird behavior when using signs outside ASCII table. I was using my native letters from my country Poland (like ł,ó,ń) - they are UTF-8 compatible but when I'm using them in WithJsonBody mock that is used under the cover is returning empty body. I cannot find in documentation is this by design or is it bug. I suspect some but when interop with native library is used.
The text was updated successfully, but these errors were encountered: