-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Address Sanitizer on Windows reports a buffer overflow in Halide::serialize_pipeline
#8426
Comments
A repro case would help :-) |
Some more detail on the complaint:
|
It looks like it has to be the insert call here:
But I don't see what could be wrong with it. It's a bit weird to use begin() instead of end(), but the spec says those are equal for an empty container. |
It's likely this is a false positive. I believe in this case Halide.dll was compiled without asan, but the generator was compiled with asan, and the std::vector in question is one that was created in the generator and passed into Halide.dll to be resized and filled. When Halide resizes it, it probably doesn't set up the right asan tracking state to catch overflows. |
This diff adds a new interface to workaround the issue. |
I think this is just a complex case of: don't allocate memory in a dll and free it in an exe (or vice versa), because those may be separate heaps with separate settings. I know there are cases where we deliberately avoided doing this. Maybe there's some way to catch all such cases instead of playing whack-a-mole. Passing stl types by reference definitely seems problematic on windows - we could forbid that, but without testing it's going to creep back in. The cleanest workaround I think is to change it to just return a pimpl intrusive pointer type like Halide::Buffer<uint8_t>, so that no reallocations happen in the exe, and the constructor and destructor both happen in the dll |
I like the idea of returning a Halide::Buffer. We could perhaps offer a type alias for that buffer for the (de)serialization interface(s). There may be other places where this problem is prone to happen ( |
Halide::serialize_pipeline
Halide::serialize_pipeline
An internal Adobe user reports that running a generator executable results in a container-overflow in the serialization code. Specifically, the offending code appears to be in
Halide::Internal::Serializer::serialize(class Halide::Pipeline const &, class std::vector<unsigned char, class std::allocator<unsigned char>> &)
.The text was updated successfully, but these errors were encountered: