Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

panic if spawn when task drop #182

Open
MrCroxx opened this issue Nov 14, 2023 · 2 comments
Open

panic if spawn when task drop #182

MrCroxx opened this issue Nov 14, 2023 · 2 comments
Assignees
@wangrunji0408

Comments

@MrCroxx
Copy link
Contributor

MrCroxx commented Nov 14, 2023

In RisingWave deterministic test, got the following panic, which is caused by tokio::spawn when some struct drop during task drop.

thread '<unnamed>' panicked at /home/mrcroxx/.cargo/registry/src/index.crates.io-6f17d22bba15001f/madsim-0.2.22/src/sim/runtime/context.rs:27:44:
there is no reactor running, must be called from the context of a Madsim runtime
stack backtrace:
   0: rust_begin_unwind
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/std/src/panicking.rs:597:5
   1: core::panicking::panic_fmt
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/panicking.rs:72:14
   2: core::panicking::panic_display
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/panicking.rs:178:5
   3: core::panicking::panic_str
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/panicking.rs:152:5
   4: core::option::expect_failed
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/option.rs:1979:5
   5: core::option::Option<T>::expect
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/option.rs:888:21
   6: madsim::sim::runtime::context::current_task::{{closure}}
             at /home/mrcroxx/.cargo/registry/src/index.crates.io-6f17d22bba15001f/madsim-0.2.22/src/sim/runtime/context.rs:27:44
   7: std::thread::local::LocalKey<T>::try_with
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/std/src/thread/local.rs:270:16
   8: std::thread::local::LocalKey<T>::with
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/std/src/thread/local.rs:246:9
   9: madsim::sim::runtime::context::current_task
             at /home/mrcroxx/.cargo/registry/src/index.crates.io-6f17d22bba15001f/madsim-0.2.22/src/sim/runtime/context.rs:27:10
  10: madsim::sim::task::Spawner::current
             at /home/mrcroxx/.cargo/registry/src/index.crates.io-6f17d22bba15001f/madsim-0.2.22/src/sim/task/mod.rs:577:20
  11: madsim::sim::task::spawn
             at /home/mrcroxx/.cargo/registry/src/index.crates.io-6f17d22bba15001f/madsim-0.2.22/src/sim/task/mod.rs:655:5
  12: foyer_storage::storage::AsyncStorageExt::insert_if_not_exists_async
             at /home/mrcroxx/.cargo/git/checkouts/foyer-e7ae575671bf01ff/ce2e222/foyer-storage/src/storage.rs:216:9
  13: <risingwave_storage::hummock::sstable_store::BlockCacheEventListener as risingwave_common::cache::LruCacheEventListener>::on_release
             at ./src/storage/src/hummock/sstable_store.rs:110:9
  14: core::ptr::drop_in_place<risingwave_common::cache::CacheableEntry<(u64,u64),alloc::boxed::Box<risingwave_storage::hummock::sstable::block::Block>>>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
  15: core::ptr::drop_in_place<risingwave_storage::hummock::block_cache::BlockEntry>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
  16: core::ptr::drop_in_place<risingwave_storage::hummock::block_cache::BlockHolder>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
  17: core::ptr::drop_in_place<risingwave_storage::hummock::sstable::block_iterator::BlockIterator>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
  18: core::ptr::drop_in_place<core::option::Option<risingwave_storage::hummock::sstable::block_iterator::BlockIterator>>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
  19: core::ptr::drop_in_place<risingwave_storage::hummock::sstable::forward_sstable_iterator::SstableIterator>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
  20: core::ptr::drop_in_place<[risingwave_storage::hummock::iterator::merge_inner::Node<risingwave_storage::hummock::iterator::HummockIteratorUnion<risingwave_storage::hummock::iterator::Forward,risingwave_storage::hummock::iterator::merge_inner::MergeIteratorInner<risingwave_storage::hummock::iterator::HummockIteratorUnion<risingwave_storage::hummock::iterator::Forward,risingwave_storage::hummock::shared_buffer::shared_buffer_batch::SharedBufferBatchIterator<risingwave_storage::hummock::iterator::Forward>,risingwave_storage::hummock::sstable::forward_sstable_iterator::SstableIterator>,usize>,risingwave_storage::hummock::sstable::forward_sstable_iterator::SstableIterator,risingwave_storage::hummock::iterator::concat_inner::ConcatIteratorInner<risingwave_storage::hummock::sstable::forward_sstable_iterator::SstableIterator>>,()>]>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
  21: <alloc::vec::Vec<T,A> as core::ops::drop::Drop>::drop
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/alloc/src/vec/mod.rs:3067:13
  22: core::ptr::drop_in_place<alloc::vec::Vec<risingwave_storage::hummock::iterator::merge_inner::Node<risingwave_storage::hummock::iterator::HummockIteratorUnion<risingwave_storage::hummock::iterator::Forward,risingwave_storage::hummock::iterator::merge_inner::MergeIteratorInner<risingwave_storage::hummock::iterator::HummockIteratorUnion<risingwave_storage::hummock::iterator::Forward,risingwave_storage::hummock::shared_buffer::shared_buffer_batch::SharedBufferBatchIterator<risingwave_storage::hummock::iterator::Forward>,risingwave_storage::hummock::sstable::forward_sstable_iterator::SstableIterator>,usize>,risingwave_storage::hummock::sstable::forward_sstable_iterator::SstableIterator,risingwave_storage::hummock::iterator::concat_inner::ConcatIteratorInner<risingwave_storage::hummock::sstable::forward_sstable_iterator::SstableIterator>>,()>>>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
  23: core::ptr::drop_in_place<alloc::collections::binary_heap::BinaryHeap<risingwave_storage::hummock::iterator::merge_inner::Node<risingwave_storage::hummock::iterator::HummockIteratorUnion<risingwave_storage::hummock::iterator::Forward,risingwave_storage::hummock::iterator::merge_inner::MergeIteratorInner<risingwave_storage::hummock::iterator::HummockIteratorUnion<risingwave_storage::hummock::iterator::Forward,risingwave_storage::hummock::shared_buffer::shared_buffer_batch::SharedBufferBatchIterator<risingwave_storage::hummock::iterator::Forward>,risingwave_storage::hummock::sstable::forward_sstable_iterator::SstableIterator>,usize>,risingwave_storage::hummock::sstable::forward_sstable_iterator::SstableIterator,risingwave_storage::hummock::iterator::concat_inner::ConcatIteratorInner<risingwave_storage::hummock::sstable::forward_sstable_iterator::SstableIterator>>,()>>>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
  24: core::ptr::drop_in_place<risingwave_storage::hummock::iterator::merge_inner::MergeIteratorInner<risingwave_storage::hummock::iterator::HummockIteratorUnion<risingwave_storage::hummock::iterator::Forward,risingwave_storage::hummock::iterator::merge_inner::MergeIteratorInner<risingwave_storage::hummock::iterator::HummockIteratorUnion<risingwave_storage::hummock::iterator::Forward,risingwave_storage::hummock::shared_buffer::shared_buffer_batch::SharedBufferBatchIterator<risingwave_storage::hummock::iterator::Forward>,risingwave_storage::hummock::sstable::forward_sstable_iterator::SstableIterator>,usize>,risingwave_storage::hummock::sstable::forward_sstable_iterator::SstableIterator,risingwave_storage::hummock::iterator::concat_inner::ConcatIteratorInner<risingwave_storage::hummock::sstable::forward_sstable_iterator::SstableIterator>>,()>>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
  25: core::ptr::drop_in_place<risingwave_storage::hummock::iterator::forward_user::UserIterator<risingwave_storage::hummock::iterator::merge_inner::MergeIteratorInner<risingwave_storage::hummock::iterator::HummockIteratorUnion<risingwave_storage::hummock::iterator::Forward,risingwave_storage::hummock::iterator::merge_inner::MergeIteratorInner<risingwave_storage::hummock::iterator::HummockIteratorUnion<risingwave_storage::hummock::iterator::Forward,risingwave_storage::hummock::shared_buffer::shared_buffer_batch::SharedBufferBatchIterator<risingwave_storage::hummock::iterator::Forward>,risingwave_storage::hummock::sstable::forward_sstable_iterator::SstableIterator>,usize>,risingwave_storage::hummock::sstable::forward_sstable_iterator::SstableIterator,risingwave_storage::hummock::iterator::concat_inner::ConcatIteratorInner<risingwave_storage::hummock::sstable::forward_sstable_iterator::SstableIterator>>,()>>>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
  26: core::ptr::drop_in_place<risingwave_storage::hummock::store::local_hummock_storage::HummockStorageIterator>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
  27: core::ptr::drop_in_place<risingwave_storage::store::into_stream_inner<risingwave_storage::hummock::store::local_hummock_storage::HummockStorageIterator>::{{closure}}>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:52:1
  28: core::ptr::drop_in_place<core::option::Option<risingwave_storage::store::into_stream_inner<risingwave_storage::hummock::store::local_hummock_storage::HummockStorageIterator>::{{closure}}>>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
  29: core::ptr::drop_in_place<futures_async_stream::try_stream::GenTryStream<risingwave_storage::store::into_stream_inner<risingwave_storage::hummock::store::local_hummock_storage::HummockStorageIterator>::{{closure}}>>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
  30: core::ptr::drop_in_place<risingwave_storage::store_impl::verify::verify_stream<futures_async_stream::try_stream::GenTryStream<risingwave_storage::store::into_stream_inner<risingwave_storage::hummock::store::local_hummock_storage::HummockStorageIterator>::{{closure}}>,futures_async_stream::try_stream::GenTryStream<risingwave_storage::store::into_stream_inner<risingwave_storage::memory::RangeKvStateStoreIter<risingwave_storage::memory::sled::SledRangeKv>>::{{closure}}>>::{{closure}}>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:296:5
  31: core::ptr::drop_in_place<core::option::Option<risingwave_storage::store_impl::verify::verify_stream<futures_async_stream::try_stream::GenTryStream<risingwave_storage::store::into_stream_inner<risingwave_storage::hummock::store::local_hummock_storage::HummockStorageIterator>::{{closure}}>,futures_async_stream::try_stream::GenTryStream<risingwave_storage::store::into_stream_inner<risingwave_storage::memory::RangeKvStateStoreIter<risingwave_storage::memory::sled::SledRangeKv>>::{{closure}}>>::{{closure}}>>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
  32: core::ptr::drop_in_place<alloc::boxed::Box<dyn futures_core::stream::Stream+Item = core::result::Result<(risingwave_hummock_sdk::key::FullKey<bytes::bytes::Bytes>,bytes::bytes::Bytes),risingwave_storage::error::StorageError>+core::marker::Send>>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
  33: core::ptr::drop_in_place<core::pin::Pin<alloc::boxed::Box<dyn futures_core::stream::Stream+Item = core::result::Result<(risingwave_hummock_sdk::key::FullKey<bytes::bytes::Bytes>,bytes::bytes::Bytes),risingwave_storage::error::StorageError>+core::marker::Send>>>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
  34: core::ptr::drop_in_place<risingwave_storage::monitor::monitored_store::MonitoredStateStoreIter<core::pin::Pin<alloc::boxed::Box<dyn futures_core::stream::Stream+Item = core::result::Result<(risingwave_hummock_sdk::key::FullKey<bytes::bytes::Bytes>,bytes::bytes::Bytes),risingwave_storage::error::StorageError>+core::marker::Send>>>::into_stream_inner::{{closure}}>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:386:5
  35: core::ptr::drop_in_place<core::option::Option<risingwave_storage::monitor::monitored_store::MonitoredStateStoreIter<core::pin::Pin<alloc::boxed::Box<dyn futures_core::stream::Stream+Item = core::result::Result<(risingwave_hummock_sdk::key::FullKey<bytes::bytes::Bytes>,bytes::bytes::Bytes),risingwave_storage::error::StorageError>+core::marker::Send>>>::into_stream_inner::{{closure}}>>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
  36: core::ptr::drop_in_place<futures_async_stream::try_stream::GenTryStream<risingwave_storage::monitor::monitored_store::MonitoredStateStoreIter<core::pin::Pin<alloc::boxed::Box<dyn futures_core::stream::Stream+Item = core::result::Result<(risingwave_hummock_sdk::key::FullKey<bytes::bytes::Bytes>,bytes::bytes::Bytes),risingwave_storage::error::StorageError>+core::marker::Send>>>::into_stream_inner::{{closure}}>>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
  37: core::ptr::drop_in_place<tracing_futures::Instrumented<futures_async_stream::try_stream::GenTryStream<risingwave_storage::monitor::monitored_store::MonitoredStateStoreIter<core::pin::Pin<alloc::boxed::Box<dyn futures_core::stream::Stream+Item = core::result::Result<(risingwave_hummock_sdk::key::FullKey<bytes::bytes::Bytes>,bytes::bytes::Bytes),risingwave_storage::error::StorageError>+core::marker::Send>>>::into_stream_inner::{{closure}}>>>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
  38: core::ptr::drop_in_place<risingwave_storage::table::batch_table::storage_table::StorageTableInnerIterInner<risingwave_storage::monitor::monitored_store::MonitoredStateStore<alloc::boxed::Box<dyn risingwave_storage::store_impl::boxed_state_store::DynamicDispatchedStateStore>>,risingwave_common::util::value_encoding::EitherSerde>::into_stream::{{closure}}>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:695:5
  39: core::ptr::drop_in_place<core::option::Option<risingwave_storage::table::batch_table::storage_table::StorageTableInnerIterInner<risingwave_storage::monitor::monitored_store::MonitoredStateStore<alloc::boxed::Box<dyn risingwave_storage::store_impl::boxed_state_store::DynamicDispatchedStateStore>>,risingwave_common::util::value_encoding::EitherSerde>::into_stream::{{closure}}>>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
  40: core::ptr::drop_in_place<futures_async_stream::try_stream::GenTryStream<risingwave_storage::table::batch_table::storage_table::StorageTableInnerIterInner<risingwave_storage::monitor::monitored_store::MonitoredStateStore<alloc::boxed::Box<dyn risingwave_storage::store_impl::boxed_state_store::DynamicDispatchedStateStore>>,risingwave_common::util::value_encoding::EitherSerde>::into_stream::{{closure}}>>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
  41: core::ptr::drop_in_place<[futures_util::stream::futures_ordered::OrderWrapper<core::result::Result<futures_async_stream::try_stream::GenTryStream<risingwave_storage::table::batch_table::storage_table::StorageTableInnerIterInner<risingwave_storage::monitor::monitored_store::MonitoredStateStore<alloc::boxed::Box<dyn risingwave_storage::store_impl::boxed_state_store::DynamicDispatchedStateStore>>,risingwave_common::util::value_encoding::EitherSerde>::into_stream::{{closure}}>,risingwave_storage::error::StorageError>>]>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
  42: <alloc::vec::Vec<T,A> as core::ops::drop::Drop>::drop
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/alloc/src/vec/mod.rs:3067:13
  43: core::ptr::drop_in_place<alloc::vec::Vec<futures_util::stream::futures_ordered::OrderWrapper<core::result::Result<futures_async_stream::try_stream::GenTryStream<risingwave_storage::table::batch_table::storage_table::StorageTableInnerIterInner<risingwave_storage::monitor::monitored_store::MonitoredStateStore<alloc::boxed::Box<dyn risingwave_storage::store_impl::boxed_state_store::DynamicDispatchedStateStore>>,risingwave_common::util::value_encoding::EitherSerde>::into_stream::{{closure}}>,risingwave_storage::error::StorageError>>>>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
  44: core::ptr::drop_in_place<alloc::collections::binary_heap::BinaryHeap<futures_util::stream::futures_ordered::OrderWrapper<core::result::Result<futures_async_stream::try_stream::GenTryStream<risingwave_storage::table::batch_table::storage_table::StorageTableInnerIterInner<risingwave_storage::monitor::monitored_store::MonitoredStateStore<alloc::boxed::Box<dyn risingwave_storage::store_impl::boxed_state_store::DynamicDispatchedStateStore>>,risingwave_common::util::value_encoding::EitherSerde>::into_stream::{{closure}}>,risingwave_storage::error::StorageError>>>>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
  45: core::ptr::drop_in_place<futures_util::stream::futures_ordered::FuturesOrdered<futures_util::future::try_future::into_future::IntoFuture<risingwave_storage::table::batch_table::storage_table::StorageTableInner<risingwave_storage::monitor::monitored_store::MonitoredStateStore<alloc::boxed::Box<dyn risingwave_storage::store_impl::boxed_state_store::DynamicDispatchedStateStore>>,risingwave_common::util::value_encoding::EitherSerde>::iter_with_encoded_key_range::{{closure}}::{{closure}}::{{closure}}>>>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
  46: core::ptr::drop_in_place<futures_util::stream::try_stream::try_collect::TryCollect<futures_util::stream::futures_ordered::FuturesOrdered<futures_util::future::try_future::into_future::IntoFuture<risingwave_storage::table::batch_table::storage_table::StorageTableInner<risingwave_storage::monitor::monitored_store::MonitoredStateStore<alloc::boxed::Box<dyn risingwave_storage::store_impl::boxed_state_store::DynamicDispatchedStateStore>>,risingwave_common::util::value_encoding::EitherSerde>::iter_with_encoded_key_range::{{closure}}::{{closure}}::{{closure}}>>,alloc::vec::Vec<futures_async_stream::try_stream::GenTryStream<risingwave_storage::table::batch_table::storage_table::StorageTableInnerIterInner<risingwave_storage::monitor::monitored_store::MonitoredStateStore<alloc::boxed::Box<dyn risingwave_storage::store_impl::boxed_state_store::DynamicDispatchedStateStore>>,risingwave_common::util::value_encoding::EitherSerde>::into_stream::{{closure}}>>>>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
  47: core::ptr::drop_in_place<futures_util::future::try_join_all::TryJoinAllKind<risingwave_storage::table::batch_table::storage_table::StorageTableInner<risingwave_storage::monitor::monitored_store::MonitoredStateStore<alloc::boxed::Box<dyn risingwave_storage::store_impl::boxed_state_store::DynamicDispatchedStateStore>>,risingwave_common::util::value_encoding::EitherSerde>::iter_with_encoded_key_range::{{closure}}::{{closure}}::{{closure}}>>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
  48: core::ptr::drop_in_place<futures_util::future::try_join_all::TryJoinAll<risingwave_storage::table::batch_table::storage_table::StorageTableInner<risingwave_storage::monitor::monitored_store::MonitoredStateStore<alloc::boxed::Box<dyn risingwave_storage::store_impl::boxed_state_store::DynamicDispatchedStateStore>>,risingwave_common::util::value_encoding::EitherSerde>::iter_with_encoded_key_range::{{closure}}::{{closure}}::{{closure}}>>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
  49: core::ptr::drop_in_place<risingwave_storage::table::batch_table::storage_table::StorageTableInner<risingwave_storage::monitor::monitored_store::MonitoredStateStore<alloc::boxed::Box<dyn risingwave_storage::store_impl::boxed_state_store::DynamicDispatchedStateStore>>,risingwave_common::util::value_encoding::EitherSerde>::iter_with_encoded_key_range::{{closure}}>
             at ./src/storage/src/table/batch_table/storage_table.rs:469:10
  50: core::ptr::drop_in_place<risingwave_storage::table::batch_table::storage_table::StorageTableInner<risingwave_storage::monitor::monitored_store::MonitoredStateStore<alloc::boxed::Box<dyn risingwave_storage::store_impl::boxed_state_store::DynamicDispatchedStateStore>>,risingwave_common::util::value_encoding::EitherSerde>::iter_with_pk_bounds<&risingwave_common::row::owned_row::OwnedRow,(core::ops::range::Bound<risingwave_common::row::owned_row::OwnedRow>,core::ops::range::Bound<risingwave_common::row::owned_row::OwnedRow>)>::{{closure}}>
             at ./src/storage/src/table/batch_table/storage_table.rs:604:10
  51: core::ptr::drop_in_place<risingwave_storage::table::batch_table::storage_table::StorageTableInner<risingwave_storage::monitor::monitored_store::MonitoredStateStore<alloc::boxed::Box<dyn risingwave_storage::store_impl::boxed_state_store::DynamicDispatchedStateStore>>,risingwave_common::util::value_encoding::EitherSerde>::batch_iter_with_pk_bounds<&risingwave_common::row::owned_row::OwnedRow,(core::ops::range::Bound<risingwave_common::row::owned_row::OwnedRow>,core::ops::range::Bound<risingwave_common::row::owned_row::OwnedRow>)>::{{closure}}>
             at ./src/storage/src/table/batch_table/storage_table.rs:618:14
  52: core::ptr::drop_in_place<risingwave_batch::executor::row_seq_scan::RowSeqScanExecutor<risingwave_storage::monitor::monitored_store::MonitoredStateStore<alloc::boxed::Box<dyn risingwave_storage::store_impl::boxed_state_store::DynamicDispatchedStateStore>>>::execute_range<risingwave_common::metrics::guarded_metrics::LabelGuardedMetric<prometheus::histogram::Histogram,4_usize>>::{{closure}}>
             at ./src/batch/src/executor/row_seq_scan.rs:421:18
  53: core::ptr::drop_in_place<core::option::Option<risingwave_batch::executor::row_seq_scan::RowSeqScanExecutor<risingwave_storage::monitor::monitored_store::MonitoredStateStore<alloc::boxed::Box<dyn risingwave_storage::store_impl::boxed_state_store::DynamicDispatchedStateStore>>>::execute_range<risingwave_common::metrics::guarded_metrics::LabelGuardedMetric<prometheus::histogram::Histogram,4_usize>>::{{closure}}>>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
  54: core::ptr::drop_in_place<futures_async_stream::try_stream::GenTryStream<risingwave_batch::executor::row_seq_scan::RowSeqScanExecutor<risingwave_storage::monitor::monitored_store::MonitoredStateStore<alloc::boxed::Box<dyn risingwave_storage::store_impl::boxed_state_store::DynamicDispatchedStateStore>>>::execute_range<risingwave_common::metrics::guarded_metrics::LabelGuardedMetric<prometheus::histogram::Histogram,4_usize>>::{{closure}}>>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
  55: core::ptr::drop_in_place<alloc::boxed::Box<futures_async_stream::try_stream::GenTryStream<risingwave_batch::executor::row_seq_scan::RowSeqScanExecutor<risingwave_storage::monitor::monitored_store::MonitoredStateStore<alloc::boxed::Box<dyn risingwave_storage::store_impl::boxed_state_store::DynamicDispatchedStateStore>>>::execute_range<risingwave_common::metrics::guarded_metrics::LabelGuardedMetric<prometheus::histogram::Histogram,4_usize>>::{{closure}}>>>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
  56: core::ptr::drop_in_place<core::pin::Pin<alloc::boxed::Box<futures_async_stream::try_stream::GenTryStream<risingwave_batch::executor::row_seq_scan::RowSeqScanExecutor<risingwave_storage::monitor::monitored_store::MonitoredStateStore<alloc::boxed::Box<dyn risingwave_storage::store_impl::boxed_state_store::DynamicDispatchedStateStore>>>::execute_range<risingwave_common::metrics::guarded_metrics::LabelGuardedMetric<prometheus::histogram::Histogram,4_usize>>::{{closure}}>>>>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
  57: core::ptr::drop_in_place<core::option::Option<core::pin::Pin<alloc::boxed::Box<futures_async_stream::try_stream::GenTryStream<risingwave_batch::executor::row_seq_scan::RowSeqScanExecutor<risingwave_storage::monitor::monitored_store::MonitoredStateStore<alloc::boxed::Box<dyn risingwave_storage::store_impl::boxed_state_store::DynamicDispatchedStateStore>>>::execute_range<risingwave_common::metrics::guarded_metrics::LabelGuardedMetric<prometheus::histogram::Histogram,4_usize>>::{{closure}}>>>>>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
  58: core::ptr::drop_in_place<futures_util::stream::stream::into_future::StreamFuture<core::pin::Pin<alloc::boxed::Box<futures_async_stream::try_stream::GenTryStream<risingwave_batch::executor::row_seq_scan::RowSeqScanExecutor<risingwave_storage::monitor::monitored_store::MonitoredStateStore<alloc::boxed::Box<dyn risingwave_storage::store_impl::boxed_state_store::DynamicDispatchedStateStore>>>::execute_range<risingwave_common::metrics::guarded_metrics::LabelGuardedMetric<prometheus::histogram::Histogram,4_usize>>::{{closure}}>>>>>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
  59: core::ptr::drop_in_place<core::option::Option<futures_util::stream::stream::into_future::StreamFuture<core::pin::Pin<alloc::boxed::Box<futures_async_stream::try_stream::GenTryStream<risingwave_batch::executor::row_seq_scan::RowSeqScanExecutor<risingwave_storage::monitor::monitored_store::MonitoredStateStore<alloc::boxed::Box<dyn risingwave_storage::store_impl::boxed_state_store::DynamicDispatchedStateStore>>>::execute_range<risingwave_common::metrics::guarded_metrics::LabelGuardedMetric<prometheus::histogram::Histogram,4_usize>>::{{closure}}>>>>>>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
  60: futures_util::stream::futures_unordered::FuturesUnordered<Fut>::release_task
             at /home/mrcroxx/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.28/src/stream/futures_unordered/mod.rs:266:13
  61: futures_util::stream::futures_unordered::FuturesUnordered<Fut>::clear_head_all
             at /home/mrcroxx/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.28/src/stream/futures_unordered/mod.rs:573:13
  62: <futures_util::stream::futures_unordered::FuturesUnordered<Fut> as core::ops::drop::Drop>::drop
             at /home/mrcroxx/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.28/src/stream/futures_unordered/mod.rs:584:9
  63: core::ptr::drop_in_place<futures_util::stream::futures_unordered::FuturesUnordered<futures_util::stream::stream::into_future::StreamFuture<core::pin::Pin<alloc::boxed::Box<futures_async_stream::try_stream::GenTryStream<risingwave_batch::executor::row_seq_scan::RowSeqScanExecutor<risingwave_storage::monitor::monitored_store::MonitoredStateStore<alloc::boxed::Box<dyn risingwave_storage::store_impl::boxed_state_store::DynamicDispatchedStateStore>>>::execute_range<risingwave_common::metrics::guarded_metrics::LabelGuardedMetric<prometheus::histogram::Histogram,4_usize>>::{{closure}}>>>>>>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
  64: core::ptr::drop_in_place<futures_util::stream::select_all::SelectAll<core::pin::Pin<alloc::boxed::Box<futures_async_stream::try_stream::GenTryStream<risingwave_batch::executor::row_seq_scan::RowSeqScanExecutor<risingwave_storage::monitor::monitored_store::MonitoredStateStore<alloc::boxed::Box<dyn risingwave_storage::store_impl::boxed_state_store::DynamicDispatchedStateStore>>>::execute_range<risingwave_common::metrics::guarded_metrics::LabelGuardedMetric<prometheus::histogram::Histogram,4_usize>>::{{closure}}>>>>>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
  65: core::ptr::drop_in_place<risingwave_batch::executor::row_seq_scan::RowSeqScanExecutor<risingwave_storage::monitor::monitored_store::MonitoredStateStore<alloc::boxed::Box<dyn risingwave_storage::store_impl::boxed_state_store::DynamicDispatchedStateStore>>>::do_execute::{{closure}}>
             at ./src/batch/src/executor/row_seq_scan.rs:299:50
  66: core::ptr::drop_in_place<core::option::Option<risingwave_batch::executor::row_seq_scan::RowSeqScanExecutor<risingwave_storage::monitor::monitored_store::MonitoredStateStore<alloc::boxed::Box<dyn risingwave_storage::store_impl::boxed_state_store::DynamicDispatchedStateStore>>>::do_execute::{{closure}}>>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
  67: core::ptr::drop_in_place<alloc::boxed::Box<dyn futures_core::stream::Stream+Item = core::result::Result<risingwave_common::array::data_chunk::DataChunk,risingwave_common::error::RwError>+core::marker::Send>>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
  68: core::ptr::drop_in_place<core::pin::Pin<alloc::boxed::Box<dyn futures_core::stream::Stream+Item = core::result::Result<risingwave_common::array::data_chunk::DataChunk,risingwave_common::error::RwError>+core::marker::Send>>>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
  69: core::ptr::drop_in_place<<risingwave_batch::executor::managed::ManagedExecutor as risingwave_batch::executor::Executor>::execute::{{closure}}>
             at ./src/batch/src/executor/managed.rs:82:5
  70: core::ptr::drop_in_place<core::option::Option<<risingwave_batch::executor::managed::ManagedExecutor as risingwave_batch::executor::Executor>::execute::{{closure}}>>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
  71: core::ptr::drop_in_place<alloc::boxed::Box<dyn futures_core::stream::Stream+Item = core::result::Result<risingwave_common::array::data_chunk::DataChunk,risingwave_common::error::RwError>+core::marker::Send>>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
  72: core::ptr::drop_in_place<core::pin::Pin<alloc::boxed::Box<dyn futures_core::stream::Stream+Item = core::result::Result<risingwave_common::array::data_chunk::DataChunk,risingwave_common::error::RwError>+core::marker::Send>>>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
  73: core::ptr::drop_in_place<risingwave_batch::executor::filter::FilterExecutor::do_execute::{{closure}}>
             at ./src/batch/src/executor/filter.rs:53:57
  74: core::ptr::drop_in_place<core::option::Option<risingwave_batch::executor::filter::FilterExecutor::do_execute::{{closure}}>>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
  75: core::ptr::drop_in_place<alloc::boxed::Box<dyn futures_core::stream::Stream+Item = core::result::Result<risingwave_common::array::data_chunk::DataChunk,risingwave_common::error::RwError>+core::marker::Send>>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
  76: core::ptr::drop_in_place<core::pin::Pin<alloc::boxed::Box<dyn futures_core::stream::Stream+Item = core::result::Result<risingwave_common::array::data_chunk::DataChunk,risingwave_common::error::RwError>+core::marker::Send>>>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
  77: core::ptr::drop_in_place<<risingwave_batch::executor::managed::ManagedExecutor as risingwave_batch::executor::Executor>::execute::{{closure}}>
             at ./src/batch/src/executor/managed.rs:82:5
  78: core::ptr::drop_in_place<core::option::Option<<risingwave_batch::executor::managed::ManagedExecutor as risingwave_batch::executor::Executor>::execute::{{closure}}>>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
  79: core::ptr::drop_in_place<alloc::boxed::Box<dyn futures_core::stream::Stream+Item = core::result::Result<risingwave_common::array::data_chunk::DataChunk,risingwave_common::error::RwError>+core::marker::Send>>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
  80: core::ptr::drop_in_place<core::pin::Pin<alloc::boxed::Box<dyn futures_core::stream::Stream+Item = core::result::Result<risingwave_common::array::data_chunk::DataChunk,risingwave_common::error::RwError>+core::marker::Send>>>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
  81: core::ptr::drop_in_place<futures_util::stream::stream::map::Map<core::pin::Pin<alloc::boxed::Box<dyn futures_core::stream::Stream+Item = core::result::Result<risingwave_common::array::data_chunk::DataChunk,risingwave_common::error::RwError>+core::marker::Send>>,risingwave_batch::executor::project::ProjectExecutor::do_execute::{{closure}}>>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
  82: core::ptr::drop_in_place<futures_util::stream::stream::fuse::Fuse<futures_util::stream::stream::map::Map<core::pin::Pin<alloc::boxed::Box<dyn futures_core::stream::Stream+Item = core::result::Result<risingwave_common::array::data_chunk::DataChunk,risingwave_common::error::RwError>+core::marker::Send>>,risingwave_batch::executor::project::ProjectExecutor::do_execute::{{closure}}>>>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
  83: core::ptr::drop_in_place<futures_util::stream::stream::buffered::Buffered<futures_util::stream::stream::map::Map<core::pin::Pin<alloc::boxed::Box<dyn futures_core::stream::Stream+Item = core::result::Result<risingwave_common::array::data_chunk::DataChunk,risingwave_common::error::RwError>+core::marker::Send>>,risingwave_batch::executor::project::ProjectExecutor::do_execute::{{closure}}>>>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
  84: core::ptr::drop_in_place<alloc::boxed::Box<dyn futures_core::stream::Stream+Item = core::result::Result<risingwave_common::array::data_chunk::DataChunk,risingwave_common::error::RwError>+core::marker::Send>>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
  85: core::ptr::drop_in_place<core::pin::Pin<alloc::boxed::Box<dyn futures_core::stream::Stream+Item = core::result::Result<risingwave_common::array::data_chunk::DataChunk,risingwave_common::error::RwError>+core::marker::Send>>>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
  86: core::ptr::drop_in_place<<risingwave_batch::executor::managed::ManagedExecutor as risingwave_batch::executor::Executor>::execute::{{closure}}>
             at ./src/batch/src/executor/managed.rs:82:5
  87: core::ptr::drop_in_place<core::option::Option<<risingwave_batch::executor::managed::ManagedExecutor as risingwave_batch::executor::Executor>::execute::{{closure}}>>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
  88: core::ptr::drop_in_place<alloc::boxed::Box<dyn futures_core::stream::Stream+Item = core::result::Result<risingwave_common::array::data_chunk::DataChunk,risingwave_common::error::RwError>+core::marker::Send>>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
  89: core::ptr::drop_in_place<core::pin::Pin<alloc::boxed::Box<dyn futures_core::stream::Stream+Item = core::result::Result<risingwave_common::array::data_chunk::DataChunk,risingwave_common::error::RwError>+core::marker::Send>>>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
  90: core::ptr::drop_in_place<risingwave_batch::task::task_execution::BatchTaskExecution<risingwave_batch::task::context::ComputeNodeContext>::run::{{closure}}>
             at ./src/batch/src/task/task_execution.rs:681:5
  91: core::ptr::drop_in_place<tracing::instrument::Instrumented<risingwave_batch::task::task_execution::BatchTaskExecution<risingwave_batch::task::context::ComputeNodeContext>::run::{{closure}}>>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
  92: core::ptr::drop_in_place<risingwave_batch::task::task_execution::BatchTaskExecution<risingwave_batch::task::context::ComputeNodeContext>::async_execute::{{closure}}::{{closure}}::{{closure}}::{{closure}}>
             at ./src/batch/src/task/task_execution.rs:474:22
  93: core::ptr::drop_in_place<tokio_metrics::task::Instrumented<risingwave_batch::task::task_execution::BatchTaskExecution<risingwave_batch::task::context::ComputeNodeContext>::async_execute::{{closure}}::{{closure}}::{{closure}}::{{closure}}>>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
  94: core::ptr::drop_in_place<core::panic::unwind_safe::AssertUnwindSafe<tokio_metrics::task::Instrumented<risingwave_batch::task::task_execution::BatchTaskExecution<risingwave_batch::task::context::ComputeNodeContext>::async_execute::{{closure}}::{{closure}}::{{closure}}::{{closure}}>>>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
  95: core::ptr::drop_in_place<futures_util::future::future::catch_unwind::CatchUnwind<core::panic::unwind_safe::AssertUnwindSafe<tokio_metrics::task::Instrumented<risingwave_batch::task::task_execution::BatchTaskExecution<risingwave_batch::task::context::ComputeNodeContext>::async_execute::{{closure}}::{{closure}}::{{closure}}::{{closure}}>>>>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
  96: core::ptr::drop_in_place<core::option::Option<futures_util::future::future::catch_unwind::CatchUnwind<core::panic::unwind_safe::AssertUnwindSafe<tokio_metrics::task::Instrumented<risingwave_batch::task::task_execution::BatchTaskExecution<risingwave_batch::task::context::ComputeNodeContext>::async_execute::{{closure}}::{{closure}}::{{closure}}::{{closure}}>>>>>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
  97: core::pin::Pin<P>::set
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/pin.rs:754:9
  98: tokio::task::task_local::_::<impl core::ops::drop::Drop for tokio::task::task_local::TaskLocalFuture<T,F>>::drop::__drop_inner::{{closure}}
             at /home/mrcroxx/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.32.0/src/task/task_local.rs:328:21
  99: tokio::task::task_local::LocalKey<T>::scope_inner
             at /home/mrcroxx/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.32.0/src/task/task_local.rs:217:19
 100: tokio::task::task_local::_::<impl core::ops::drop::Drop for tokio::task::task_local::TaskLocalFuture<T,F>>::drop::__drop_inner
             at /home/mrcroxx/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.32.0/src/task/task_local.rs:327:25
 101: tokio::task::task_local::_::<impl core::ops::drop::Drop for tokio::task::task_local::TaskLocalFuture<T,F>>::drop
             at /home/mrcroxx/.cargo/registry/src/index.crates.io-6f17d22bba15001f/pin-project-lite-0.2.13/src/lib.rs:1300:17
 102: core::ptr::drop_in_place<tokio::task::task_local::TaskLocalFuture<(),futures_util::future::future::catch_unwind::CatchUnwind<core::panic::unwind_safe::AssertUnwindSafe<tokio_metrics::task::Instrumented<risingwave_batch::task::task_execution::BatchTaskExecution<risingwave_batch::task::context::ComputeNodeContext>::async_execute::{{closure}}::{{closure}}::{{closure}}::{{closure}}>>>>>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
 103: core::ptr::drop_in_place<risingwave_batch::task::task_execution::BatchTaskExecution<risingwave_batch::task::context::ComputeNodeContext>::async_execute::{{closure}}::{{closure}}>
             at ./src/batch/src/task/task_execution.rs:481:73
 104: core::ptr::drop_in_place<madsim::sim::task::Spawner::spawn_inner<risingwave_batch::task::task_execution::BatchTaskExecution<risingwave_batch::task::context::ComputeNodeContext>::async_execute::{{closure}}::{{closure}}>::{{closure}}::{{closure}}>
             at /home/mrcroxx/.cargo/registry/src/index.crates.io-6f17d22bba15001f/madsim-0.2.22/src/sim/task/mod.rs:629:28
 105: core::mem::manually_drop::ManuallyDrop<T>::drop
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/mem/manually_drop.rs:144:18
 106: <async_task::runnable::Builder<M>::spawn_local::Checked<F> as core::ops::drop::Drop>::drop
             at /home/mrcroxx/.cargo/registry/src/index.crates.io-6f17d22bba15001f/async-task-4.4.0/src/runnable.rs:442:21
 107: core::ptr::drop_in_place<async_task::runnable::Builder<M>::spawn_local::Checked<madsim::sim::task::Spawner::spawn_inner<risingwave_batch::task::task_execution::BatchTaskExecution<risingwave_batch::task::context::ComputeNodeContext>::async_execute::{{closure}}::{{closure}}>::{{closure}}::{{closure}}>>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
 108: core::ptr::drop_in_place<alloc::boxed::Box<async_task::runnable::Builder<M>::spawn_local::Checked<madsim::sim::task::Spawner::spawn_inner<risingwave_batch::task::task_execution::BatchTaskExecution<risingwave_batch::task::context::ComputeNodeContext>::async_execute::{{closure}}::{{closure}}>::{{closure}}::{{closure}}>>>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
 109: core::ptr::drop_in_place<core::pin::Pin<alloc::boxed::Box<async_task::runnable::Builder<M>::spawn_local::Checked<madsim::sim::task::Spawner::spawn_inner<risingwave_batch::task::task_execution::BatchTaskExecution<risingwave_batch::task::context::ComputeNodeContext>::async_execute::{{closure}}::{{closure}}>::{{closure}}::{{closure}}>>>>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
 110: core::ptr::mut_ptr::<impl *mut T>::drop_in_place
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mut_ptr.rs:1461:18
 111: async_task::raw::RawTask<F,T,S,M>::drop_future::{{closure}}
             at /home/mrcroxx/.cargo/registry/src/index.crates.io-6f17d22bba15001f/async-task-4.4.0/src/raw.rs:454:24
 112: async_task::utils::abort_on_panic
             at /home/mrcroxx/.cargo/registry/src/index.crates.io-6f17d22bba15001f/async-task-4.4.0/src/utils.rs:34:13
 113: async_task::raw::RawTask<F,T,S,M>::drop_future
             at /home/mrcroxx/.cargo/registry/src/index.crates.io-6f17d22bba15001f/async-task-4.4.0/src/raw.rs:453:9
 114: <async_task::runnable::Runnable<M> as core::ops::drop::Drop>::drop
             at /home/mrcroxx/.cargo/registry/src/index.crates.io-6f17d22bba15001f/async-task-4.4.0/src/runnable.rs:850:13
 115: core::ptr::drop_in_place<async_task::runnable::Runnable<alloc::sync::Weak<madsim::sim::task::TaskInfo>>>
             at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/ptr/mod.rs:498:1
 116: madsim::sim::task::Executor::run_all_ready
             at /home/mrcroxx/.cargo/registry/src/index.crates.io-6f17d22bba15001f/madsim-0.2.22/src/sim/task/mod.rs:306:9
 117: madsim::sim::task::Executor::block_on
             at /home/mrcroxx/.cargo/registry/src/index.crates.io-6f17d22bba15001f/madsim-0.2.22/src/sim/task/mod.rs:238:13
 118: madsim::sim::runtime::Runtime::block_on
             at /home/mrcroxx/.cargo/registry/src/index.crates.io-6f17d22bba15001f/madsim-0.2.22/src/sim/runtime/mod.rs:126:9
 119: madsim::sim::runtime::builder::Builder::run::{{closure}}::{{closure}}::{{closure}}
             at /home/mrcroxx/.cargo/registry/src/index.crates.io-6f17d22bba15001f/madsim-0.2.22/src/sim/runtime/builder.rs:128:35
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
thread '<unnamed>' panicked at /home/mrcroxx/.cargo/registry/src/index.crates.io-6f17d22bba15001f/async-task-4.4.0/src/utils.rs:17:5:
aborting the process
stack backtrace:
   0:     0x55cc0209ce4c - std::backtrace_rs::backtrace::libunwind::trace::he8dc9e634c3d5b69
                               at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/std/src/../../backtrace/src/backtrace/libunwind.rs:93:5
   1:     0x55cc0209ce4c - std::backtrace_rs::backtrace::trace_unsynchronized::hcc1fc094cc2c1843
                               at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
   2:     0x55cc0209ce4c - std::sys_common::backtrace::_print_fmt::he6c8e9059e21630d
                               at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/std/src/sys_common/backtrace.rs:67:5
   3:     0x55cc0209ce4c - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h145361bce15f4a6a
                               at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/std/src/sys_common/backtrace.rs:44:22
   4:     0x55cc020d2ee0 - core::fmt::rt::Argument::fmt::hb890552c7d6e8c2b
                               at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/fmt/rt.rs:142:9
   5:     0x55cc020d2ee0 - core::fmt::write::h97d71643815ece8a
                               at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/core/src/fmt/mod.rs:1117:17
   6:     0x55cc020979ef - std::io::Write::write_fmt::h9145d12d2fb312aa
                               at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/std/src/io/mod.rs:1762:15
   7:     0x55cc0209cc34 - std::sys_common::backtrace::_print::h34d8c7ca46068513
                               at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/std/src/sys_common/backtrace.rs:47:5
   8:     0x55cc0209cc34 - std::sys_common::backtrace::print::h69de4357a88b4f9d
                               at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/std/src/sys_common/backtrace.rs:34:9
   9:     0x55cc0209e3e7 - std::panicking::default_hook::{{closure}}::hbf7d8b588847f0d6
  10:     0x55cc0209e14f - std::panicking::default_hook::h797e21049b5f2e1d
                               at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/std/src/panicking.rs:292:9
  11:     0x55cc0209e8d8 - std::panicking::rust_panic_with_hook::h4b1939882c095f9c
                               at /rustc/249624b5043013d18c00f0401ca431c1a6baa8cd/library/std/src/panicking.rs:731:13
malloc_consolidate(): unaligned fastbin chunk detected
@MrCroxx MrCroxx mentioned this issue Nov 14, 2023
Merged
8 tasks
@wangrunji0408
Copy link
Member

The reason of this panic is that madsim needs to know which node is spawning the task. But when a task is dropping, the global context is empty. In other words, madsim doesn't run any task at that time, so it can't know who is spawning the task. This seems to be a bug since this behavior is normal on tokio. But I don't have a clear solution yet.

@wangrunji0408
Copy link
Member

Let me construct a minimum reproducible example first.

@wangrunji0408 wangrunji0408 self-assigned this Nov 14, 2023
@kwannoel kwannoel mentioned this issue Oct 7, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@wangrunji0408 wangrunji0408

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@wangrunji0408 @MrCroxx