reserved_space larger than allocated entry error in write heavy scenarios

[alexghr]

I'm seeing these errors reserved_space larger than allocated entry 2 15 507 0 2 2 pop up in write heavy scenarios where I'm using lmdb in a queue-like fashion. Eventually this leads to a crash in the process where it can't write to the underlying database anymore.

I have prepared this isolated test case that's exhibiting the same behaviour alexghr/reserved_space_error and I'd be interested if anyone has any insights into what I'm doing wrong here (or ideas to debug this further).

I have tested this both on Node 18.19 (old LTS) and 22.12 (current) on my PC running Linux.

One workaround I have found is to delete items one-by-one instead of batches of 100 items at a time (ie. use asyncPool(1, ...) on this line in test.mjs)

[kriszyp]

Thank you for putting this together! However, I haven't been able to reproduce with this test case yet. Does this consistently reproduce with this test case? I have tried bumping up the run length and queues and still haven't seen the issue manifest (and I am actually running on linux/Ubuntu as well). And are there any other error messages or codes when it "leads to a crash in the process"?

[alexghr]

Interestingly enough, I've restarted my computer yesterday and now running the test script I can't get it to print the errors even after increasing the batch size, max items and the frequency of writes.

I'm wondering if this might be due to a long running system? My development env is based on a long-running devcontainer so that's where I ran the script in. I've attached logs from before one of the crashes.

This was using lmdb-js 3.2.0

logs.json