Skip to content

Commit

Permalink
[Mongo] Clear the streamID from the request/response maps after stitc…
Browse files Browse the repository at this point in the history
…hing (#1878)

Summary: This PR modifies mongo's stitcher logic to clear streamID’s
from request/response maps once all frames of a streamID have been
consumed. We observed high CPU use allocated to `FramesSize()` &
`EraseExpiredFrames()`, this was due to the size of the maps increasing
with new streamIDs and having to continuously loop over those growing
maps to cleanup. Clearing the streamID's from the maps after stitching
significantly reduces the CPU allocated to the cleanup logic for mongo,
the exact details for mongo's streamID reuse needs to be determined to
further adapt the stitcher logic.

Type of change: /kind bug

Test Plan: Existing tests still pass, ran the px-mongo demo and observed
lower CPU allocated to mongo in the PEM through flamegraph.

---------

Signed-off-by: Kartik Pattaswamy <[email protected]>
  • Loading branch information
kpattaswamy authored Apr 24, 2024
1 parent 48d20c7 commit e5605b9
Showing 1 changed file with 10 additions and 2 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -183,7 +183,13 @@ RecordsWithErrorCount<mongodb::Record> StitchFrames(
++erase_until_iter;
}

req_deque.erase(req_deque.begin(), erase_until_iter);
// Determine whether to clear the StreamID from the map or only the frames that have been
// consumed.
if (erase_until_iter == req_deque.end()) {
reqs->erase(stream_id);
} else {
req_deque.erase(req_deque.begin(), erase_until_iter);
}
stream_id_pair.second = true;
}

Expand All @@ -195,9 +201,11 @@ RecordsWithErrorCount<mongodb::Record> StitchFrames(
error_count++;
}
}
resp_deque.clear();
}

// Clear the response map.
resps->clear();

// Clear the state.
auto it = state->stream_order.begin();
while (it != state->stream_order.end()) {
Expand Down

0 comments on commit e5605b9

Please sign in to comment.