Skip to content

Commit

Permalink
Throw exception when request period larger than max tokens rather tha…
Browse files Browse the repository at this point in the history
…n infinite loop
  • Loading branch information
matthewkotila committed Sep 28, 2023
1 parent e10c266 commit c49b056
Showing 1 changed file with 9 additions and 0 deletions.
9 changes: 9 additions & 0 deletions src/c++/perf_analyzer/periodic_concurrency_worker.cc
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,16 @@ PeriodicConcurrencyWorker::WorkerCallback(uint32_t infer_context_id)
request_period_) {
period_completed_callback_();
}

if (ctxs_.at(infer_context_id)->HasReceivedFinalResponse()) {
bool has_not_completed_period{
ctxs_.at(infer_context_id)->GetNumResponsesForCurrentRequest() <
request_period_};
if (has_not_completed_period) {
throw std::runtime_error(
"Request received final response before request period was reached. "
"Request period parameter must be less than or equal to max tokens.");
}
request_completed_callback_();
}
}
Expand Down

0 comments on commit c49b056

Please sign in to comment.