Skip to content

Commit

Permalink
Throw exception when request period larger than max tokens rather tha…
Browse files Browse the repository at this point in the history
…n infinite loop
  • Loading branch information
matthewkotila committed Sep 28, 2023
1 parent e10c266 commit d36a4c8
Showing 1 changed file with 8 additions and 0 deletions.
8 changes: 8 additions & 0 deletions src/c++/perf_analyzer/periodic_concurrency_worker.cc
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,14 @@ PeriodicConcurrencyWorker::WorkerCallback(uint32_t infer_context_id)
period_completed_callback_();
}
if (ctxs_.at(infer_context_id)->HasReceivedFinalResponse()) {
bool has_not_completed_period{
ctxs_.at(infer_context_id)->GetNumResponsesForCurrentRequest() <
request_period_};
if (has_not_completed_period) {
throw std::runtime_error(
"Request received final response before request period was reached. "
"Request period parameter must be less than or equal to max tokens.");
}
request_completed_callback_();
}
}
Expand Down

0 comments on commit d36a4c8

Please sign in to comment.