Skip to content

Commit

Permalink
Exception if remote server is not in explicit mode (#754)
Browse files Browse the repository at this point in the history
* Throw exception if remote server is not in explicit mode

* Fix based on review

* Remove comment
  • Loading branch information
nv-braf authored Aug 30, 2023
1 parent 27da17f commit d708869
Show file tree
Hide file tree
Showing 3 changed files with 12 additions and 10 deletions.
8 changes: 0 additions & 8 deletions docs/config_search.md
Original file line number Diff line number Diff line change
Expand Up @@ -148,14 +148,6 @@ _This will perform an Automatic Brute Search with instance group counts: 3-5, ba

---

### **Interaction with Remote Triton Launch Mode**

When the triton launch mode is remote, _\*\*only concurrency or request rate values can be swept._\*\*<br>

Model Analyzer will ignore any model config parameters because we have no way of accessing and modifying the model repository of the remote Triton Server.

---

## Manual Brute Search

**Default brute search mode when any model config parameters or parameters are specified**
Expand Down
4 changes: 4 additions & 0 deletions model_analyzer/triton/client/client.py
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,10 @@ def load_model(self, model_name, variant_name="", config_str=None):
return None
except Exception as e:
logger.info(f"Model {variant_name} load failed: {e}")
if "polling is enabled" in e.message():
raise TritonModelAnalyzerException(
"The remote Tritonserver needs to be launched in EXPLICIT mode"
)
return -1

def unload_model(self, model_name):
Expand Down
10 changes: 8 additions & 2 deletions tests/mocks/mock_client.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,8 @@

from unittest.mock import ANY, MagicMock, Mock, patch

from tritonclient.http import InferenceServerException

from .mock_base import MockBase


Expand Down Expand Up @@ -140,8 +142,12 @@ def raise_exception_on_load(self):
InferenceServerException
"""

self.grpc_mock.return_value.load_model.side_effect = Exception
self.http_mock.return_value.load_model.side_effect = Exception
self.grpc_mock.return_value.load_model.side_effect = InferenceServerException(
""
)
self.http_mock.return_value.load_model.side_effect = InferenceServerException(
""
)

def raise_exception_on_unload(self):
"""
Expand Down

0 comments on commit d708869

Please sign in to comment.