Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Exception if remote server is not in explicit mode #754

Merged
merged 3 commits into from
Aug 30, 2023
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 0 additions & 8 deletions docs/config_search.md
Original file line number Diff line number Diff line change
Expand Up @@ -148,14 +148,6 @@ _This will perform an Automatic Brute Search with instance group counts: 3-5, ba

---

### **Interaction with Remote Triton Launch Mode**

When the triton launch mode is remote, _\*\*only concurrency or request rate values can be swept._\*\*<br>

Model Analyzer will ignore any model config parameters because we have no way of accessing and modifying the model repository of the remote Triton Server.

---

## Manual Brute Search

**Default brute search mode when any model config parameters or parameters are specified**
Expand Down
5 changes: 5 additions & 0 deletions model_analyzer/triton/client/client.py
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,11 @@ def load_model(self, model_name, variant_name="", config_str=None):
return None
except Exception as e:
logger.info(f"Model {variant_name} load failed: {e}")
if "polling is enabled" in e.message():
# if hasattr(e, "_msg") and "polling is enabled" in e._msg:
nv-braf marked this conversation as resolved.
Show resolved Hide resolved
raise TritonModelAnalyzerException(
"The remote Tritonserver needs to be launched in EXPLICIT mode"
)
return -1

def unload_model(self, model_name):
Expand Down
10 changes: 8 additions & 2 deletions tests/mocks/mock_client.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,8 @@

from unittest.mock import ANY, MagicMock, Mock, patch

from tritonclient.http import InferenceServerException

from .mock_base import MockBase


Expand Down Expand Up @@ -140,8 +142,12 @@ def raise_exception_on_load(self):
InferenceServerException
"""

self.grpc_mock.return_value.load_model.side_effect = Exception
self.http_mock.return_value.load_model.side_effect = Exception
self.grpc_mock.return_value.load_model.side_effect = InferenceServerException(
""
)
self.http_mock.return_value.load_model.side_effect = InferenceServerException(
""
)

def raise_exception_on_unload(self):
"""
Expand Down
Loading