[Do not merge!] Build: Remove TRT model generation for V100 #7712
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* name changes * updated names
* Add documentation on generative sequence * Address comment * Reflect the "iterative" change
Co-authored-by: Ryan McCormick <[email protected]>
…generate_stream endpoint.
…#6620) * Extend request objects lifetime * Remove explicit TRITONSERVER_InferenceRequestDelete * Format fix * Include the inference_request_ initialization to cover RequestNew --------- Co-authored-by: Neelay Shah <[email protected]>
This fixes the issue where python client has `AttributeError: 'NoneType' object has no attribute 'enum_types_by_name' errors after python version is updated.
* Update README and versions for 2.40.0 / 23.11 (#6544) * Removing path construction to use SymLink alternatives * Update version for PyTorch * Update windows Dockerfile configuration * Update triton version to 23.11 * Update README and versions for 2.40.0 / 23.11 * Fix typo * Ading 'ldconfig' to configure dynamic linking in container (#6602) * Point to tekit_backend (#6616) * Point to tekit_backend * Update version * Revert tekit changes (#6640) --------- Co-authored-by: Kris Hung <[email protected]>
* New testing to confirm large request timeout values can be passed and retrieved within Python BLS models.
* Add test for optional internal tensor within an ensemble * Fix up
* Set CMake version to 3.27.7 * Set CMake version to 3.27.7 * Fix double slash typo
* Mlflow plugin fix
* Unify iGPU test build with x86 ARM * adding TRITON_IGPU_BUILD to core build definition; adding logic to skip caffe2plan test if TRITON_IGPU_BUILD=1 * re-organizing some copies in Dockerfile.QA to fix igpu devel build * Pre-commit fix --------- Co-authored-by: kyle <[email protected]>
* adding default value for TRITON_IGPU_BUILD=OFF * fix newline --------- Co-authored-by: kyle <[email protected]>
* Add test case for decoupled model raising exception * Remove unused import * Address comment
Don't Build `tritonfrontend` for Windows.
Co-authored-by: GuanLuo <[email protected]>
…revent race condition (#7617)
Co-authored-by: Sai Kiran Polisetty <[email protected]>
pvijayakrish
changed the title
mc-nv
reviewed
Oct 16, 2024
| @@ -512,7 +512,7 @@ chmod -R 777 $FORMATDESTDIR | |||
| python3 $SRCDIR/gen_qa_trt_data_dependent_shape.py --models_dir=$DATADEPENDENTDIR | |||
| chmod -R 777 $DATADEPENDENTDIR | |||
| # Make shared library for custom Hardmax plugin. | |||
| (git clone -b release/10.0 https://github.com/NVIDIA/TensorRT.git && \ | |||
| (git clone -b release/10.5 https://github.com/NVIDIA/TensorRT.git && \ | |||
There was a problem hiding this comment.
Just to avoid us from managing the version
Suggested change
| (git clone -b release/10.5 https://github.com/NVIDIA/TensorRT.git && \ | |
| docker pull ${TENSORRT_IMAGE} | |
| TENSORRT_VERSION="$(docker inspect ${TENSORRT_IMAGE} --format '{{index .Config.Labels "com.nvidia.tensorrt.version"}}' | cut -d . -f -2)" | |
| (git clone -b release/${TENSORRT_VERSION} https://github.com/NVIDIA/TensorRT.git && \ |
There was a problem hiding this comment.
Thanks @mc-nv! Ack the suggestions. This is not to be merged and is still under development and is only a work around for generating models on V100 to unblock testing.
20 tasks
20 tasks
pvijayakrish
force-pushed
the
pvijayakrish-24.10-V100
branch
from
68cf1fe to
6835fe7
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does the PR do?
Remove TRT model generation on V100.
Checklist
<commit_type>: <Title>Commit Type:
Check the conventional commit type
box here and add the label to the github PR.