test: Client-side input shape/element validation#7427

Draft

yinggeh wants to merge 3378 commits intomainfrom yinggeh-DLIS-6657-client-input-byte-size-check

+203,098-31,781

This pull request is big! We're only showing the most recent 250 commits

Commits on Oct 13, 2023

Add gRPC AsyncIO request cancellation tests (#6408 )
kthui
authored

Commits on Oct 16, 2023

Commits on Oct 17, 2023

Switch Jetson model TensorRT models generation to container (#6378 )
mc-nv
authored

Commits on Oct 18, 2023

Commits on Oct 19, 2023

Commits on Oct 20, 2023

Commits on Oct 24, 2023

Commits on Oct 25, 2023

Test Python BLS with different sizes of CUDA memory pool (#6276 )
krishung5
authored

Commits on Oct 26, 2023

Add documentation for K8s-onprem StartupProbe (#5257 )

authored

Commits on Oct 27, 2023

Update main post-23.10 release (#6484 )

authored

Commits on Oct 28, 2023

Adding structure reference to the new document (#6493 )
mc-nv
authored

Commits on Oct 30, 2023

Improve L0_backend_python test stability (ensemble / gpu_tensor_lifecycle) (#6490 )
rmccorm4
authored

Commits on Nov 1, 2023

Commits on Nov 3, 2023

Commits on Nov 6, 2023

Upgrade DCGM from 2.4.7 to 3.2.6 (#6515 )
kthui
authored

Commits on Nov 9, 2023

Commits on Nov 10, 2023

Commits on Nov 13, 2023

Add error during rescheduling test to L0_generative_sequence (#6550 )
GuanLuo
authored

Commits on Nov 15, 2023

Commits on Nov 19, 2023

Change name generative -> iterative (#6601 )
nnshah1
authored

Commits on Nov 20, 2023

Commits on Nov 29, 2023

Update protobuf after python update for testing (#6638 )
jbkyang-nvi
authored

Commits on Dec 4, 2023

Commits on Dec 5, 2023

Add note on lack of ensemble support (#6648 )
rmccorm4
authored

Commits on Dec 6, 2023

Added request id to span attributes (#6667 )
oandreeva-nv
authored

Commits on Dec 8, 2023

Add test for optional internal tensor within an ensemble (#6663 )
GuanLuo
authored

Commits on Dec 10, 2023

Set CMake version to 3.27.7 (#6675 )
mc-nv
authored

Commits on Dec 11, 2023

restore typo (#6680 )
mc-nv
authored

Commits on Dec 12, 2023

Commits on Dec 13, 2023

Commits on Dec 14, 2023

adding default value for TRITON_IGPU_BUILD=OFF (#6705 )

nv-kmcgill53
and
kyle
authored

Commits on Dec 19, 2023

Modify HTTP frontend to return error code reflecting Triton error. Add test (#6713 )
GuanLuo
authored

Commits on Dec 26, 2023

Update main branch post-23.12 release (#6726 )

authored

Commits on Dec 28, 2023

trtllm CI fixes (#6687 )
mc-nv
authored

Commits on Jan 2, 2024

Remove double unit test (#6714 )
kthui
authored

Commits on Jan 3, 2024

Commits on Jan 4, 2024

Fix L0_lifecycle on insufficient hardware concurrency (#6762 )
kthui
authored

Commits on Jan 5, 2024

Commits on Jan 8, 2024

Commits on Jan 11, 2024

Commits on Jan 12, 2024

Commits on Jan 17, 2024

Commits on Jan 18, 2024

Commits on Jan 19, 2024

Commits on Jan 22, 2024

Remove boost::filesystem (#6810 )
rmccorm4
authored

Commits on Jan 23, 2024

Generate unittest xml reports from L0_python_api (#6822 )
rmccorm4
authored

Commits on Jan 27, 2024

Add test for shutdown while unloading in background (#6835 )
kthui
authored

Commits on Jan 29, 2024

Commits on Jan 30, 2024

Commits on Jan 31, 2024

Add test for shutdown while loading model (#6837 )
kthui
authored

Commits on Feb 1, 2024

Commits on Feb 7, 2024

Updating vllm version to 0.3.0 (#6858 )
oandreeva-nv
authored

Commits on Feb 8, 2024

Python Backend Windows Support (#6830 )
fpetrini15
authored

Commits on Feb 9, 2024

Add support for Oracle Cloud in deploy (#6850 )
bruno-garbaccio
authored

Commits on Feb 13, 2024

Add link to TRTLLM metrics docs (#6874 )
rmccorm4
authored

Commits on Feb 16, 2024

Update 'main' to track development of 2.44.0 / 24.03 (#6892 )
mc-nv
authored

Commits on Feb 17, 2024

Add response statistics (#6869 )

kthui
and
rmccorm4
authored

Commits on Feb 22, 2024

Fix busyop test for L0_memory_growth (#6900 )
krishung5
authored

Commits on Feb 23, 2024

Add cancellation into response statistics (#6904 )
kthui
authored

Commits on Feb 24, 2024

Install required pip pkgs (#6906 )
krishung5
authored

Commits on Feb 27, 2024

Match forward headers case insensitively. (#6889 )
yinggeh
authored

Commits on Mar 1, 2024

Add note on --cache-config spacing and fix typos (#6929 )
rmccorm4
authored

Commits on Mar 2, 2024

Commits on Mar 4, 2024

Expose tritonserver args in values.yaml (#5582 )
okyspace
authored

Commits on Mar 6, 2024

Parameterize git repository (#6934 )

nv-kmcgill53
and
kyle
authored

Commits on Mar 9, 2024

Test improved input validation errors (#6933 )
indrajit96
authored

Commits on Mar 13, 2024

Commits on Mar 15, 2024

Commits on Mar 19, 2024

Add documentation for mapping between Triton Errors and HTTP status codes (#6992 )
Tabrizian
authored

Commits on Mar 21, 2024

Commits on Mar 25, 2024

Commits on Mar 27, 2024

Add meetup invite banner (#7049 )
rmccorm4
authored

Commits on Apr 1, 2024

Update 'main' post-24.03 (#7051 )

authored

Commits on Apr 4, 2024

Commits on Apr 8, 2024

Fix html image rendering in sphinx documentation (#7084 )
tanmayv25
authored

Commits on Apr 10, 2024

Updated vllm version (#7095 )
oandreeva-nv
authored

Commits on Apr 12, 2024

Commits on Apr 13, 2024

Commits on Apr 23, 2024

Use a lower concurrency with more repetition for L0_memory_growth (#7127 )
krishung5
authored

Commits on Apr 24, 2024

Replace deprecated tritongrpcclient package (#7061 )
Tabrizian
authored

Commits on Apr 25, 2024

Commits on Apr 26, 2024

Commits on Apr 30, 2024

Update main post-24.04 (#7160 )

mc-nv
and
krishung5
authored

Commits on May 3, 2024

Remove meetup note now that the event has completed (#7179 )
Tabrizian
authored

Commits on May 8, 2024

Rename triton_tensorrtllm_worker -> trtllmExecutorWorker (#7194 )
krishung5
authored

Commits on May 9, 2024

Tests for Top Level Request Caching for Ensemble Models (#7074 )
lkomali
authored

Commits on May 10, 2024

Test cuda shared memory offset and byte size out of bounds(#7202 )
jbkyang-nvi
authored

Commits on May 13, 2024

Upgrade the golang version to 1.22.3 (#7208 )
tanmayv25
authored

Commits on May 14, 2024

Update 'Dockerfile' Python path to include DALI (#7216 )
mc-nv
authored

Commits on May 15, 2024

Remove the dependency on CUDA driver (#7224 )
krishung5
authored

Commits on May 16, 2024

Commits on May 17, 2024

Fix the L0_simple_go_client (#7239 )
tanmayv25
authored

Commits on May 18, 2024

Add section on ensemble model caching (#7234 )
rmccorm4
authored

Commits on May 20, 2024

Add testing for escaped log messages

nnshah1
and
rmccorm4
authored

Commits on May 21, 2024

Commits on May 22, 2024

Return an error if --load-model is specified without explicit model control mode (#7259 )
rmccorm4
authored

Commits on May 23, 2024

Exclude Jax example from Python 3.8 (#7260 )
krishung5
authored

Commits on May 24, 2024

Commits on May 28, 2024

Fix Python 3.11 env (#7274 )
krishung5
authored

Commits on May 29, 2024

Commits on Jun 3, 2024

Commits on Jun 4, 2024

fix: Fix L0_input_validation--base (#7304 )
yinggeh
authored

Commits on Jun 18, 2024

fix: Fix version for setuptools and grpcio-tools. Remove cudnn 8 installation (#7331 )
krishung5
authored

Commits on Jun 20, 2024

ci: Add INT64 Datatype Support for Shape Tensors in TensorRT Backend (#7329 )
pskiran1
authored

Commits on Jun 26, 2024

Update 15-container-copyright.txt (#7375 )
Tabrizian
authored

Commits on Jun 28, 2024

Update main post -24.06 (#7380 )
mc-nv
authored

Commits on Jul 3, 2024

test: Add input byte size tests using C APIs (#7372 )
yinggeh
authored

Commits on Jul 8, 2024

Commits on Jul 10, 2024

Commits on Jul 12, 2024

Commits on Jul 25, 2024

Commits on Jul 27, 2024

ci: Fix shape and reformat free tensor handling in the input byte size check (#7444 )
pskiran1
authored

Commits on Jul 29, 2024

Commits on Jul 30, 2024

test: Add BF16 test for python backend (#7483 )
rmccorm4
authored

Commits on Jul 31, 2024

Commits on Aug 2, 2024

Including 'tritonserver.lib' into final package (#7491 )
mc-nv
authored

Commits on Aug 4, 2024

Workaround with L0_trt_reformat_free by removing shm checks
yinggeh
committed

Commits on Aug 6, 2024

Update tests
yinggeh
committed