2.5.0-rc1

TensorFlow Serving using TensorFlow 2.5.0-rc1

Major Features and Improvements

• Upgrade to CUDA 11.2 and and cuDNN 8.1.0 (commit: 1975e3e)
• Update TF Text to v2.4.3. (commit: ccfb606)
• Experimental support for serving JAX and XLA/CPU models. (commit: 3c1b2b3)
• Add latency and availability metrics to the Prometheus API (#1623) (commit: dfb41f1)
• Support URL reserved characters for REST API (#1726) (commit: dd9c467)
• Add Cross-Origin Resource Sharing (CORS) headers to REST API (#1817)

Breaking Changes

• No breaking changes

Bug Fixes and Other Changes

• Fix typo in REQUIRED_PACKAGES for grpcio (commit: b9ed0f8)
• Update resnet_k8s.yaml file (commit: e7b7b33)
• Fix a compile warning thrown by gcc-9 (commit: 38a017d)
• Fix typo (commit: dbcd54f)
• Update json_tensor.cc (commit: a0a9d14)
• Add TfLiteInterpreterPool to make concurrent use of TfliteSession better (commit: d9efa43)
• Enable download of TF Serving sources at arbitrary commit for CPU docker image. (commit: de1ab9e)
• Updated tests to newer API (commit: 30dd2fe)
• Control number of grpc threads for request handling to avoid OOM (Fixes #1785). (commit: ac0eb73)
• Add dedicated aliases field to ModelServerConfig. (commit: 358f7d1)
• Update docker command line to work with GPUs (Fixes #1768). (commit: b41a28b)
• Option to disable grpc over http (Fixes #1764) (commit: f087290)
• Remove an unused experimental config option "experimental_fixed_input_tensors_filepath". (commit: 3234fca)
• Removing CurriedSession, since it is no longer used. (commit: 87793ad)
• Improve error message for file not exists. (commit: 78d47f7)
• Fix inference request delay when model is switched (Fixes #1796). (commit: 803dd42)
• Transition TensorFlow Serving to TensorFlow's new WORKSPACE protocol. (commit: 50a7ef3)
• Clarifying object values in REST requests to include B64 encoding and similar key:value pair objects. (commit: 0536678)
• Remove experimental comment on TfLiteSession (commit: ab7f9a5)
• Register custom TfLite ParseExample and add benchmark (commit: 20fe3ca)
• Use respectful terms. (commit: b73bd7b)
• Pre-allocate memory for certain vectors where the size is known. (commit: e208b6e)
• Updating serving_basic for adjusting serving_basic.md file and making it up2date with TF2.x - including: (commit: cea306a)
• Use NullSafeStringView for potentially null pointer returned from libevent (commit: a46fdb2)
• Replace nullptr constructor for string_view with empty strings (commit: a98d164)
• Fixing MKL builds due to missing 'build_with_openmp' option (commit: 0ed23df)
• Implement batch parallelism for tflite sessions (commit: fec1d5d)
• Fix GPU docker image massive increase in size (#1813) (commit: 5a0dfd9)
• Fix TensorFlow Serving build with MKL+OpenMP (commit: ddad074)
• Remove hashtable custom op dependencies (commit: bb51722)
• Enable aspired version which failed to load to attempt reload. (commit: 2530a33)
• Fixed a compilation error in aspired_versions_manager.cc (commit: 4ca9a4b)
• Add "_r" root event annotation to ProcessBatch events. (commit: e5c3aec)
• Bump minimum bazel version 3.7.2. (commit: 5edcd13)
• Dont hardcode path to python3. (commit: 63b2d1c)
• Fix package build due to config move in: (commit: 18dd766)
• Add model_service_cc_grpc_proto (commit: a670ff5)
• Fix memory leak from allocating input tensors (commit: 2f9b6a0)
• Allowing lossy floating point conversions for JSON inputs (commit: 57dac6c)
• Adding enable_profiler command line flag. (commit: 7e8720d)
• Add logging in ServerCore. (commit: 623da67)
• Removes mention of ASCII (commit: 8e97b59)

Thanks to our Contributors

This release contains contributions from many people at Google, as well as:

Abhinav Pundir, Abolfazl Shahbazi, Aurélien Geron, Bairen Yi, gbaned, handong, Hao Ziyu, Junqin Zhang, kiddos, Oliver Sampson, OniB, Runzhen Wang, skawasak, zou000