Investigate TF batching more thoroughly

[mylesagray]

Currently we are using a set of numbers for batching in our tf-inference server that we have arrived at through trial and error.

There must be a more comprehensive way to look at these with a view to improving throughput and reliability.

refs:

• https://github.com/tensorflow/serving/blob/master/tensorflow_serving/batching/README.md
• https://mux.com/blog/tuning-performance-of-tensorflow-serving-pipeline/
• https://techblog.realtor.com/building-tensorflow-serving-aws-gpu-instances/
• https://stackoverflow.com/questions/42519010/how-to-do-batching-in-tensorflow-serving