Update on the development branch

[kaiyux]

Hi,

The TensorRT-LLM team is pleased to announce that we are pushing an update to the development branch (and the Triton backend) this December 27th, 2023.

This update includes:

• Features
  • Baichuan
    • P-tuning support
    • INT4-AWQ and INT4-GPTQ support
  • The batch manager can now return logits from context and generation phases
  • Support for combining repetition_penalty and presence_penalty Support for combining repetition_penalty, presence_penalty #274
  • Support for frequency_penalty Support for frequency_penalty #275
• Bug fixes
  • Fix LLaMA with LoRA enabled build failure Failed to build Llama-2-13b-hf engine with Lora adapter #673
  • Fix InternLM SmoothQuant build failure [0.6.1] InternLM SmoothQuant does not work #705
  • Fix Bloom int8_kv_cache functionality 【bloom】convert_checkpoint.py local variable 'int8_weights' referenced before assignment #741
• Documentation
  • Batch manager arguments documentation updates
  • Add v0.7 change log

Happy holidays and advanced wishes to a happy new year!

Thanks,
The TensorRT-LLM Engineering Team