OpenNMT-tf 2.17.1

Fixes and improvements

• Fix some cases where batch size autotuning failed
• Raise an explicit error when batch size autotuning failed

OpenNMT-tf 2.17.0

Changes

• The tensorflow package is no longer a direct dependency of OpenNMT-tf and should be installed separately. This is to facilitate support of other TensorFlow packages such as intel-tensorflow, tensorflow-rocm, etc. Use pip install OpenNMT-tf[tensorflow] to keep the previous behavior.

New features

• Use the Keras mixed precision API instead of the auto_mixed_precision graph optimization pass
• Support mixed precision training in eager execution (for debugging)
• Support batch size autotuning with the "examples" batch type

Fixes and improvements

• Fix some TensorFlow Lite conversion errors
• Improve batch size autotuning accuracy by building batches with the maximum sequence lengths
• Reduce batch size autotuning time by running less iterations and disabling dataset buffering
• Scale the loss instead of the gradients when aggregating multiple batches
• Address some deprecation warnings in TensorFlow 2.4

OpenNMT-tf 2.16.0

Changes (non breaking)

• Rename evaluation parameter external_evaluators to scorers
• Rename export command line option --export_dir to --output_dir
• Rename export command line option --export_format to --format

New features

• Add TensorFlow Lite exporters tflite and tflite_float16 (only RNN-based sequence to sequence models are currently supported)
• Add argument pre_norm in the Transformer constructor to disable the pre-norm architecture and use the post-norm architecture as described in the original paper
• Add argument attention_reduction in the Transformer constructor to define how multi-head attention matrices are reduced into a single attention matrix
• Support batch type "tokens" and length bucketing in score task
• Support initializing the decoder output layer from its constructor (until now a separate call to decoder.initialize(...) was required)

Fixes and improvements

• Fix error on 0-length inputs in MultiHeadAttention with relative position representations
• Fix CTranslate2 export for models that enable BOS or EOS tokens on the source side (see source_sequence_controls data parameter)
• Load YAML configuration with the safe_load function to prevent arbitrary code execution

OpenNMT-tf 2.15.0

New features

• Add sentence weighting: the data parameter example_weights accepts a file with one weight per line that are used to scale the loss of the corresponding training example
• Summarize out of vocabulary tokens at the end of the training logs:
  • ratio of OOV tokens in the training data
  • 10 most frequent OOV tokens
• [API] Add argument fallback_to_cpu to Runner.train to declare whether CPU training is allowed or not (defaults to True for backward compatibility)

Fixes and improvements

• Fix error when computing BLEU score with SacreBLEU
• Fix vocabulary generation when using SentencePiece with a pre-tokenization
• Remove verbose checkpoint warnings when an exception occurs before checkpoint weights are actually loaded
• Enable the pyonmttok and ctranslate2 dependencies on macOS
• Reformat the entire codebase with Black

OpenNMT-tf 2.14.0

New features

• Support pre-tokenization when training a SentencePiece model and vocabulary with onmt-build-vocab
• Add quantized CTranslate2 exporters: ctranslate2_int8, ctranslate2_int16, ctranslate2_float16
• Re-add decay_step_duration training parameter from OpenNMT-tf V1 to delay the updates of learning rate schedules

Fixes and improvements

• Fix error when training with multiple GPUs and TensorFlow 2.4
• Fix error when setting a Keras mixed precision policy and using high-level APIs such as Runner or Trainer
• Fix error when computing the dataset size and using a MixedInputter
• Fix the Python wheel pushed to PyPI that incorrectly declared Python 2 compatibility
• Return the learning rate and the losses as float values and not tensors in the training summary
• Remove pyter3 dependency and compute TER with SacreBleu
• Remove unclear warning when setting --model or --model_type and a checkpoint already exists
• Raise error when training a SentencePiece model and vocabulary with onmt-build-vocab and using --min_frequency which is incompatible

OpenNMT-tf 2.13.0

New features

• [API] opennmt.Encoder now accepts tf.RaggedTensor as input
• [API] opennmt.Runner.train can return a summary of the training (e.g. average loss, last step, etc.)
• [API] Add function opennmt.data.create_lookup_tables to create tf.lookup tables from a vocabulary file

Fixes and improvements

• Raise errors (not warnings) when starting the training but end conditions are met
• Ignore non training time when computing training throughput: ignore initialization, evaluation, and checkpoint saving
• Print OpenNMT-tf version in the training logs
• Improve error message when model.initialize() is not called

OpenNMT-tf 2.12.1

Fixes and improvements

• Update sacrebleu to 1.4.14 to fix Python 3.5 compatibility

OpenNMT-tf 2.12.0

New features

• Update to TensorFlow 2.3 and TensorFlow Addons 0.11
• Support running onmt-main with eager execution for development or debugging (see --eager_execution command line argument)

Fixes and improvements

• Fix --data_dir prefix when the data block contains non string values
• Replace usage of the custom op Addons>GatherTree in beam search with a pure TensorFlow implementation to make model export and serving easier
• Raise an error when enabling case_feature for tokenization (unsupported in OpenNMT-tf)

OpenNMT-tf 2.11.1

Fixes and improvements

• Fix undefined gradient shape error in relative MultiHeadAttention
• Do not require --labels_file argument when evaluating language models

OpenNMT-tf 2.11.0

New features

• Dump a PyTorch-style description of the model in the logs
• Support bucketing the evaluation dataset by length for improved efficiency (parameter length_bucket_width, usually enabled by default with --auto_config)
• Accept "tokens" batch type for evaluation and inference (parameter batch_type)
• Accept passing a flat list of files on the command line for nested inputters
• Add "checkpoint" export format to export models as TensorFlow checkpoints
• Add opennmt.layers.DenseReducer to reduce inputs with a linear transformation
• Add function opennmt.utils.average_checkpoints_into_layer to average a list of checkpoints into an existing layer instance

Fixes and improvements

• Improve training performance when not using gradient accumulation
• Simplify graph generated by opennmt.layers.MultiplyReducer
• Update the online documentation and add a list of direct children in each class reference, allowing to navigate up and down in the class inheritance