OpenNMT-tf 1.20.0

OpenNMT-tf 1.20.0

New features

• More embeddings sharing combinations:
  • Share embeddings of multi source inputs (set share_parameters=True to ParallelInputter)
  • Share target embeddings and softmax weights (set share_embeddings=onmt.models.EmbeddingsSharingLevel.TARGET to a seq2seq model)
  • Share all embeddings (set share_embeddings=onmt.models.EmbeddingsSharingLevel.ALL to a seq2seq model, see the example model in config/models/transformer_shared_embedding.py)
• Support converting SentencePiece vocabularies in onmt-build-vocab

Fixes and improvements

• Remove the --dtype option of onmt-ark-to-records: this is considered a bug fix as the records should always be saved in float32
• Fix output dtype of SequenceRecordInputter which was always float32
• Fix guided alignment training for TensorFlow versions older than 1.11
• Refactor the Inputter API
• Increase coverage of TensorFlow 2.0 tests and remove temporary namespace opennmt.v2

OpenNMT-tf 1.19.2

OpenNMT-tf 1.19.2

Fixes and improvements

• Fix error when passing the tokenization configuration as a file in the training configuration

OpenNMT-tf 1.19.1

OpenNMT-tf 1.19.1

Fixes and improvements

• Revert default model exporter to "last" because "best" is causing issues for some users

OpenNMT-tf 1.19.0

OpenNMT-tf 1.19.0

New features

• Experimental Horovod support
• Experimental opennmt.v2 namespace that will expose modules compatible with TensorFlow 2.0
• sacreBLEU external evaluator (requires Python 3)
• Simplify configuration of non structural arguments: tokenization and pretrained embedding can now be configured in the YAML file directly

Fixes and improvements

• In distributed training, only the master should save checkpoints
• Clarify logging of training throughput, use source/target terminology instead of features/labels
• Do not save the learning rate and words per second counters in the checkpoints

OpenNMT-tf 1.18.0

OpenNMT-tf 1.18.0

New features

• Argument --size_multiple to the onmt-build-vocab script to constrain the vocabulary size used during the training
• TransformerBigFP16 in model catalog

Fixes and improvements

• Improve FP16 training speed by making the batch size a multiple of 8
• In training logs, dump final run configuration in YAML format instead of JSON

OpenNMT-tf 1.17.1

OpenNMT-tf 1.17.1

Fixes and improvements

• Fix offset in alignment vectors introduced by the <s> special token
• Fix crash when using the AdafactorOptimizer and setting beta1

OpenNMT-tf 1.17.0

OpenNMT-tf 1.17.0

New features

• Experimental batch size autotuning: automatically find the largest supported batch size based on the current configuration and available memory (for token-based batch type only)
• effective_batch_size training parameter to automatically configure gradients accumulation based on the current batch size and the number of training replicas
• Add var_list argument to the optimize_loss function
• save_checkpoints_secs training parameter as an alternative to save_checkpoints_steps

Fixes and improvements

• Change default model exporter to "best" for compatible TensorFlow versions

OpenNMT-tf 1.16.0

OpenNMT-tf 1.16.0

New features

• rnmtplus_decay learning rate decay function described in Chen et al. 2018

Fixes and improvements

• Fix error when exporting models containing a SequenceRecordInputter
• Fix error when using rsqrt_decay
• Step logging should respect save_summary_steps even when gradients accumulation is used

OpenNMT-tf 1.15.0

OpenNMT-tf 1.15.0

New features

• Parameter sampling_topk to sample predictions from the output distribution (from Edunov et al. 2018)

Fixes and improvements

• Checkpoint utilities now save a relative path instead of absolute in the generated checkpoint state
• Fix error on missing configuration fields that should be optional
• Fix error on gradient accumulation in TensorFlow versions <= 1.9
• Fix optimizer variable names mismatch introduced by gradient accumulation which prevented to continue from an existing checkpoint trained without

OpenNMT-tf 1.14.1

OpenNMT-tf 1.14.1

Fixes and improvements

• Fix inference error when using parallel inputs and the parameter bucket_width
• Fix size mismatch error when decoding from multi-source models