Automatically fallback to dense updates in Adafactor

OpenNMT · Mar 27, 2018 · b2df462 · b2df462
1 parent 3beff36
commit b2df462
Show file tree

Hide file tree

Showing 2 changed files with 3 additions and 2 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -16,6 +16,7 @@ OpenNMT-tf follows [semantic versioning 2.0.0](https://semver.org/). The API cov
 
 * Fix the encoder state structure when RNN encoders are combined (e.g. in `SequentialEncoder`)
 * Fix `CharConvEmbedder` error on empty sequences
+* Fix `Adafactor` crash on sparse updates, automatically fallback to dense updates instead
 
 ## [1.0.1](https://github.com/OpenNMT/OpenNMT-tf/releases/tag/v1.0.1) (2018-03-14)
 

diff --git a/opennmt/utils/adafactor.py b/opennmt/utils/adafactor.py
@@ -175,9 +175,9 @@ def _create_slots(self, var_list):
 
   def _apply_dense(self, grad, var):
     return self._resource_apply_dense(grad, var)
+
   def _apply_sparse(self, grad, var):
-    print(grad, var)
-    raise NotImplementedError()
+    return self._apply_dense(tf.convert_to_tensor(grad), var)
 
   def _parameter_scale(self, var):
     """Estimate the scale of the parameters from the current values.