Skip to content

Tokenizer 1.32.0
Compare
Choose a tag to compare
@guillaumekln guillaumekln released this 25 Jul 09:56
· 35 commits to master since this release
v1.32.0
4807909

New features

  • Add property pyonmttok.Vocab.counters to retrieve the number of occurrences of each token

Fixes and improvements

  • Update pybind11 to 2.10.0
  • Update cxxopts to 3.0.0
Assets 2
Loading