Skip to content

v1.2.2: Word Segmenter Cascades, Unigram word segmenter
Compare
Choose a tag to compare
@ruanchaves ruanchaves released this 12 Feb 10:15
· 68 commits to master since this release
v1.2.2
a49e144
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Verified
Learn about vigilant mode.

Features:

  • Introduces word segmenter cascades that allow us to chain rerankers ( ad infinitum ).
  • Replaces ekphrasis by an unigram segmenter based on wordfreq. It can run on all languages supported by the wordfreq library.

Breaking changes:

  • WordSegmenter has been renamed to TransformerWordSegmenter.
Assets 2
Loading