Commit History

Autor SHA1 Mensaxe Data
  Christoph Büscher 3827918417 Add configurable `maxTokenLength` parameter to whitespace tokenizer (#26749) %!s(int64=8) %!d(string=hai) anos
  Shubham Aggarwal e07e4cc4dd Fix incorrect heading for Whitespace Tokenizer (#22883) %!s(int64=8) %!d(string=hai) anos
  Clinton Gormley 5da9e5dcbc Docs: Improved tokenizer docs (#18356) %!s(int64=9) %!d(string=hai) anos
  Clinton Gormley 822043347e Migrated documentation into the main repo %!s(int64=12) %!d(string=hai) anos