uaxurlemail-tokenizer.asciidoc 614 B

12345678910111213141516
  1. [[analysis-uaxurlemail-tokenizer]]
  2. === UAX Email URL Tokenizer
  3. A tokenizer of type `uax_url_email` which works exactly like the
  4. `standard` tokenizer, but tokenizes emails and urls as single tokens.
  5. The following are settings that can be set for a `uax_url_email`
  6. tokenizer type:
  7. [cols="<,<",options="header",]
  8. |=======================================================================
  9. |Setting |Description
  10. |`max_token_length` |The maximum token length. If a token is seen that
  11. exceeds this length then it is discarded. Defaults to `255`.
  12. |=======================================================================