pathhierarchy-tokenizer.asciidoc 879 B

1234567891011121314151617181920212223242526272829303132
  1. [[analysis-pathhierarchy-tokenizer]]
  2. === Path Hierarchy Tokenizer
  3. The `path_hierarchy` tokenizer takes something like this:
  4. -------------------------
  5. /something/something/else
  6. -------------------------
  7. And produces tokens:
  8. -------------------------
  9. /something
  10. /something/something
  11. /something/something/else
  12. -------------------------
  13. [cols="<,<",options="header",]
  14. |=======================================================================
  15. |Setting |Description
  16. |`delimiter` |The character delimiter to use, defaults to `/`.
  17. |`replacement` |An optional replacement character to use. Defaults to
  18. the `delimiter`.
  19. |`buffer_size` |The buffer size to use, defaults to `1024`.
  20. |`reverse` |Generates tokens in reverse order, defaults to `false`.
  21. |`skip` |Controls initial tokens to skip, defaults to `0`.
  22. |=======================================================================