1234567891011121314151617181920212223242526272829303132 |
- [[analysis-pathhierarchy-tokenizer]]
- === Path Hierarchy Tokenizer
- The `path_hierarchy` tokenizer takes something like this:
- -------------------------
- /something/something/else
- -------------------------
- And produces tokens:
- -------------------------
- /something
- /something/something
- /something/something/else
- -------------------------
- [cols="<,<",options="header",]
- |=======================================================================
- |Setting |Description
- |`delimiter` |The character delimiter to use, defaults to `/`.
- |`replacement` |An optional replacement character to use. Defaults to
- the `delimiter`.
- |`buffer_size` |The buffer size to use, defaults to `1024`.
- |`reverse` |Generates tokens in reverse order, defaults to `false`.
- |`skip` |Controls initial tokens to skip, defaults to `0`.
- |=======================================================================
|