浏览代码

[DOCS] Add a lowercase email example to keyword tokenizer docs (#53257)

jureaky 5 年之前
父节点
当前提交
4fe8ad357c
共有 1 个文件被更改,包括 48 次插入0 次删除
  1. 48 0
      docs/reference/analysis/tokenizers/keyword-tokenizer.asciidoc

+ 48 - 0
docs/reference/analysis/tokenizers/keyword-tokenizer.asciidoc

@@ -44,6 +44,54 @@ The above sentence would produce the following term:
 [ New York ]
 ---------------------------
 
+[discrete]
+[[analysis-keyword-tokenizer-token-filters]]
+=== Combine with token filters
+You can combine the `keyword` tokenizer with token filters to normalise
+structured data, such as product IDs or email addresses.
+
+For example, the following <<indices-analyze,analyze API>> request uses
+`keyword` tokenizer and <<analysis-lowercase-tokenfilter,`lowercase`>> filter to
+convert an email address to lowercase.
+
+[source,console]
+---------------------------
+POST _analyze
+{
+  "tokenizer": "keyword",
+  "filter": [ "lowercase" ],
+  "text": "john.SMITH@example.COM"
+}
+---------------------------
+
+/////////////////////
+
+[source,console-result]
+----------------------------
+{
+  "tokens": [
+    {
+      "token": "john.smith@example.com",
+      "start_offset": 0,
+      "end_offset": 22,
+      "type": "word",
+      "position": 0
+    }
+  ]
+}
+----------------------------
+
+/////////////////////
+
+
+The request produces the following token:
+
+[source,text]
+---------------------------
+[ john.smith@example.com ]
+---------------------------
+
+
 [float]
 === Configuration