|
@@ -1049,6 +1049,125 @@ understands this to mean `2016-04-01` as is explained in the <<date-math-index-n
|
|
|
| `index_name_format` | no | yyyy-MM-dd | The format to be used when printing the parsed date into the index name. An valid Joda pattern is expected here.
|
|
|
|======
|
|
|
|
|
|
+[[dot-expand-processor]]
|
|
|
+=== Dot Expander Processor
|
|
|
+
|
|
|
+Expands a field with dots into an object field. This processor allows fields
|
|
|
+with dots in the name to be accessible by other processors in the pipeline.
|
|
|
+Otherwise these <<accessing-data-in-pipelines,fields>> can't be accessed by any processor.
|
|
|
+
|
|
|
+[[dot-expender-options]]
|
|
|
+.Dot Expand Options
|
|
|
+[options="header"]
|
|
|
+|======
|
|
|
+| Name | Required | Default | Description
|
|
|
+| `field` | yes | - | The field to expand into an object field
|
|
|
+| `path` | no | - | The field that contains the field to expand. Only required if the field to expand is part another object field, because the `field` option can only understand leaf fields.
|
|
|
+|======
|
|
|
+
|
|
|
+[source,js]
|
|
|
+--------------------------------------------------
|
|
|
+{
|
|
|
+ "dot_expander": {
|
|
|
+ "field": "foo.bar"
|
|
|
+ }
|
|
|
+}
|
|
|
+--------------------------------------------------
|
|
|
+// NOTCONSOLE
|
|
|
+
|
|
|
+For example the dot expand processor would turn this document:
|
|
|
+
|
|
|
+[source,js]
|
|
|
+--------------------------------------------------
|
|
|
+{
|
|
|
+ "foo.bar" : "value"
|
|
|
+}
|
|
|
+--------------------------------------------------
|
|
|
+// NOTCONSOLE
|
|
|
+
|
|
|
+into:
|
|
|
+
|
|
|
+[source,js]
|
|
|
+--------------------------------------------------
|
|
|
+{
|
|
|
+ "foo" : {
|
|
|
+ "bar" : "value"
|
|
|
+ }
|
|
|
+}
|
|
|
+--------------------------------------------------
|
|
|
+// NOTCONSOLE
|
|
|
+
|
|
|
+If there is already a `bar` field nested under `foo` then
|
|
|
+this processor merges the `foo.bar` field into it. If the field is
|
|
|
+a scalar value then it will turn that field into an array field.
|
|
|
+
|
|
|
+For example, the following document:
|
|
|
+
|
|
|
+[source,js]
|
|
|
+--------------------------------------------------
|
|
|
+{
|
|
|
+ "foo.bar" : "value2",
|
|
|
+ "foo" : {
|
|
|
+ "bar" : "value1"
|
|
|
+ }
|
|
|
+}
|
|
|
+--------------------------------------------------
|
|
|
+// NOTCONSOLE
|
|
|
+
|
|
|
+is transformed by the `dot_expander` processor into:
|
|
|
+
|
|
|
+[source,js]
|
|
|
+--------------------------------------------------
|
|
|
+{
|
|
|
+ "foo" : {
|
|
|
+ "bar" : ["value1", "value2"]
|
|
|
+ }
|
|
|
+}
|
|
|
+--------------------------------------------------
|
|
|
+// NOTCONSOLE
|
|
|
+
|
|
|
+If any field outside of the leaf field conflicts with a pre-existing field of the same name,
|
|
|
+then that field needs to be renamed first.
|
|
|
+
|
|
|
+Consider the following document:
|
|
|
+
|
|
|
+[source,js]
|
|
|
+--------------------------------------------------
|
|
|
+{
|
|
|
+ "foo": "value1",
|
|
|
+ "foo.bar": "value2"
|
|
|
+}
|
|
|
+--------------------------------------------------
|
|
|
+// NOTCONSOLE
|
|
|
+
|
|
|
+Then the `foo` needs to be renamed first before the `dot_expander`
|
|
|
+processor is applied. So in order for the `foo.bar` field to properly
|
|
|
+be expanded into the `bar` field under the `foo` field the following
|
|
|
+pipeline should be used:
|
|
|
+
|
|
|
+[source,js]
|
|
|
+--------------------------------------------------
|
|
|
+{
|
|
|
+ "processors" : [
|
|
|
+ {
|
|
|
+ "rename" : {
|
|
|
+ "field" : "foo",
|
|
|
+ "target_field" : "foo.bar""
|
|
|
+ }
|
|
|
+ },
|
|
|
+ {
|
|
|
+ "dot_expander": {
|
|
|
+ "field": "foo.bar"
|
|
|
+ }
|
|
|
+ }
|
|
|
+ ]
|
|
|
+}
|
|
|
+--------------------------------------------------
|
|
|
+// NOTCONSOLE
|
|
|
+
|
|
|
+The reason for this is that Ingest doesn't know how to automatically cast
|
|
|
+a scalar field to an object field.
|
|
|
+
|
|
|
[[fail-processor]]
|
|
|
=== Fail Processor
|
|
|
Raises an exception. This is useful for when
|
|
@@ -2058,125 +2177,6 @@ Converts a string to its uppercase equivalent.
|
|
|
--------------------------------------------------
|
|
|
// NOTCONSOLE
|
|
|
|
|
|
-[[dot-expand-processor]]
|
|
|
-=== Dot Expander Processor
|
|
|
-
|
|
|
-Expands a field with dots into an object field. This processor allows fields
|
|
|
-with dots in the name to be accessible by other processors in the pipeline.
|
|
|
-Otherwise these <<accessing-data-in-pipelines,fields>> can't be accessed by any processor.
|
|
|
-
|
|
|
-[[dot-expender-options]]
|
|
|
-.Dot Expand Options
|
|
|
-[options="header"]
|
|
|
-|======
|
|
|
-| Name | Required | Default | Description
|
|
|
-| `field` | yes | - | The field to expand into an object field
|
|
|
-| `path` | no | - | The field that contains the field to expand. Only required if the field to expand is part another object field, because the `field` option can only understand leaf fields.
|
|
|
-|======
|
|
|
-
|
|
|
-[source,js]
|
|
|
---------------------------------------------------
|
|
|
-{
|
|
|
- "dot_expander": {
|
|
|
- "field": "foo.bar"
|
|
|
- }
|
|
|
-}
|
|
|
---------------------------------------------------
|
|
|
-// NOTCONSOLE
|
|
|
-
|
|
|
-For example the dot expand processor would turn this document:
|
|
|
-
|
|
|
-[source,js]
|
|
|
---------------------------------------------------
|
|
|
-{
|
|
|
- "foo.bar" : "value"
|
|
|
-}
|
|
|
---------------------------------------------------
|
|
|
-// NOTCONSOLE
|
|
|
-
|
|
|
-into:
|
|
|
-
|
|
|
-[source,js]
|
|
|
---------------------------------------------------
|
|
|
-{
|
|
|
- "foo" : {
|
|
|
- "bar" : "value"
|
|
|
- }
|
|
|
-}
|
|
|
---------------------------------------------------
|
|
|
-// NOTCONSOLE
|
|
|
-
|
|
|
-If there is already a `bar` field nested under `foo` then
|
|
|
-this processor merges the `foo.bar` field into it. If the field is
|
|
|
-a scalar value then it will turn that field into an array field.
|
|
|
-
|
|
|
-For example, the following document:
|
|
|
-
|
|
|
-[source,js]
|
|
|
---------------------------------------------------
|
|
|
-{
|
|
|
- "foo.bar" : "value2",
|
|
|
- "foo" : {
|
|
|
- "bar" : "value1"
|
|
|
- }
|
|
|
-}
|
|
|
---------------------------------------------------
|
|
|
-// NOTCONSOLE
|
|
|
-
|
|
|
-is transformed by the `dot_expander` processor into:
|
|
|
-
|
|
|
-[source,js]
|
|
|
---------------------------------------------------
|
|
|
-{
|
|
|
- "foo" : {
|
|
|
- "bar" : ["value1", "value2"]
|
|
|
- }
|
|
|
-}
|
|
|
---------------------------------------------------
|
|
|
-// NOTCONSOLE
|
|
|
-
|
|
|
-If any field outside of the leaf field conflicts with a pre-existing field of the same name,
|
|
|
-then that field needs to be renamed first.
|
|
|
-
|
|
|
-Consider the following document:
|
|
|
-
|
|
|
-[source,js]
|
|
|
---------------------------------------------------
|
|
|
-{
|
|
|
- "foo": "value1",
|
|
|
- "foo.bar": "value2"
|
|
|
-}
|
|
|
---------------------------------------------------
|
|
|
-// NOTCONSOLE
|
|
|
-
|
|
|
-Then the `foo` needs to be renamed first before the `dot_expander`
|
|
|
-processor is applied. So in order for the `foo.bar` field to properly
|
|
|
-be expanded into the `bar` field under the `foo` field the following
|
|
|
-pipeline should be used:
|
|
|
-
|
|
|
-[source,js]
|
|
|
---------------------------------------------------
|
|
|
-{
|
|
|
- "processors" : [
|
|
|
- {
|
|
|
- "rename" : {
|
|
|
- "field" : "foo",
|
|
|
- "target_field" : "foo.bar""
|
|
|
- }
|
|
|
- },
|
|
|
- {
|
|
|
- "dot_expander": {
|
|
|
- "field": "foo.bar"
|
|
|
- }
|
|
|
- }
|
|
|
- ]
|
|
|
-}
|
|
|
---------------------------------------------------
|
|
|
-// NOTCONSOLE
|
|
|
-
|
|
|
-The reason for this is that Ingest doesn't know how to automatically cast
|
|
|
-a scalar field to an object field.
|
|
|
-
|
|
|
[[urldecode-processor]]
|
|
|
=== URL Decode Processor
|
|
|
URL-decodes a string
|