浏览代码

[DOCS] EQL/SQL: Document `runtime_fields` parameter (#71487)

James Rodewig 4 年之前
父节点
当前提交
07fade1d27

+ 2 - 0
docs/reference/eql/eql-search-api.asciidoc

@@ -294,6 +294,8 @@ command].
 NOTE: This parameter may change the set of returned hits. However, it does not
 NOTE: This parameter may change the set of returned hits. However, it does not
 change the sort order of hits in the response.
 change the sort order of hits in the response.
 
 
+include::{es-repo-dir}/search/search.asciidoc[tag=runtime-mappings-def]
+
 `size`::
 `size`::
 (Optional, integer or float)
 (Optional, integer or float)
 For <<eql-basic-syntax,basic queries>>, the maximum number of matching events to
 For <<eql-basic-syntax,basic queries>>, the maximum number of matching events to

+ 70 - 1
docs/reference/eql/eql.asciidoc

@@ -442,7 +442,7 @@ GET /my-data-stream/_eql/search?filter_path=-hits.events._source
     "process.*",                <1>
     "process.*",                <1>
     {
     {
       "field": "@timestamp",    <2>
       "field": "@timestamp",    <2>
-      "format": "epoch_millis"  
+      "format": "epoch_millis"
     }
     }
   ]
   ]
 }
 }
@@ -532,6 +532,75 @@ The values are returned as a flat list in the `fields` section of each hit:
 // TESTRESPONSE[s/"_id": "OQmfCaduce8zoHT93o4H"/"_id": $body.hits.events.0._id/]
 // TESTRESPONSE[s/"_id": "OQmfCaduce8zoHT93o4H"/"_id": $body.hits.events.0._id/]
 // TESTRESPONSE[s/"_id": "xLkCaj4EujzdNSxfYLbO"/"_id": $body.hits.events.1._id/]
 // TESTRESPONSE[s/"_id": "xLkCaj4EujzdNSxfYLbO"/"_id": $body.hits.events.1._id/]
 
 
+[discrete]
+[[eql-use-runtime-fields]]
+=== Use runtime fields
+
+Use the `runtime_mappings` parameter to extract and create <<runtime,runtime
+fields>> during a search. Use the `fields` parameter to include runtime fields
+in the response.
+
+The following search creates a `day_of_week` runtime field from the `@timestamp`
+and returns it in the response.
+
+[source,console]
+----
+GET /my-data-stream/_eql/search?filter_path=-hits.events._source
+{
+  "runtime_mappings": {
+    "day_of_week": {
+      "type": "keyword",
+      "script": "emit(doc['@timestamp'].value.dayOfWeekEnum.toString())"
+    }
+  },
+  "query": """
+    process where process.name == "regsvr32.exe"
+  """,
+  "fields": [
+    "@timestamp",
+    "day_of_week"
+  ]
+}
+----
+// TEST[setup:sec_logs]
+
+The API returns:
+
+[source,console-result]
+----
+{
+  "is_partial": false,
+  "is_running": false,
+  "took": 60,
+  "timed_out": false,
+  "hits": {
+    "total": {
+      "value": 2,
+      "relation": "eq"
+    },
+    "events": [
+      {
+        "_index": ".ds-my-data-stream-2099.12.07-000001",
+        "_id": "OQmfCaduce8zoHT93o4H",
+        "fields": {
+          "@timestamp": [
+            "2099-12-07T11:07:09.000Z"
+          ],
+          "day_of_week": [
+            "MONDAY"
+          ]
+        }
+      },
+      ...
+    ]
+  }
+}
+----
+// TESTRESPONSE[s/"took": 60/"took": $body.took/]
+// TESTRESPONSE[s/"_index": ".ds-my-data-stream-2099.12.07-000001"/"_index": $body.hits.events.0._index/]
+// TESTRESPONSE[s/"_id": "OQmfCaduce8zoHT93o4H"/"_id": $body.hits.events.0._id/]
+// TESTRESPONSE[s/\.\.\./$body.hits.events.1/]
+
 [discrete]
 [discrete]
 [[specify-a-timestamp-or-event-category-field]]
 [[specify-a-timestamp-or-event-category-field]]
 === Specify a timestamp or event category field
 === Specify a timestamp or event category field

+ 19 - 8
docs/reference/search/search.asciidoc

@@ -428,36 +428,47 @@ Period of time used to extend the life of the PIT.
 <<query-dsl,Query DSL>>.
 <<query-dsl,Query DSL>>.
 
 
 [[search-api-body-runtime]]
 [[search-api-body-runtime]]
+// tag::runtime-mappings-def[]
 `runtime_mappings`::
 `runtime_mappings`::
-(Optional, object)
-Defines a <<runtime-search-request,runtime field>> in the search request that
-exist only as part of the query. Fields defined in the search request take
-precedence over fields defined with the same name in the index mappings.
+(Optional, object of objects)
+Defines one or more <<runtime-search-request,runtime fields>> in the search
+request. These fields take precedence over mapped fields with the same name.
 +
 +
 .Properties of `runtime_mappings` objects
 .Properties of `runtime_mappings` objects
 [%collapsible%open]
 [%collapsible%open]
 ====
 ====
+
+`<field-name>`::
+(Required, object)
+Configuration for the runtime field. The key is the field name.
++
+.Properties of `<field-name>`
+[%collapsible%open]
+=====
+
 `type`::
 `type`::
 (Required, string)
 (Required, string)
-<<mapping-types,Data type>> of the object, which can be any of the following:
+<<mapping-types,Field type>>, which can be any of the following:
 +
 +
 include::{es-ref-dir}/mapping/runtime.asciidoc[tag=runtime-data-types]
 include::{es-ref-dir}/mapping/runtime.asciidoc[tag=runtime-data-types]
 
 
 `script`::
 `script`::
 (Optional, string)
 (Optional, string)
-<<modules-scripting-using,Painless script>> that is executed at query time. The
+<<modules-scripting-using,Painless script>> executed at query time. The
 script has access to the entire context of a document, including the original
 script has access to the entire context of a document, including the original
 `_source` and any mapped fields plus their values.
 `_source` and any mapped fields plus their values.
 +
 +
-Your script must include `emit` to return calculated values. For
-example:
+This script must include `emit` to return calculated values. For example:
 +
 +
 [source,js,indent=0]
 [source,js,indent=0]
 ----
 ----
 include::search.asciidoc[tag=runtime-script]
 include::search.asciidoc[tag=runtime-script]
 ----
 ----
 // NOTCONSOLE
 // NOTCONSOLE
+
+=====
 ====
 ====
+// end::runtime-mappings-def[]
 
 
 ////
 ////
 [source,console]
 [source,console]

+ 45 - 1
docs/reference/sql/endpoints/rest.asciidoc

@@ -9,6 +9,7 @@
 * <<sql-rest-filtering>>
 * <<sql-rest-filtering>>
 * <<sql-rest-columnar>>
 * <<sql-rest-columnar>>
 * <<sql-rest-params>>
 * <<sql-rest-params>>
+* <<sql-runtime-fields>>
 * <<sql-rest-fields>>
 * <<sql-rest-fields>>
 
 
 [[sql-rest-overview]]
 [[sql-rest-overview]]
@@ -522,6 +523,44 @@ POST /_sql?format=txt
 [IMPORTANT]
 [IMPORTANT]
 The recommended way of passing values to a query is with question mark placeholders, to avoid any attempts of hacking or SQL injection.
 The recommended way of passing values to a query is with question mark placeholders, to avoid any attempts of hacking or SQL injection.
 
 
+[[sql-runtime-fields]]
+=== Use runtime fields
+
+Use the `runtime_mappings` parameter to extract and create <<runtime,runtime
+fields>>, or columns, from existing ones during a search.
+
+The following search creates a `release_day_of_week` runtime field from
+`release_date` and returns it in the response.
+
+[source,console]
+----
+POST _sql?format=txt
+{
+  "runtime_mappings": {
+    "release_day_of_week": {
+      "type": "keyword",
+      "script": """
+        emit(doc['release_date'].value.dayOfWeekEnum.toString())
+      """
+    }
+  },
+  "query": """
+    SELECT * FROM library WHERE page_count > 300 AND author = 'Frank Herbert'
+  """
+}
+----
+// TEST[setup:library]
+
+The API returns:
+
+[source,txt]
+----
+    author     |     name      |  page_count   |      release_date      |release_day_of_week
+---------------+---------------+---------------+------------------------+-------------------
+Frank Herbert  |Dune           |604            |1965-06-01T00:00:00.000Z|TUESDAY
+----
+// TESTRESPONSE[non_json]
+
 [[sql-rest-fields]]
 [[sql-rest-fields]]
 === Supported REST parameters
 === Supported REST parameters
 
 
@@ -530,7 +569,7 @@ the request time-outs or localization information (such as timezone).
 
 
 The table below lists the supported parameters:
 The table below lists the supported parameters:
 
 
-[cols="^m,^m,^5"]
+[cols="<m,<m,<5"]
 
 
 |===
 |===
 
 
@@ -579,6 +618,11 @@ More information available https://docs.oracle.com/javase/8/docs/api/java/time/Z
 |none
 |none
 |Optional list of parameters to replace question mark (`?`) placeholders inside the query.
 |Optional list of parameters to replace question mark (`?`) placeholders inside the query.
 
 
+|runtime_mappings
+|none
+|Defines one or more <<runtime-search-request,runtime fields>> in the search
+request. These fields take precedence over mapped fields with the same name.
+
 |===
 |===
 
 
 Do note that most parameters (outside the timeout and `columnar` ones) make sense only during the initial query - any follow-up pagination request only requires the `cursor` parameter as explained in the <<sql-pagination, pagination>> chapter.
 Do note that most parameters (outside the timeout and `columnar` ones) make sense only during the initial query - any follow-up pagination request only requires the `cursor` parameter as explained in the <<sql-pagination, pagination>> chapter.