[DOCS] Add info about GET/POST to the API conventions. (#74529)

* [DOCS] Add info about GET/POST to the API conventions.

* Fixed anchor

* Added redirect

* Incorporated review suggestions

docs/reference/api-conventions.asciidoc

@@ -2,81 +2,39 @@
 == API conventions
 
 The {es} REST APIs are exposed over HTTP.
+Except where noted, the following conventions apply across all APIs.
 
-The conventions listed in this chapter can be applied throughout the REST
-API, unless otherwise specified.
-
-* <<multi-index>>
-* <<date-math-index-names>>
-* <<cron-expressions>>
-* <<common-options>>
-* <<url-access-control>>
-
-[[multi-index]]
-=== Multi-target syntax
-
-Most APIs that accept a `<data-stream>`, `<index>`, or `<target>` request path
-parameter also support _multi-target syntax_.
-
-In multi-target syntax, you can use a comma-separated list to run a request on
-multiple resources, such as data streams, indices, or aliases:
-`test1,test2,test3`. You can also use {wikipedia}/Glob_(programming)[glob-like]
-wildcard (`*`) expressions to target resources that match a pattern: `test*` or
-`*test` or `te*t` or `*test*`.
-
-You can exclude targets using the `-` character: `test*,-test3`.
-
-IMPORTANT: Aliases are resolved after wildcard expressions. This can result in a
-request that targets an excluded alias. For example, if `test3` is an index
-alias, the pattern `test*,-test3` still targets the indices for `test3`. To
-avoid this, exclude the concrete indices for the alias instead.
-
-Multi-target APIs that can target indices support the following query
-string parameters:
-
-include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=index-ignore-unavailable]
-
-include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=allow-no-indices]
-
-include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=expand-wildcards]
-
-The defaults settings for the above parameters depend on the API being used.
-
-Some multi-target APIs that can target indices also support the following query
-string parameter:
-
-include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=ignore_throttled]
-
-NOTE: APIs with a single target, such as the <<docs-get,get document API>>, do
-not support multi-target syntax.
-
-[[hidden]]
-==== Hidden data streams and indices
-
-For most APIs, wildcard expressions do not match hidden data streams and indices
-by default. To match hidden data streams and indices using a wildcard
-expression, you must specify the `expand_wildcards` query parameter.
+[discrete]
+=== Content-type requirements
 
-You can create hidden data streams by setting `data_stream.hidden` to `true` in
-the stream's matching <<indices-put-template,index template>>. You can hide
-indices using the <<index-hidden,`index.hidden`>> index setting.
+The type of the content sent in a request body must be specified using
+the `Content-Type` header. The value of this header must map to one of
+the supported formats that the API supports. Most APIs support JSON,
+YAML, CBOR, and SMILE. The bulk and multi-search APIs support NDJSON,
+JSON, and SMILE; other types will result in an error response.
 
-The backing indices for data streams are hidden automatically. Some features,
-such as {ml}, store information in hidden indices.
+When using the `source` query string parameter, the content type must be
+specified using the `source_content_type` query string parameter.
 
-Global index templates that match all indices are not applied to hidden indices.
+{es} only supports UTF-8-encoded JSON. {es} ignores any other encoding headings
+sent with a request. Responses are also UTF-8 encoded.
 
-[[system-indices]]
-==== System indices
+[discrete]
+[[get-requests]]
+=== GET and POST requests
 
-{es} modules and plugins can store configuration and state information in internal _system indices_. 
-You should not directly access or modify system indices 
-as they contain data essential to the operation of the system.
+A number of {es} GET APIs--most notably the search API--support a request body. 
+While the GET action makes sense in the context of retrieving information, 
+GET requests with a body are not supported by all HTTP libraries.
+All {es} GET APIs that require a body can also be submitted as POST requests.
+Alternatively, you can pass the request body as the 
+<<api-request-body-query-string, `source` query string parameter>>
+when using GET.
 
-IMPORTANT: Direct access to system indices is deprecated and 
-will no longer be allowed in the next major version.
+include::rest-api/cron-expressions.asciidoc[]
 
-[[date-math-index-names]]
+[discrete]
+[[api-date-math-index-names]]
 === Date math support in index and index alias names
 
 Date math name resolution lets you to search a range of time series indices or
@@ -172,345 +130,123 @@ GET /%3Clogstash-%7Bnow%2Fd-2d%7D%3E%2C%3Clogstash-%7Bnow%2Fd-1d%7D%3E%2C%3Clogs
 // TEST[s/^/PUT logstash-2016.09.20\nPUT logstash-2016.09.19\nPUT logstash-2016.09.18\n/]
 // TEST[s/now/2016.09.20%7C%7C/]
 
-include::rest-api/cron-expressions.asciidoc[]
-
-[[common-options]]
-=== Common options
-
-The following options can be applied to all of the REST APIs.
-
-[discrete]
-==== Pretty Results
-
-When appending `?pretty=true` to any request made, the JSON returned
-will be pretty formatted (use it for debugging only!). Another option is
-to set `?format=yaml` which will cause the result to be returned in the
-(sometimes) more readable yaml format.
-
-
 [discrete]
-==== Human readable output
-
-Statistics are returned in a format suitable for humans
-(e.g. `"exists_time": "1h"` or `"size": "1kb"`) and for computers
-(e.g. `"exists_time_in_millis": 3600000` or `"size_in_bytes": 1024`).
-The human readable values can be turned off by adding `?human=false`
-to the query string. This makes sense when the stats results are
-being consumed by a monitoring tool, rather than intended for human
-consumption. The default for the `human` flag is
-`false`.
-
-[[date-math]]
-[discrete]
-==== Date Math
-
-Most parameters which accept a formatted date value -- such as `gt` and `lt`
-in <<query-dsl-range-query,`range` queries>>, or `from` and `to`
-in <<search-aggregations-bucket-daterange-aggregation,`daterange`
-aggregations>> -- understand date maths.
-
-The expression starts with an anchor date, which can either be `now`, or a
-date string ending with `||`. This anchor date can optionally be followed by
-one or more maths expressions:
-
-* `+1h`: Add one hour
-* `-1d`: Subtract one day
-* `/d`: Round down to the nearest day
-
-The supported time units differ from those supported by <<time-units, time units>> for durations.
-The supported units are:
-
-[horizontal]
-`y`:: Years
-`M`:: Months
-`w`:: Weeks
-`d`:: Days
-`h`:: Hours
-`H`:: Hours
-`m`:: Minutes
-`s`:: Seconds
-
-Assuming `now` is `2001-01-01 12:00:00`, some examples are:
-
-[horizontal]
-`now+1h`:: `now` in milliseconds plus one hour. Resolves to: `2001-01-01 13:00:00`
-`now-1h`:: `now` in milliseconds minus one hour. Resolves to: `2001-01-01 11:00:00`
-`now-1h/d`:: `now` in milliseconds minus one hour, rounded down to UTC 00:00. Resolves to: `2001-01-01 00:00:00`
- `2001.02.01\|\|+1M/d`:: `2001-02-01` in milliseconds plus one month. Resolves to: `2001-03-01 00:00:00`
-
-[discrete]
-[[common-options-response-filtering]]
-==== Response Filtering
-
-All REST APIs accept a `filter_path` parameter that can be used to reduce
-the response returned by Elasticsearch. This parameter takes a comma
-separated list of filters expressed with the dot notation:
-
-[source,console]
---------------------------------------------------
-GET /_search?q=kimchy&filter_path=took,hits.hits._id,hits.hits._score
---------------------------------------------------
-// TEST[setup:my_index]
-
-Responds:
-
-[source,console-result]
---------------------------------------------------
-{
-  "took" : 3,
-  "hits" : {
-    "hits" : [
-      {
-        "_id" : "0",
-        "_score" : 1.6375021
-      }
-    ]
-  }
-}
---------------------------------------------------
-// TESTRESPONSE[s/"took" : 3/"took" : $body.took/]
-// TESTRESPONSE[s/1.6375021/$body.hits.hits.0._score/]
-
-It also supports the `*` wildcard character to match any field or part
-of a field's name:
-
-[source,console]
---------------------------------------------------
-GET /_cluster/state?filter_path=metadata.indices.*.stat*
---------------------------------------------------
-// TEST[s/^/PUT my-index-000001\n/]
+[[api-multi-index]]
+=== Multi-target syntax
 
-Responds:
+Most APIs that accept a `<data-stream>`, `<index>`, or `<target>` request path
+parameter also support _multi-target syntax_.
 
-[source,console-result]
---------------------------------------------------
-{
-  "metadata" : {
-    "indices" : {
-      "my-index-000001": {"state": "open"}
-    }
-  }
-}
---------------------------------------------------
+In multi-target syntax, you can use a comma-separated list to run a request on
+multiple resources, such as data streams, indices, or aliases:
+`test1,test2,test3`. You can also use {wikipedia}/Glob_(programming)[glob-like]
+wildcard (`*`) expressions to target resources that match a pattern: `test*` or
+`*test` or `te*t` or `*test*`.
 
-And the `**` wildcard can be used to include fields without knowing the
-exact path of the field. For example, we can return the Lucene version
-of every segment with this request:
+You can exclude targets using the `-` character: `test*,-test3`.
 
-[source,console]
---------------------------------------------------
-GET /_cluster/state?filter_path=routing_table.indices.**.state
---------------------------------------------------
-// TEST[s/^/PUT my-index-000001\n/]
+IMPORTANT: Aliases are resolved after wildcard expressions. This can result in a
+request that targets an excluded alias. For example, if `test3` is an index
+alias, the pattern `test*,-test3` still targets the indices for `test3`. To
+avoid this, exclude the concrete indices for the alias instead.
 
-Responds:
+Multi-target APIs that can target indices support the following query
+string parameters:
 
-[source,console-result]
---------------------------------------------------
-{
-  "routing_table": {
-    "indices": {
-      "my-index-000001": {
-        "shards": {
-          "0": [{"state": "STARTED"}, {"state": "UNASSIGNED"}]
-        }
-      }
-    }
-  }
-}
---------------------------------------------------
+include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=index-ignore-unavailable]
 
-It is also possible to exclude one or more fields by prefixing the filter with the char `-`:
+include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=allow-no-indices]
 
-[source,console]
---------------------------------------------------
-GET /_count?filter_path=-_shards
---------------------------------------------------
-// TEST[setup:my_index]
+include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=expand-wildcards]
 
-Responds:
+The defaults settings for the above parameters depend on the API being used.
 
-[source,console-result]
---------------------------------------------------
-{
-  "count" : 5
-}
---------------------------------------------------
+Some multi-target APIs that can target indices also support the following query
+string parameter:
 
-And for more control, both inclusive and exclusive filters can be combined in the same expression. In
-this case, the exclusive filters will be applied first and the result will be filtered again using the
-inclusive filters:
+include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=ignore_throttled]
 
-[source,console]
---------------------------------------------------
-GET /_cluster/state?filter_path=metadata.indices.*.state,-metadata.indices.logstash-*
---------------------------------------------------
-// TEST[s/^/PUT my-index-000001\nPUT my-index-000002\nPUT my-index-000003\nPUT logstash-2016.01\n/]
+NOTE: APIs with a single target, such as the <<docs-get,get document API>>, do
+not support multi-target syntax.
 
-Responds:
+[discrete]
+[[multi-hidden]]
+==== Hidden data streams and indices
 
-[source,console-result]
---------------------------------------------------
-{
-  "metadata" : {
-    "indices" : {
-      "my-index-000001" : {"state" : "open"},
-      "my-index-000002" : {"state" : "open"},
-      "my-index-000003" : {"state" : "open"}
-    }
-  }
-}
---------------------------------------------------
+For most APIs, wildcard expressions do not match hidden data streams and indices
+by default. To match hidden data streams and indices using a wildcard
+expression, you must specify the `expand_wildcards` query parameter.
 
-Note that Elasticsearch sometimes returns directly the raw value of a field,
-like the `_source` field. If you want to filter `_source` fields, you should
-consider combining the already existing `_source` parameter (see
-<<get-source-filtering,Get API>> for more details) with the `filter_path`
-parameter like this:
+You can create hidden data streams by setting `data_stream.hidden` to `true` in
+the stream's matching <<indices-put-template,index template>>. You can hide
+indices using the <<index-hidden,`index.hidden`>> index setting.
 
-[source,console]
---------------------------------------------------
-POST /library/_doc?refresh
-{"title": "Book #1", "rating": 200.1}
-POST /library/_doc?refresh
-{"title": "Book #2", "rating": 1.7}
-POST /library/_doc?refresh
-{"title": "Book #3", "rating": 0.1}
-GET /_search?filter_path=hits.hits._source&_source=title&sort=rating:desc
---------------------------------------------------
-
-[source,console-result]
---------------------------------------------------
-{
-  "hits" : {
-    "hits" : [ {
-      "_source":{"title":"Book #1"}
-    }, {
-      "_source":{"title":"Book #2"}
-    }, {
-      "_source":{"title":"Book #3"}
-    } ]
-  }
-}
---------------------------------------------------
+The backing indices for data streams are hidden automatically. Some features,
+such as {ml}, store information in hidden indices.
 
+Global index templates that match all indices are not applied to hidden indices.
 
 [discrete]
-==== Flat Settings
+[[system-indices]]
+==== System indices
 
-The `flat_settings` flag affects rendering of the lists of settings. When the
-`flat_settings` flag is `true`, settings are returned in a flat format:
+{es} modules and plugins can store configuration and state information in internal _system indices_. 
+You should not directly access or modify system indices 
+as they contain data essential to the operation of the system.
 
-[source,console]
---------------------------------------------------
-GET my-index-000001/_settings?flat_settings=true
---------------------------------------------------
-// TEST[setup:my_index]
+IMPORTANT: Direct access to system indices is deprecated and 
+will no longer be allowed in the next major version.
 
-Returns:
+[discrete]
+[[api-conventions-parameters]]
+=== Parameters
 
-[source,console-result]
---------------------------------------------------
-{
-  "my-index-000001" : {
-    "settings": {
-      "index.number_of_replicas": "1",
-      "index.number_of_shards": "1",
-      "index.creation_date": "1474389951325",
-      "index.uuid": "n6gzFZTgS664GUfx0Xrpjw",
-      "index.version.created": ...,
-      "index.routing.allocation.include._tier_preference" : "data_content",
-      "index.provided_name" : "my-index-000001"
-    }
-  }
-}
---------------------------------------------------
-// TESTRESPONSE[s/1474389951325/$body.my-index-000001.settings.index\\\\.creation_date/]
-// TESTRESPONSE[s/n6gzFZTgS664GUfx0Xrpjw/$body.my-index-000001.settings.index\\\\.uuid/]
-// TESTRESPONSE[s/"index.version.created": \.\.\./"index.version.created": $body.my-index-000001.settings.index\\\\.version\\\\.created/]
+Rest parameters (when using HTTP, map to HTTP URL parameters) follow the
+convention of using underscore casing.
 
-When the `flat_settings` flag is `false`, settings are returned in a more
-human readable structured format:
+[discrete]
+[[api-request-body-query-string]]
+=== Request body in query string
 
-[source,console]
---------------------------------------------------
-GET my-index-000001/_settings?flat_settings=false
---------------------------------------------------
-// TEST[setup:my_index]
+For libraries that don't accept a request body for non-POST requests,
+you can pass the request body as the `source` query string parameter
+instead. When using this method, the `source_content_type` parameter
+should also be passed with a media type value that indicates the format
+of the source, such as `application/json`.
 
-Returns:
+[discrete]
+[[api-url-access-control]]
+=== URL-based access control
 
-[source,console-result]
---------------------------------------------------
-{
-  "my-index-000001" : {
-    "settings" : {
-      "index" : {
-        "number_of_replicas": "1",
-        "number_of_shards": "1",
-        "creation_date": "1474389951325",
-        "uuid": "n6gzFZTgS664GUfx0Xrpjw",
-        "version": {
-          "created": ...
-        },
-        "routing": {
-          "allocation": {
-            "include": {
-              "_tier_preference": "data_content"
-            }
-          }
-        },
-        "provided_name" : "my-index-000001"
-      }
-    }
-  }
-}
---------------------------------------------------
-// TESTRESPONSE[s/1474389951325/$body.my-index-000001.settings.index.creation_date/]
-// TESTRESPONSE[s/n6gzFZTgS664GUfx0Xrpjw/$body.my-index-000001.settings.index.uuid/]
-// TESTRESPONSE[s/"created": \.\.\./"created": $body.my-index-000001.settings.index.version.created/]
+Many users use a proxy with URL-based access control to secure access to
+{es} data streams and indices. For <<search-multi-search,multi-search>>,
+<<docs-multi-get,multi-get>>, and <<docs-bulk,bulk>> requests, the user has
+the choice of specifying a data stream or  index in the URL and on each individual request
+within the request body. This can make URL-based access control challenging.
 
-By default `flat_settings` is set to `false`.
+To prevent the user from overriding the data stream or index specified in the
+URL, set `rest.action.multi.allow_explicit_index` to `false` in `elasticsearch.yml`.
 
-[discrete]
-[[api-conventions-parameters]]
-==== Parameters
 
-Rest parameters (when using HTTP, map to HTTP URL parameters) follow the
-convention of using underscore casing.
+This causes  {es} to
+reject requests that explicitly specify a data stream or index in the request body.
 
 [discrete]
-==== Boolean Values
+=== Boolean Values
 
 All REST API parameters (both request parameters and JSON body) support
 providing boolean "false" as the value `false` and boolean "true" as the
 value `true`. All other values will raise an error.
 
 [discrete]
-==== Number Values
+=== Number Values
 
 All REST APIs support providing numbered parameters as `string` on top
 of supporting the native JSON number types.
 
-[[time-units]]
-[discrete]
-==== Time units
-
-Whenever durations need to be specified, e.g. for a `timeout` parameter, the duration must specify
-the unit, like `2d` for 2 days. The supported units are:
-
-[horizontal]
-`d`::       Days
-`h`::       Hours
-`m`::       Minutes
-`s`::       Seconds
-`ms`::      Milliseconds
-`micros`::  Microseconds
-`nanos`::   Nanoseconds
-
 [[byte-units]]
 [discrete]
-==== Byte size units
+=== Byte size units
 
 Whenever the byte size of data needs to be specified, e.g. when setting a buffer size
 parameter, the value must specify the unit, like `10kb` for 10 kilobytes. Note that
@@ -524,25 +260,9 @@ these units use powers of 1024, so `1kb` means 1024 bytes. The supported units a
 `tb`::  Terabytes
 `pb`::  Petabytes
 
-[[size-units]]
-[discrete]
-==== Unit-less quantities
-
-Unit-less quantities means that they don't have a "unit" like "bytes" or "Hertz" or "meter" or "long tonne".
-
-If one of these quantities is large we'll print it out like 10m for 10,000,000 or 7k for 7,000. We'll still print 87
-when we mean 87 though. These are the supported multipliers:
-
-[horizontal]
-`k`::  Kilo
-`m`::  Mega
-`g`::  Giga
-`t`::  Tera
-`p`::  Peta
-
 [[distance-units]]
 [discrete]
-==== Distance Units
+=== Distance Units
 
 Wherever distances need to be specified, such as the `distance` parameter in
 the <<query-dsl-geo-distance-query>>), the default unit is meters if none is specified.
@@ -562,151 +282,37 @@ Centimeter::    `cm` or `centimeters`
 Millimeter::    `mm` or `millimeters`
 Nautical mile:: `NM`, `nmi`, or `nauticalmiles`
 
-[[fuzziness]]
 [discrete]
-==== Fuzziness
-
-Some queries and APIs support parameters to allow inexact _fuzzy_ matching,
-using the `fuzziness` parameter.
-
-When querying `text` or `keyword` fields, `fuzziness` is interpreted as a
-{wikipedia}/Levenshtein_distance[Levenshtein Edit Distance]
--- the number of one character changes that need to be made to one string to
-make it the same as another string.
+[[time-units]]
+=== Time units
 
-The `fuzziness` parameter can be specified as:
+Whenever durations need to be specified, e.g. for a `timeout` parameter, the duration must specify
+the unit, like `2d` for 2 days. The supported units are:
 
 [horizontal]
-`0`, `1`, `2`::
-
-The maximum allowed Levenshtein Edit Distance (or number of edits)
-
-`AUTO`::
-+
---
-Generates an edit distance based on the length of the term.
-Low and high distance arguments may be optionally provided `AUTO:[low],[high]`. If not specified,
-the default values are 3 and 6, equivalent to `AUTO:3,6` that make for lengths:
-
-`0..2`:: Must match exactly
-`3..5`:: One edit allowed
-`>5`:: Two edits allowed
-
-`AUTO` should generally be the preferred value for `fuzziness`.
---
-
-[discrete]
-[[common-options-error-options]]
-==== Enabling stack traces
-
-By default when a request returns an error Elasticsearch doesn't include the
-stack trace of the error. You can enable that behavior by setting the
-`error_trace` url parameter to `true`. For example, by default when you send an
-invalid `size` parameter to the `_search` API:
-
-[source,console]
-----------------------------------------------------------------------
-POST /my-index-000001/_search?size=surprise_me
-----------------------------------------------------------------------
-// TEST[s/surprise_me/surprise_me&error_trace=false/ catch:bad_request]
-// Since the test system sends error_trace=true by default we have to override
-
-The response looks like:
-
-[source,console-result]
-----------------------------------------------------------------------
-{
-  "error" : {
-    "root_cause" : [
-      {
-        "type" : "illegal_argument_exception",
-        "reason" : "Failed to parse int parameter [size] with value [surprise_me]"
-      }
-    ],
-    "type" : "illegal_argument_exception",
-    "reason" : "Failed to parse int parameter [size] with value [surprise_me]",
-    "caused_by" : {
-      "type" : "number_format_exception",
-      "reason" : "For input string: \"surprise_me\""
-    }
-  },
-  "status" : 400
-}
-----------------------------------------------------------------------
-
-But if you set `error_trace=true`:
-
-[source,console]
-----------------------------------------------------------------------
-POST /my-index-000001/_search?size=surprise_me&error_trace=true
-----------------------------------------------------------------------
-// TEST[catch:bad_request]
-
-The response looks like:
-
-[source,console-result]
-----------------------------------------------------------------------
-{
-  "error": {
-    "root_cause": [
-      {
-        "type": "illegal_argument_exception",
-        "reason": "Failed to parse int parameter [size] with value [surprise_me]",
-        "stack_trace": "Failed to parse int parameter [size] with value [surprise_me]]; nested: IllegalArgumentException..."
-      }
-    ],
-    "type": "illegal_argument_exception",
-    "reason": "Failed to parse int parameter [size] with value [surprise_me]",
-    "stack_trace": "java.lang.IllegalArgumentException: Failed to parse int parameter [size] with value [surprise_me]\n    at org.elasticsearch.rest.RestRequest.paramAsInt(RestRequest.java:175)...",
-    "caused_by": {
-      "type": "number_format_exception",
-      "reason": "For input string: \"surprise_me\"",
-      "stack_trace": "java.lang.NumberFormatException: For input string: \"surprise_me\"\n    at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)..."
-    }
-  },
-  "status": 400
-}
-----------------------------------------------------------------------
-// TESTRESPONSE[s/"stack_trace": "Failed to parse int parameter.+\.\.\."/"stack_trace": $body.error.root_cause.0.stack_trace/]
-// TESTRESPONSE[s/"stack_trace": "java.lang.IllegalArgum.+\.\.\."/"stack_trace": $body.error.stack_trace/]
-// TESTRESPONSE[s/"stack_trace": "java.lang.Number.+\.\.\."/"stack_trace": $body.error.caused_by.stack_trace/]
-
-[discrete]
-==== Request body in query string
-
-For libraries that don't accept a request body for non-POST requests,
-you can pass the request body as the `source` query string parameter
-instead. When using this method, the `source_content_type` parameter
-should also be passed with a media type value that indicates the format
-of the source, such as `application/json`.
+`d`::       Days
+`h`::       Hours
+`m`::       Minutes
+`s`::       Seconds
+`ms`::      Milliseconds
+`micros`::  Microseconds
+`nanos`::   Nanoseconds
 
+[[size-units]]
 [discrete]
-==== Content-type requirements
+=== Unit-less quantities
 
-The type of the content sent in a request body must be specified using
-the `Content-Type` header. The value of this header must map to one of
-the supported formats that the API supports. Most APIs support JSON,
-YAML, CBOR, and SMILE. The bulk and multi-search APIs support NDJSON,
-JSON, and SMILE; other types will result in an error response.
-
-When using the `source` query string parameter, the content type must be
-specified using the `source_content_type` query string parameter.
-
-{es} only supports UTF-8-encoded JSON. {es} ignores any other encoding headings
-sent with a request. Responses are also UTF-8 encoded.
+Unit-less quantities means that they don't have a "unit" like "bytes" or "Hertz" or "meter" or "long tonne".
 
-[[url-access-control]]
-=== URL-based access control
+If one of these quantities is large we'll print it out like 10m for 10,000,000 or 7k for 7,000. We'll still print 87
+when we mean 87 though. These are the supported multipliers:
 
-Many users use a proxy with URL-based access control to secure access to
-{es} data streams and indices. For <<search-multi-search,multi-search>>,
-<<docs-multi-get,multi-get>>, and <<docs-bulk,bulk>> requests, the user has
-the choice of specifying a data stream or  index in the URL and on each individual request
-within the request body. This can make URL-based access control challenging.
+[horizontal]
+`k`::  Kilo
+`m`::  Mega
+`g`::  Giga
+`t`::  Tera
+`p`::  Peta
 
-To prevent the user from overriding the data stream or index specified in the
-URL, set `rest.action.multi.allow_explicit_index` to `false` in `elasticsearch.yml`.
 
 
-This causes  {es} to
-reject requests that explicitly specify a data stream or index in the request body.

docs/reference/redirects.asciidoc

@@ -1598,3 +1598,28 @@ include::redirects.asciidoc[tag=frozen-index-redirect]
 === Glossary
 
 See the {glossary}/terms.html[Elastic glossary].
+[role="exclude",id="multi-index"]
+=== Multi-target syntax
+
+See <<api-multi-index>>. 
+
+[float]
+[[hidden]]
+==== Hidden data streams and indices
+
+See <<multi-hidden>>. 
+
+[role="exclude",id="date-math-index-names"]
+=== Date math support in system and index alias names 
+
+See <<api-date-math-index-names>>.
+
+[role="exclude",id="cron-expressions"]
+=== Cron expressions
+
+See <<api-cron-expressions>>.
+
+[role="exclude",id="url-access-control"]
+=== URL-based access control
+
+See <<api-url-access-control>>.

docs/reference/rest-api/common-options.asciidoc

@@ -0,0 +1,406 @@
+[[common-options]]
+== Common options
+
+All {es} REST APIs support the following options.
+
+[discrete]
+=== Pretty Results
+
+When appending `?pretty=true` to any request made, the JSON returned
+will be pretty formatted (use it for debugging only!). Another option is
+to set `?format=yaml` which will cause the result to be returned in the
+(sometimes) more readable yaml format.
+
+
+[discrete]
+=== Human readable output
+
+Statistics are returned in a format suitable for humans
+(e.g. `"exists_time": "1h"` or `"size": "1kb"`) and for computers
+(e.g. `"exists_time_in_millis": 3600000` or `"size_in_bytes": 1024`).
+The human readable values can be turned off by adding `?human=false`
+to the query string. This makes sense when the stats results are
+being consumed by a monitoring tool, rather than intended for human
+consumption. The default for the `human` flag is
+`false`.
+
+[[date-math]]
+[discrete]
+=== Date Math
+
+Most parameters which accept a formatted date value -- such as `gt` and `lt`
+in <<query-dsl-range-query,`range` queries>>, or `from` and `to`
+in <<search-aggregations-bucket-daterange-aggregation,`daterange`
+aggregations>> -- understand date maths.
+
+The expression starts with an anchor date, which can either be `now`, or a
+date string ending with `||`. This anchor date can optionally be followed by
+one or more maths expressions:
+
+* `+1h`: Add one hour
+* `-1d`: Subtract one day
+* `/d`: Round down to the nearest day
+
+The supported time units differ from those supported by <<time-units, time units>> for durations.
+The supported units are:
+
+[horizontal]
+`y`:: Years
+`M`:: Months
+`w`:: Weeks
+`d`:: Days
+`h`:: Hours
+`H`:: Hours
+`m`:: Minutes
+`s`:: Seconds
+
+Assuming `now` is `2001-01-01 12:00:00`, some examples are:
+
+[horizontal]
+`now+1h`:: `now` in milliseconds plus one hour. Resolves to: `2001-01-01 13:00:00`
+`now-1h`:: `now` in milliseconds minus one hour. Resolves to: `2001-01-01 11:00:00`
+`now-1h/d`:: `now` in milliseconds minus one hour, rounded down to UTC 00:00. Resolves to: `2001-01-01 00:00:00`
+ `2001.02.01\|\|+1M/d`:: `2001-02-01` in milliseconds plus one month. Resolves to: `2001-03-01 00:00:00`
+
+[discrete]
+[[common-options-response-filtering]]
+=== Response Filtering
+
+All REST APIs accept a `filter_path` parameter that can be used to reduce
+the response returned by Elasticsearch. This parameter takes a comma
+separated list of filters expressed with the dot notation:
+
+[source,console]
+--------------------------------------------------
+GET /_search?q=kimchy&filter_path=took,hits.hits._id,hits.hits._score
+--------------------------------------------------
+// TEST[setup:my_index]
+
+Responds:
+
+[source,console-result]
+--------------------------------------------------
+{
+  "took" : 3,
+  "hits" : {
+    "hits" : [
+      {
+        "_id" : "0",
+        "_score" : 1.6375021
+      }
+    ]
+  }
+}
+--------------------------------------------------
+// TESTRESPONSE[s/"took" : 3/"took" : $body.took/]
+// TESTRESPONSE[s/1.6375021/$body.hits.hits.0._score/]
+
+It also supports the `*` wildcard character to match any field or part
+of a field's name:
+
+[source,console]
+--------------------------------------------------
+GET /_cluster/state?filter_path=metadata.indices.*.stat*
+--------------------------------------------------
+// TEST[s/^/PUT my-index-000001\n/]
+
+Responds:
+
+[source,console-result]
+--------------------------------------------------
+{
+  "metadata" : {
+    "indices" : {
+      "my-index-000001": {"state": "open"}
+    }
+  }
+}
+--------------------------------------------------
+
+And the `**` wildcard can be used to include fields without knowing the
+exact path of the field. For example, we can return the Lucene version
+of every segment with this request:
+
+[source,console]
+--------------------------------------------------
+GET /_cluster/state?filter_path=routing_table.indices.**.state
+--------------------------------------------------
+// TEST[s/^/PUT my-index-000001\n/]
+
+Responds:
+
+[source,console-result]
+--------------------------------------------------
+{
+  "routing_table": {
+    "indices": {
+      "my-index-000001": {
+        "shards": {
+          "0": [{"state": "STARTED"}, {"state": "UNASSIGNED"}]
+        }
+      }
+    }
+  }
+}
+--------------------------------------------------
+
+It is also possible to exclude one or more fields by prefixing the filter with the char `-`:
+
+[source,console]
+--------------------------------------------------
+GET /_count?filter_path=-_shards
+--------------------------------------------------
+// TEST[setup:my_index]
+
+Responds:
+
+[source,console-result]
+--------------------------------------------------
+{
+  "count" : 5
+}
+--------------------------------------------------
+
+And for more control, both inclusive and exclusive filters can be combined in the same expression. In
+this case, the exclusive filters will be applied first and the result will be filtered again using the
+inclusive filters:
+
+[source,console]
+--------------------------------------------------
+GET /_cluster/state?filter_path=metadata.indices.*.state,-metadata.indices.logstash-*
+--------------------------------------------------
+// TEST[s/^/PUT my-index-000001\nPUT my-index-000002\nPUT my-index-000003\nPUT logstash-2016.01\n/]
+
+Responds:
+
+[source,console-result]
+--------------------------------------------------
+{
+  "metadata" : {
+    "indices" : {
+      "my-index-000001" : {"state" : "open"},
+      "my-index-000002" : {"state" : "open"},
+      "my-index-000003" : {"state" : "open"}
+    }
+  }
+}
+--------------------------------------------------
+
+Note that Elasticsearch sometimes returns directly the raw value of a field,
+like the `_source` field. If you want to filter `_source` fields, you should
+consider combining the already existing `_source` parameter (see
+<<get-source-filtering,Get API>> for more details) with the `filter_path`
+parameter like this:
+
+[source,console]
+--------------------------------------------------
+POST /library/_doc?refresh
+{"title": "Book #1", "rating": 200.1}
+POST /library/_doc?refresh
+{"title": "Book #2", "rating": 1.7}
+POST /library/_doc?refresh
+{"title": "Book #3", "rating": 0.1}
+GET /_search?filter_path=hits.hits._source&_source=title&sort=rating:desc
+--------------------------------------------------
+
+[source,console-result]
+--------------------------------------------------
+{
+  "hits" : {
+    "hits" : [ {
+      "_source":{"title":"Book #1"}
+    }, {
+      "_source":{"title":"Book #2"}
+    }, {
+      "_source":{"title":"Book #3"}
+    } ]
+  }
+}
+--------------------------------------------------
+
+
+[discrete]
+=== Flat Settings
+
+The `flat_settings` flag affects rendering of the lists of settings. When the
+`flat_settings` flag is `true`, settings are returned in a flat format:
+
+[source,console]
+--------------------------------------------------
+GET my-index-000001/_settings?flat_settings=true
+--------------------------------------------------
+// TEST[setup:my_index]
+
+Returns:
+
+[source,console-result]
+--------------------------------------------------
+{
+  "my-index-000001" : {
+    "settings": {
+      "index.number_of_replicas": "1",
+      "index.number_of_shards": "1",
+      "index.creation_date": "1474389951325",
+      "index.uuid": "n6gzFZTgS664GUfx0Xrpjw",
+      "index.version.created": ...,
+      "index.routing.allocation.include._tier_preference" : "data_content",
+      "index.provided_name" : "my-index-000001"
+    }
+  }
+}
+--------------------------------------------------
+// TESTRESPONSE[s/1474389951325/$body.my-index-000001.settings.index\\\\.creation_date/]
+// TESTRESPONSE[s/n6gzFZTgS664GUfx0Xrpjw/$body.my-index-000001.settings.index\\\\.uuid/]
+// TESTRESPONSE[s/"index.version.created": \.\.\./"index.version.created": $body.my-index-000001.settings.index\\\\.version\\\\.created/]
+
+When the `flat_settings` flag is `false`, settings are returned in a more
+human readable structured format:
+
+[source,console]
+--------------------------------------------------
+GET my-index-000001/_settings?flat_settings=false
+--------------------------------------------------
+// TEST[setup:my_index]
+
+Returns:
+
+[source,console-result]
+--------------------------------------------------
+{
+  "my-index-000001" : {
+    "settings" : {
+      "index" : {
+        "number_of_replicas": "1",
+        "number_of_shards": "1",
+        "creation_date": "1474389951325",
+        "uuid": "n6gzFZTgS664GUfx0Xrpjw",
+        "version": {
+          "created": ...
+        },
+        "routing": {
+          "allocation": {
+            "include": {
+              "_tier_preference": "data_content"
+            }
+          }
+        },
+        "provided_name" : "my-index-000001"
+      }
+    }
+  }
+}
+--------------------------------------------------
+// TESTRESPONSE[s/1474389951325/$body.my-index-000001.settings.index.creation_date/]
+// TESTRESPONSE[s/n6gzFZTgS664GUfx0Xrpjw/$body.my-index-000001.settings.index.uuid/]
+// TESTRESPONSE[s/"created": \.\.\./"created": $body.my-index-000001.settings.index.version.created/]
+
+By default `flat_settings` is set to `false`.
+
+[[fuzziness]]
+[discrete]
+=== Fuzziness
+
+Some queries and APIs support parameters to allow inexact _fuzzy_ matching,
+using the `fuzziness` parameter.
+
+When querying `text` or `keyword` fields, `fuzziness` is interpreted as a
+{wikipedia}/Levenshtein_distance[Levenshtein Edit Distance]
+-- the number of one character changes that need to be made to one string to
+make it the same as another string.
+
+The `fuzziness` parameter can be specified as:
+
+[horizontal]
+`0`, `1`, `2`::
+
+The maximum allowed Levenshtein Edit Distance (or number of edits)
+
+`AUTO`::
++
+--
+Generates an edit distance based on the length of the term.
+Low and high distance arguments may be optionally provided `AUTO:[low],[high]`. If not specified,
+the default values are 3 and 6, equivalent to `AUTO:3,6` that make for lengths:
+
+`0..2`:: Must match exactly
+`3..5`:: One edit allowed
+`>5`:: Two edits allowed
+
+`AUTO` should generally be the preferred value for `fuzziness`.
+--
+
+[discrete]
+[[common-options-error-options]]
+=== Enabling stack traces
+
+By default when a request returns an error Elasticsearch doesn't include the
+stack trace of the error. You can enable that behavior by setting the
+`error_trace` url parameter to `true`. For example, by default when you send an
+invalid `size` parameter to the `_search` API:
+
+[source,console]
+----------------------------------------------------------------------
+POST /my-index-000001/_search?size=surprise_me
+----------------------------------------------------------------------
+// TEST[s/surprise_me/surprise_me&error_trace=false/ catch:bad_request]
+// Since the test system sends error_trace=true by default we have to override
+
+The response looks like:
+
+[source,console-result]
+----------------------------------------------------------------------
+{
+  "error" : {
+    "root_cause" : [
+      {
+        "type" : "illegal_argument_exception",
+        "reason" : "Failed to parse int parameter [size] with value [surprise_me]"
+      }
+    ],
+    "type" : "illegal_argument_exception",
+    "reason" : "Failed to parse int parameter [size] with value [surprise_me]",
+    "caused_by" : {
+      "type" : "number_format_exception",
+      "reason" : "For input string: \"surprise_me\""
+    }
+  },
+  "status" : 400
+}
+----------------------------------------------------------------------
+
+But if you set `error_trace=true`:
+
+[source,console]
+----------------------------------------------------------------------
+POST /my-index-000001/_search?size=surprise_me&error_trace=true
+----------------------------------------------------------------------
+// TEST[catch:bad_request]
+
+The response looks like:
+
+[source,console-result]
+----------------------------------------------------------------------
+{
+  "error": {
+    "root_cause": [
+      {
+        "type": "illegal_argument_exception",
+        "reason": "Failed to parse int parameter [size] with value [surprise_me]",
+        "stack_trace": "Failed to parse int parameter [size] with value [surprise_me]]; nested: IllegalArgumentException..."
+      }
+    ],
+    "type": "illegal_argument_exception",
+    "reason": "Failed to parse int parameter [size] with value [surprise_me]",
+    "stack_trace": "java.lang.IllegalArgumentException: Failed to parse int parameter [size] with value [surprise_me]\n    at org.elasticsearch.rest.RestRequest.paramAsInt(RestRequest.java:175)...",
+    "caused_by": {
+      "type": "number_format_exception",
+      "reason": "For input string: \"surprise_me\"",
+      "stack_trace": "java.lang.NumberFormatException: For input string: \"surprise_me\"\n    at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)..."
+    }
+  },
+  "status": 400
+}
+----------------------------------------------------------------------
+// TESTRESPONSE[s/"stack_trace": "Failed to parse int parameter.+\.\.\."/"stack_trace": $body.error.root_cause.0.stack_trace/]
+// TESTRESPONSE[s/"stack_trace": "java.lang.IllegalArgum.+\.\.\."/"stack_trace": $body.error.stack_trace/]
+// TESTRESPONSE[s/"stack_trace": "java.lang.Number.+\.\.\."/"stack_trace": $body.error.caused_by.stack_trace/]

docs/reference/rest-api/cron-expressions.asciidoc

@@ -1,4 +1,5 @@
-[[cron-expressions]]
+[discrete]
+[[api-cron-expressions]]
 === Cron expressions
 
 A cron expression is a string of the following form:
@@ -16,7 +17,7 @@ All schedule times are in coordinated universal time (UTC); other timezones are
 
 TIP: You can use the <<elasticsearch-croneval>> command line tool to validate your cron expressions.
 
-
+[discrete]
 [[cron-elements]]
 ==== Cron expression elements
 
@@ -51,6 +52,7 @@ Valid values: `1`-`7`, `SUN`-`SAT`, `sun`-`sat`,  and the special characters `,`
 (Optional)
 Valid values: `1970`-`2099` and the special characters `,` `-` `*` `/`
 
+[discrete]
 [[cron-special-characters]]
 ==== Cron special characters
 
@@ -122,9 +124,11 @@ Friday of the month. Note that if you specify `3#5` and
 there are not 5 Tuesdays in a particular month, the
 schedule won't trigger that month.
 
+[discrete]
 [[cron-expression-examples]]
 ==== Examples
 
+[discrete]
 [[cron-example-daily]]
 ===== Setting daily triggers
 
@@ -134,6 +138,7 @@ Trigger at 9:05 a.m. UTC every day.
 `0 5 9 * * ? 2020`::
 Trigger at 9:05 a.m. UTC every day during the year 2020.
 
+[discrete]
 [[cron-example-range]]
 ===== Restricting triggers to a range of days or times
 
@@ -143,6 +148,7 @@ Trigger at 9:05 a.m. UTC Monday through Friday.
 `0 0-5 9 * * ?`::
 Trigger every minute starting at 9:00 a.m. UTC and ending at 9:05 a.m. UTC every day.
 
+[discrete]
 [[cron-example-interval]]
 ===== Setting interval triggers
 
@@ -152,6 +158,7 @@ Trigger every 15 minutes starting at 9:00 a.m. UTC and ending at 9:45 a.m. UTC e
 `0 5 9 1/3 * ?`::
 Trigger at 9:05 a.m. UTC every 3 days every month, starting on the first day of the month.
 
+[discrete]
 [[cron-example-day]]
 ===== Setting schedules that trigger on a particular day
 
@@ -169,6 +176,7 @@ Trigger at 9:05 a.m. UTC on the nearest weekday to the 15th of every month.
 `0 5 9 ? * 6#1`::
 Trigger at 9:05 a.m. UTC on the first Friday of every month.
 
+[discrete]
 [[cron-example-last]]
 ===== Setting triggers using last
 

docs/reference/rest-api/index.asciidoc

@@ -11,6 +11,7 @@ We are working on including more {es} APIs in this section. Some content might
 not be included yet.
 
 * <<api-conventions, API conventions>>
+* <<common-options, Common Options>>
 * <<autoscaling-apis, Autoscaling APIs>>
 * <<cat, cat APIs>>
 * <<cluster, Cluster APIs>>
@@ -46,6 +47,7 @@ not be included yet.
 --
 
 include::{es-repo-dir}/api-conventions.asciidoc[]
+include::{es-repo-dir}/rest-api/common-options.asciidoc[]
 include::{es-repo-dir}/autoscaling/apis/autoscaling-apis.asciidoc[]
 include::{es-repo-dir}/cat.asciidoc[]
 include::{es-repo-dir}/cluster.asciidoc[]

docs/reference/search.asciidoc

@@ -4,7 +4,7 @@
 Search APIs are used to search and aggregate data stored in {es} indices and
 data streams. For an overview and related tutorials, see <<search-your-data>>.
 
-Most search APIs support <<multi-index,multi-target syntax>>, with the
+Most search APIs support <<api-multi-index,multi-target syntax>>, with the
 exception of the <<search-explain,explain API>>.
 
 [discrete]

docs/reference/search/count.asciidoc

@@ -37,7 +37,7 @@ that query. The query can either
 be provided using a simple query string as a parameter, or using the
 <<query-dsl,Query DSL>> defined within the request body.
 
-The count API supports <<multi-index,multi-target syntax>>. You can run a single
+The count API supports <<api-multi-index,multi-target syntax>>. You can run a single
 count API search across multiple data streams and indices.
 
 The operation is broadcast across all shards. For each shard id group, a replica

docs/reference/snapshot-restore/restore-snapshot.asciidoc

@@ -29,7 +29,7 @@ POST /_snapshot/my_backup/snapshot_1/_restore
 
 By default, all data streams and indices in the snapshot are restored, but the cluster state is
 *not* restored. Use the `indices` parameter to restore only specific data streams or indices. This parameter
-supports <<multi-index,multi-target syntax>>. To include the global cluster state, set
+supports <<api-multi-index,multi-target syntax>>. To include the global cluster state, set
 `include_global_state` to `true` in the restore request body.
 
 Because all indices in the snapshot are restored by default, all system indices will be restored

docs/reference/snapshot-restore/take-snapshot.asciidoc

@@ -54,7 +54,7 @@ PUT /_snapshot/my_backup/snapshot_2?wait_for_completion=true
 // TEST[skip:cannot complete subsequent snapshot]
 
 Use the `indices` parameter to list the data streams and indices that should be included in the snapshot. This parameter supports
-<<multi-index,multi-target syntax>>, although the options that control the behavior of multi-index syntax
+<<api-multi-index,multi-target syntax>>, although the options that control the behavior of multi-index syntax
 must be supplied in the body of the request, rather than as request parameters.
 
 Data stream backups include the stream's backing indices and metadata, such as

docs/reference/sql/language/indices.asciidoc

@@ -9,7 +9,7 @@
 [discrete]
 ==== {es} multi-index
 
-The {es} notation for enumerating, including or excluding <<multi-index,multi-target syntax>>
+The {es} notation for enumerating, including or excluding <<api-multi-index,multi-target syntax>>
 is supported _as long_ as it is quoted or escaped as a table identifier.
 
 For example:

docs/reference/sql/language/syntax/commands/show-tables.asciidoc

@@ -27,7 +27,7 @@ patterns.
 include-tagged::{sql-specs}/docs/docs.csv-spec[showTables]
 ----
 
-Match multiple indices by using {es} <<multi-index,multi-target syntax>>
+Match multiple indices by using {es} <<api-multi-index,multi-target syntax>>
 notation:
 
 [source, sql]